Do you have the shape layer selected before you start drawing the mask with the pen tool? If no layer is selected, you're actually making a shape layer, not a mask
Sorry, I meant command+click. To be clear here, are you talking about the "multiply" transfer mode that you see similiar in...
That's the purpose for lossless codecs. They are mathematically lossless. You incur no generation losses if you remain in the same color model/ color space (you can re-encode it a zillion times, and decoded image will be bit for bit identical) . If you're familiar with audio, the analogy would...
It depends how your shot is setup - if you need alpha channel preserved or not
Most formats that preserve alpha and are lossless will be RGBA (4:4:4:4)
There are plenty of lossless 4:2:0 codecs for windows , but I don't know of any that have alpha 4:2:0:4
There maybe a few 4:2:2:4 codecs...
Are you saying you want to "dumb down" the generated RGB material to match the other 4:2:0 footage ?
You could export a 4:2:0 format then reimport it. AE uses nearest neighbor algorithm to upsample chroma, so it will apply that to the reimported footage
Another approach is to use another...