Custom curves and JPEG artifacts

[bearing]

I've been thinking about the possibilities given with custom curves. Can a curve be made to make the JPEG-compression remove less information and show less artifacts?

My first thought was to use a curve that preserves as much of the raw data as possible. It is possible to make a curve that saves the raw 12-bit values into 8-bits using a logarithmic curve combined with some pure raw values. My idea is to map raw values 0-63 into 8-bit values 0-63 and put raw 64-4096 into 8-bit 64-256 using a log curve with 32 values per stop. When converting back to 12-bit colorspace the shadows will contain much more details and the highlights less, compared to a pure linear mapping.

Will a curve similar to above also help to decrease JPEG-compression artifacts?
We know that pictures containing much details gets higher bitrates. Saving more color details should thus give higher bitrates. Whether those high bitrate pictures will have less artifacts I'm not sure of.

How will the optimal curve that minimizes JPEG-artifacts look like?

[seejay1031]

I have no idea, but I love the thought process.

And it got me to thinking about B&W. Would the mjpeg codec have an easier time and less artifacts in a B&W mode?

I hope one of the codec gurus stop by this thread.

[ando]

don't know about compression, you'll have to test that either with video or with lowest quality jpegs. but in a way could work. jpeg will average areas of the image. if there is more detail in there it won't make a "mud" out of it. i think you had a very smart idea to test.

but curves' main purpose is to extract a different exposure from the raw.
so in the area of your main interest (say shadows and midtones) you could draw a curve which provides more values for interpolation than the linear one. the resulting jpeg range is expanded that way. basically you fit more dynamic range which is exactly what you want. you want for example finer highligts so you won't get clipping as fast as jpeg usually does, that is possible.

it would be interesting to test if b&w d90 videos show less compression and they could, i can't say without testing.

i'll post again some links

http://planetneil.com/tangents/2008/03/16/nikon-custom-curves/
http://fotogenetic.dearingfilm.com/custom_tone_curves.html
http://www5e.biglobe.ne.jp/~komin/curve/D70/d70download.htm

so not only you could fix your jpeg exposure but you could also mimic film types, color or b&w.
the easiest way is to get the demo for 2 nikon plugins for photoshop and NX. one is silver efex and the other is color efex pro.
they have presets for lots of film types with tone curves. by looking at a preset's curve you can understand how you should draw your custom curve to make it look like that, as exposure. then you must tweak the other parameters from the picture controls for the colors.

anther thing to know is that digital has more usable dynamic range in midtones-highlights than shadows.
so don't underexpose your movie, never. or your image even if raw. you underexpose later in post if you need that but always try to shoot with good exposure.
the photo rule of exposing to the right was based on that.
also the higher the iso the less usable dynamic range you get. if you take a picture at iso 100 you can push and pull that raw heavily. the same image at iso 800 has much less processing range.

[Matthew Bennett]

I'm personally a huge fan of underexposure but this camera has no interest in it.

[Matthew Bennett]

I've been thinking about the possibilities given with custom curves. Can a curve be made to make the JPEG-compression remove less information and show less artifacts?

My first thought was to use a curve that preserves as much of the raw data as possible. It is possible to make a curve that saves the raw 12-bit values into 8-bits using a logarithmic curve combined with some pure raw values. My idea is to map raw values 0-63 into 8-bit values 0-63 and put raw 64-4096 into 8-bit 64-256 using a log curve with 32 values per stop. When converting back to 12-bit colorspace the shadows will contain much more details and the highlights less, compared to a pure linear mapping.

Will a curve similar to above also help to decrease JPEG-compression artifacts?
We know that pictures containing much details gets higher bitrates. Saving more color details should thus give higher bitrates. Whether those high bitrate pictures will have less artifacts I'm not sure of.

How will the optimal curve that minimizes JPEG-artifacts look like?

This camera is so intent on holding a JPEG smokehouse on your images, that I'm sure any master plan regarding 'gentling' the compression will be met with absolute and bitter frustration.
Go for it, though, what do I know about jpegs.

At this point in the discussion, it would interesting to have a representative from the Nikon design team here... although that will very likely never happen.

It's like these products drop from the heavens and we're left, like primitive savages, to figure out how to bring them to a high level of use.

[kevm14]

If you wanted clean shadows, it seems to me you could just set your curve up to bring them up, which will also compress highlights, then push it back down in post. You just have to prioritize what you want for a particular scene.

[Demistate]

Theres also a piece of software called NeatVideo (http://neatvideo.com) that i've been using to clean up Noisy HVX-200 video. It makes wonderful results, and you can create profiles for it to speed up your de-noising. It works on Jpeg Artificats, and Camera noise.

[bearing]

I have been playing with curves with my D40. How come curves affect RAW? i thought RAW was straight from the sensor.

Is there a way to load a table instead of a curve?. I would like to map raw 12-bit sensor values to 8-bit.

[ando]

it doesn't affect raw
it affects the default values in the raw file. since your display and printer are 8 bit with an srgb-type color profile they can't show the 12 bit raw which doesn't have a color profile. default values are used taken from current camera settings. but there is no property like sharpness, WB, saturation, color modes etc in the raw file. you pick any you like.

you're free to change them without quality loss. either in view nx or capture nx. and also with 3rd party raw developers but most replace them with generic defaults. only the latest ACR beta has some user defined raw conversion parameters so you can make presets for your camera instead of using adobe's generic.

sure some have the money for hp's adobergb lcd display with 10bit per channel depth. but that's another story