How Image Compression Really Works

[combatentropy]

Wrong, Chevy Chase! There will be lots of math. But it is presented by someone who explains things better than I have seen most people explain things.

The Discrete Cosine Transform is the basis for most of the codecs we use: JPEG, MPEG, ProRes, etc. (The only other major kind is wavelets [ JPEG2000, Cineform, and Redcode ] but I haven't found a video for that yet.) If right now your answer to how these codecs compress images by a factor of 10 is "magic," but are mathematically inclined, you might enjoy this set of videos.

Part 0: Color spaces (7:30)
Part 1: Color downsampling and JPEG (7:17)
Part 2: The Discrete Cosine Transform (15:11)

 

[jcs]

Cool animation from: https://en.m.wikipedia.org/wiki/Discrete_cosine_transform

Coded the DCT when doing spectral image filtering. Very cool how it works. Very similar to the DFT/FFT used for spectral audio filtering (iDCT and iFFT converts back after processing in the frequency domain). h.264 uses variable sized macroblocks and not just square. For IPB, motion vectors describe how to move around macroblocks.

Wavelets and the DWT are cascaded filters which operate on the entire image instead of using macroblocks. The advantage is there will never be any macroblock edge artifacts or aliasing caused by macroblocks (which require special expensive filters to fix). This makes wavelets fast to decode, and variable quality decompression (blurrier images) is easy for real-time previews of very high res content. Disadvantages are only competitive with DCT for light compression- no good solutions to compete with interframe DCT/IPB.

Future AI compression will use a large database of patterns to provide very efficient compression (will need AI hardware to compute).

 

[cyvideo]

Future AI compression will use a large database of patterns to provide very efficient compression (will need AI hardware to compute).

As is already being used by Sony on their proprietary "Clear Image Zoom"

"What actually powers Clear Image Zoom technology is as surprising as it is impressive. There is a database of common objects held in the camera, and the processor uses this data to recognise objects in the scene, and to help it to zoom more accurately.

Let’s say that the camera recognises that the object in a scene is a bicycle and not, for example, a fish. It therefore knows that the two round objects are wheels with spokes. Knowing this, it can recreate the scene, based on what it thinks it is, more accurately, at a higher level of zoom.

You probably realise that none of this is trivially easy. In fact, to do this, you need elements of artificial intelligence and machine vision. It’s not the sort of thing you’d expect to find in a camera."

Warning! AI is already amongst us.

https://www.redsharknews.com/techno...-really-going-on-with-sony’s-clear-image-zoom

Chris Young

 

[jcs]

As is already being used by Sony on their proprietary "Clear Image Zoom"

"What actually powers Clear Image Zoom technology is as surprising as it is impressive. There is a database of common objects held in the camera, and the processor uses this data to recognise objects in the scene, and to help it to zoom more accurately.

Let’s say that the camera recognises that the object in a scene is a bicycle and not, for example, a fish. It therefore knows that the two round objects are wheels with spokes. Knowing this, it can recreate the scene, based on what it thinks it is, more accurately, at a higher level of zoom.

You probably realise that none of this is trivially easy. In fact, to do this, you need elements of artificial intelligence and machine vision. It’s not the sort of thing you’d expect to find in a camera."

Warning! AI is already amongst us.

https://www.redsharknews.com/techno...-really-going-on-with-sony’s-clear-image-zoom

Chris Young

Here's a photo I took in 2009 on a Canon SD900ti camera at Mount Rushmore:

You can explore the EXIF data here: https://www.flickr.com/photos/johncschultz/3913214713/in/dateposted-public/ (unedited photo)

This was with a simple point and shoot digital camera, no double exposure etc. What was most puzzling was the apparent correct perspective and location of the artifacts (ceiling, floor, something odd above window center screen) and 'alpha blending' quality. Was there some form of AI present in 2009 that built this strange composite (bug?), amazing coincidence framebuffer / blending bug, or something more entertaining related to advanced physics?