Compressing three individual jpeg pics containing temporal redundancy?

Question

I am interfacing an embedded device with a camera module that returns a single jpeg compressed frame each time I trigger it.

I would like to take three successive shots (approx 1 frame per 1/4 second) and further compress the images into a single file. The assumption here is that there is a lot of temporal redundancy, therefore lots of room for more compression across the three frames (compared to sending three separate jpeg images).

I will be implementing the solution on an embedded device in C without any libraries and no OS.

The camera will be taking pics in an area with very little movement (no visitors or screens in the background, maybe a tree with swaying branches), so I think my assumption about redundancy is pretty solid.

When the file is finally viewed on a pc/mac, I don't mind having to write something to extract the three frames (so it can be a nonstandard cluge)

So I guess the actual question is: What is the best way to compress these three images together given the fact that they are already in JPEG format (it is a possibly to convert back to a raw image, but if i dont have too...)

Answer 1

While I've not studied signals since uni, I think you're looking for a lossless video codec.

Huffyuv is one that's been around, and has source code available. The basic concept is to predict the pixel changes between each frame, and encode (and compress) the difference between predicted and actual changes.

Lagarith is another open source codec.

You'll need to feed the decoded JPEG frames into each of these codecs.

Answer 2

If I were you, I'd use your system to take three pictures by hand right now so you can check your assumptions before going much further.

My guess is that you will need a slight translation even if you don't intend any movement. Vibration of equipment, wind and even heat expansion might be enough to throw you off by a pixel or two which would ruin a straight pixel-to-pixle compression.

Other factors could be light change due to a cloud passing across the sun or heat magnification wafting off the ground or even jpeg compression artifacts.

I'm not saying it's not going to work, just that I'd run one by hand first.

Storage is so cheap, you are going to get much more bang for the buck by adding a larger sim card (or whatever) to your camera.

Answer 3

Any decode/modify/recode on the JPEG images may lower the image quality, but as your camera can only capture JPEGs, I'm guessing ultimate image quality is unlikely to be a key requirement...

I can't think of an easy way you can do this in the JPEG frequency domain, but you can decompress then SUBTRACT images 2 and 3 from image 1 to get delta images. These should compress a lot better, and would be added back to image #1 by the receiver.

It turns out there are some operations you can do in the compressed domain that might help. You'd need to uncompress the Huffman/RLE stages of the jpeg, and then work on the DCT coefficients directly. You could well be able to do image subtraction this way, and it should not introduce further artefacts.

Answer 4

I'm adding this as a second answer because it's VERY different from my first now that I better understand your problem.

I find it HIGHLY unlikely that you will be able to work with the jpeg files directly. In compressed files, a small change tends to be propagated across a large portion of the file, causing the two files to fail to compare in many places.

I have two suggestions.

1: zip the images up. Seems too simple, you probably already thought of it, but the zip protocol is well known and freely available and will automatically take advantage of any similarities it can. Again just get a camera take three pictures; zip them up and see how it goes.

2: A little more complex but you could uncompress the three jpegs into bmps, concatenate the bmps (line them up one after the other) then re-compress to a jpeg. The jpeg protocol should take full advantage of the similarities in the three images and the work is pretty minimal from your point of view.