Parallelization and H.264 (or probably any compression codec)? Why's it so hard?

Question

My limited (and probably wrong?) understanding of video compression is that intra-frames are completely independent. In other words, all the picture data for a intra frame (key frame) is stored in its entirety for that frame. It's the following inter frames (P and B frames in H.264, I think) that depend on the data in other frames to be "drawn."

If these intra frames are completely independent, why isn't encoding an embarrassingly parallel problem? If you had N processors and X I-frames, you could just give X/N chunks of the source to each processor to encode independently, then patch them all together at the end, right? But it seems that this isn't the case--or at least, I haven't seen any encoders that have the kind of parallelization that being able to do this would give. What am I not understanding?

Answer 1

You are right. Parallelization makes it work faster.

In fact, the x264 encoder provides parallel encoding ability.

http://www.videolan.org/developers/x264.html

Answer 2

The first thing to consider is where you want to put your intra frames. For the best compression you need to choose this wisely, preferring scene changes to the middle of a static sequence for example. To find the best places you need to analyse the raw video, which can either be done in an extra pass (expensive) or it can be decided on the fly as you encode.

So to parcel up the stream into chunks you either need to make an extra pass to analyse it or just arbitrarily divide it up and lose some compression efficiency.

Then you have to think about how you've got this raw video to encode. Either it's streaming into your compression program from somewhere or the whole thing is available on disk.

If it's streaming then you're out of luck as you don't have random access to different parts of the stream. Yes, you could buffer it but a quick calculation shows that this would either need a LOT of memory or you'd have to buffer to disk, which leads on to the next point:

If you have the whole raw file stored locally then you could parcel out portions to different processes or threads. Except that your problem would now be disk access! Consider that the data rate for raw 1080p, 24fps video is about 4 GB per minute. With a single process encoding it the disk will be heavily occupied providing the raw data. It could even be the slowest part of the process (though probably not unless your hard-drive is very fragmented!)

Now think about getting 4 processes to access the same file, all trying to grab the raw data at extremely high rates. This hard-drive just won't be able to keep the encoders fed with data - the weak link won't be slow processors but slow data access.

So unless you've got some really professional kit to store your uncompressed video, parcelling up different sections for parallel encoding just isn't practical yet.