ffmpeg vstats problems, Missing frames and misleading information!

Question

The Problem I have is when using ffmpeg to encode a YUV using libx264 I don't get all the frame information in -vstats output. It raises the question of how reliable ffmpeg is, and therefore can any 'codec benchmark' review based on ffmpeg be trusted?

I am analysing codec's to determine how they perform. I am using ffmpeg and its -vstats option to look at an encoded movie frame by frame. the process I use:

RAW YUV -> bar-code each frame with frame number -> Bar-coded YUV

Bar-coded YUV -> encoded (e.g. with libx264) -> MKV -> Decoded to YUV

I can compare the two outputs ('Bar-coded YUV' & 'Decoded to YUV') using the bar-code in each frame. I can then compare, exactly, an original frame with an encoded frame using PSNR etc.

When encoding using libx264 and libdirac, there are some frame information which is missing. Other codecs, such as mpeg2video or even libvpx, don't have this problem.

I have found that libx264 vstats are missing for the first 40 to 50 frames. I have since proved that the missing information is actually the last 40 to 50 frames.

It also looks like ffmpeg calculates average bitrate based on the information in vstats. But as there is missing frames the average bitrate is less than what it should be.

Below are links to the average bitrate error example:

http://dl.dropbox.com/u/6743276/ffmpeg_probs/ffmpeg_av_bitrate_error.png

http://dl.dropbox.com/u/6743276/ffmpeg_probs/ffmpeg_av_bitrate_error.xlsx

Below is a link to the PSNR & f_size graph:

http://dl.dropbox.com/u/6743276/ffmpeg_probs/frame_mismatch.png

Below is a link to the output & command line options:

http://dl.dropbox.com/u/6743276/ffmpeg_probs/stderr.txt

I think this is also a bug, anyone clever enough to work it out might want to follow this tracker: http://roundup.ffmpeg.org/issue2248

Answer 1

I have just discovered something which makes me very red in the face!! quite annoyed, but never mind :)

A fellow ffmpeg user pointed out that ffprobe should output more frame info, which it did. here is a link to his handy tip:

http://forums.creativecow.net/thread/291/71

Using this I found the following:

Actual average bitrate (ffprobe data): 8355.2776056338
Actual average bitrate (ffmpeg vstats data): 8406.23275471698
Ffmpeg -vstats avg_br: 7816.3
Reproduced above: 7816.32168421053
Ffmpeg standard error output 'bitrate=': 8365.8

Below is a link to my workings out:

http://dl.dropbox.com/u/6743276/ffmpeg_probs/ffprobe_vs_ffmpeg-vstats.xlsx

What I have discovered is I should have been using the average bitrate info from ffmpeg standard error output, it looks like the most reliable!