How to attach metadata to each frame of a video

Question

I would like to be able to store metadata on each frame of a video file (a key to value map with ~10 entries per frame).

Which video format allows me to store such data ?

Where can I find a c++ library to do this ?

Thank you very much for your answers.

Answer 1

FFmpeg does such thing.

ffmpeg -i in.avi -metadata title="my title" out.flv

So if you want a library to do this, you can take a look at libavcodec (the leading audio/video codec library).