Ok, I get the basics of video format - there are some container formats and then you have core video/audio formats. I would like to write a web based application that determines what video/audio codec a file is using.
How best can I programmatically determine a video codec? Would it be best to use a standard library via system calls and parse its output? (eg ffmpeg, transcode, etc?)
mplayer -identify will do the trick. Just calling ffmpeg on a file will also work--it will automatically print a set of info at the start about the input file regardless of what you're telling ffmpeg to actually do.
Of course, if you want to do it from your program without an exec call to an external program, you can just include the avcodec libraries and run its own identify routine directly.
While you could implement your own detection, it will surely be inferior to existing routines given the absolutely enormous number of formats that libav* supports. And it would be a rather silly case of reinventing the wheel.
Linux's "file" command can also do the trick, but the amount of data it prints out depends on the video format. For example, on AVI it gives all sorts of data about resolution, FOURCC, fps, etc, while for an MKV file it just says "Matroska data," telling you nothing about the internals, or even the video and audio formats used.
You need to start further down the line. You need to know the container format and how it specifies the codec.
So I'd start with a program that identifies the container format (not just from the extension, go into the header and determine the real container).
Then figure out which containers your program will support, and put in the functions required to parse the meta data stored in the container, which will include the codecs.
You really want a big database of binary identifying markers to look for near the start of the file. Luckily, your question is tagged "Linux", and such a dabase already exists there; file(1) will do the job for you.
I have used FFMPEG in a perl script to achieve this.
$info = `ffmpeg -i $path$file 2>&1 /dev/null`;
@fields = split(/\n/, $info);
And just find out what items in @fields you need to extract.