Would like help thinking algorithmically about server-side video encoding.

views:

answers:

+4 Q:

Would like help thinking algorithmically about server-side video encoding.

I've been learning PHP and MySQL and have been thinking about exercises to help me learn the language better. I've used ffmpeg for a few years now to process video files and thought I could probably work out a way to manage this from the web. I've read up on using ffmpeg from php but I'm wondering how to manage the files on the backend. Algorithmically speaking how does one make it so that a users file stays connected to their account during the processing of the file in the queue? Ideally I would like to have it where people in my small department could submit jobs to it in a queue.

Would you keep track of the files based on a generated file name and store that in the database?
If this is the case how does one keep track of whether the file processed correctly?
What's the best way to do process checking? I've googled around and I haven't found a clear explanation on how one keeps track of the ffmpeg process for reporting purposes.
Is it best to manage the queuing of encodes from a cron job or is there a better way?

+2 A:

Once upon a time, I created a very similar system with Ruby/sqlite

First of all forget about keeping the users connected - encoding takes a very long time. Its more appropriate to email a user once the operation is complete, or for some other process to poll on status. Think of this more as a web service.

You can store information about the client or a request id, as well as the input and output file paths in a job structure in a database.
The jobs form a queue, each one has a status (waiting/processing/finshed-good/finished-fail). The encoder task (or tasks) take jobs from the queue and process them, updating the status as necessary.
You have to do some horrible command-line scanning to get progress. There's a few SO questions about it: http://stackoverflow.com/search?q=ffmpeg+progress
There needs to be some kind of server or service running, that is waiting for jobs to arrive, adding them to the queue, and notifying users of complete jobs.

Justicle 2010-02-16 04:00:03

Thanks for the great reply. I like your idea about keeping status updates. I'm assuming that you can just run a test to see if the file completes conversion successfully? Is there any advantage to storing the converted files in a folder for each user or to keep them in one directory and just store the path for each user? Thanks again.

rschapman 2010-02-16 04:42:27

You can't make any assumptions about the output file because you don't know how big it should be or at what point ffmpeg "flushes" work out. FFmpeg does output a "complete" command line message when finished, and a separate message when there is an error. Where you store the files depends on your particular needs - is convenience more important than security? If so dump them in one folder.

Justicle 2010-02-16 05:49:01

Thanks again. Is it possible for the executed ffmpeg script to output that "complete" command back to PHP or will it be necessary to pull that data from a log file each time? I've dumped the logs from ffmpeg and have seen the errors but the errors can vary depending on the problem so I'm wondering if there's a generic "complete" for "fail" that can be counted on to be returned or if I have to just code it to accept multiple error outputs.

rschapman 2010-02-16 16:42:45

I can't remember what the output looks like to be honest. Depending on how you spawn ffmpeg the executable may return an error code (EXIT_FAILURE, EXIT_SUCCESS) to caller process. You've got a good architecture now, this is an implementation detail.

Justicle 2010-02-16 22:11:05

+1 A:

yes i did all the things http://vipinkrsahu.blogspot.com/2010/01/how-to-make-video-streaming-sites-like.html please have a look

vipinsahu 2010-02-17 09:23:52

ansaurus

tags:

views:

answers:

Would like help thinking algorithmically about server-side video encoding.

related questions