MJPEG internet streaming - accurate fps

Question

Hello,

I want to write MJPEG picture internet stream viewer. I think getting jpeg images using sockets it's not very hard problem. But i want to know how to make accurate streaming.

while (1)
{
     get_image()
     show_image()
     sleep (SOME_TIME) // how to make it accurate?
}  

Any suggestions would be great.

Answer 1

In order to make it accurate, there are two possibilities:

Using framerate from the streaming server. In this case, the client needs to keep the same framerate (calculate each time when you get frame, then show and sleep for a variable amount of time using feedback: if the calculated framerate is higher than on server -> sleep more; if lower -> sleep less; then, the framerate on the client side will drift around the original value from server). It can be received from server during the initialization of streaming connection (when you get picture size and other parameters) or it can be configured.

The most accurate approach, actually, is using of timestamps from server per each frame (which is either taken from file by demuxer or generated in image sensor driver in case of camera device). If MJPEG is packeted into RTP stream, these timestamps are already in RTP header. So, client's task is trivial: show picture using time calculating from time offset, current timestamp and time base.

Update For the first solution:

time_to_sleep = time_to_sleep_base = 1/framerate;
number_of_frames = 0;
time = current_time();
while (1)
{
     get_image();
     show_image(); 
     sleep (time_to_sleep);

     /* update time to sleep */
     number_of_frames++;
     cur_time = current_time();
     cur_framerate = number_of_frames/(cur_time - time); 
     if (cur_framerate > framerate)        
         time_to_sleep += alpha*time_to_sleep;
     else 
         time_to_sleep -= alpha*time_to_sleep;
     time = cur_time;
} 

, where alpha is a constant parameter of reactivity of the feedback (0.1..0.5) to play with.

However, it's better to organize queue for input images to make the process of showing smoother. The size of queue can be parametrized and could be somewhere around 1 sec time of showing, i.e. numerically equal to framerate.