How hard is dynamic rendering on video?

Question

I'd like to make an app that takes a video input (e.g. using your webcam), and dynamically render something fun on top of it based on its content. For example, I'll detect user's face in the video and change the color of his/her lips. Probably I can ask user to specify the contour of his/her lip using some closed curve with a few controller knobs, and track its change in live video stream, then render some colors on it. How hard is this? Ideally it would evolve into something bigger that allows you to do more content-dependent special effects, like those seen in the movie, but for now I just want to make a toy app that modifies real time video stream with a human face in it. I believe this kind of tools are already well developed in profession video softwares, but I want something fun and runnable on a normal computer, maybe even a phone (iphone, android, etc) with a user facing camera.

Although I am a programmer but I don't have any experience handling video, I was reading something about OpenCV library, looks it's a good start. But I am not sure about the feasibility of this. Is there anything else that might help with this task?

Thanks a lot.

Answer 1

95% of the hard work has already been done. The remaining 5% is up to you.

http://opencv.willowgarage.com/wiki/FaceDetection

There are lots of face recognition examples with OpenCV available on the web.

Processing video can be quite demanding on your CPU. Remember that your algorithm must be executed on 25~30 frames per second in order to give the "real-time" effect.

Answer 2

OpenCV is indeed the best place to start. I would check out the lkdemo.c file in the sample folder for a gentle introduction to drawing things on top of live video. It's under 200 lines of c code and will show you how to read from a webcam or video file, accept keyboard & mouse input, track feature points in video & draw simple shapes on top of them dynamically.

If you started from there, you could probably start experimenting with your own ideas in under an hour, depending on your prior knowledge.