Combine multiple videos into one.

Question

I have three videos:

• a lecture that was filmed with a video camera
• a video of the desktop capture of the computer used in the lecture
• and the video of the whiteboard

I want to create a final video with those three components taking up a certain region of the screen.

Is open-source software that would allow me to do this (mencoder, ffmpeg, virtualdub..)? Which do you recommend?

Or is there a C/C++ API that would enable me to create something like that programmatically?

Edit
There will be multiple recorded lectures in the future. This means that I need a generic/automated solution.

I'm currently checking out if I could write an application with GStreamer to do this job. Any comments on that?

Solved!
I succeeded in doing this with GStreamer's videomixer element. I use the gst-launch syntax to create a pipeline and then load it with gst_parse_launch. It's a really productive way to implement complex pipelines.

Here's a pipeline that takes two incoming video streams and a logo image, blends them into one stream and the duplicates it so that it simultaneously displayed and saved to disk.

  desktop. ! queue
           ! ffmpegcolorspace
           ! videoscale
           ! video/x-raw-yuv,width=640,height=480
           ! videobox right=-320
           ! ffmpegcolorspace
           ! vmix.sink_0
  webcam. ! queue
          ! ffmpegcolorspace
          ! videoscale
          ! video/x-raw-yuv,width=320,height=240
          ! vmix.sink_1
  logo. ! queue
        ! jpegdec
        ! ffmpegcolorspace
        ! videoscale
        ! video/x-raw-yuv,width=320,height=240
        ! vmix.sink_2
  vmix. ! t.
  t. ! queue
     ! ffmpegcolorspace
     ! ffenc_mpeg2video
     ! filesink location="recording.mpg"
  t. ! queue
     ! ffmpegcolorspace
     ! dshowvideosink
  videotestsrc name="desktop"
  videotestsrc name="webcam"
  multifilesrc name="logo" location="logo.jpg"
  videomixer name=vmix
             sink_0::xpos=0 sink_0::ypos=0 sink_0::zorder=0
             sink_1::xpos=640 sink_1::ypos=0 sink_1::zorder=1
             sink_2::xpos=640 sink_2::ypos=240 sink_2::zorder=2
  tee name="t"

Answer 1

If you just want to combine footage into a single video and crop the video, I'd use virtual dub.

Answer 2

avisynth can do it rather easily. Look here under the Mosaic section for an example.

I've used ffmpeg quite a bit and I have never stumbled upon this functionality, but that doesn't mean it isn't there. You can always do it yourself in C or C++ with libavformat and libavcodec (ffmpeg libraries) if you're looking for a project, but you will have to get your hands very dirty with compositing the video yourself. If you are just looking to get the video done and not tinker with code, definitely use a pre-made tool like avisynth or virtualdub.

Answer 3

It can be done with ffmpeg; I've done it myself. That said, it is technically complex. That said, again, it is what any other software you might use is going to do in its core essence.

The process works like this:

1. Demux audio from source 1 to raw wav
2. Demux audio from source 2
3. Demux audio from source 3
4. Demux video from source 1 to MPEG1
5. Demux video from source 2
6. Demux video from source 3
7. Concatenate audio 1 + audio 2 + audio 3
8. Concatenate video 1 + video 2 + video 3
9. Mux audio 123 and video 123 into target
10. encode to target format

I think what surprises folks is that you can literally concatenate two raw PCM wav audio files, and the result is valid. What really, really surprises people is that you can do the same with MPEG1/h.261 video.

Like I've said, I've done it. There are some specifics left out, but it most definately works. My program was done in a bash script with ffmpeg. While I've never used the ffmpeg C API, I don't see why you could not use it to do the same thing.

It's a highly educational project to do, if you are inclined. If your goal is just to slap some videos together for a one off project, then maybe using a GUI tool is a better idea.