Can I use the Gstreamer API to merge 2 videos?

Question

I'd like to write a simple linux CLI application that can take 2 video sources (1 of a presenter talking and 1 with their slides and no audio) and merge them.

I'd like the entire output video to be the two original videos, side by side. Failing that, my second best option would be a "picture in picture" style video, with the presenter in a small frame in the corner.

From a few hours research, GStreamer looks like it might be able to do this. Can anyone confirm it before I spend more time trying it out?

If it can't, are there other APIs out there that I might be able to use?

Answer 1

AviSynth comes to my mind. I’ve used it many years ago under Windows and it’s pretty good at arbitrary post-processing. AviSynth v3 is supposed to run natively under Linux but is still far from ready. There are tools to run previous version with Wine, though.

Answer 2

MEncoder can do that natively on linux. You can fork their code, or invoke the binary.

Answer 3

It turns out gstreamer can merge two videos, placing them side by side into an output video using the videomixer filter.

A basic pipeline that takes two input files, scales them to be the same size, then merges them and encodes them into a theora video might look like this:

filesrc -> decodebin -> ffmpegcolourspace -> videoscale ->  videobox -> videorate
                                                                                  \
filesrc -> decodebin ->  ffmpegcolourspace  ->  videoscale  ->  videorate   ->    videomixer -> ffmpegcolourspace -> theoraenc -> oggmux -> filesink

How you implement this pipeline depends on the language. I prototyped with the Ruby bindings, and it works really well.