How can I tile videos/create a video montage?

Question

I have four videos that I would like to tile in a 2x2 fashion to make a new video. Is there a way I can do this easily, preferably free and under Linux? I am willing to program a moderate amount, perhaps in order to interact with some library, but unwilling to write an entire video-processing program myself. You may assume that the input and output videos are in whatever commonly-occurring format is most convenient.

An analogue of the gm montage command (for images) would be fantastic.

Threewheeler

Answer 1

I am currently using GStreamer for a similar project (lecture-capture) myself. You are probably looking for the videomixer element. Check out this example: Video 4-way split screen gstreamer pipeline (script is located here).

GStreamer works perfectly fine on Windows too. You may want to check out the GStreamer WinBuilds if you are interested.

Example
Here's a basic script that works for me on Windows (it doesn't have the backslashes because I use the gst_parse_launch call from C code to parse the pipeline description):

  v0. ! queue
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox right=-320 bottom=-240
      ! ffmpegcolorspace
      ! vmix.sink_0
  v1. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox bottom=-240
      ! ffmpegcolorspace
      ! vmix.sink_1
  v2. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! videobox right=-240
      ! ffmpegcolorspace
      ! vmix.sink_2
  v3. ! queue   
      ! decodebin
      ! ffmpegcolorspace
      ! videoscale
      ! video/x-raw-yuv,width=320,height=240
      ! ffmpegcolorspace
      ! vmix.sink_3
  vmix. ! queue 
        ! ffmpegcolorspace
        ! dshowvideosink
  filesrc location="c:/test.mpg" name="v0"
  filesrc location="c:/test.mpg" name="v1"
  filesrc location="c:/test.mpg" name="v2"
  filesrc location="c:/test.mpg" name="v3"
  videomixer name=vmix
             sink_0::xpos=0   sink_0::ypos=0   sink_0::zorder=0
             sink_1::xpos=320 sink_1::ypos=0   sink_1::zorder=1
             sink_2::xpos=0   sink_2::ypos=240 sink_2::zorder=2
             sink_3::xpos=320 sink_3::ypos=240 sink_3::zorder=3

Selfinduced

That looks great, but it didn't work for me. However, the other pipeline he linked to DID work, although as claimed, it's inefficient and lacks sound. That's definitely better than what I've got so far. Thanks! Meanwhile, I'll try to figure out why Ray's doesn't work. – Threewheeler

I posted a simpler example. If you want audio then you'll have to choose from which video you want to use audio. Then you'll need to use a named decodebin element for that video and connect it to an audioconvert -> audioresample -> audiosink. If you need any more help feel free to ask. – Selfinduced

Okay, that worked great! Re audio: I don't need it that much, but if you have a way of mixing all of the audio, that would be nice. Re Windows: as my question says, I'm on Linux. I assume that's why "dshowvideosink" doesn't work for me, but I just changed that to filesink. – Threewheeler

In any case, I think this was the best answer because it included complete instructions, not just a pointer to a language that should -- in principle -- be able to complete the task. Thanks for your help! – Threewheeler

Great :) Btw, you can use autovideosink instead of dshowvideosink on Linux. You also may need to add a muxer element before the filesink to have the playback speed correct. – Selfinduced

Answer 2

The following ffmpeg command will do exactly what the questioner wanted:

ffmpeg -i input1.mp4 -i input2.mp4 -i input3.mp4 -i input4.mp4 -filter_complex \
'[0:v]pad=iw*2:ih*2:0:0[int2];[int2][1:v]overlay=0:H/2[int3];[int3][2:v]overlay=W/2:0[int4];[int4][3:v]overlay=W/2:H/2[out]' \
-map [out] -c:v libx264 -crf 23 -preset veryfast output.mp4

First, the pad filter doubles the size of the first input video, leaving the original video in the top-left corner. The serial overlay filters then place the other inputs over the the black padding added by the pad filter.

If the videos are of different resolutions, the command will require some modification.

Pollinize

works like a charm :D runs perfectly with avconv too (same args) if ffmpeg is unavailable – Joinder

Answer 3

This sounds like the sort of problem that AviSynth was designed to solve.

AviSynth is essentially a scripting language for manipulating video streams. A text file describes that operations you want to apply to one or more input video streams. The text file is handed to the AviSynth engine, which provides a virtual .AVI file that manipulates the source streams one frame at a time as you fetch them.

Combine AviSynth with a separate tool that reads from the virtual .AVI file and writes to a new file to save the changes.

Ubangishari

Thanks for your help. Since I had to pick an answer to reward the bounty to, I chose the one with the most complete instructions. Nonetheless, I hope you can get some upvotes for your assistance. – Threewheeler

Sorry to hear that you couldn't install it. It just worked when I installed it a couple years ago. That's the way I remember it anyway ;) – Ubangishari

Answer 4

One possible solution would be to describe the layout of your video montage with SMIL, a multimedia markup language. This requires a text editor for writing your SMIL document and a SMIL video player (e.g., Ambulant, Quicktime or Realplayer) for displaying it.

Greywacke