How can I script video playback with output to multiple screens?

Question

Background

I'm attempting to craft a simple video playback script for a small cinema that automates the playing of videos and control of the projector, sound and lighting systems. I have two video outputs, one goes to a monitor in the projection booth, and the other directly to the projector. I desire to play video (and only video) fullscreen to the projector while putting controls and a small (~1/4 screen) preview on the monitor. This will allow the projectionist to view the video being output and control the playback from the monitor in the booth while all the audience ever sees is the video output.

Problem

I am currently using Python to control VLC player (with libvlc Python bindings) to playback videos. I have everything working fine except that I can't figure out how to get a preview (direct copy) of the video being played fullscreen on the projector output into my GUI.

I have tried using the clone filter, but I cant get the cloned window to automagically appear full screen nor in my GUI. The clone filter seems like the logical choice but it seems to be VERY inflexible when it comes to specifying destination screens, fullscreen, etc. I must be able to open video windows full screen on the projector monitor. Professionalism is key and it would look bad if the projectionist had to drag a window over and double click on it when the movie started.

Currently Using:

• Debian Linux
• Python 2.7
• wxPython
• libvlc

I would like to continue using Python as I already have the code for controlling the projector, sound processor, lighting and curtain written and tested. I chose VLC because it really seems bulletproof when it comes to video playback but am not committed to it's continued use. I also chose wxWidgets for my GUI as a result of past experience but I am not stuck on that either.

Phytopathology

I don't think it is a good idea to have a "GUI" at all for this purpose. The machine that plays the video should not be touched by a user at all. If you want to double video signals, you typically invest into the needed hardware. However this is a software board, so may i ask if you made some thoughts about transcoding a preview and in your gui you receive and show the transcoded stream? Is your GUI the owner of the decoding process for main output? – Other

While I would generally tend to agree with you my client is looking to replicate a high-end "show player" that also has a GUI and supports interfacing with other automation equipment in a sort of cue list. The original goal was to show a small preview of the playing video within that window while also rendering the full size version on a separate "screen". Our less than ideal workaround has been to split the output to the projector and display that on a second monitor for a preview to the projectionist. I'm still bewildered that I cannot simply copy the video output to another frame buffer. – Phytopathology

Even if you had a way to copy within your gui, the result would not ever be as accurate as splitting the final output signal as you do currently. However, what you do by now is to use a very high level api, controling a software player just as you would do with mouse clicks. If you go one level deeper, developing your own player, you end up in months of work. i can give you some tipps for that but is that really what you want to do?... especially because you already have a pretty well working system. Also do not forget my idea of live encoding to the gui... WebRTC could help here – Other

I am in full agreement, I do not want to get into the business of creating my own player or even tweaking the business logic of an existing, open source player. The only thing that I potentially have on my side is that the "preview" window that would be visible to the projectionist does not have to be 100% accurate. My biggest requirement, however, is that the primary video playback be seamless which has proven to be a bigger issue and is probably worthy of it's own SO question. – Phytopathology

Answer 1

This describes the direct solution and does not concentrate on any alternative or the overall design of your application.

As Your Application and VLC media player are separate processes, you will not be able to get what you want directly because there is no "shared memory" between those 2 applications. The best shot to "copy" the decoded frames from VLC will be to e.g. send a RAW Video .mts stream (ts is usually used for this kind of usecase) and send e.g. to udp://localhost:1234.

In your application, you will need to be able to receive the ts stream, "decode" it and display at the spot of interest.

For start, i would try if you are able to do this using 2 vlc players that you control manually. When you achieved that the first VLC streams to udp and outputs on the main display at the same time, and the other VLC player receives and plays the udp stream you can go on:

Find a player library that you can use directly in your wxpython application and check if it can receive the udp stream as well E.g.

https://wxpython.org/Phoenix/docs/html/wx.media.MediaCtrl.html

This player lib for example requires gstreamer as a base.

As a result, main display and the picture in your applicatoin might have a latency of some seconds. To come around this latency, the best way that i currently know is using WebRTC but this is a lot more complex setup than the above.

https://www.sipwise.org/news/technical/tv-over-webrt/

Sure in case you do some "encoding" for WebRTC or even for UDP, you would need to utilize some hardware encoder, e.g. Nvidia NVENC in order to be able to guarantee the needed resources are always there.

Other