Streaming video in memory with OpenCV VideoWriter and Python BytesIO

Question

I was wondering if it is possible to 'stream' data using the OpenCV VideoWriter class in Python?

Normally for handling data in memory that would otherwise go to disk I use BytesIO (or StringIO).

My attempt to use BytesIO fails though:

import cv2
from io import BytesIO

stream = cv2.VideoCapture(0)
fourcc = cv2.VideoWriter_fourcc('x264')

data = BytesIO()

# added these to try to make data appear more like a string
data.name = 'stream.{}'.format('av1')
data.__str__ = lambda x: x.name

try:
    video = cv2.VideoWriter(data, fourcc=fourcc, fps=30., frameSize=(640, 480))
    start = data.tell()

        # Check if camera opened successfully
        if (stream.isOpened() == False): 
            print("Unable to read camera feed", file=sys.stderr)
            exit(1)

        # record loop
        while True:
            _, frame = stream.read()
            video.write(frame)
            data.seek(start)
            # do stuff with frame bytes
            # ...

            data.seek(start)

    finally:
        try:
            video.release()
        except:
            pass
finally:
    stream.release()

However instead of writing the BytesIO object I end up with the following message:

Traceback (most recent call last):
  File "video_server.py", line 54, in talk_to_client
    video = cv2.VideoWriter(data, fourcc=fourcc, fps=fps, frameSize=(width, height))
TypeError: Required argument 'apiPreference' (pos 2) not found

... So when I modify the VideoWriter call to be cv2.VideoWriter(data, apiPreference=0, fourcc=fourcc, fps=30., frameSize=(640, 480)) (I read that 0 means auto, but I also tried cv2.CAP_FFMPEG), I instead get the following error:

Traceback (most recent call last):
  File "video_server.py", line 54, in talk_to_client
    video = cv2.VideoWriter(data, apiPreference=0, fourcc=fourcc, fps=fps, frameSize=(width, height))
TypeError: bad argument type for built-in operation

So my question is, is it possible to write encoded video using the cv2.VideoWriter class in memory and if so how is it done?

At this point I'm fresh out of ideas, so any help would be most welcome :-)

Engler

Are you trying to encode video without writing to disk? It's not possible in OpenCV. – Inferior

Answer 1

Unfortunately, OpenCV doesn't support encoding to (or decoding from) memory. You must write to (or read from) disk for VideoWriter (or VideoCapture) to work.

Crucifer

I was afraid this was the case. I wanted to encode the video before piping over the network to reduce the amount of data sent, however I was trying to get around having to write the data to disk... I guess I'll fallback to sending individual images over the network. – Engler

Yes, it is unfortunate OpenCV doesn't support this feature as I'm sure it would be used heavily. – Crucifer

Answer 2

If you want some convenience to load a video that is already in memory, use a temporary file: https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile

import tempfile
import cv2

my_video_bytes = download_video_in_memory()

with tempfile.NamedTemporaryFile() as temp:
   temp.write(my_video_bytes)

   video_stream = cv2.VideoCapture(temp.name)

   # do your stuff.

This will still create a file on disk unfortunately. But hey, at least you don't have to manage that yourself. There is a SpooledTemporaryFile() implementation that will stay in memory, but, unfortunately, it won't create a file system name that OpenCV can reference.

EDIT UPDATE:

Exploring the Linux interface a bit more, in looks like you can very much utilize a temporary file and have it only exist in memory by utilizing the tmpfs utility.

Tested on an Ubuntu machine, a NamedTemporaryFile() object without specifying a dir will follow the TMPDIR (which happens to point to /tmp on my machine. Now, not sure about other OS systems, but my machine did not have /tmp mounted with tmpfs. It was mounted on the main partition.

But, it's super easy to just create your own "in memory file system" like so: sudo mount -t tmpfs -o size=1G tmpfs my_folder

If we check the output of df my_folder -h I see this:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.0G     0  1.0G   0% my_folder

Fricken wicked!!

So now if I do this in Python:

with tempfile.NamedTemporaryFile(dir="my_folder") as temp:

I just created a temporary file in linux with mkstemp in a completely in memory file system. Of course, I do incur the cost of copying the data from my python's memory into another memory location. But hey, on big files, that's probably cheaper than writing to disk.

Make sure that your in-memory file system has enough ram to hold the video file otherwise, by design, tmpfs will utilize swap space to put the file on disk anyways.

EDIT UPDATE

Kubernetes also allows you to do this directly on a Pod or a Deployment. So you can make a TMPFS directory directly on the container's filesystem and away you go!

volumes:
- name: data
  emptyDir:
    medium: Memory
    sizeLimit: 64Mi
volumeMounts:
- mountPath: /tmp/data
  name: data

Aramaic

This is very neat and will be very useful in the ML context. – Neubauer

Answer 3

If you use Linux, you can create a ramdisk and write to it.

mount -t tmpfs -o size=512m tmpfs /mnt/ramdisk

Alary

will this be measurably quicker than say and SDD ? – Occult