Flask video stream using OpenCV images

Question

I am trying to use Flask to show a stream of OpenCV images. I am using ROS and the Zed stereo camera.

The issue is that the flask server only shows a broken image icon. I am guessing that the issue is in the gen() method because the cv2.imwrite('t.jpg', img) method is the wrong way to go. I have very little experience with OpenCV.

The image data that the Flask server receives is an InputArray. I need a way of converting this and showing the image in the Flask server.

I am using Python 2.7 and rospy (ROS).

Any advice?

UPDATE:

The code for the ROS node accessing ZED cam:

    #!/usr/bin/env python

# ROS imports
import rospy
from sensor_msgs.msg import Image

# Utils
import numpy as np
import cv2
from cv_bridge import CvBridge, CvBridgeError
import stream


def callback(data):
    """
    param data: data from zed/rgb/image_rect_color topic
    """
    # convert from ROS sensor_msgs/Image to cv2
    bridge = CvBridge()
    try:
        cv_img = bridge.imgmsg_to_cv2(data, desired_encoding='passthrough')
        stream.img = cv_img
    except CvBridgeError as e:
        print(e)
    # show image stream
    # cv2.imshow('zed', cv_img)
    # cv2.waitKey(3)


def zed_sub():
    # initialize ROS node 'zed_sub'
    rospy.init_node('zed_sub')
    # subscribe to the ROS topic 'zed/rgb/image_rect_color'
    rospy.Subscriber('zed/rgb/image_rect_color', Image, callback)
    # keep python from exiting until this node is stopped
    try:
        rospy.spin()
    except KeyboardInterrupt:
        cv2.destroyAllWindows()


if __name__ == '__main__':
    zed_sub()

The code for the Flask server:

#!/usr/bin/env python

from flask import Flask, render_template, Response
import cv2


app = Flask(__name__)
HOST = '192.168.1.3'    # on-board computer's IP address
PORT = 8080

img = None


@app.route('/')
def index():
    return render_template('index.html')


@app.route('/video_feed')
def video_feed():
    return Response(gen(), mimetype='multipart/x-mixed-replace; boundary=frame')


def gen():
    """Video streaming generator function."""
    global img
    while True:
        try:
            cv2.imwrite('t.jpg', img)
            yield(b'--frame\r\n'
                b'Content-Type: image/jpeg\r\n\r\n' + open('t.jpg', 'rb').read() + b'\r\n')
        except NameError as e:
            print(e)


if __name__ == '__main__':
    app.run(host=HOST, port=PORT, debug=True, threaded=True)

Schoenfelder

why are you doing cv2.imwrite('t.jpg', img) ? – Springer

how are you accessing video camera? – Longish

can you share your full code of accessing video? – Longish

I am using imwrite because I found it in a guide. As I said, I suspect that is the issue and I need some help. – Schoenfelder

The ROS node that subscribes to the ZED cam is added in the update. – Schoenfelder

To summarize: The ZED data is converted to a cv2 InputArray. I therefore need something that can interpret this and show it as an image or stream in a Flask server. – Schoenfelder

Please see update – Schoenfelder

Answer 1

for accessing camera other than your laptop's webcam, you can use RTSP link like this rtsp://admin:[email protected]:554/h264/ch1/main/av_stream"

where

>        username:admin
>         password:12345
>         your camera ip address and port
>         ch1 is first camera on that DVR

replace cv2.VideoCamera(0) with this link like this for your camera and it will work

camera.py

import cv2

class VideoCamera(object):
    def __init__(self):
        # Using OpenCV to capture from device 0. If you have trouble capturing
        # from a webcam, comment the line below out and use a video file
        # instead.
        self.video = cv2.VideoCapture(0)
        # If you decide to use video.mp4, you must have this file in the folder
        # as the main.py.
        # self.video = cv2.VideoCapture('video.mp4')

    def __del__(self):
        self.video.release()

    def get_frame(self):
        success, image = self.video.read()
        # We are using Motion JPEG, but OpenCV defaults to capture raw images,
        # so we must encode it into JPEG in order to correctly display the
        # video stream.
        ret, jpeg = cv2.imencode('.jpg', image)
        return jpeg.tobytes()

main.py

from flask import Flask, render_template, Response
from camera import VideoCamera

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

def gen(camera):
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_feed')
def video_feed():
    return Response(gen(VideoCamera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True)

then you can follow this blog to increase your FPS

https://www.pyimagesearch.com/2015/12/21/increasing-webcam-fps-with-python-and-opencv/

Springer