How can I capture iSight frames with Python in Snow Leopard?

Question

I have the following PyObjC script:

from Foundation import NSObject
import QTKit
error = None
capture_session = QTKit.QTCaptureSession.alloc().init()
print 'capture_session', capture_session
device = QTKit.QTCaptureDevice.defaultInputDeviceWithMediaType_(QTKit.QTMediaTypeVideo)
print 'device', device, type(device)
success = device.open_(error)
print 'device open success', success, error
if not success:
    raise Exception(error)
capture_device_input = QTKit.QTCaptureDeviceInput.alloc().initWithDevice_(device)
print 'capture_device_input', capture_device_input, capture_device_input.device()
success = capture_session.addInput_error_(capture_device_input, error)
print 'session add input success', success, error
if not success:
    raise Exception(error)
capture_decompressed_video_output = QTKit.QTCaptureDecompressedVideoOutput.alloc().init()
print 'capture_decompressed_video_output', capture_decompressed_video_output
class Delegate(NSObject):
    def captureOutput_didOutputVideoFrame_withSampleBuffer_fromConnection_(self, captureOutput, videoFrame, sampleBuffer, connection):
        print videoFrame, sampleBuffer, connection
delegate = Delegate.alloc().init()
print 'delegate', delegate
capture_decompressed_video_output.setDelegate_(delegate)
print 'output delegate:', capture_decompressed_video_output.delegate()
success = capture_session.addOutput_error_(capture_decompressed_video_output, error)
print 'capture session add output success', success, error
if not success:
    raise Exception(error)
print 'about to run session', capture_session, 'with inputs', capture_session.inputs(), 'and outputs', capture_session.outputs()
capture_session.startRunning()
print 'capture session is running?', capture_session.isRunning()
import time
time.sleep(10)

The program reports no errors, but iSight's green light is never activated and the delegate's frame capture callback is never called. Here's the output I get:

$ python prueba.py 
capture_session <QTCaptureSession: 0x1006c16f0>
device Built-in iSight <objective-c class QTCaptureDALDevice at 0x7fff70366aa8>
device open success (True, None) None
capture_device_input <QTCaptureDeviceInput: 0x1002ae010> Built-in iSight
session add input success (True, None) None
capture_decompressed_video_output <QTCaptureDecompressedVideoOutput: 0x104239f10>
delegate <Delegate: 0x10423af50>
output delegate: <Delegate: 0x10423af50>
capture session add output success (True, None) None
about to run session <QTCaptureSession: 0x1006c16f0> with inputs (
    "<QTCaptureDeviceInput: 0x1002ae010>"
) and outputs (
    "<QTCaptureDecompressedVideoOutput: 0x104239f10>"
)
capture session is running? True

PS: Please don't answer I should try PySight, I have but it won't work because Xcode can't compile CocoaSequenceGrabber in 64bit.

Nickienicklaus

Answer 1

Your problem here is that you don't have an event loop. If you want to do this as a standalone script, you'll have to figure out how to create one. The PyObjC XCode templates automatically set that up for you with:

from PyObjCTools import AppHelper
AppHelper.runEventLoop()

Trying to insert that at the top of your script, however, shows that something inside AppHelper (probably NSApplicationMain) expects a plist file to extract the main class from. You can get that by creating a setup.py file and using py2app, something like this example from a PyObjc talk:

from distutils.core import setup
import py2app
plist = dict(
    NSPrincipalClass='SillyBalls',
)
setup(
    plugin=['SillyBalls.py'],
    data_files=['English.lproj'],
    options=dict(py2app=dict(
        extension='.saver',
        plist=plist,
    )),
)

Conlin

@Dan: Thanks for the pointer! It's my first experience with Mac OS X programming and I was absolutely clueless. I got it to work invoking AppHelper.runConsoleEventLoop() instead at the end of the script, no need for Plist. Now my problem is that it takes over the main thread and never returns. I was hoping to wrap it nicely in a module in a non-intrusive way. – Nickienicklaus

You could spawn off a thread and handle it within the thread, probably. QT is not threadsafe, but in this context all it means is you have to do all of your QT stuff in one thread, which is not necessarily the main thread. You might also look into timers, but I think you probably still need a main loop for that. – Conlin

Apparently, it has to be the main thread. If I do Thread(target=AppHelper.runConsoleEventLoop).start() instead, I get a bunch of errors and nothing works: 2009-10-20 12:58:32.075 Python[2054:4903] *** __NSAutoreleaseNoPool(): Object 0x1018065b0 of class NSCFString autoreleased with no pool in place - just leaking 2009-10-20 12:58:32.078 Python[2054:4903] *** __NSAutoreleaseNoPool(): Object 0x101821130 of class NSCFString autoreleased with no pool in place - just leaking 2009-10-20 12:58:32.078 Python[2054:4903] *** __NSAutoreleaseNoPool(): Object 0x101828df0 of class NSCFString autorelease – Nickienicklaus

Answer 2

You should give a try to motmot's camiface library from Andrew Straw. It also works with firewire cameras, but it works also with the isight, which is what you are looking for.

From the tutorial:

import motmot.cam_iface.cam_iface_ctypes as cam_iface
import numpy as np

mode_num = 0
device_num = 0
num_buffers = 32

cam = cam_iface.Camera(device_num,num_buffers,mode_num)
cam.start_camera()
frame = np.asarray(cam.grab_next_frame_blocking())
print 'grabbed frame with shape %s'%(frame.shape,)

Unalienable

you can see some simple examples on incm.cnrs-mrs.fr/LaurentPerrinet/SimpleCellDemo – Unalienable

is there any way to directly record to a file through this library? – Monumentalize

motmot and camiface are great but I wanted a low-dependency solution, specially to directly record video. So created this: github.com/dashesy/pyavfcam using AVFoundation for direct video and image capturing and recording to file. – Monumentalize