Reading and writing to USB (HID) interrupt endpoints on Mac

Question

I am attempting to communicate with a rather specific USB device and developing both Windows and Mac code to do so.

The device is a USB device with a HID interface (class 3) with two endpoints, an interrupt input and an interrupt output. The nature of the device is such that data is sent out from the device on the input endpoint only when data is requested from the host: the host sends it data which the device responds to on its input interrupt endpoint. Getting data to the device (a write) is much more simple...

The code for Windows is rather straight-forward: I get a handle to the device and then call either ReadFile or WriteFile. Apparently much of the underlying asynchronous behavior is abstracted out. It appears to work fine.

On Mac, however, it is a bit stickier. I have tried a number of things, none which have been fully successful, but here are the two things which seemed most promising...

1.) Attempt to get access to the device (as USB) via IOUSBInterfaceInterface, iterate through the endpoints to determine the input and output endpoints, and (hopefully) use ReadPipe and WritePipe to communicate. Unfortunately I am unable to open the interface once I have it, with the return value (kIOReturnExclusiveAccess) noting that something already has the device open exclusively. I have tried using IOUSBinterfaceInterface183, so that I could call USBInterfaceOpenSeize, but that results in the same return error value.

--- update 7/30/2010 ---
Apparently, the Apple IOUSBHIDDriver matches early to the device and this is what likely is preventing opening the IOUSBInterfaceInterface. From some digging about it seems that the common way to prevent the IOUSBHIDDriver from matching is to write a code-less kext (kernel extension) with a higher probe score. This would match early, preventing the IOUSBHIDDriver from opening the device, and should, in theory, permit me to open the interface and to write and read to endpoints directly. This is OK, but I would much prefer not having to install something additional on the user machine. If anyone knows of a solid alternative I would be thankful for the information.

2.) Open the device as an IOHIDDeviceInterface122 (or later). To read, I set up an async port, event source and callback method to be called when data is ready - when data is sent from the device on the input interrupt endpoint. However, to write the data — that the device needs — to initialize a response I can't find a way. I'm stumped. setReport typically writes to the control endpoint, plus I need a write that does not expect any direct response, no blocking.

I have looked around online and have tried many things, but none of them is giving me success. Any advice? I can not use much of the Apple HIDManager code since much of that is 10.5+ and my application must work on 10.4 as well.

Rollandrollaway

Answer 1

I have now a working Mac driver to a USB device that requires communication through interrupt endpoints. Here is how I did it:

Ultimately the method that worked well for me was option 1 (noted above). As noted, I was having issues opening the COM-style IOUSBInterfaceInterface to the device. It became clear over time that this was due to the HIDManager capturing the device. I was unable to wrest control of the device from the HIDManager once it was captured (not even the USBInterfaceOpenSeize call or the USBDeviceOpenSeize calls would work).

To take control of the device I needed to grab it before the HIDManager. The solution to this was to write a codeless kext (kernel extension). A kext is essentially a bundle that sits in System/Library/Extensions that contains (usually) a plist (property list) and (occasionally) a kernel-level driver, among other items. In my case I wanted only the plist, which would give the instructions to the kernel on what devices it matches. If the data gives a higher probe score than the HIDManager then I could essentially capture the device and use a user-space driver to communicate with it.

The kext plist written, with some project-specific details modified, is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>OSBundleLibraries</key>
    <dict>
        <key>com.apple.iokit.IOUSBFamily</key>
        <string>1.8</string>
        <key>com.apple.kernel.libkern</key>
        <string>6.0</string>
    </dict>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>CFBundleGetInfoString</key>
    <string>Demi USB Device</string>
    <key>CFBundleIdentifier</key>
    <string>com.demiart.mydevice</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleName</key>
    <string>Demi USB Device</string>
    <key>CFBundlePackageType</key>
    <string>KEXT</string>
    <key>CFBundleSignature</key>
    <string>????</string>
    <key>CFBundleVersion</key>
    <string>1.0.0</string>
    <key>IOKitPersonalities</key>
    <dict>
        <key>Device Driver</key>
        <dict>
            <key>CFBundleIdentifier</key>
            <string>com.apple.kernel.iokit</string>
            <key>IOClass</key>
            <string>IOService</string>
            <key>IOProviderClass</key>
            <string>IOUSBInterface</string>
            <key>idProduct</key>
            <integer>12345</integer>
            <key>idVendor</key>
            <integer>67890</integer>
            <key>bConfigurationValue</key>
            <integer>1</integer>
            <key>bInterfaceNumber</key>
            <integer>0</integer>
        </dict>
    </dict>
    <key>OSBundleRequired</key>
    <string>Local-Root</string>
</dict>
</plist>

The idVendor and idProduct values give the kext specificity and increase its probe score sufficiently.

In order to use the kext, the following things need to be done (which my installer will do for clients):

1. Change the owner to root:wheel (sudo chown root:wheel DemiUSBDevice.kext)
2. Copy the kext to Extensions (sudo cp DemiUSBDevice.kext /System/Library/Extensions)
3. Call the kextload utility to load the kext for immediate use without restart (sudo kextload -vt /System/Library/Extensions/DemiUSBDevice.kext)
4. Touch the Extensions folder so that the next restart will force a cache rebuild (sudo touch /System/Library/Extensions)

At this point the system should use the kext to keep the HIDManager from capturing my device. Now, what to do with it? How to write to and read from it?

Following are some simplified snippets of my code, minus any error handling, that illustrate the solution. Before being able to do anything with the device, the application needs to know when the device attaches (and detaches). Note that this is merely for purposes of illustration — some of the variables are class-level, some are global, etc. Here is the initialization code that sets the attach/detach events up:

#include <IOKit/IOKitLib.h>
#include <IOKit/IOCFPlugIn.h>
#include <IOKit/usb/IOUSBLib.h>
#include <mach/mach.h>

#define DEMI_VENDOR_ID 12345
#define DEMI_PRODUCT_ID 67890

void DemiUSBDriver::initialize(void)
{
    IOReturn                result;
    Int32                   vendor_id = DEMI_VENDOR_ID;
    Int32                   product_id = DEMI_PRODUCT_ID;
    mach_port_t             master_port;
    CFMutableDictionaryRef  matching_dict;
    IONotificationPortRef   notify_port;
    CFRunLoopSourceRef      run_loop_source;
    
    //create a master port
    result = IOMasterPort(bootstrap_port, &master_port);
    
    //set up a matching dictionary for the device
    matching_dict = IOServiceMatching(kIOUSBDeviceClassName);
    
    //add matching parameters
    CFDictionarySetValue(matching_dict, CFSTR(kUSBVendorID),
        CFNumberCreate(kCFAllocatorDefault, kCFNumberInt32Type, &vendor_id));
    CFDictionarySetValue(matching_dict, CFSTR(kUSBProductID),
        CFNumberCreate(kCFAllocatorDefault, kCFNumberInt32Type, &product_id));
      
    //create the notification port and event source
    notify_port = IONotificationPortCreate(master_port);
    run_loop_source = IONotificationPortGetRunLoopSource(notify_port);
    CFRunLoopAddSource(CFRunLoopGetCurrent(), run_loop_source, 
      kCFRunLoopDefaultMode);
    
    //add an additional reference for a secondary event 
    //  - each consumes a reference...
    matching_dict = (CFMutableDictionaryRef)CFRetain(matching_dict);
    
    //add a notification callback for detach event
    //NOTE: removed_iter is a io_iterator_t, declared elsewhere
    result = IOServiceAddMatchingNotification(notify_port, 
      kIOTerminatedNotification, matching_dict, device_detach_callback, 
      NULL, &removed_iter);
    
    //call the callback to 'arm' the notification
    device_detach_callback(NULL, removed_iter);
    
    //add a notification callback for attach event
    //NOTE: added_iter is a io_iterator_t, declared elsewhere
    result = IOServiceAddMatchingNotification(notify_port, 
      kIOFirstMatchNotification, matching_dict, device_attach_callback, 
      NULL, &g_added_iter);
    if (result)
    {
      throw Exception("Unable to add attach notification callback.");
    }
    
    //call the callback to 'arm' the notification
    device_attach_callback(NULL, added_iter);
    
    //'pump' the run loop to handle any previously added devices
    service();
}

There are two methods that are used as callbacks in this initialization code: device_detach_callback and device_attach_callback (both declared at static methods). device_detach_callback is straightforward:

//implementation
void DemiUSBDevice::device_detach_callback(void* context, io_iterator_t iterator)
{
    IOReturn       result;
    io_service_t   obj;

    while ((obj = IOIteratorNext(iterator)))
    {
        //close all open resources associated with this service/device...
        
        //release the service
        result = IOObjectRelease(obj);
    }
}

device_attach_callback is where most of the magic happens. In my code I have this broken into multiple methods, but here I'll present it as a big monolithic method...:

void DemiUSBDevice::device_attach_callback(void * context, 
    io_iterator_t iterator)
{
    IOReturn                   result;
    io_service_t           usb_service;
    IOCFPlugInInterface**      plugin;   
    HRESULT                    hres;
    SInt32                     score;
    UInt16                     vendor; 
    UInt16                     product;
    IOUSBFindInterfaceRequest  request;
    io_iterator_t              intf_iterator;
    io_service_t               usb_interface;

    UInt8                      interface_endpoint_count = 0;
    UInt8                      pipe_ref = 0xff;
    
    UInt8                      direction;
    UInt8                      number;
    UInt8                      transfer_type;
    UInt16                     max_packet_size;
    UInt8                      interval;

    CFRunLoopSourceRef         m_event_source;
    CFRunLoopSourceRef         compl_event_source;
    
    IOUSBDeviceInterface245** dev = NULL;
    IOUSBInterfaceInterface245** intf = NULL;
    
    while ((usb_service = IOIteratorNext(iterator)))
    {
      //create the intermediate plugin
      result = IOCreatePlugInInterfaceForService(usb_service, 
        kIOUSBDeviceUserClientTypeID, kIOCFPlugInInterfaceID, &plugin, 
        &score);
      
      //get the device interface
      hres = (*plugin)->QueryInterface(plugin, 
        CFUUIDGetUUIDBytes(kIOUSBDeviceInterfaceID245), (void**)&dev);
      
      //release the plugin - no further need for it
      IODestroyPlugInInterface(plugin);
      
      //double check ids for correctness
      result = (*dev)->GetDeviceVendor(dev, &vendor);
      result = (*dev)->GetDeviceProduct(dev, &product);
      if ((vendor != DEMI_VENDOR_ID) || (product != DEMI_PRODUCT_ID))
      {
        continue;
      }
      
      //set up interface find request
      request.bInterfaceClass     = kIOUSBFindInterfaceDontCare;
      request.bInterfaceSubClass  = kIOUSBFindInterfaceDontCare;
      request.bInterfaceProtocol  = kIOUSBFindInterfaceDontCare;
      request.bAlternateSetting   = kIOUSBFindInterfaceDontCare;
    
      result = (*dev)->CreateInterfaceIterator(dev, &request, &intf_iterator);
    
      while ((usb_interface = IOIteratorNext(intf_iterator)))
      {
        //create intermediate plugin
        result = IOCreatePlugInInterfaceForService(usb_interface, 
          kIOUSBInterfaceUserClientTypeID, kIOCFPlugInInterfaceID, &plugin, 
          &score);
      
        //release the usb interface - not needed
        result = IOObjectRelease(usb_interface);
      
        //get the general interface interface
        hres = (*plugin)->QueryInterface(plugin, CFUUIDGetUUIDBytes(
          kIOUSBInterfaceInterfaceID245), (void**)&intf);
      
        //release the plugin interface
        IODestroyPlugInInterface(plugin);
      
        //attempt to open the interface
        result = (*intf)->USBInterfaceOpen(intf);
      
        //check that the interrupt endpoints are available on this interface
        //calling 0xff invalid...
        m_input_pipe = 0xff;  //UInt8, pipe from device to Mac
        m_output_pipe = 0xff; //UInt8, pipe from Mac to device
    
        result = (*intf)->GetNumEndpoints(intf, &interface_endpoint_count);
        if (!result)
        {
          //check endpoints for direction, type, etc.
          //note that pipe_ref == 0 is the control endpoint (we don't want it)
          for (pipe_ref = 1; pipe_ref <= interface_endpoint_count; pipe_ref++)
          {
            result = (*intf)->GetPipeProperties(intf, pipe_ref, &direction,
              &number, &transfer_type, &max_packet_size, &interval);
            if (result)
            {
              break;
            }
        
            if (transfer_type == kUSBInterrupt)
            {
              if (direction == kUSBIn)
              {
                m_input_pipe = pipe_ref;
              }
              else if (direction == kUSBOut)
              {
                m_output_pipe = pipe_ref;
              }
            }
          }
        }

        //set up async completion notifications
        result = (*m_intf)->CreateInterfaceAsyncEventSource(m_intf, 
          &compl_event_source);
        CFRunLoopAddSource(CFRunLoopGetCurrent(), compl_event_source, 
          kCFRunLoopDefaultMode);
        
        break;
      }

      break;
    }
}

At this point we should have the numbers of the interrupt endpoints and an open IOUSBInterfaceInterface to the device. An asynchronous writing of data can be done by calling something like:

result = (intf)->WritePipeAsync(intf, m_output_pipe, 
          data, OUTPUT_DATA_BUF_SZ, device_write_completion, 
          NULL);

where data is a char buffer of data to write, the final parameter is an optional context object to pass into the callback, and device_write_completion is a static method with the following general form:

void DemiUSBDevice::device_write_completion(void* context, 
    IOReturn result, void* arg0)
{
  //...
}

reading from the interrupt endpoint is similar:

result = (intf)->ReadPipeAsync(intf, m_input_pipe, 
          data, INPUT_DATA_BUF_SZ, device_read_completion, 
          NULL);

where device_read_completion is of the following form:

void DemiUSBDevice::device_read_completion(void* context, 
    IOReturn result, void* arg0)
{
  //...
}

Note that to receive these callbacks the run loop must be running (see this link for more information about the CFRunLoop). One way to achieve this is to call CFRunLoopRun() after calling the async read or write methods at which point the main thread blocks while the run loop runs. After handling your callback you can call CFRunLoopStop(CFRunLoopGetCurrent()) to stop the run loop and hand execution back to the main thread.

Another alternative (which I do in my code) is to pass a context object (named 'request' in the following code sample) into the WritePipeAsync/ReadPipeAsync methods - this object contains a boolean completion flag (named 'is_done' in this example). After calling the read/write method, instead of calling CFRunLoopRun(), something like the following can be executed:

while (!(request->is_done))
{
  //run for 1/10 second to handle events
  Boolean returnAfterSourceHandled = false;
  CFTimeInterval seconds = 0.1;
  CFStringRef mode = kCFRunLoopDefaultMode;
  CFRunLoopRunInMode(mode, seconds, returnAfterSourceHandled);
}

This has the benefit that if you have other threads that use the run loop you won't prematurely exit should another thread stop the run loop...

I hope that this is helpful to people. I had to pull from many incomplete sources to solve this problem and this required considerable work to get running well...

Rollandrollaway

+1. +100 This answer is brilliant, and I'm very grateful for your hard work. – Conwell

Answer 2

After reading this question a few times and thinking about it for a bit, I thought of another solution for emulating blocking read behavior, but using the HID manager instead of replacing it.

A blocking read function can register an input callback for the device, register the device on the current run loop, and then block by calling CFRunLoopRun(). The input callback can then copy the report into a shared buffer and call CFRunLoopStop(), which causes CFRunLoopRun() to return, thereby unblocking read(). Then, read() can return the report to the caller.

The first issue I can think of is the case where the device is already scheduled on a run loop. Scheduling and then unscheduling the device in the read function may have adverse affects. But that would only be a problem if the application is trying to use both synchronous and asynchronous calls on the same device.

The second thing that comes to mind is the case where the calling code already has a run loop running (Cocoa and Qt apps for example). But, the documentation for CFRunLoopStop() seems to indicate that nested calls to CFRunLoopRun() are handled properly. So, it should be ok.

Here's a bit of simplified code to go with that. I just implemented something similar in my HID Library and it seems to work, although I haven't tested it extensively.

/* An IN report callback that stops its run loop when called. 
   This is purely for emulating blocking behavior in the read() method */
static void input_oneshot(void*           context,
                          IOReturn        result,
                          void*           deviceRef,
                          IOHIDReportType type,
                          uint32_t        reportID,
                          uint8_t*        report,
                          CFIndex         length)
{
    buffer_type *const buffer = static_cast<HID::buffer_type*>(context);

    /* If the report is valid, copy it into the caller's buffer
         The Report ID is prepended to the buffer so the caller can identify
         the report */
    if( buffer )
    {
        buffer->clear();    // Return an empty buffer on error
        if( !result && report && deviceRef )
        {
            buffer->reserve(length+1);
            buffer->push_back(reportID);
            buffer->insert(buffer->end(), report, report+length);
        }
    }

    CFRunLoopStop(CFRunLoopGetCurrent());
}

// Block while waiting for an IN interrupt report
bool read(buffer_type& buffer)
{
    uint8_t _bufferInput[_lengthInputBuffer];

    // Register a callback
    IOHIDDeviceRegisterInputReportCallback(deviceRef, _bufferInput, _lengthInputBuffer, input_oneshot, &buffer);

    // Schedule the device on the current run loop
    IOHIDDeviceScheduleWithRunLoop(deviceRef, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);

    // Trap in the run loop until a report is received
    CFRunLoopRun();

    // The run loop has returned, so unschedule the device
    IOHIDDeviceUnscheduleFromRunLoop(deviceRef, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode);

    if( buffer.size() )
        return true;
    return false;
}

Monomer

Answer 3

I ran into this same kIOReturnExclusiveAccess. Instead of fighting it (building a kext, etc). I found the device and used the POSIX api's.

//My funcation was named differently, but I'm using this for continuity..
void DemiUSBDevice::device_attach_callback(void * context, 
    io_iterator_t iterator)
{
DeviceManager *deviceManager = (__bridge DADeviceManager *)context;
  io_registry_entry_t device;
  while ((device = IOIteratorNext(iterator))) {

    CFTypeRef prop;
    prop = IORegistryEntrySearchCFProperty(device,
                                           kIOServicePlane,
                                           CFSTR(kIODialinDeviceKey),
                                           kCFAllocatorDefault,
                                           kIORegistryIterateRecursively);
    if(prop){
      deviceManager->devPath = (__bridge NSString *)prop;
      [deviceManager performSelector:@selector(openDevice)];
    }
  }
}

once devPath is set you can call open and read/write..

int dfd;
dfd = open([devPath UTF8String], O_RDWR | O_NOCTTY | O_NDELAY);
  if (dfd == -1) {
    //Could not open the port.
    NSLog(@"open_port: Unable to open %@", devPath);
    return;
  } else {
    fcntl(fd, F_SETFL, 0);
  }

Gaffe