USB Mass Storrage on File Level

Question

The Challenge: I have an Linux hand-held device, which records data and stores it to the disc. It should exchange these data with a Windows application via USB. When this data is accessible by the user - e.g. via USB-mass-storage - it has to be encrypted. It should work out-of-the-box, with a variety of OS, also for citrix terminal sessions, etc.

The Plan: I create a file-system in user-space with FUSE, and offer it to windows via mass-storage. Whenever Windows accesses one file, I get a callback and encrypt the data on the fly. Further more, we can have some dynamic content - e.g. when some password is written into a file, more content is shown.

The Problem: When using the mass-storage gadget (e.g. g_file_storage) it only accepts files or block-devices - but no file-systems (directories). Why?

[...] it provides a simple interface to read and write sectors of data—much like the low-level interface used to access any hard drive [...]. Operating systems may treat the USB drive like a hard drive, and can format it with any file system they like. (from wikipedia)

So there is no chance to have a dynamic file-system via mass-storage... and this seems to be the reason, why my android mobile phone un-mounts all data on the phone, when I connect it to the PC.

Options:

• Create a 'block-device in userspace' - similar to FUSE (need a reverse-FAT-driver, when I want to offer files dynamically)
• Implement my own nbd-server to create a block-device (need a reverse-FAT-driver as well?)
• I save encrypted files to a partition, and pass this partition to the mass-storage gadget (Problem would be the performance and the lack of dynamic interaction)
• Do not offer a mass-storrage device and watch out for other ideas (eth over USB)

At the moment, only the last option seems to be realistic - or do you have another tip for me?

I would be grateful!

Charly

Mclaren

Probably not helpful, but anyway: what's the point of encrypting the USB connection when the data is on the device (presumably) unencrypted anyway? Would it not be more secure to save the data encrypted, then you don't have to worry about doing it on the fly either? – Kerf

It's a performance issue and it's easier for the development when having ASCII files readable. Of course, with a high effort you can also break the device and get the data from there. – Mclaren

Answer 1

The USB mass storage protocol is a block device protocol; it does not operate at the level of files or directories. The Windows host expects to see a raw VFAT filesystem exposed by the g_mass_storage driver, and will make writes and reads to VFAT metadata as appropriate to figure out how directories are structured.

Because of this, it's nontrivial to expose a FUSE filesystem to the windows host. You'd have to emulate VFAT, assigning blocks in the virtual filesystem to metadata and to data, and since the windows host is free to cache any data or metadata it reads, once you assign some metadata or data it cannot change (so changes to your FUSE data could not be reflected in the windows filesystem). The windows host can also delay and reorder writes to both metadata and data - it's all real mess if you try to emulate.

Now, there are some things you can do:

1. You can write a custom IFS driver on the windows side to interact with your linux device over a custom protocol that works at the file/directory level.
2. You can treat the USB device as a virtual ethernet port, and speak CIFS to the windows host
3. You can somehow create a static VFAT layout on connection time to expose to the windows host; not-yet-decryptable data can return I/O errors in order to avoid the Windows host caching raw encrypted data.
4. You can just encrypt a raw block device using dm-crypt, and expose this entire block device (encrypted as one chunk) to windows.
5. You could implement a MTP gadget.

These approaches come with their own problems:

1. Requires a windows driver to be installed, and to be signed by microsoft etc. Can't be used on a machine without administrative access to install the driver.
2. Won't autoplay; the user would need to browse through the network browser to get access to the files. Firewall settings may interfere. May have significant overhead.
3. Very complex. Handling metadata updates on the backend may be extremely difficult. Surprise unplug events may be devastating. Windows behavior on receiving an IO error may be a problem if the user tries to access a locked file.
4. No file-level encryption is available, but otherwise should work well.
5. I'm unsure exactly how much support for non-media files MTP has; support is not as widespread as mass storage support.

Citronella

Hmmm... thanks a lot, but that is what I feared. The great thing of the mass-storage would be, to need no custom-driver, to need no configuration (eth address, gateway, etc), need no custom protocol.. and it's working everywhere. With 1) we already made bad experiences - libusb is some kind of buggy. 2) seems to have similar disadvantages as using the real network. 3) Maybe never will stable 5) I'm not sure if it's working on all different windows variants in an acceptable way. – Mclaren

Answer 2

It may interest you to know that Android, which previously exposed itself as a USB mass storage device to the host, is instead acting as an MTP device (as of Honeycomb).

That being said, somebody has already implemented your option 1, though with the "device" and "host" a bit reversed. QEMU has an insane hack called vvfat which is able to create a fake block device which to the VM looks like it contains a VFAT filesytem just from a directory/file tree on the host. It requires a full recursive scan before starting, is dependent on details of how OSes write to filesystems, and falls over if you independently change any files while it's in use, but (somehow) manages to (sometimes) work.

Canoewood

Answer 3

It may be possible to translate the NBD protocol to USB, and "bit-bang" usb to appear to the USB Host as a USB mass storage device.

• NBD and USB msas storage are both block level devices, so it may be possible to translate one protocol to the other. However, that would almost certainly require programming, as I don't think this exists. However, http://travisgoodspeed.blogspot.com/2012/07/emulating-usb-devices-with-python.html looks fairly useful for the USB side, and https://bitbucket.org/hirofuchi/xnbd/wiki/Home should give you a good example for the NBD client side.
• Using VFAT means that Windows can see it as a normal USB disk drive, with no drivers.
  • as another poster suggested, qemu's vvfat module could be used to automate some of this.
• Encryption of the traffic would require tunneling the NBD traffic through SSH or OpenVPN.

The final setup would look something like this:

+---------------------------------+                +------------------+
| data collection device          |                | client device    |
| +----------------+              |                |                  |
| | collected data |              |                |  SSH -> NBD      |
| | -> linux fs    |              |                |         client   |
| +--+-------------+--------+     |                |         |        |
| -> | qemu vvfat           | SSH +-{some network}-+         V        |--> client PC
|    | run NBD inside the VM|     |                | USB Mass Storage |    USB port 
+----+----------------------+-----+                +------------------+   

Censure

Answer 4

Looking at the options, I'd consider using the ethernet-gadget, and doing IP autoconfiguration in the device, then running Samba to export the filesystem to the Windows host. This would give you the level of control you need.

Ambsace

Yes, but the disadvantage is that the user still has to configure the mount - e.g. with choosing the network share in the network manager. A mass storage would be much easier to handle - from user side. – Mclaren