How to use tun/tap interface to split packets, tunnel and then reassemble. (similar to MLPPP)

Question

I am looking to create a client/server application that I can use to slit network packets in half, tunnel each half of the packet over a separate udp connection (because each udp connection will be going over a different wifi link) and then reassemble the split packets on the other end. In addition to splitting the packets each half packet will also have to have an ID and sequence number so that they can be reassembled properly.

Basically I am trying to do something similar to MLPPP

I am looking to do this using python and the TUN/TAP network driver. I have found the following python code samples and modules that I think might be helpful for this project.

Python tun/tap

• http://www.secdev.org/projects/tuntap_udp/files/tunproxy.py
• http://twistedmatrix.com/trac/browser/trunk/twisted/pair/tuntap.py
• http://pastebin.com/gMB8zUfj

Python raw packet manipulation

• http://libdnet.sourceforge.net/
• http://pypi.python.org/pypi/pyip/
• http://code.google.com/p/python-packet/

My question is can the necessary packet modification be done using python and what would be a possible way to approach this? Can I use the modules above to do this or is there a better solution? I am looking for some input that will steer me in the right direction as I am not an experienced programmer. Any code samples or additional links are welcome.

Abingdon

I don't have an answer for you- but I will say, if you are changing every packet in userspace, I think you will be very disappointed with the performance. Both in terms of throughput and latency. I'd done something similar with scapy (similar in processing every packet in a python userland) and the performance was just terrible. FWIW – Round

Would using a different programing language make a difference or is it only possible to have acceptable performance using something like a custom kernel module? – Abingdon

I haven't done enough to know- but I've read some others research about the cost of a context switch from the kernel to userland and I can only imagine if you have to make the switch for every packet, you will feel it. That's not to say you can't make it quick enough to meet your needs, just have realistic expectations. – Round

Answer 1

We are doing something like this in production and it works quite well. We don't split individual packets though. We set fractional weights for each connection (unlimited) and send the packets out. We have some code in place to deal with different latencies on each line. On the other end we buffer them and reorder. Performance is pretty good - we have sites with 5+ ADSL lines and get good speeds, 40+ Mbps on the download.

Splitting packets (eg 1500/2 = 750) would introduce unnecessary overhead... keep your packets as big as possible.

We have developed our own protocol (header format) for the UDP packets. We have done loopback testing on the tun/tap up to 200 Mbps, so definitely the kernel to user space interaction works well. Previously we used NFQUEUE but that had reliability issues.

All of the above was written in Python.

Telephoto

Did you do it also in Python? – Debris

Yes, we wrote everything in Python – Telephoto

More than a year later and I still haven't done anything with this :-(. I'm assuming since you were not doing packet splitting the MTU size of the final bonded link was below the standard 1500 size. Has this been a problem for certain applications? – Abingdon

Sorry for the delay in replying... we lose around 24 bytes so our MTU is slightly smaller. All applications run fine since we do TCP MSS clamping. Different MTU sizes are common on the Internet. Just think, PPPoE is often 1492 and many other VPN systems have overhead as well. – Telephoto

Answer 2

It looks perfectly possible to me.

The tun/tap modules you've discovered look like they would do the job. Twisted will be high performance and the cost of hurting your head working it all out.

As for splitting the packets, you don't need to interpret the data in any way, just treat it as a blob of binary data, split it in two and add a header - I wouldn't use any 3rd party modules for that, just normal python string handling.

Or you could use netstrings if you want an easy to use packet encapsulation format.

I don't suppose it would go like a rocket, but I'm sure you would learn lots doing it!

Halliehallman

I'm not really looking for this to go like a rocket. I guess I should have added my performance requirements to the question. Ideally I'd like this application to be able to transmit data at a minimum of 2Mbps. If this is not realistic with python or any other user space implementation then I should probably be going down a different road. – Abingdon