How can I transfer binary data over xmlrpc (python)?

Question

As the name xmlrpc implies, this transfer protocol relies on XML to carry data, and cannot transfer binary data, or non-printable ASCII-characters (\n, \b, chr(2),...) [or can it?].

I would like to know if there is a way to transfer a character string safely from a client to a server with minimal impact on the coding (i.e. ONLY on the client side). I tried the xmlrpclib.Binary class but this only seem to work with files.

Testcode, server.py:

def output(text):
    print "-".join([str(ord(x)) for x in text])

from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()

client.py:

import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(".\n."+chr(2))

Expected outcome:

46-10-46-2

Seen outcome (on server side):

xmlrpclib.Fault: <Fault 1: "<class 'xml.parsers.expat.ExpatError'>:not well-formed (invalid token): line 7, column 1">

Resident

And what outcome did you get? – Minne

@Ned: see updated question. I get the error because of the chr(2) character. – Resident

Answer 1

You could try encoding your binary data in a text format in the client and decoding it back into binary in the server. One encoding you could use is base64.

In your client:

import xmlrpclib
import base64
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(base64.b64encode(".\n."+chr(2)))

In your server:

import base64
def output(text):
    print "-".join([str(ord(x)) for x in base64.b64decode(text)])

from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()

Koto

This requires coding on both, the server AND the client side. – Resident

Isn't that what the code shows? ;) However, got your point, will correct the expression. Or do you mean you don't control the server? – Koto

I am looking for a solution to just change the client side. I was hoping, you could use the Binary object and have the server side decode it automatically without code change on the server side. – Resident

That's going to prove since XML is a text-based format. Have you tried wrapping the binary data in CDATA block? – Koto

How to change the code to wrap in CDATA? In fact, I am in control of the server, but I would have liked to reduce the amount of work to the minimum. – Resident

Just tried you can't... expat still fails. The idea was to use a string like "<![CDATA[.\n."+chr(2)+"]]>". – Koto

If you control the server too, then the easiest is encoding/decoding the binary stuff (IMHO). Perhaps base64 is overkill and you could use some sort of escape encoding (e.g. URL encoding scheme or such like). – Koto

I have a fact: Large data conversions using pythons's base64 can be really painful on your system memory and resources. I tried it on a 100 MB thing and things started to lag (crash for my system). Tried with 10MB == No Problems. – Earpiece

Answer 2

I think the expected answer was using xml-rpc base64 type. In python, on client side, you have to manually specify that a string contains binary data, using the xmlrpclib.Binary type.

import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(xmlrpclib.Binary(".\n."+chr(2)))

Xanthine