how to transport an object in twisted?

Question

I know with "transport.write()" I can transport string object but I want to know is this possible to transport other type of data, something like a python class object ? if It is possible how can I do this?

Cogitable

Answer 1

You can devise some arbitrary custom protocol, but it may be more productive to use a framework that already exists for this purpose, like google's protocol buffers that allow you to easily define an efficient message passing structure, and support python already.

JSON is an easy alternative, and plenty of people simply use zipped json objects, because its easy and built in to python by default, but the results are slow, have unpredictable size artifacts (zipped is typically larger than uncompressed output for numerous small messages), and are a poor solution for transferring binary data.

edit: Oh yes, and don't use pickle.

Edan

Answer 2

You can use json to serialize the Objects. Almost all Python objects are json serializable and if not then you can write your own encoder-decoder to handle them. Don't use eval to decode the input though.

Sloth

Answer 3

The role of the transport layer is to write data onto a physical connection. It's at a low level of abstraction, and the mapping of python class objects to bytes is better handled further up the protocol stack.

So, one way to handle this would be to write a custom protocol that accepts a python object, converts it to a byte representation and then pushes it onto the transport, as I write this Ashwini's answer has just popped up suggesting pickle, so we'll run with that to define our custom protocol

import pickle 
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ClientEndpoint


class ClassSender(Protocol):

    def dataReceived(self, data):
        print pickle.loads(data)

    def writeObject(self, _object):
        pickle.dump(_object,self.transport)


class ClassSenderFactory(ClientFactory):
    def buildProtocol(self, addr):
        return ClassSender()

class MyClass(object):
   def __init__(self,data):
       self.data = data

def SendObject(protocol):
    print "Sending"
    protocol.writeObject(MyClass('some sample data'))


point = TCP4ClientEndpoint(reactor, "localhost", 8000)
d = point.connect(ClassSenderFactory())
d.addCallback(SendObject)

reactor.run()

This sample code tries to make a connection to port 8000 (localhost) on TCP. Once a connection is established, we take a sample object, pickle it and push it onto the transport.

Reciprocally, when data is received, we try and unpickle it and print it to the console.

To see this in action, run a echo server (http://twistedmatrix.com/documents/current/core/examples/echoserv.py) and run then run the sample code to see a sample object get bounced off the echo server.

The problem with this is that there's a lot of corner cases that need to be considered. What if the data gets broken into two blocks so we can't unpickle it in one call?Fortunately, twisted provides a ready made class to deal with this called perspective broker. Where you have control both ends of the wire, this can deal with a lot of these issues for you. Take a look at : https://twistedmatrix.com/documents/12.2.0/core/howto/pb-intro.html

Ante

Do not use pickle. Do not use pickle. Do not use pickle. Do not ever suggest the use of pickle to novice network programmers. See the big red box on top of docs.python.org/2/library/pickle.html . – Cyruscyst