Python Twisted integration with Cmd module

Question

I like Python's Twisted and Cmd. I want to use them together.

I got some things working, but so far I haven't figured out how to make tab-completion work, because I don't see how to receive tab keypres events right away (without pressing Enter) in Twisted's LineReceiver.

Here's my code so far:

#!/usr/bin/env python

from cmd import Cmd
from twisted.internet import reactor
from twisted.internet.stdio import StandardIO
from twisted.protocols.basic import LineReceiver

class CommandProcessor(Cmd):
    def do_EOF(self, line):
        return True

class LineProcessor(LineReceiver):
    from os import linesep as delimiter # makes newline work

    def __init__(self):
        self.processor = CommandProcessor()
        self.setRawMode()

    def connectionMade(self):
        self.transport.write('>>> ')

    def rawDataReceived(self, data):
        self.processor.onecmd(data)
        self.transport.write('>>> ')

StandardIO(LineProcessor())
reactor.run()

Apart from tab completion, this somewhat works. I can enter a command like "help" and the Cmd module will print the results. But I've lost the nifty tab-complete functionality of the Cmd module, because Twisted is buffering one line at a time. I tried setting LineProcessor.delimiter to the empty string, to no avail. Maybe I need to find some other piece of Twisted to use instead of LineReceiver? Or maybe there's a simpler approach that will avoid my having to process every character one-by-one?

I can't use Cmd alone, because I want to make this a network application, where some commands will result in sending data, and receiving data from the network will happen asynchronously (and be displayed to the user).

So whether we start from the above code or something completely different, I'd like to build a nice, friendly terminal application in Python that responds to network events and also to tab completion. I hope I can use what's already out there and not have to implement too much myself.

Pandybat

take a look at Twisted Manhole, example – Bevon

I noticed Manhole and Conch but they didn't make much sense to me for what I'm doing. The documentation describes Conch as an SSHv2 implementation, both client and server, and illustrates how you can make an SSH server which does colorization for its clients. My needs are both similar and different to that. If you have a more specific suggestion of how I can use Manhole, I'm all ears...one of the problems with it is a pronounced lack of documentation. – Pandybat

Answer 1

You have a couple of difficulties with this approach:

• Cmd.onecmd is not going to do any tab processing.
• Even if it did, your terminal needs to be in cbreak mode in order for individual keystrokes to make it to the Python interpreter (tty.setcbreak can take care of that).
• As you know, Cmd.cmdloop is not reactor aware and will block waiting for input.
• Yet, to get all of the cool line-editing you want, Cmd (actually readline) needs to have direct access to stdin and stdout.

Given all of these difficulties, you might want to look at letting the CommandProcessor run in its own thread. For example:

#!/usr/bin/env python

from cmd import Cmd
from twisted.internet import reactor

class CommandProcessor(Cmd):
    def do_EOF(self, line):
        return True

    def do_YEP(self, line):
        reactor.callFromThread(on_main_thread, "YEP")

    def do_NOPE(self, line):
        reactor.callFromThread(on_main_thread, "NOPE")

def on_main_thread(item):
    print "doing", item

def heartbeat():
    print "heartbeat"
    reactor.callLater(1.0, heartbeat)

reactor.callLater(1.0, heartbeat)
reactor.callInThread(CommandProcessor().cmdloop)
reactor.run()

Footage

I was thinking of calling the completion methods in the Cmd module myself on Tab keypresses. Thank you for pointing out cbreak--I didn't know about that. I will try your approach, thank you for writing it up. – Pandybat

You're welcome. Basically, readline is a mini curses application. I think you will end up having to reimplement most of its functionality if you try to mediate access to stdin/stdout. This threaded approach let's it directly manage stdin/stdout (including cbreak mode) which should keep things simple. – Footage

twisted.internet.threads.blockingCallFromThread is a very useful twisted function that allows the CommandProcessor to get the return value. – Helga