Load/reload a portion of code in Python without restarting main script

Question

Intro

I've been tinkering with Twisted for the past few days, having picked up python less than a month ago. My first inclination was to play with something I know and use every day, IRC. I've gotten a basic IRC connection up and running thanks to the ircLogBot.py example.

Question

I want to have some arbitrary code that runs whenever an IRC event (PRIVMSG/CTCP/JOIN/PART) is received, and for purposes of debugging I'd like to be able to make changes to that piece of code and then reload it without shutting down the whole script and reconnecting to the IRC server.

Final Notes

It doesn't have to be a solution that incorporates Twisted, as I do not fully understand it yet. Though I assume this is the sort of thing that twisted, being an event-driven framework, is likely designed to do well.

Quintie

Answer 1

Schedule a recurring event (every few seconds) to reload your module being debugged. Use the builtin reload() function for this purpose.

You might want to wrap it in a try/except to handle the case where you introduce an error in the module being debugged. That will keep the server alive in the face of errors :-)

Arun

Accepted! I didn't think to use modules like that. – Quintie

Be sure to read reload's docs, what it does isn't too intuitive. – Catechism

Answer 2

Twisted has some built-in functionality in twisted.python.rebuild which provides a more comprehensive implementation of Python's built-in reload function. There are still some limitations, but its chief difference from Python's built-in reload is that it will find old instances of objects and replace their classes with the new version. (The main limitation is that you have to be aware that your instances may have old state that doesn't match with your current version of __init__, if you've changed it, which is what rebuild.Sensitive is for.)

If you want to make something really fancy and automatic, you can set up a file-system monitor that detects when files change, and re-load the associated modules with rebuild when it changes. On Linux, you can use Twisted's inotify support for the change notifications, and on OS X, you can use cfreactor along with the FSEvents API via PyObjC. (I don't know which file alteration monitoring schemes work on win32 but it may be possible there as well.)

Branscum

Answer 3

You might want to take a look at 'livecoding' which does handle code reloading ; it might suit your needs. Note, though, that 'livecoding' needs code files (folders) to follow its own conventions that are not necessarily pythonic.

Another possibility is to roll your own system using reload, like Raymond suggested. If your code is well detached from your state data, it may be easy to implement.

Earthling

Answer 4

Run in the interactive interpreter. You can start a background thread and you're still in interactive mode. Then you can do reload as desired. That gives you more control than having it automatic on a timer.

I used to occasionally start up my whole enterprise server in interactive mode.

Depside