Callback to python function from Tkinter Tcl crashes in windows

Question

This is not exactly my application, but very similar. I have created this test code to show the problem. Basically I am trying to call tcl proc from python thread. Tcl proc will callback to python function when result is ready. This result will be posted as an event to wx frame. When I run as pure python code, it works fine. When I use tcl proc, the whole app crashes without any info. If I increase wait_time (say 100) then it works fine even with tcl. Is it the high rate of callback a problem or am I missing something else. This app runs on windows by the way.

import wx
from Tkinter import Tcl
from threading import Thread
import wx.lib.newevent
from time import sleep

CountUpdateEvent, EVT_CNT_UPDATE = wx.lib.newevent.NewEvent()

tcl_code = 'proc tcl_worker {name max_count delay_time callback} { while {$max_count>0} {after $delay_time; $callback $name $max_count; incr max_count -1}}'

# Option to use Tcl or not for counter
# When enabled, Tcl will callback to python to upate counter value
use_tcl = True

# Option to create tcl interpreter per thread. 
# Test shows single interpreter for all threads will fail.
use_per_thread_tcl = True 

count = 5000 
wait_time = 1 ;# in milliseconds

class Worker:
    def __init__(self,name,ui,tcl):
        global use_per_thread_tcl
        self.name = name
        self.ui = ui
        if use_per_thread_tcl:
            self.tcl = Tcl()
            self.tcl.eval(tcl_code)
        else:
            self.tcl = tcl
        self.target = ui.add_textbox(name)
        self.thread = Thread(target=self.run)
        self.thread.daemon = True
        self.thread.start()

    def callback(self, name, val):
        evt = CountUpdateEvent(name=self.name, val=val, target=self.target)
        wx.PostEvent(self.ui,evt)        
    def run(self):
        global count, wait_time, use_tcl

        if use_tcl:
            # Register a python function to be called back from tcl
            tcl_cmd = self.tcl.register(self.callback)

            # Now call tcl proc
            self.tcl.call('tcl_worker', self.name, str(count), str(wait_time), tcl_cmd)
        else:
            # Convert milliseconds to seconds for sleep
            py_wait_time = wait_time / 1000
            while count > 0:
                # Directly call the callback from here
                self.callback(self.name, str(count))
                count -= 1
                sleep(py_wait_time)


class MainWindow(wx.Frame):
    def __init__(self, parent):
        wx.Frame.__init__(self, parent, title="Decrement Counter", size=(600, 100))

        self._DoLayout()
        self.Bind(EVT_CNT_UPDATE, self.on_count_update)

    def _DoLayout(self):
        self.sizer = wx.BoxSizer(wx.HORIZONTAL)

        self.panels = []
        self.tbs = []
        self.xpos = 0

    def add_textbox(self,name):
        panel = wx.Panel(self, pos=(self.xpos, 0), size=(60,40))
        self.panels.append(panel)
        tb = wx.StaticText(panel, label=name)
        tb.SetFont(wx.Font(16,wx.MODERN,wx.NORMAL,wx.NORMAL))
        self.sizer.Add(panel, 1, wx.EXPAND, 7)
        self.tbs.append(tb)
        self.xpos = self.xpos + 70
        return tb

    def on_count_update(self,ev):
        ev.target.SetLabel(ev.val)
        del ev

if __name__ == '__main__':
    app = wx.App(False)
    frame = MainWindow(None)
    tcl = Tcl()
    tcl.eval(tcl_code)
    w1 = Worker('A', frame, tcl)
    w2 = Worker('B', frame, tcl)
    w3 = Worker('C', frame, tcl)
    w4 = Worker('D', frame, tcl)
    w5 = Worker('E', frame, tcl)
    w6 = Worker('F', frame, tcl)
    w7 = Worker('G', frame, tcl)
    w8 = Worker('H', frame, tcl)
    frame.Show()
    app.MainLoop()

Tumbrel

Answer 1

Each Tcl interpreter object (i.e., the context that knows how to run a Tcl procedure) can only be safely used from the OS thread that creates it. This is because Tcl doesn't use a global interpreter lock like Python, and instead makes extensive use of thread-specific data to reduce the number of locks required internally. (Well-written Tcl code can take advantage of this to scale up very large on suitable hardware.)

Because of this, you must make sure that you only ever run Tcl commands or Tkinter operations from a single thread; that's typically the main thread, but I'm not sure if that's the real requirement for integrating with Python. You can launch worker threads if you want, but they'll be unable to use Tcl or Tkinter (well, not without very special precautions which are more trouble than it's likely worth). Instead, they need to send messages to the main thread for it to handle the interaction with the GUI; there are many different ways to do that.

Bilingual

Thanks for your response. By setting 'use_per_thread_tcl' to True in my code, every thread creates its own tcl interpreter. 'tcl_worker' proc is eval'd by each thread individually. So I guess there is no interactions between tcl interpreters. – Tumbrel

You create the Tcl interpreters before you enter your run() method, so they are all created in the main thread. Maybe move the creation to your run method. – Tanney

Thanks schlenk for the interesting suggestion. I did give a try as per your suggestion. Tcl interpreter doesn't even call the 'tcl_worker' proc. The whole app hangs without doing anything. It is happy when I cut down the number of workers to one. It seems like tcl interpreter can be created in main or child thread, but only one tcl interpreter is permitted. Perhaps Tkinter may be using some global variables which restrict using multiple interpreter objects. Next I will give a try using multiprocessing module, creating tcl objects in child processes. – Tumbrel

Answer 2

I hesitate to contradict Donal and AFAICT he's correct with respect to Tcl, however the situation in cPython is, as the source code admits, more complicated. In short, cPython creates an extra lock around the Tcl interpreter and manages the call method to do as Donal suggests, sending a message to have the main thread call the desired command.

Because much of the Tk() methods are implemented in terms of call, this means a large footprint of the tkinter API is thread-safe (but not everything). Of particular interest is the after method which queues up work in the event loop without blocking the current thread for long.

Finally, I'd like to point out that the eval method seems to be thread-unsafe, as it simply sends a string to the Tcl interpreter for evaluation from whichever thread it's in.

Vouvray