Example of the right way to use QThread in PyQt?

Question

I'm trying to learn how to use QThreads in a PyQt Gui application. I have stuff that runs for a while, with (usually) points where I could update a Gui, but I would like to split the main work out to its own thread (sometimes stuff gets stuck, and it would be nice to eventually have a cancel/try again button, which obviously doesn't work if the Gui is frozen because the Main Loop is blocked).

I've read https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/. That page says that re-implementing the run method is not the way to do it. The problem I am having is finding a PyQt example that has a main thread doing the Gui and a worker thread that does not do it that way. The blog post is for C++, so while it's examples do help, I'm still a little lost. Can someone please point me to an example of the right way to do it in Python?

Ciracirca

This looks like a dup of Background thread with QThread in PyQt. The second example in the accepted answer looks like a straightforward translation of the C++ code from the blog you linked. – Cochabamba

Also, have you written any native Python threading code (with threading.Thread, etc.)? If not, you may want to work through some examples of that first. (Also see Threading in a PyQt application: Use Qt threads or Python threads to see if you even need QThread here.) – Cochabamba

@Cochabamba Thanks, I think that link was just what I was looking for. I had seen the second link, and decided that I should use QThreads because I wanted to be able to send slots/signals between threads. I was aware that threading.Thread exists, but have not used it before. I did a lot of searching, and even saw the first link, skimmed it, saw def run and moved on, not realizing the showed both ways! – Ciracirca

Answer 1

Here is a working example of a separate worker thread which can send and receive signals to allow it to communicate with a GUI.

I made two simple buttons, one which starts a long calculation in a separate thread, and one which immediately terminates the calculation and resets the worker thread.

Forcibly terminating a thread as is done here is not generally the best way to do things, but there are situations in which always gracefully exiting is not an option.

from PyQt4 import QtGui, QtCore
import sys
import random

class Example(QtCore.QObject):

    signalStatus = QtCore.pyqtSignal(str)

    def __init__(self, parent=None):
        super(self.__class__, self).__init__(parent)

        # Create a gui object.
        self.gui = Window()

        # Create a new worker thread.
        self.createWorkerThread()

        # Make any cross object connections.
        self._connectSignals()

        self.gui.show()


    def _connectSignals(self):
        self.gui.button_cancel.clicked.connect(self.forceWorkerReset)
        self.signalStatus.connect(self.gui.updateStatus)
        self.parent().aboutToQuit.connect(self.forceWorkerQuit)


    def createWorkerThread(self):

        # Setup the worker object and the worker_thread.
        self.worker = WorkerObject()
        self.worker_thread = QtCore.QThread()
        self.worker.moveToThread(self.worker_thread)
        self.worker_thread.start()

        # Connect any worker signals
        self.worker.signalStatus.connect(self.gui.updateStatus)
        self.gui.button_start.clicked.connect(self.worker.startWork)


    def forceWorkerReset(self):      
        if self.worker_thread.isRunning():
            print('Terminating thread.')
            self.worker_thread.terminate()

            print('Waiting for thread termination.')
            self.worker_thread.wait()

            self.signalStatus.emit('Idle.')

            print('building new working object.')
            self.createWorkerThread()


    def forceWorkerQuit(self):
        if self.worker_thread.isRunning():
            self.worker_thread.terminate()
            self.worker_thread.wait()


class WorkerObject(QtCore.QObject):

    signalStatus = QtCore.pyqtSignal(str)

    def __init__(self, parent=None):
        super(self.__class__, self).__init__(parent)

    @QtCore.pyqtSlot()        
    def startWork(self):
        for ii in range(7):
            number = random.randint(0,5000**ii)
            self.signalStatus.emit('Iteration: {}, Factoring: {}'.format(ii, number))
            factors = self.primeFactors(number)
            print('Number: ', number, 'Factors: ', factors)
        self.signalStatus.emit('Idle.')

    def primeFactors(self, n):
        i = 2
        factors = []
        while i * i <= n:
            if n % i:
                i += 1
            else:
                n //= i
                factors.append(i)
        if n > 1:
            factors.append(n)
        return factors


class Window(QtGui.QWidget):

    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.button_start = QtGui.QPushButton('Start', self)
        self.button_cancel = QtGui.QPushButton('Cancel', self)
        self.label_status = QtGui.QLabel('', self)

        layout = QtGui.QVBoxLayout(self)
        layout.addWidget(self.button_start)
        layout.addWidget(self.button_cancel)
        layout.addWidget(self.label_status)

        self.setFixedSize(400, 200)

    @QtCore.pyqtSlot(str)
    def updateStatus(self, status):
        self.label_status.setText(status)


if __name__=='__main__':
    app = QtGui.QApplication(sys.argv)
    example = Example(app)
    sys.exit(app.exec_())

Plaintive

When I run this, I get. Qt has caught an exception thrown from an event handler. Throwing exceptions from an event handler is not supported in Qt. You must reimplement QApplication::notify() and catch all exceptions there. – Meerkat

Any idea what the exception was? I tested this snippet using PyQt4 with Python 2.7 on OS X without any errors. When I try to use this with Python 3.4, I occasionally get segmentation faults. I am not sure if this issue is with the code as written or the specific (version specific) implementation. – Plaintive

I haven't investigated what is throwing but apparently you have something which occasionally throws an exception into a message loop, which must never happen. – Meerkat

Important to note if you're trying to copy this example: be sure to store both the QThread and the Worker if you create them in a function. Otherwise, at the end of the function, the garbage collector will dump both and the Worker won't run – Thadeus

I found that there was a problem with my example in that I was attempting to reuse the thread and worker objects after a forced termination. This appears to be what was leading to the segmentation faults I was seeing. This no longer segfaults, but occasionally the thread is not terminated after the cancel button is pressed and the application hangs. – Plaintive

I'm puzzling over this... it seems a little confusing to have two signals, both called signalStatus, one belonging to Example and the other to WorkerObject, and incidentally both of which call Window.updateStatus. For sake of simplicity it might be better to get rid of one. Apart from this, this seems very good, but it would appear that you have to be careful with the scope of ... something, because trying to apply this to the architecture of my app I keep getting those mysterious uncatchable "Python has had a problem" crashes on App.quit(). NB using PyQt5 (5.15) on W10 OS. – Dockyard

Answer 2

You are right that it is a good thing to have a worker thread doing the processing while main thread is doing the GUI. Also, PyQt is providing thread instrumentation with a signal/slot mechanism that is thread safe.

This may sound of interest. In their example, they build a GUI

import sys, time
from PyQt4 import QtCore, QtGui

class MyApp(QtGui.QWidget):
 def __init__(self, parent=None):
  QtGui.QWidget.__init__(self, parent)

  self.setGeometry(300, 300, 280, 600)
  self.setWindowTitle('threads')

  self.layout = QtGui.QVBoxLayout(self)

  self.testButton = QtGui.QPushButton("test")
  self.connect(self.testButton, QtCore.SIGNAL("released()"), self.test)
  self.listwidget = QtGui.QListWidget(self)

  self.layout.addWidget(self.testButton)
  self.layout.addWidget(self.listwidget)

 def add(self, text):
  """ Add item to list widget """
  print "Add: " + text
  self.listwidget.addItem(text)
  self.listwidget.sortItems()

 def addBatch(self,text="test",iters=6,delay=0.3):
  """ Add several items to list widget """
  for i in range(iters):
   time.sleep(delay) # artificial time delay
   self.add(text+" "+str(i))

 def test(self):
  self.listwidget.clear()
  # adding entries just from main application: locks ui
  self.addBatch("_non_thread",iters=6,delay=0.3)

(simple ui containing a list widget which we will add some items to by clicking a button)

You may then create our own thread class, one example is

class WorkThread(QtCore.QThread):
 def __init__(self):
  QtCore.QThread.__init__(self)

 def __del__(self):
  self.wait()

 def run(self):
  for i in range(6):
   time.sleep(0.3) # artificial time delay
   self.emit( QtCore.SIGNAL('update(QString)'), "from work thread " + str(i) )

  self.terminate()

You do redefine the run() method. You may find an alternative to terminate(), see the tutorial.

Hyaline

The OP specifically said he wants to use the moveToThread mechanism, rather than the QThread.run mechanism. I'm not sure if he has a good reason for that, but still, you're not answering his question. – Cochabamba

The OP is right: you should not subclass QThread. See blog.qt.io/blog/2010/06/17/youre-doing-it-wrong – Eastsoutheast

The Qt docs on threading do mention subclassing QThread though ... – Waterside

And this blog explains why the one mentioned by @Eastsoutheast was partially wrong. – Waterside

I recommend wrapping run() code in a try/catch block. That's because (at least on Windows and py3.7) any exception raisec from run method and not catched leads to application silently closing, without any message in console. – Stoller

The QThread 5.15 docs, has subclassing & moveToThread() example code, plus reasons when each is appropriate. And there is this interesting bit: "It is important to remember that a QThread instance lives in the old thread that instantiated it, not in the new thread that calls run(). This means that all of QThread's queued slots and invoked methods will execute in the old thread. Thus, a developer who wishes to invoke slots in the new thread must use the worker-object approach; new slots should not be implemented directly into a subclassed QThread." – Tan

Answer 3

In my opinion, by far the best explanation, with example code which is initially unresponsive, and is then improved, is to be found here.

Note that this does indeed use the desired (non-subclassed) QThread and moveToThread approach, which the article claims to be the preferred approach.

The above linked page also provides the PyQt5 equivalent to the C Qt page giving the definitive explanation by Maya Posch from 2011. I think she was probably using Qt4 at the time, but that page is still applicable in Qt5 (hence PyQt5) and well worth studying in depth, including many of the comments (and her replies).

Just in case the first link above one day goes 404 (which would be terrible!), this is the essential Python code which is equivalent to Maya's C code:

self.thread = QtCore.QThread()
# Step 3: Create a worker object
self.worker = Worker()
# Step 4: Move worker to the thread
self.worker.moveToThread(self.thread)
# Step 5: Connect signals and slots
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.thread.finished.connect(self.thread.deleteLater)
self.worker.progress.connect(self.reportProgress)
# Step 6: Start the thread
self.thread.start()

# Final resets
self.longRunningBtn.setEnabled(False)
self.thread.finished.connect(
    lambda: self.longRunningBtn.setEnabled(True)
)
self.thread.finished.connect(
    lambda: self.stepLabel.setText("Long-Running Step: 0")
)   

NB self in the example on that page is the QMainWindow object. I think you may have to be quite careful about what you attach QThread instances to as properties: instances which are destroyed when they go out of scope, but which have a QThread property, or indeed a local QThread instance which goes out of scope, seem to be capable of causing some inexplicable Python crashes, which aren't picked up by sys.excepthook (or the sys.unraisablehook). Caution advised.

... where Worker looks something like this:

class Worker(QtCore.QObject):
    finished = QtCore.pyqtSignal()
    progress = QtCore.pyqtSignal(int)

    def run(self):
        """Long-running task."""
        for i in range(5):
            sleep(1)
            self.progress.emit(i + 1)
        self.finished.emit()

Dockyard