Pyinstaller doesn't work properly with threading

Question

A program that runs a bat file which contains instruction for running an executable(longTask) using Qthread but it doesn't work as expected when I create an executable using Pyinstaller with the following command. I gave '--windowed' to do not provide a console window for standard i/o

pyinstaller --onefile --windowed main.py

But the interesting thing is it works as expected when I remove the --windowed argument

pyinstaller --onefile main.py

Here is the code:

from PyQt4.Qt import *
import subprocess


def callSubprocess():
    page = QWizardPage()
    page.setTitle("Run myLongTask")

    runButton = QPushButton("Run")
    progressBar = QProgressBar()
    procLabel = QLabel()
    procLabel1 = QLabel()
    progressBar.setRange(0, 1)

    layout = QGridLayout()
    layout.addWidget(runButton, 0, 0)
    layout.addWidget(progressBar, 0, 1)
    layout.addWidget(procLabel)
    layout.addWidget(procLabel1)

    # Calls thread class
    myLongTask = TaskThread()
    runButton.clicked.connect(lambda: OnStart(myLongTask, progressBar, procLabel1))
    myLongTask.taskFinished.connect(lambda: onFinished(progressBar, procLabel))
    page.setLayout(layout)
    return page


def OnStart(myLongTask, progressBar, procLabel1):
    progressBar.setRange(0, 0)
    myLongTask.start()
    # I am waiting until my subprocess completes
    while not myLongTask.isFinished():
        QCoreApplication.processEvents()
    procLabel1.setText("Hello This is main")


def onFinished(progressBar, procLabel):
        # Stop the pulsation
        progressBar.setRange(0, 1)
        procLabel.setText("longTask finished")


class TaskThread(QThread):
    taskFinished = pyqtSignal()
    def __init__(self):
        QThread.__init__(self)

    def run(self):
        proc = subprocess.Popen(r'C:\Users\Desktop\runInf.bat', bufsize=0, shell=True,  stdout=subprocess.PIPE)
        proc.wait()
        self.taskFinished.emit()

    def __del__(self):
        self.wait()


if __name__ == '__main__':
    import sys
    app = QApplication(sys.argv)
    wizard = QWizard()
    wizard.addPage(callSubprocess())
    wizard.setWindowTitle("Example Application")
    wizard.show()

sys.exit(wizard.exec_())

When executed the above code in Pycharm it works as expected. But when built with PyInstaller, the main thread doesn't wait until the subprocess completion.

Any idea how to create an executable where threading works as expected. Thanks in advance

Guipure

What evidence do you have that the problem that qthreads don't work? I would think it more likely that your program can't find runInf.bat and so the thread ends earlier than you expect (when using pyinstaller). – Artemisia

@Artemisia Thank you for your reply. I gave the absolute path but still it doesn't work. – Guipure

I would encourage you to do some more debugging to see exactly what is failing and with what error message. I'm not sure that I can help without more information – Artemisia

@Artemisia Thank you. I really tried a lot and exhausted couldn't find the solution. If you could run the above code you can see the problem. – Guipure

Answer 1

I just resolved this issue. My findings as follows.

The culprit is subprocess. So I replaced subprocess with some file handling (let's say opening a file 10000 times and write some content) and it works as expected which means the main thread waits until child process completes.

So Pyinstaller does work properly with threading

I added few more arguments

proc = subprocess.Popen(os.getcwd() + r'\runInf.bat',
                            shell=True,
                            stdout=subprocess.PIPE,
                            stderr=subprocess.STDOUT,
                            stdin=subprocess.PIPE,
                            cwd=os.getcwd(),
                            env=os.environ)
proc.stdin.close()

This change made the subprocess to run and the main thread also waits until its completion

Guipure

Looks like this is a known bug with pyinstaller, subprocess and the --noconsole flag (which is the same as the --windowed flag). Glad you managed to figure it out! – Artemisia

Answer 2

I had a similar problem, which I resolved by using the "console=True" statement in the .spec file. Without the console the threading is not executed.

Femmine

seems more like a comment – Maximinamaximize