Paho MQTT Python Client: No exceptions thrown, just stops

Question

I try to setup a mqtt client in python3. This is not the first time im doing this, however i came across a rather odd behaviour. When trying to call a function, which contains a bug, from one of the callback functions (on_connect or on_message), python does not throw an exception (at least it is not printed), it just stops there. I tied together a short example, that reproduces that behaviour.

Does someone has an idea?

import paho.mqtt.client as mqtt

import re
import os.path

import json
from termcolor import colored

client = mqtt.Client()

def func():
    test = 1 + "1"
    print("Should never reach that")

def on_connect(client, userdata, flags, rc):
    """Establishes connection to broker
    """
    print("Connected to broker with result code " + str(rc))
    client.subscribe("test")

def on_message(client,userdata,msg):
    print("Recieved message on " + msg.topic)
    params = {}
    if msg.topic == "test":

        print("Invoke func")
        func()

if __name__ == "__main__":
    client.on_connect = on_connect
    client.on_message = on_message

    client.connect("localhost",1883,60)

    client.loop_forever()

This is the output when sending a message to the topic "test":

Connected to broker with result code 0
Recieved message on test
Invoke func

When calling func() from main, i get the correct TypeError thrown. So something catches this exception in paho. I had a look at an olderproject (python2 though) and tried to recreate the behaviour. There the exception gets thrown correctly. What do i miss?

EDIT I can catch the exception by putting the func() call in a try block. How ever, it does not stop the execution of the program when not catched. I dont get why

Longford

Answer 1

This will be due to the fact that the on_message function is called by the network thread and it will be wrapping that call in a try block to stop errors in on_message from stopping that thread.

If you want to an error to stop the app then you should use your own try block in on_message and behave appropriately.

Elyse

Just dived into the paho module code and saw the same. This changed between the python2 and python3 version of paho. Thanks for your answer anyway, your explanation makes a lot of sense. Just have to make sure, to get the log of mqtt printed somehow, to actually see what failed – Longford

@ralvarez ask a new question and reference this one, tag it with python so the language experts can help you. – Elyse

Answer 2

For anybody who comes across this and wonders why all exceptions inside of a mqtt callback are not thrown or at least not visible: In contrast to the python2 version of paho, the clients already catches ALL exceptions that occur when calling on of the user set callback functions. The output of this catch is then outputted to the on_log callback function. If this is not implemented by the user, there will be no visible output. So just add

def on_log(client, userdata, level, buff):
    print(buff)

mqttc.on_log = on_log

to your code, to print out the exception.

Longford

Answer 3

You can catch the errors using try + expect and then manually print the error message and pointer to the source of error using the traceback. This will give you mode details than using the on_log function.

import traceback

def on_message(client, userdata, msg):
    try:
        do_something(msg)
    except:
        traceback.print_exc()
        quit(0)

Cherice