Pytest: Windows fatal exception: access violation

Question

I'm using pytest to run some tests for my project. Sometimes (about 30 to 50%) I get an error after the test finished. But this is preventing the testengine to create the testreport, which is really a pain.

Error:

Windows fatal exception: access violation

Current thread 0x000019e0 (most recent call first):
  File "C:\Python38\lib\threading.py", line 1200 in invoke_excepthook
  File "C:\Python38\lib\threading.py", line 934 in _bootstrap_inner
  File "C:\Python38\lib\threading.py", line 890 in _bootstrap

Thread 0x00001b0c (most recent call first):
  File "C:\Python38\lib\site-packages\serial\serialwin32.py", line 240 in _close
  File "C:\Python38\lib\site-packages\serial\serialwin32.py", line 246 in close
  File "C:\NoBackup\svn\test_system_smets2\pyets\plugins\plugin_zigbee_dongle.py", line 141 in disconnect
  File "C:\NoBackup\svn\test_system_smets2\pyets\plugins\plugin_zigbee_dongle.py", line 232 in stop
  File "C:\NoBackup\svn\test_system_smets2\pyets\testengine\plugin_manager.py", line 286 in stop
  File "C:\NoBackup\svn\test_system_smets2\pyets\testengine\testengine.py", line 112 in session_finalize
  File "C:\NoBackup\svn\test_system_smets2\pyets\testengine\pytest_test_engine.py", line 58 in test_engine
  File "C:\Python38\lib\site-packages\_pytest\fixtures.py", line 800 in _teardown_yield_fixture
  File "C:\Python38\lib\site-packages\_pytest\fixtures.py", line 871 in finish
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 318 in _callfinalizers
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 328 in _teardown_with_finalization
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 310 in _pop_and_teardown
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 350 in _teardown_towards
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 342 in teardown_exact
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 148 in pytest_runtest_teardown
  File "C:\Python38\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 83 in <lambda>
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 92 in _hookexec
  File "C:\Python38\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 217 in <lambda>
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 244 in from_call
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 216 in call_runtest_hook
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 186 in call_and_report
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 101 in runtestprotocol
  File "C:\Python38\lib\site-packages\_pytest\runner.py", line 85 in pytest_runtest_protocol
  File "C:\Python38\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 83 in <lambda>
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 92 in _hookexec
  File "C:\Python38\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Python38\lib\site-packages\_pytest\main.py", line 272 in pytest_runtestloop
  File "C:\Python38\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 83 in <lambda>
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 92 in _hookexec
  File "C:\Python38\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Python38\lib\site-packages\_pytest\main.py", line 247 in _main
  File "C:\Python38\lib\site-packages\_pytest\main.py", line 191 in wrap_session
  File "C:\Python38\lib\site-packages\_pytest\main.py", line 240 in pytest_cmdline_main
  File "C:\Python38\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 83 in <lambda>
  File "C:\Python38\lib\site-packages\pluggy\manager.py", line 92 in _hookexec
  File "C:\Python38\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Python38\lib\site-packages\_pytest\config\__init__.py", line 124 in main
  File "testrun.py", line 1184 in run_single_test
  File "testrun.py", line 1548 in main
  File "testrun.py", line 1581 in <module>

Has somebody any idea how to fix or debug that?

I'm using pytest 5.4.1 with python 3.8.0 on win10. But this is also reproducable with older pytest versions.

The plugin plugin_zigbee_dongle.py uses pyserial (3.4) to communicate with an usb-rf-dongle in a thread. The following code is a snippet of this plugin:

import serial
import threading

class ZigbeeDongleSerial(object):

    def __init__(self, test_engine):
        self.ser = serial.Serial()
        self.test_engine = test_engine

    # ----------------------------------------------------------
    def connect(self,
                port,
                baud,
                timeout):

        self.ser.baudrate = baud
        self.ser.timeout = timeout
        self.ser.port = port
        self.ser.open()
    
    # ----------------------------------------------------------
    def disconnect(self):

        try:
            if self.is_connected():
                self.ser.close() # <------- This is line 141 ----------
        except:
            pass

# ----------------------------------------------------------
# ----------------------------------------------------------
class PluginZigbeeDongle(PluginBase):

    # ----------------------------------------------------------
    def __init__(self, test_engine):
        super(PluginZigbeeDongle, self).__init__()

        self.test_engine        = test_engine
        
        self.dongle_serial = ZigbeeDongleSerial(self.test_engine)

        self._startup_lock = threading.Lock()
        self._startup_lock.acquire()

        self.reader_thread = threading.Thread(target=self._read_worker, name="ZigbeeDongleThread")

    # ----------------------------------------------------------
    def start(self):
        super(PluginZigbeeDongle, self).start()

        # connect the serial port
        self.dongle_serial.connect()

        # start the reader thread
        if self.dongle_serial.is_connected():
            self.reader_thread.start()

    # ----------------------------------------------------------
    def stop(self):
        super(PluginZigbeeDongle, self).stop()

        # disconnect the serial port
        if self.dongle_serial.is_connected():
            self.dongle_serial.disconnect() # <------- This is line 232 ----------

        # stop the reader thread
        if self.reader_thread.is_alive():
            self.reader_thread.join()

    # ----------------------------------------------------------
    def _read_worker(self):

        # start-up is now complete
        self._startup_lock.release()

        # handle the incoming character stream
        done = False
        while not done:

            try:
                c = self.dongle_serial.read_byte()

            except serial.SerialException:
                done = True

            except AttributeError:
                done = True

            if not done:
                self._read_parser(c)

Muntin

The error is raised in plugin_zigbee_dongle.py which looks like a module from your project. Without seeing what's happening there, it's not possible to say what the issue is. Can you provide a minimal reproducible example? – Wretch

I also face access violation problem when using fixture and underlying c-library wrapped with cython – Stonecrop

@hoefling: Thank you for your comment, I added a code snippet – Muntin

I get this without having installed pyserial. OTOH, I have installed pytest-qt, and this sort of error seems to occur when I do things using the qtbot fixture, but only occasionally: at the moment I have no understanding of what sort of coding causes it. It seems to be connected with using unittest.mock.patch to mock out methods on PyQt5 objects. But other tests handle that perfectly well, so at the moment I'm mystified. – Shontashoo

Answer 1

Downgrading pyserial from Version 3.4 to Version 2.7 fixed the problem

Muntin