Catching `KeyboardInterrupt` without closing Selenium Webdriver sessions in Python

Question

A Python program drives Firefox via Selenium WebDriver. The code is embedded in a try/except block like this:

session = selenium.webdriver.Firefox(firefox_profile)
try:
    # do stuff
except (Exception, KeyboardInterrupt) as exception:
    logging.info("Caught exception.")
    traceback.print_exc(file=sys.stdout)

If the program aborts because of an error, the WebDriver session is not closed and hence the Firefox window is left open. But if the program aborts with a KeyboardInterrupt exception, the Firefox window gets closed (I suppose because the WebDriver sessions are released, too) and I would like to avoid this.

I know that both exceptions go through the same handler because I see the "Caught exception" message in both cases.

How could I avoid the closing of the Firefox window with KeyboardInterrupt?

Gotcher

Because you have included Exception which is a very general and wide exception clause in the except statement. Try restricting yourself to KeyboardInterrupt and tell me if it works. – Callen

I cannot reproduce this on Windows 7 with Firefox 52.1, geckodriver 0.16.1, and Selenium 3.9.0. Can you please post what OS you are using and which versions of Firefox, geckodriver, and Selenium? – Fibrinolysin

I can't reproduce this either. In my testing, the code you have already behaves as desired on Windows 7, using latest firefox, geckodriver, and selenium. Edit: However, chrome behaves differently in that it is closed in both cases. – Decosta

In hopes of saving somebody a bit of time: The signal.signal approach does not circumvent this problem. – Kremenchug

Answer 1

I've got a solution, but it's pretty ugly.

When Ctrl+C is pressed, python receives a Interrupt Signal (SIGINT), which is propagated throughout your process tree. Python also generates a KeyboardInterrupt, so you can try to handle something that is bound to the logic of your process, but logic that is coupled to child processes cannot be influenced.

To influence which signals are passed on to your child processes, you'd have to specify how signals should be handled, before the process is spawned through subprocess.Popen.

There are various options, this one is taken from another answer:

import subprocess
import signal

def preexec_function():
    # Ignore the SIGINT signal by setting the handler to the standard
    # signal handler SIG_IGN.
    signal.signal(signal.SIGINT, signal.SIG_IGN)

my_process = subprocess.Popen(
    ["my_executable"],
    preexec_fn = preexec_function
)

Problem is, you're not the one calling Popen, that is delegated to selenium. There are various discussions on SO. From what I've gathered other solutions that try to influence signal masking are prone to failure when the masking is not executed right before the call to Popen.

Also keep in mind, there is a big fat warning regarding the use of preexec_fn in the python documentation, so use that at your own discretion.

"Luckily" python allows to override functions at runtime, so we could do this:

>>> import monkey
>>> import selenium.webdriver
>>> selenium.webdriver.common.service.Service.start = monkey.start
>>> ffx = selenium.webdriver.Firefox()
>>> # pressed Ctrl+C, window stays open.
KeyboardInterrupt
>>> ffx.service.assert_process_still_running()
>>> ffx.quit()
>>> ffx.service.assert_process_still_running()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 107, in assert_process_still_running
    return_code = self.process.poll()
AttributeError: 'NoneType' object has no attribute 'poll'

with monkey.py as follows:

import errno
import os
import platform
import subprocess
from subprocess import PIPE
import signal
import time
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common import utils

def preexec_function():
    signal.signal(signal.SIGINT, signal.SIG_IGN)

def start(self):
  """
        Starts the Service.
        :Exceptions:
         - WebDriverException : Raised either when it can't start the service
           or when it can't connect to the service
        """
  try:
    cmd = [self.path]
    cmd.extend(self.command_line_args())
    self.process = subprocess.Popen(cmd, env=self.env,
                                    close_fds=platform.system() != 'Windows',
                                    stdout=self.log_file,
                                    stderr=self.log_file,
                                    stdin=PIPE,
                                    preexec_fn=preexec_function)
#                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  except TypeError:
    raise
  except OSError as err:
    if err.errno == errno.ENOENT:
      raise WebDriverException(
        "'%s' executable needs to be in PATH. %s" % (
          os.path.basename(self.path), self.start_error_message)
      )
    elif err.errno == errno.EACCES:
      raise WebDriverException(
        "'%s' executable may have wrong permissions. %s" % (
          os.path.basename(self.path), self.start_error_message)
      )
    else:
      raise
  except Exception as e:
    raise WebDriverException(
      "The executable %s needs to be available in the path. %s\n%s" %
      (os.path.basename(self.path), self.start_error_message, str(e)))
  count = 0
  while True:
    self.assert_process_still_running()
    if self.is_connectable():
      break
    count += 1
    time.sleep(1)
    if count == 30:
      raise WebDriverException("Can not connect to the Service %s" % self.path)

the code for start is from selenium, with the added line as highlighted. It's a crude hack, it might as well bite you. Good luck :D

Stoller

You are missing this in the imports: from selenium.common.exceptions import WebDriverException – Drivein

Answer 2

I was inpired by @einsweniger answer, thanks a lot! This code worked for me:

import subprocess, functools, os
import selenium.webdriver

def new_start(*args, **kwargs):
    def preexec_function():
        # signal.signal(signal.SIGINT, signal.SIG_IGN) # this one didn't worked for me
        os.setpgrp()
    default_Popen = subprocess.Popen
    subprocess.Popen = functools.partial(subprocess.Popen, preexec_fn=preexec_function)
    try:
        new_start.default_start(*args, **kwargs)
    finally:
        subprocess.Popen = default_Popen
new_start.default_start = selenium.webdriver.common.service.Service.start
selenium.webdriver.common.service.Service.start = new_start

It's less intrusive than the previous answer as it does not rewrite the code of the full function. But on the other hand it modifies the subprocess.Popen function itself, which can be called a pretty ugly move by some.

It does the job anyway, and you don't have to update the code when the source code of Service.start changes.

Palestine

I'm getting a "maximum recursion depth" error on new_start.default_start(*args, **kwargs) via subprocess.Popen = functools.partial(subprocess.Popen, preexec_fn=preexec_function) – Tiger

@Tiger You should run this code only once. This code stores the original start method in new_start.default_start, if you execute it twice it will store new_start instead, which would explain the recursion error. – Palestine

To make the code run only once, you could put it inside if not hasattr(selenium.webdriver.common.service.Service.start, 'default_start'): ... – Palestine

Answer 3

I was inspired by @User9123's answer, and have cleaned it up a bit:

import functools
import subprocess
from selenium.webdriver import Firefox

subprocess_Popen = subprocess.Popen
subprocess.Popen = functools.partial(subprocess_Popen, process_group=0)
driver = Firefox()
subprocess.Popen = subprocess_Popen  # Undo the monkey patch

I've only tried this on Ubuntu so far, but it does exactly what I wanted!

Antenatal