Requests-html results in OSError: [Errno 8] Exec format error when calling html.render()

Question

I am using requests-html and trying the render function, with little success. When I run this script using python3.8

#!/usr/bin/python3
from requests_html import HTML
file = "scrape/temp_file2.html"
with open(file) as html_file:
   source = html_file.read()
   html = HTML(html=source)
   html.render()
   match = html.find('#footer', first=True)
   try:
      print(match.hmtl)
   except:
      print('not found')

it results in a traceback:

python3 scrape/test1.py
Traceback (most recent call last):
  File "scrape/test1.py", line 10, in <module>
    html.render()
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 730, in browser
    self._browser = self.loop.run_until_complete(super().browser)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/pi/.local/lib/python3.8/site-packages/requests_html.py", line 714, in browser
    self._browser = await pyppeteer.launch(ignoreHTTPSErrors=not(self.verify), headless=True, args=self.__browser_args)
  File "/home/pi/.local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 306, in launch
    return await Launcher(options, **kwargs).launch()
  File "/home/pi/.local/lib/python3.8/site-packages/pyppeteer/launcher.py", line 147, in launch
    self.proc = subprocess.Popen(  # type: ignore
  File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome'

The chromium file is an executable:

ls -l /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome
-rwxr-xr-x 1 pi pi 214121928 Mar  9 17:21 /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome

If I remove the html.render() line it works fine (but won't render the javascript). Any ideas?

Lowell

If you do file /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome, does it identify it as an executable that matches your CPU? – Logistic

You're right, the executable does not match the CPU: file /home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, not stripped It should be, for example: file /usr/local/bin/python3.8 python3.8: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=11f271b583048eca129a464244876a525e5f92f0, not stripped. Any idea how to correct it? – Lowell

Answer 1

After much searching I found a solution. The problem is that pyppeteer installs the x86-64 version of chromium on a 32-bit ARM processor. The solution below is to install the chromium files from Ubuntu and link to those files from the pyppeteer directory. From https://github.com/miyakogi/pyppeteer/issues/250:

install chromium-codecs-ffmpeg:
   https://launchpad.net/ubuntu/trusty/armhf/chromium-codecs-ffmpeg
   ➜ chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb
   sudo dpkg --force-all -i chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb

install chromium-browser:
   https://launchpad.net/ubuntu/trusty/armhf/chromium-browser
   ➜ chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb
   sudo dpkg --force-all -i chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb

Other notes:

• make sure that you download a version >= 62. I've used version 65 and it works fine.
• install chromium-codecs-ffmpeg first, then chromium-browser by using the command "dpkg --force-all -i".
• after the installation of Chromium from Ubuntu packages, grab its path by running "whereis chromium-browser"
• most probably, it's going to be on "/usr/bin/chromium-browser".
• go to the directory where pyppeteer downloaded x86 Chromium on it.
• in your case it is "/home/pi/.local/share/pyppeteer/local-chromium/588429/chrome-linux/".
• delete all its files.
• in the same directory. Create a symbol link which has the name "chrome" and points to Ubuntu's Chromium (you have got its path in step 3).
• for example: ln -s /usr/bin/chromium-browser chrome
• after performing those steps, everything should work fine.

Lowell

If Chromium is already installed on Raspberry Pi you can just link to the existing installation. – Augite