Can Selenium be run on a server to automate tasks, or does it need a browser to run the task inside of

Question

I was reading over the Selenium docs and i couldn't quite work out whether you could run it on a server - as part of a larger web app.

ie. something happens in the web app that triggers the automated Selenium script to run, then returns result to web app.

Is that a possible use of Selenium or does it always need an actual browser to run the tasks inside of, or can it run a virtual browser for itself ?

---

Craniometry

Answer 1

Selenium always need an instance of a browser to control.

Luckily, there are browsers out there that aren't that heavy as the usual browsers you know. You don't have to open IE / Firefox / Chrome / Opera. You can use HtmlUnitDriver which controls HTMLUnit - a headless Java browser that does not have any UI. Or a PhantomJsDriver which drives PhantomJS - another headless browser running on WebKit.

Those headless browsers are much less memory-heavy, usually are faster (since they don't have to render anything), they don't require a graphical interface to be available for the computer they run at and are therefore easily usable server-side.

Mirilla

actually, PhantomJS renders the page in memory as far as i know. you can even take screenshots with it! – Walking

do they also save cookies for login sessions? – Shouse

@Pengin The answer is 8 years old now, and a lot has changes. Selenium still rules, PhantomJS no longer exists. And there are new things, like github.com/microsoft/Playwright-Java, and many more. I do no longer know which in-memory library is the best. Sorry. – Christabella

Answer 2

I managed to run Selenium on CentOS server. I have used Chromedriver. As I found the Chromedriver perhaps needs a browser (although the latest Chrome reportedly can run without browser...I didn't try it). But in server you don't need a GUI to run a browser (Google-Chrome) for this purpose.

Following are the steps I have followed:

• We need Google Chrome to be installed (IF NOT INSTALLED):

# FOR CENTOS
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
yum localinstall google-chrome-stable_current_x86_64.rpm

# FOR UBUNTU (I didn't validate it myself, as I only worked with CentOS)
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb
sudo apt-get install -f

• Now check if Google Chrome is installed by checking its version:

google-chrome --version

• If the above steps are okay, now download Chromedriver for your distribution and according to the Google Chrome's version you have. To get Chromedriver go: Chrome Driver | Downloads
• Now you have to include several options to run it. (without --no-sandbox the Chrome browser won't run. Some also say to keep this option as the first option. Some other say only this option is enough....read the comments of the post mentioned in the third reference.):

from selenium.webdriver.chrome.options import Options
from selenium import webdriver

options = Options()
options.add_argument('--no-sandbox')
options.add_argument("--headless")
options.add_argument('--disable-dev-shm-usage')

• Now show the Chromedriver's directory (the following shows for the current directory) and also pass these options:

driver = webdriver.Chrome('./chromedriver', options = options)

• Finally start Selenium's crawling:

driver.get(YOUR_DESIRED_LINK)

Reference:

• Install Google Chrome | CentOS 7
• Install Google Chrome | Ubuntu
• To make Selenium to run on server (also read the comments of this post).

Ratter

Answer 3

Using Selenium, WebDriver, and ChromeDriver

Right now, Selenium opens a full instance of Chrome. In other words, it's an automated solution but not completely headless. However, Selenium can be configured to run headless Chrome with a little work. I recommend Running Selenium with Headless Chrome if you want the full instructions on how to set things up yourself, but I've dropped in some examples below to get you started.

Disavow

where are examples? – Marrilee

@SarveshwarPM Follow the link – Disavow

Answer 4

You can use Selenium Grid for this. You can setup Grid into your server, actually Grid has docker images. So it can be easily installed. You can route test scripts to grid url

this docker-compose.yml file can be used

# To execute this docker-compose yml file use `docker-compose -f docker-compose-v3.yml up`
# Add the `-d` flag at the end for detached execution
# To stop the execution, hit Ctrl+C, and then `docker-compose -f docker-compose-v3.yml down`
version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.14.1-20231020
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  edge:
    image: selenium/node-edge:4.14.1-20231020
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  firefox:
    image: selenium/node-firefox:4.14.1-20231020
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443

  selenium-hub:
    image: selenium/hub:4.14.1-20231020
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

Whipstall