Nightmare.js not working as expected on Ubuntu Linux cloud server
Asked Answered
J

4

8

I can't seem to get nightmare.js to work on an Ubuntu Linux 14.04 server [via DigitalOcean].

I've installed PhantomJS (1.9.8) and Node (4.2.4), and they are working well as far as I can tell.

For example, when I run this:

phantomjs loadspeed.js http://www.yahoo.com

with loadspeed.js containing this:

"use strict";
var page = require('webpage').create(),
    system = require('system'),
    t, address;

if (system.args.length === 1) {
    console.log('Usage: loadspeed.js <some URL>');
    phantom.exit(1);
} else {
    t = Date.now();
    address = system.args[1];
    page.open(address, function (status) {
        if (status !== 'success') {
            console.log('FAIL to load the address');
        } else {
            t = Date.now() - t;
            console.log('Page title is ' + page.evaluate(function () {
                return document.title;
            }));
            console.log('Loading time ' + t + ' msec');
        }
        phantom.exit();
    });
}

I get the following output:

Page title is Yahoo
Loading time 700 msec

However, when I try running a simple nightmare:

node --harmony hello_nightmare.js

with hello_nightmare.js containing this:

var Nightmare = require('nightmare');

var google = new Nightmare()
  .goto('http://google.com')
  .wait()
  .run(function(err, nightmare) {
    if (err) return console.log(err);
    console.log('Done!');
  });

I get no output whatsoever; it feels like I just pressed 'Enter' on the command line.

I also tried the example on the nightmare github site:

npm install nightmare vo
node --harmony hello_nightmare_main.js

with hello_nightmare_main.js containing this:

var Nightmare = require('nightmare');
var vo = require('vo');

vo(function* () {
  var nightmare = Nightmare({ show: true });
  var link = yield nightmare
    .goto('http://yahoo.com')
    .type('input[title="Search"]', 'github nightmare')
    .click('.searchsubmit')
    .wait('.ac-21th')
    .evaluate(function () {
      return document.getElementsByClassName('ac-21th')[0].href;
    });
  yield nightmare.end();
  return link;
})(function (err, result) {
  if (err) return console.log(err);
  console.log(result);
});

And it still doesn't work.

How do I fix this nightmare?

Jarrod answered 14/1, 2016 at 16:44 Comment(1)
As of Oct 2017 it seems to be the best to use chromeless instead (same API / runs headless-chrome)Amianthus
T
11

Your issue is most likely described by https://github.com/segmentio/nightmare/issues/224

Nightmare uses Electron which requires an X display; since your server doesn't have a display, you can use Xvfb to provide a virtual one. Install xvfb, and run

xvfb-run node --harmony hello_nightmare.js

Thevenot answered 14/1, 2016 at 18:21 Comment(8)
Thanks. I just ran this apt-get install -y xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libgconf2-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev gcc-multilib g++-multilib, and then tried the xvfb command but I didn't get any output either.Jarrod
I've found that for debugging this, you can run node_modules/nightmare/node_modules/electron-prebuilt/dist/electron --help to see whether Electron is starting at all (you should get a help message; if it prints nothing then it's failing, most likely for lack of display). Do you get any output running that command with xvfb-run?Thevenot
Ok. So I ran that apt-get again, this time removing the -y switch and doing everything manually because I noticed some errors relating to using -y. After that I ran xvfb again and it seems to be working. I'll create a new droplet and try it again (using -y --force-yes when installing xvfb) to make sure I have everything worked out. Cheers.Jarrod
I believe of those packages, only xvfb is really necessary (apt-get will pull in all of its dependencies). You might still get some warnings about missing fonts but xvfb will run just fine. Best of luck.Thevenot
And here I was thinking why it wasn't running in my raspberry pi yesterday on arch linux. I thought it was just because electron didn't support ARM yet but it did downloaded the binaries successfully. Thanks for clarifying this @Thevenot and Obinwanne Hill for bringing this up.Calycle
@Yoz Thanks. I'm trying this out right now on a new droplet. I'll post the full bash script when everything checks out. Shriek, I'm intrigued about your use case, care to share?Jarrod
@ObinwanneHill, I was in a similar boat as you were (no gui or X display on OS which electron needed) but I was also running on rasberry pi 2 on ARM7 chip rather than traditional x86 or x64 which most of the apps run on. ARM is new so many apps doesn't yet have support for it which led me to believe that electron doesn't yet have support for ARM however I didn't know electron needed X display to work so maybe that was the issue instead of ARM that I had previous thought. I'll check tonight and update on my finding.Calycle
Okay, back with my findings. It works although I had to fiddle around electron to make it work (lot of libs missing). Also, you can actually install some window manager to see what's going on your gui-less machine. I'll just leave this link here which I followed.Calycle
J
9

I'm just posting this for posterity.

Below is the bash script to install nightmarejs with node (4.2.4) on a clean Ubuntu Linux machine. I've tested this on a DigitalOcean droplet running 14.04.

apt-get -y update
apt-get -y upgrade
apt-get -y --force-yes install make unzip g++ libssl-dev git xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libgconf2-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev gcc-multilib g++-multilib
mkdir src
cd src
wget https://nodejs.org/dist/v4.2.4/node-v4.2.4.tar.gz
tar xzf node-v4.2.4.tar.gz
cd node-v4.2.4
./configure
make -j2
make install
cd ..
mkdir nightmarejs
cd nightmarejs
npm -f init
npm install --save nightmare vo

Then you simply create the .js file (e.g. hello_nightmare.js) (in the same directory where nightmarejs is installed) and then run it using the command below (as already mentioned in @yoz's answer):

xvfb-run node --harmony hello_nightmare.js

I hope this helps.

Jarrod answered 14/1, 2016 at 19:49 Comment(4)
Thank you, i have got my nightmare script working, but do you know how to get my node project run with PM2 ?Mudslinging
@Mudslinging It's a little bit more involved when using PM2. Essentially, you want to create a simple API (using something like ExpressJS for routing) with routes linked to functions in your JS file, then create a function that does something with Nightmare, then use PM2 to run the node app. This means that your script can now be activated from a HTTP request, instead of command line. I would suggest you explore using HorsemanJS (which is what I prefer these days) instead of Nightmare.Jarrod
Cheers mate, I ended up using node-xvfb. so, just need to xvfb.startSync() before my nightmarejs code, and xvfb.stopSync() after nightmarejs finished.Mudslinging
it seems installing xvfb on its own does not fix the problem - adding the other packages makes it work but adds 1.2 GbExpectant
D
7

Since electron requires X display you need to install all the following packages

sudo apt-get install -y xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libgconf2-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev gcc-multilib g++-multilib

Tested in ubuntu server in aws ec2 and it worked

then run your script:

xvfb-run node --harmony script.js

Disjuncture answered 30/10, 2016 at 2:26 Comment(0)
A
1

Nightmare.js uses the Electron browser and requires an X server. Install xvfb and its dependencies so that you can run graphical applications without display hardware:

sudo apt-get install -y xvfb \
x11-xkb-utils \
xfonts-100dpi \
xfonts-75dpi \
xfonts-scalable \
xfonts-cyrillic \
x11-apps \
clang libdbus-1-dev \
libgtk2.0-dev libnotify-dev \
libgconf2-dev \
libasound2-dev libcap-dev \
libcups2-dev libxtst-dev \
libxss1 libnss3-dev \
gcc-multilib g++-multilib

Create nightmare.js file and add the following:

const Nightmare = require("nightmare");    
const outputFile = "test.png";

const nightmare = Nightmare({ show: true })

nightmare
  .goto('https://duckduckgo.com')
  .type('#search_form_input_homepage', 'github nightmare')
  .click('#search_button_homepage')
  .wait('#r1-0 a.result__a')
  .evaluate(() => document.querySelector('#r1-0 a.result__a').href)
  .end()
  .then(res => {
    console.log(res)
  })
  .catch(error => {
    console.error('Search failed:', error)
  })

run the script:

$ xvfb-run node --harmony nightmare.js
// output: https://github.com/segmentio/nightmare
Attune answered 23/2, 2021 at 4:0 Comment(1)
Thanks! This only works for me, ubuntu 20.04, AWSWallasey

© 2022 - 2024 — McMap. All rights reserved.