CasperJS/PhantomJS doesn't load https page

Question

I know there are certain web pages PhantomJS/CasperJS can't open, and I was wondering if this one was one of them: https://maizepages.umich.edu. CasperJS gives an error: PhantomJS failed to open page status=fail.

I tried ignoring-ssl-errors and changing my user agent but I'm not sure how to determine which ones to use.

All I'm doing right now is the basic casper setup with casper.start(url, function () { ... }) where url=https://maizepages.umich.edu;

Varion

possible duplicate of PhantomJS failing to open HTTPS site – Clausewitz

Answer 1

The problem may be related to the recent discovery of a SSLv3 vulnerability (POODLE). Website owners were forced to remove SSLv3 support from their websites. Since PhantomJS < v1.9.8 uses SSLv3 by default, you should use TLSv1:

casperjs --ssl-protocol=tlsv1 yourScript.js

The catchall solution would be to use any for when newer PhantomJS versions come along with other SSL protocols. But this would make the POODLE vulnerability exploitable on sites which haven't yet disabled SSLv3.

casperjs --ssl-protocol=any yourScript.js

Alternative method: Update to PhantomJS 1.9.8 or higher. Note that updating to PhantomJS 1.9.8 leads to a new bug, which is especially annoying for CasperJS.

How to verify: Add a resource.error event handler like this at the beginning of your script:

casper.on("resource.error", function(resourceError){
    console.log('Unable to load resource (#' + resourceError.id + 'URL:' + resourceError.url + ')');
    console.log('Error code: ' + resourceError.errorCode + '. Description: ' + resourceError.errorString);
});

If it is indeed a problem with SSLv3 the error will be something like:

Error code: 6. Description: SSL handshake failed

---

As an aside, you also might want to run with the --ignore-ssl-errors=true commandline option, when there is something wrong with the certificate.

Clausewitz

Unfortunately even with TLSv1 I get the same error. Are there other common SSL protocols accepted by CasperJS? – Varion

That is strange since this produces a proper screenshot for me. If you're using an older PhantomJS version than 1.9.7, you should probably update. – Clausewitz

This gives me hope! I checked my installed version using phantomjs --version and it gave me 1.8.2, but when I did brew info phantomjs it said 1.9.7. I'm not sure how to go about finding where the 1.8.2 version is installed. I'll investigate if I can update PhantomJS from the command line. My Casper is 1.0.4. As a side note, do you know how to format the cookies.json file for PhantomJS/CasperJS to use? – Varion

adding --ssl-protocol=tlsv1 worked for me, thank you! – Ictinus

Same problem, switching to tlsv1 resolved it for me. Error I was receiving: "Loading resource failed with status=fail:" – Marquess

It's not working for me somehow ! Arguments passed with phantomjs worked but when passed with casperjs it fails to load with error {"errorCode":2,"errorString":"Connection closed"} as server doesn't support sslv2/3 ! Working : "phantomjs --ssl-protocol=tlsv1 sslCheck.js" NotWorking :"casperjs --ssl-protocol=tlsv1 test LoadTesting.js" – Tetrastichous

@SurenderSinghMalik That is strange. I don't know what could be the issue. You can ask a new question, but don't forget to include all version information when you do. If you have installed CasperJS through NPM, you can check if the commandline option works with the PhantomJS version under node_modules. – Clausewitz

@ArtjomB. It started working now ! Actually there were two ways to execute casper one is from batchbin and other in bin. batchbin one was not passing the argument to phantom ! Changing the path to bin/casperjs helped ! – Tetrastichous

I'm getting 'unknown option --ssl-protocol=tlsv1' error :( – Ciapas

@Ima That can't be. What CasperJS and PhantomJS versions do you have? – Clausewitz

@ArtjomB. Casperjs 1.1.0-beta3 and phantomjs-2.0.0 – Ciapas

@ArtjomB. Here is my command : casperjs test scenarios\article\create-article.js --engine=slimerjs --ssl-protocol=tlsv1 – Ciapas

@Ima Native options must come before the script file not after. – Clausewitz

@ArtjomB. Same result :( test> casperjs --ssl-protocol=any --engine=slimerjs test scenarios\article\create-article.js unknown option --ssl-protocol=any – Ciapas

@Ima Oh, right. Didn't recognise the engine. SlimerJS doesn't support changing this value. It's set to SSLv3 for all time. :( #26531434 – Clausewitz

I am using casper 1.0.4 and phantom 2.0.0, I was able to load the page on phantom but not casper, and casper did not print out the error from this resource load page, the options did no work – Lehr

@FullDecent I have never used CasperJS 1.0.4, but I would think that it doesn't work with PhantomJS 2. Since CasperJS 1.1-beta3 specifically supports only PhantomJS < 1.x. Ask a new question. – Clausewitz

I am using phantom 1.9.8 but still need to add --ssl-protocol=any to solve the https issue – Gypsie