PHP Mink/Zombie - page visit returns status code 0?

Question

I am supposed to test a login at a server, not under my control, which is HTTPS, using Mink and Zombie driver in PHP. The problem is that I mostly get status code 0 whenever I visit the login page, and thus I cannot work with any of the elements. I've tried using Wireshark to inspect the network traffic - but since it's HTTPS, it's encrypted, and I have no idea what is going on in the Wireshark captures; I can only see some data has been exchanged with the server.

I cannot share the URL to the actual page I'm trying to access, but I found that I can demonstrate it with the public demo of Centreon (more on https://www.centreon.com/en/centreon-demo/ ; apologies to Centreon in advance). So here is an example that replicates this problem (my install is as on nodejs cannot find module 'zombie' with PHP mink):

<?php
$nodeModPath = "/home/USERNAME/.nvm/versions/node/v4.0.0/lib/node_modules";

# composer autoload:
require_once __DIR__ . '/vendor/autoload.php';

$URL = "http://demo.centreon.com/centreon";
$USERNAME = "admin";
$PASSWORD = "centreon";
$LFID = "useralias";
$PFID = "password";
$BFID = "submitLogin";


$zsrv = new \Behat\Mink\Driver\NodeJS\Server\ZombieServer();
$zsrv->setNodeModulesPath($nodeModPath . "/"); # needs to end with a trailing '/'
$driver = new \Behat\Mink\Driver\ZombieDriver( $zsrv );
$session = new \Behat\Mink\Session($driver);

// start the session
$session->start();
$session->setRequestHeader('User-Agent', 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36');

//~ $session->visit($URL);

$statcode = 0;
while ($statcode != 200) {
  $isleep = rand(2, 7); echo "sleeping $isleep sec...\n";
  sleep($isleep);
  $session->visit($URL);
  $statcode = $session->getStatusCode();
  echo "  current URL: " . $session->getCurrentUrl() ."\n";
  echo "  status code: " . $statcode ."\n";
}


$page = $session->getPage();
$el_login = $page->findField($LFID);
$el_password = $page->findField($PFID);
$el_button = $page->find('xpath', '//*[@name="'.$BFID.'"]');//findById($BFID);//findField($BFID);
//~ var_dump($el_button);

$el_login->setValue($USERNAME);
$el_password->setValue($PASSWORD);

echo "  pressing/clicking login button\n";
$el_button->click();

echo "Page URL after click: ". $session->getCurrentUrl() . "\n";
$page = $session->getPage();
?>

Now, when I run this script, I typically get:

$ php test_php_mink.php 
sleeping 4 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 0
sleeping 4 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 0
sleeping 5 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 0
...

Notably, while this is going on, if I refresh the same URL in a browser, it returns the page properly (so status 200).

Very, very rarely, I can actually get this to result with status 200 and full page load in the script; but as soon as I try it once more after that, again I get status 0 - here is a snippet of that (with an additional JavaScript error, which seems site related):

$ php test_php_mink.php 
sleeping 7 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 0
sleeping 4 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 200
  pressing/clicking login button
PHP Fatal error:  Uncaught exception 'Behat\Mink\Exception\DriverException' with message 'Error while processing event 'click': "ReferenceError: Effect is not defined\n    at https://demo.centreon.com/centreon/include/common/javascript/modalbox.js:517:1\n    at Object.exports.runInContext (vm.js:44:17)\n ...

Can anyone explain what could be the reason for this behavior - and is there anything I could do retrieve the page properly in script in a reliable manner?

Sklar

Answer 1

Well, I found something, but I'm really not sure about how things should work - so a more learned answer will be much appreciated.

Anyways, I was experimenting a bit, and eventually I found that $session->getStatusCode() is (most likely) defined in ./vendor/behat/mink-zombie-driver/src/ZombieDriver.php as:

public function getStatusCode()
{
    return (int) $this->server->evalJS('browser.statusCode', 'json');
}

So, basically, it reads the JavaScript property browser.statusCode. Turns out, if we wait long enough for this status code to change, then page load proceeds and in the end I obtain status code 200 (again, this is for the case where a normal browser always reads the page properly, thus confirming network is in order). This is the change I made:

$statcode = 0;
while ($statcode != 200) {
  $isleep = rand(2, 7); echo "sleeping $isleep sec...\n";
  sleep($isleep);
  $session->visit($URL);
  // $session->wait(20000, '(0 === jQuery.active)'); # Uncaught exception 'RuntimeException' with message 'Could not establish connection: Connection refused (111)'
  $session->wait(20000, '(browser.statusCode > 0)'); ### THIS makes things work?!
  $statcode = $session->getStatusCode();
  echo "  current URL: " . $session->getCurrentUrl() ."\n";
  echo "  status code: " . $statcode ."\n";
}

Note:

• I found the wait condition (0 === jQuery.active) on How to make Behat wait for an AJAX call? ; here however it seems to cause 'RuntimeException' with message 'Could not establish connection: Connection refused (111)'
• Even with the (browser.statusCode > 0) wait condition, it failed as in the OP if I just used the random sleep delay i.e. wait($isleep, '(browser.statusCode > 0)'); - that is likely because $isleep is in seconds, while wait() expects milliseconds; script started working after I explicitly set the wait to 20000 (milliseconds = 20 sec; script completes sooner than that though).

Now, the past 10-15 times I've run the script, I get this:

$ php test_php_mink_timeout.php 
sleeping 6 sec...
  current URL: https://demo.centreon.com/centreon/
  status code: 200
  pressing/clicking login button
PHP Fatal error:  Uncaught exception 'Behat\Mink\Exception\DriverException' with message 'Error while processing event 'click': "ReferenceError: Effect is not defined\n    at https://demo.centreon.com/centreon/include/common/javascript/modalbox.js:517:1\n    at Object.exports.runInContext (vm.js:44:17)\n    at window._evaluate (/home/USERNAME/.nvm/versions/node/v4.0.0/lib/node_modules/zombie/lib/document.js:253:75)\n ...

... which makes it seem as if the connection / status code 0 problem is solved with this wait(). However, there is another problem - one of the JavaScript classes that the page uses cannot be found, and the program crashes - and what's worse, there are mink processes hanging in the background, and until I kill them, I cannot run the script again. But I'll post another question for that ... and did so, here: PHP Mink/Zombie - handling hanging processes after a fatal exception?

Sklar