Filling log in form with zombie in node.js

Question

Evening! I'm trying to log in into a website with zombie.js, but I don't seem to be able to make it work. Oh and the website is in Finnish, but it's not very hard to understand, two text fields and a button. First is for username, second for password and the button is the log in button.

At the moment my log in code is as follows:

var Browser = require("zombie");
browser = new Browser();
browser.visit("https://www.nordnet.fi/mux/login/startFI.html?cmpi=start-loggain",
    function () {
        // Here I check the title of the page I'm on.
        console.log(browser.text("title"));
        // Here I fill the needed information.
        browser.document.getElementById("input1").value ="MYUSERNAME";
        browser.document.getElementById("pContent").value ="MYPASSWORD";
        // And here it fails. I try to submit the form in question.
        browser.document.getElementById("loginForm").submit();
        setTimeout(function () {
            // This is here to check that we've submitted the info and have been
            // redirected to a new website.
            console.log(browser.text("title"));
        }, 2000);
});

Now I know that I maybe should have used zombie's own "fill" method, but I tried that with no luck so I tried something new.

All I get from this is an error:

Y:\IMC\Development\Web\node_modules\zombie\lib\zombie\forms.js:72
  return history._submit(_this.getAttribute("action"), _this.getAttribute(
                 ^
TypeError: Cannot call method '_submit' of undefined

Now if I log that browser.document.getElementById("loginForm") it clearly does find the form, but alas, it doesn't like it for some reason.

I also tried the "conventional" method with zombie, which is using that log in button on the web page and pressing it. The problem is that it's not actually a button, just an image which has a link attached to it, and it's all inside <span>. And I have no idea how I can "click" that button.

It has no ID on it, so I can't use that, then I tried to use the text on it, but because it has umlauts on it I can't get it to work. Escaping the ä with /344 only gave an error:

throw new Error("No BUTTON '" + selector + "'");
        ^
Error: No BUTTON 'Kirjaudu sisään'

So yeah, that didn't work, though I have no idea why it doesn't recognize the escaped umlaut correctly.

This is my first question, the second one is a minor one, but I though why not ask it here too now that I've written this text.

If I get all this to work, can I somehow copy the cookie that this log in gives me, and use that in my YQL for screen scraping? Basically I'm trying to scrape stock market values, but without the log in the values are 15min deferred, which isn't too bad, but I'd like it to be live anyhow.

Nab

See zombie.labnotes.org – Convoluted

Answer 1

After couple of tests using zombie I came to the conclusion that it's still to early to use it for serious testing. Nevertheless, I came up with working example of form submit (using regular .submit() method).

var Browser = require("zombie");
var assert = require("assert");

browser = new Browser()
browser.visit("http://duckduckgo.com/", function () {
    // fill search query field with value "zombie"
    browser.fill('input[name=q]', 'mouse');
    // **how** you find a form element is irrelevant - you can use id, selector, anything you want
    // in this case it was easiest to just use built in forms collection - fire submit on element found
    browser.document.forms[0].submit();
    // wait for new page to be loaded then fire callback function
    browser.wait().then(function() {
        // just dump some debug data to see if we're on the right page
        console.log(browser.dump());
    })
});

As you can see, the clue is to use construct browser.wait().then(...) after submitting the form, otherwise browser object will still refer to the initial page (the one passed as an argument to visit method). Note: history object will contain address of page you submitted your form to even if you don't wait for the page to load - it confused me for a bit, as I was sure that I should already see the new page.

---

Edit: For your site, the zombie seems to be working ok (I could submit the form and get "wrong login or password" alert). There are some JS errors but zombie isn't concerned with them (you should debug those however to see if the script are working ok for regular users). Anyhow, here's the script I used:

var Browser = require("zombie");
var assert = require("assert");

browser = new Browser()
browser.visit("https://www.nordnet.fi/mux/login/startFI.html?cmpi=start-loggain", function () {
    // fill in login field
    browser.fill('#input1', 'zombie');
    // fill in password field
    browser.fill('#pContent', 'commingyourway');
    // submit the form
    browser.document.forms[0].submit();
    // wait for new page to be loaded then fire callback function
    browser.wait().then(function() {
        console.log('Form submitted ok!');
        // the resulting page will be displayed in your default browser
        browser.viewInBrowser();
    })
});

---

As side note: while I was trying to come up with working example I've tried to user following pages (all have failed for different reasons):

• google.com - even though I filled query box with a string and submitted the form I didn't get search results . Reason? Probably google took some measures to prevent automatic tools (such as zombie) to browse through search results.
• bing.com - same as google - after submitting the form I didn't get search results. Reason? Probably same as for google.
• paulirish.com - After filling in the search query box and submitting the form zombie encountered script errors that prevent it from completing the page (something about missing ActiveX from charts script).
• perfectionkills.com - Surprisingly here I've encountered the same problems as with Paul Irish site - page with search results couldn't be loaded due to javascript errors.

Conclusion: It's not so easy to force zombie into doing your work after all... :)

Sthenic

Indeed zombie seems to be rather incomplete. I couldn't get that piece of code to work because of some Hierarchy error which jsdom shoots at me, but I still have no clue why I cannot use the same technique on that my page. – Nab

By "that piece of code" you mean code from my example? Anyhow post a link to your page if it's available to general public and I'll try to roam it with zombie :) – Sthenic

Yes I did mean your code. Here's the page if you're interested: nordnet.fi/mux/login/startFI.html?cmpi=start-loggain It's in Finnish though, but the page isn't too complicated. Username and password and a button to log in :p – Nab

No worries - google translate to the rescue ;) I'll try to zombie your site later on today. – Sthenic

Something fishy is going on here. First of all that browser.wait().then seems to do nothing, it just never logs that "Form submitted" part, and neither does the viewInBrowser(), though the error did say that it's not supported on Windows. When I try to circumvent the problem by just using the normal setTimeout(), it doesn't seem to change the site. On both occasions (before and after setTimeout()) the browser.text("title") gives the same result, and if it would've worked, the later one should've given a different answer. I'll try to play with this some more, but I don't like how this behaves. – Nab

Ah! Apparently I do go to the error page, although I have no clue why because the login information is correct. But thank you very much! This helped me a ton! – Nab