How to click a "select option" and then evaluate loaded content with casperjs

Question

I'm trying to crawl the sizes for this product:

Link to product

The problem: The sizes are loaded after the color of the product is selected.

In the product page's source code, I can see that the dropdown has a onchange-method: It clicks the form #postColor onchange.

The select dropdown:

<select name="color" id="color" class="cposelect" onchange="document.getElementById('postColor').click();" style="width:150px;margin-right: 20px; float: left;">
    <option selected="selected" onfocus="if (this.storeCurrentControl != null) storeCurrentControl(event, this);" value="0">Select colour</option>
    <option onfocus="if (this.storeCurrentControl != null) storeCurrentControl(event, this);" value="-8027">Light Camel</option>
    <option onfocus="if (this.storeCurrentControl != null) storeCurrentControl(event, this);" value="-9999">black</option>
</select>

The #postColor form, which is clicked onchange:

<input type="submit" name="postColor" value="" onclick="location.href=('./?model=10344-4180&amp;color='+document.forms[0].color.value+'&amp;size='+document.forms[0].size.value+'&amp;addbread=OUTLET&amp;addbread2=DRIZIA&amp;currentimage='+document.getElementById('currentimage').value+'&amp;selectedmi=a1_INDEX_14&amp;prev=10850-4314&amp;next=10413-4183'); return false;" id="postColor" class="cpobutton " style="display: none;">

This is my code so far and it's not working:

casper.start('http://shop.baumundpferdgarten.com/showmodel/?model=10344-4180&addbread=OUTLET&addbread2=DRIZIA&color=0&currentimage=1&selectedmi=a1_INDEX_14', function() {
    this.test.assertExists('select[name="color"] option:nth-child(2)');
    this.click('select[name="color"] option:nth-child(2)');
    this.waitForSelector('select[name="size"] option:nth-child(2)', function() {
        this.test.pass('selector is !');
        var sizes = this.evaluate(function() {
            console.log("======== evaluating ========");
            // var sizes = document.querySelectorAll('#size option');
            return document.querySelectorAll('#size option');
        });
        for (var i = sizes.length - 1; i >= 0; i--) {
            console.log(sizes[i].innerText);
        }
    });
});

I suspect that problem is that a totally new page is loaded when a color is clicked (the sizes are not dynamically appended).

How would you solve this problem?

Stone

Answer 1

This is how I do it

this.evaluate(function() {
    $('#select_element_selector').val('value').change();
});

The change() is very important

I'm assuming that you have jQuery on the page

Gard

This doesn't work for me. It changes the option drop-down visually (and that can be seen with casper.capture), but it doesn't actually trigger the selected value. – Outmarch

Answer 2

Got same issue here. My solution is:

casper.then(function(){
    this.evaluate(function() {
        document.querySelector('select.select-category').selectedIndex = 2; //it is obvious
    });
    this.capture('screenshot.png');
});

Menstruum

I tried with the mouse element but you solution seems to be the better one. – Clamorous

Just try the code get the data that is what i want. – Tussis

Answer 3

The recommended jQuery solution doesn't actually work for me.

casper.evaluate(function() {
    $('#select_element_selector').val('value').change();
});

While, the capture() command shows the select option as selected visually, it doesn't actually trigger the event. Try using this with the waitForText() command for example; the program will timeout.

casper
  .start('http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=DEC_00_SF1_DP1&prodType=table')
  .waitForText('Add/Remove Geographies', function () {
    casper.click('#addRemoveGeo_btn');
  })
  .waitForText('Select a geographic type:', function () {
    casper.evaluate(function () {
      $('#summaryLevel').val('050').change();
    });
  })
  .waitForText('Select a state:', function () {
    casper.capture('test.png');
  })
  .run();

What did work for me, was the code provided below (thanks @ArtjomB). How to fill a select element which is not embedded in a form with CasperJS?

casper.selectOptionByValue = function(selector, valueToMatch){
    this.evaluate(function(selector, valueToMatch){
        var select = document.querySelector(selector),
            found = false;
        Array.prototype.forEach.call(select.children, function(opt, i){
            if (!found && opt.value.indexOf(valueToMatch) !== -1) {
                select.selectedIndex = i;
                found = true;
            }
        });
        // dispatch change event in case there is some kind of validation
        var evt = document.createEvent("UIEvents"); // or "HTMLEvents"
        evt.initUIEvent("change", true, true);
        select.dispatchEvent(evt);
    }, selector, valueToMatch);
};

casper.selectOptionByValue('#summaryLevel', '050');

Although, I think CasperJS should provide native support to select options from a drop-down when they aren't apart of a form (http://docs.casperjs.org/en/latest/modules/index.html). Selenium offers the select and addSelection commands (https://seleniumhq.github.io/selenium/docs/api/javascript/index.html). I've also filed a pending issue ticket on the CasperJS GitHub page to implement this natively (https://github.com/n1k0/casperjs/issues/1390).

Outmarch

Answer 4

this is simple code to enter credit card details in casper js

casper.evaluate(function(CC,security_code) {

      document.querySelector('input#receipt_creditcard_number').value = CC;
      document.querySelector('select#receipt_creditcard_month').selectedIndex = 10;
      document.querySelector('select#receipt_creditcard_year').selectedIndex = 10;
      document.querySelector('input#receipt_creditcard_verification_value').value = security_code;
      document.querySelector('input#receipt_save_creditcard_in_profile').click();
  }, '4242424242424242','123');

Unipersonal

Answer 5

Tricky, but looking at the URL, I think Casper can handle this nicely.

Let me know if you need help with the code for this but it will take a bit longer so to give you a bit of pseudo-code,

• create an empty object of colors and color IDs { color: colorId }
• as you're already kind of doing, find the select menu with [name='color'] and loop through those options. push anything that doesn't have a value of 0 to your object
• write a new function that will go to the url http://shop.baumundpferdgarten.com/showmodel/?model=10344-4180&color=-9999&size=0&addbread=OUTLET&addbread2=DRIZIA&currentimage=1&selectedmi=a1_INDEX_14&prev=10850-4314&next=10413-4183 and substitute the color ID where color=-9999.

example:

'http://shop.baumundpferdgarten.com/showmodel/?model=10344-4180&color=' + object.colorId + '&size=0&addbread=OUTLET&addbread2=DRIZIA&currentimage=1&selectedmi=a1_INDEX_14&prev=10850-4314&next=10413-4183'

• Either add the sizes to the existing colors object or make a new one, or console.log them. The world is yours!

Thimblerig

Answer 6

I don't know if you found a solution to your problem, but here is how I would solve it:

casper.click('#color');
casper.then(function() {
casper.waitFor(function check() {
  return this.evaluate(function() {
    return document.querySelector('select.select-category').selectedIndex === 2; 
  });
}, function then() {
     /* do the rest that you would want to do!*/
   });

}

Candlestand

Answer 7

A slightly hacky way to do this without using jQuery utilising built-in casper methods is:

// Assumes the select box is on the first item at index 0
chooseSelectOption = (friendlyName : string, selectLocator : string, optionIndex : number) => {
    casper.test.assertExists(selectLocator, "then select index " + optionIndex + " in the " + friendlyName + " select box");
    casper.click(selectLocator);
    this.goDown(selectLocator, optionIndex);
};

// recursive funtion to go down various levels
private goDown = (locator: string, depth : number, currentLevel : number = 0) => {
    if (currentLevel >= depth) { return; }
    casper.sendKeys(locator, casper.page.event.key.Down, { keepFocus: true });
    this.goDown(locator, depth, currentLevel + 1);
};

This is in TypeScript but you can edit for vanilla JS if you need to. You need to use a recursive function because a normal for loop gets into difficulties with capser's queuing system.

Drogheda

Answer 8

vanilla javascript solution (triggering onchange method):

casper.evaluate(function() {
    var select_element = document.getElementById('select_element_selector');
    select_element.value = 'value';
    select_element.onchange();
});

Refresh

Answer 9

tejesh95's solution worked for me with a few minor changes, #1 I couldn't get findElementById to work so switched it to 'document.querySelector('#selectorId)' . I also had to change 'onchange' to 'onclick'. Here's the code within the casperjs test harness:

casper.then(function() {
  this.evaluate(function() {
  var select_element = document.querySelector('#selectorId')
  select_element.value = 'stringValue'
  select_element.onclick
})

})

Follow on note: Your mileage may vary on the above, I found inconsistent results elsewhere, the text value would update, but the underlying control wouldn't fire. Frustrating!

Pennsylvanian