So probably the most important thing to preface this with is that I'm using c9. It's an IDE in the cloud and so that is giving me a lot of trouble when trying to use Chrome or Firefox with Watir, because I can't write a path to the Chrome or Firefox browser. I've also tried every variation of wait methods I could find but none of them work.
def save
require 'watir'
require 'phantomjs'
@browser = Watir::Browser.new :phantomjs
@browser.goto "https://kroger.softcoin.com/programs/kroger/digital_coupons/?origin=DigitalCoupons&banner=Smiths#contentBox"
@browser.div(id: "contentBox").wait_until(&:present?).text
@products = @browser.divs
end
Error
timed out after 30 seconds, waiting for true condition on #"contentBox", :tag_name=>"div"}>
The way I want to fix this problem of not being able to scrape data from the Smiths website is to use a chrome browser, but I get the error "unable to connect to chromedriver 127.0.0.1:9515"
Watir::Browser.new :chrome, headless: true
as a replacement for phantomjs. It is much faster and phantomjs is being deprecated. – Nammu