Geb/Selenium tests hang loading new page

My Selenium tests frequently hang indefinitely on CI, always at an attempt to load a new page in driver.get. I am using PhantomJS 1.9.8.

After several rounds of debugging, I think I've traced the problem to the webpage load not completing in PhantomJS.

Selenium RemoteWebDriver has sent a request to PhantomJS/GhostDriver and it is waiting on a response.

GhostDriver is still accepting requests. I can see what page it's stuck on by hitting curl http://localhost:port/session/:sessionId/url and further, if I re-request the same page from the command line (curl -d '{"url": ...}' http://localhost:port/session/:sessionId/url) then the blocked Selenium request magically resumes from where it left off.

UPDATE: Ghostdriver itself is not hanging -- its default timeout is essentially infinite, though. If I specify a page load timeout, e.g., manage().timeouts().pageLoadTimeout(60, TimeUnit.SECONDS) then when the problem happens in PhantomJS, the GhostDriver will return an error response with a timeout, the test will fail, and the build will proceed.

If I debug PhantomJS itself (--debug=TRUE) then I see this as the last thing in the logs at the point of failure

2015-03-21T21:26:39 [DEBUG] WebPage - updateLoadingProgress: 86 (then nothing until timeout reached)

The stack trace from the Java side at the point of hanging is like this:

"Forwarding get on session fd1ac2c0-ccd4-11e4-a596-a1f7b09caa5d to remote" prio=10 tid=0x0000000001f74800 nid=0x5cc3 runnable [0x00002b87c3039000]
 java.lang.Thread.State: RUNNABLE
     at java.net.SocketInputStream.socketRead0(Native Method)
     at java.net.SocketInputStream.read(SocketInputStream.java:150)
     at java.net.SocketInputStream.read(SocketInputStream.java:121)
     at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
     at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
     at horg.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
     at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
     at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
     at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
     at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
     at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
     at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
     at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
     at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
     at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
     at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
     at org.openqa.selenium.remote.HttpCommandExecutor.fallBackExecute(HttpCommandExecutor.java:316)
     at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:295)
     at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:66)
     at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527)
     at org.openqa.selenium.remote.RemoteWebDriver.get(RemoteWebDriver.java:276)
     at org.openqa.selenium.WebDriver$get.call(Unknown Source)
     at geb.Browser.go(Browser.groovy:371)
     at geb.Browser$go.call(Unknown Source)
     at geb.Page.to(Page.groovy:169)

It feels similar to this but with PhantomJS instead of Firefox:

Firefox WebDriver hangs waiting for page to load

I hope you can find a way to implement this into your code, but this is what worked for me when I was having a similar situation with PhantomJS hanging.

I traced it to be hanging on a driver.get() call, which for me was saying something wasn't going through or the webdriver just wasn't - for some reason - giving the load successful command back to the driver, allowing the script to continue.

So, I added the following:

driver = webdriver.PhantomJS()

# set timeout information
driver.set_page_load_timeout(15)

I've tested this at a time of 5 (seconds) and it just didn't wait long enough and nothing would happen. 15 seconds worked great for me, but that's maybe something you should test.

On top of this, I also created a loop whenever there was an option for the webdriver to timeout, so that the driver.get() could attempt to re-send the .get() command. Implementing a try / except stacked scenario, I was able to approach this:

while finished == 0:
    try:
        driver.get(url3)
        finished = 1
    except:
        sleep(5)

I have seen an except handle as:

except TimeoutException as e:
    #Handle your exception here
    print(e)

but I had no use for this. It might be nice to know how to catch specific exceptions, though.

See this solution for more options for a timeout: Setting timeout on selenium webdriver.PhantomJS

Recommended topics

Hot tags