webkit_server hangs periodically when run from Capybara in Ruby

Question

I am having a problem where an instance of webkit_server with Capybara and capybara-webkit running headless connected to a local Xvfb screen hangs when visiting a URL. It seems to happen after several minutes of repeatedly visiting different URLs and executing finders. (I'm using capybara for a screen scraping application in vanilla Ruby, not for testing.)

I've confirmed that when it hangs the site is still accessible (for example, through curl or wget on the command line). I've also tried wrapping the Ruby code that invokes the visit and subsequent finders in a Timeout block so that after 60 seconds of waiting a new URL is visited, but any visit() attempt fails after the first time this occurs. The only way to fix the problem is to kill both the Ruby process invoking Capybara/capybara-webkit and the webkit_server process and restart.

When I strace the webkit_server process, I see output like this repeatedly:

clock_gettime(CLOCK_MONOTONIC, {5821, 680279627}) = 0
gettimeofday({1330890176, 712033}, {0, 33052112}) = 0
gettimeofday({1330890176, 712087}, {0, 140736435864256}) = 0
gettimeofday({1330890176, 712137}, {0, 33108640}) = 0
clock_gettime(CLOCK_MONOTONIC, {5821, 680486036}) = 0
clock_gettime(CLOCK_MONOTONIC, {5821, 680530091}) = 0
read(7, 0x1fac1b4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)

And if I strace the Ruby process that invokes it, it is hung on a read():

Process 3331 attached - interrupt to quit
read(5, 
^C <unfinished ...>
Process 3331 detached

I know that the Ruby code hangs on the Capybara visit() method.

Any ideas on what I can do to troubleshoot or correct this is appreciated. I'm assuming the problem has something to do with some resource webkit_server needs to visit the URL but am not sure what to try next.

Thanks!

Invective

Also, for what it's worth, I've tried using the webkit_debug Capybara driver and turned up Capybara's own logging. The last lines from debug output are: >> Visit >> 1 >> 109 >> http://[the site URL] I've tried doing using xwd to screen capture what is happening at this point: xwd -root -display :1 | xwdtopnm | pnmtopng > screenshot.png But I only get a black rectangle. – Invective

Answer 1

The capybara-webkit 1.0 randomly hangs on entire suite for me as well.

Simply adding 'thin' was not enough. But explicit use of Thin handler worked. Added to env.rb:

Capybara.server do |app, port|
  require 'rack/handler/thin'
  Rack::Handler::Thin.run(app, :Port => port)
end

Note Thin usage warning: https://github.com/thoughtbot/capybara-webkit/issues/399#issuecomment-22328028

Exasperation

If I could give you a 1000 upvotes I'd do it. After hours of trying out everything, this made it. You can also disable the annoying Thin logging at start by adding: Thin::Logging.silent = true – Alum

Answer 2

I've found that calling Capybara.reset_sessions! after an error seems to correct this problem. I'm not sure why, but I haven't had the problem since.

Invective

can you please clarify where you add the reset_sessions? Is it in an after hook? How do you condition its invocation on the occurrence of an error? – Hying

Answer 3

I've seen this problem when Capybara is using Webrick to run the server. Add "gem 'thin'" to your Gemfile & it will automatically use that instead. See if that helps?

Izzy

Thanks. Unfortunately I have to use webkit_server, so I don't think thin would help in this particular case. It may help with other servers though. – Invective

Maybe I'm wrong, but I thought webkit_server was just a headless browser. Capybara is still running Thin/Webrick/whatever to give the browser something to talk to, right? – Izzy

I found that I had to explicitly add thin handler, as Grimmo did. – Steenbok