Chromedriver / Capybara Too many open files - socket(2) for "127.0.0.1" port 9518

Question

I'm getting this error (details below) when a capybara selector is malformed or can't find. Why is it happening, and how can I prevent the error?

This is new, I think, since our recent upgrade to the latest driver.

The command that triggers this:

find('.panel.#synopsis .btn', text: /new email/i)

Before such an error sudo lsof -i :9518 returns 6 items, after there seem to be hundreds of these.

Error:

 Failure/Error: TCPSocket.open(conn_addr, conn_port, @local_host, @local_port)
 
 Errno::EMFILE:
   Failed to open TCP connection to 127.0.0.1:9518 (Too many open files - socket(2) for "127.0.0.1" port 9518)
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:960:in `initialize'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:960:in `open'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:960:in `block in connect'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/timeout.rb:78:in `timeout'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:958:in `connect'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:943:in `do_start'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/webmock-3.7.6/lib/webmock/http_lib_adapters/net_http.rb:136:in `start_with_connect_without_finish'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/webmock-3.7.6/lib/webmock/http_lib_adapters/net_http.rb:104:in `request'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/default.rb:129:in `response_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/default.rb:82:in `request'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:64:in `call'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:167:in `execute'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:567:in `execute'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:173:in `window_handles'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:187:in `window_handles'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara/selenium/driver.rb:208:in `window_handles'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara/selenium/driver_specializations/chrome_driver.rb:43:in `reset!'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara/session.rb:130:in `reset!'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara.rb:322:in `block in reset_sessions!'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara.rb:322:in `reverse_each'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara.rb:322:in `reset_sessions!'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/capybara-3.30.0/lib/capybara/rspec.rb:18:in `block (2 levels) in <top (required)>'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:450:in `instance_exec'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:450:in `instance_exec'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:363:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:515:in `block in run_owned_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:514:in `each'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:514:in `run_owned_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:601:in `block in run_example_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:600:in `each'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:600:in `run_example_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:471:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:510:in `run_after_example'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:276:in `block in run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:503:in `block in with_around_and_singleton_context_hooks'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:460:in `block in with_around_example_hooks'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:472:in `block in run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:612:in `block in run_around_example_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:345:in `call'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-rails-3.9.0/lib/rspec/rails/adapters.rb:127:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:450:in `instance_exec'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:450:in `instance_exec'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:381:in `execute_with'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:614:in `block (2 levels) in run_around_example_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:345:in `call'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:615:in `run_around_example_hooks_for'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/hooks.rb:472:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:460:in `with_around_example_hooks'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:503:in `with_around_and_singleton_context_hooks'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example.rb:254:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:633:in `block in run_examples'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:629:in `map'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:629:in `run_examples'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:595:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `block in run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `map'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `block in run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `map'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/example_group.rb:596:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `map'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/configuration.rb:2031:in `with_suite_hooks'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:116:in `block in run_specs'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/reporter.rb:74:in `report'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:115:in `run_specs'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:89:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:71:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/lib/rspec/core/runner.rb:45:in `invoke'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/rspec-core-3.9.1/exe/rspec:4:in `<top (required)>'
 # /Users/myuser/.rbenv/versions/2.7.0/bin/rspec:23:in `load'
 # /Users/myuser/.rbenv/versions/2.7.0/bin/rspec:23:in `<top (required)>'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in `load'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli/exec.rb:28:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli.rb:476:in `exec'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli.rb:30:in `dispatch'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/cli.rb:24:in `start'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/bundler-2.1.2/libexec/bundle:46:in `block in <top (required)>'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/bundler/friendly_errors.rb:123:in `with_friendly_errors'
 # /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/bundler-2.1.2/libexec/bundle:34:in `<top (required)>'
 # /Users/myuser/.rbenv/versions/2.7.0/bin/bundle:23:in `load'
 # /Users/myuser/.rbenv/versions/2.7.0/bin/bundle:23:in `<main>'
 # 
 #   Showing full backtrace because every line was filtered out.
 #   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
 #   RSpec::Configuration#backtrace_inclusion_patterns for more information.
 # ------------------
 # --- Caused by: ---
 # Errno::EMFILE:
 #   Too many open files - socket(2) for "127.0.0.1" port 9518
 #   /Users/myuser/.rbenv/versions/2.7.0/lib/ruby/2.7.0/net/http.rb:960:in `initialize'

Harding

Answer 1

If you're using WebMock in your specs, then add the following to your rails_helper.rb:

WebMock.allow_net_connect!(net_http_connect_on_start: true) or WebMock.disable_net_connect!(net_http_connect_on_start: true)

source: https://github.com/bblimke/webmock/blob/master/README.md#connecting-on-nethttpstart

Topdrawer

This is what worked. I don't underestand why it became neccessary only in ruby 2.7, or why it's not just the default setting. – Styles

Answer 2

I have been getting the same error messages with Capybara feature specs since upgrading to ruby 2.7, and it has nothing to do with open files, per se. When Capybara misses a selector (in your case, probably find('.panel.#synopsis .btn', text: /new email/i)), it throws a socket(2) too many files error.

Edit: I had a Webmock call that seemed to be opening all the files (in other words: yeah, you get too many files open - socket(2) when something can’t be found ... but while Capybara was looking for the thing that couldn’t be found, Webmock was running amok).

Here’s the link: https://github.com/bblimke/webmock/blob/master/README.md#connecting-on-nethttpstart, which is a piggyback off of Pepa’s response below (it’s a reference inside the Capybara 'gotchas').

Noto

look like it's not an answer for this question. if you want to comment on this question there are comment button below the question. – Giblet

I can confirm that this is the correct answer to the question. I have also recently upgraded to Ruby 2.7, and now I am also seeing the same misleading "too many open files" error when Capybara can't find an element: Failed to open TCP connection to 127.0.0.1:9515 (Too many open files - socket(2) for "127.0.0.1" port 9515) This seems like a bug in Capybara that should be fixed – Prizefight

This is explained here github.com/teamcapybara/capybara#gotchas. And also solution. – Ingar

why did this start happening only in ruby 2.7? – Styles

Answer 3

ulimit -Sn 10240 did the trick for me

   -S   Set a soft limit for the given resource.
   -n   The maximum number of open file descriptors. 

Donnie

I added that line to my ~/.zshrc profile and it worked for new terminal windows as well. (In case you're using regular terminal add that line in this file instead: ~/.bash_profile) – Chromate

note: spring needs to be restarted if you use it! – Johnsson

Answer 4

You need to increase limit of file descriptors that are allowed to be open.

The simplest approach to do that is to add a line ulimit -n 65536 to your ~/.bash_profile or ~/.zshrc. That line will set ulimit to 65536 for new sessions.

Rosannarosanne

ulimit -n 65536 => ulimit: setrlimit failed: invalid argument – Donnie

A lower limit works: ulimit -n 24576 (highest possible value reported by sysctl kern.maxfilesperproc on macOS 10.14.6). – Kreis

Answer 5

an alternative solution would be cleaning tmp/cache

a computer restart may help as well

Donnie

Answer 6

In my case, I was missing the id set in the HTML. Adding the id fixed it immediately.

Coronary