Problem
In my selenium grid setup, my session requests (that are waiting for empty slots to become available) timeout before hitting the newSessionWaitTimeout because they are hitting the RemoteWebDriver command timeout.
Details
I sometimes have more tests trying to run than available slots on my nodes. This is supposed to be fine as the tests trying to get a new session on an available slot get queued up until a slot is free. I can see this happening correctly in my grid console (example message below).
3 requests waiting for a slot to be free. Capabilities [{browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]Capabilities [{browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]Capabilities [{browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]
However, these requests timeout after the webdriver's command timeout period, effectively ignoring the newSessionWaitTimeout setting. My newSessionWaitTimeout is set to the default -1 to allow indefinite waiting for a slot. The webdriver command timeout defaults to 60 seconds.
OneTimeSetUp: OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL http://{myHubIP}:4444/wd/hub/session timed out after 60 seconds. ----> System.Net.WebException : The operation has timed out Exception doesn't have a stacktrace
I can increase the driver's timeout to some large amount that's greater than what my tests might possibly need to wait for to get a slot, but that will affect every command the driver executes. It seems unreasonable to have to basically remove command timeouts to make sure my tests don't timeout while just waiting for an open slot, especially when there's a specific setting that's supposed to handle the timeout for a waiting session.
So, I feel like I'm somehow missing something...
Testing / Sanity Checks
I tested to make sure my newSessionWaitTimeout was actually working by dropping the value under the 60 second timeout I was seeing (dropped it to 10 seconds). Doing this while running more tests than open slots gave me the expected error after 10 seconds, right on cue.
System.InvalidOperationException : Error forwarding the new session Request timed out waiting for a node to become available. at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse) at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters) at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities) at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(ICommandExecutor commandExecutor, ICapabilities desiredCapabilities) at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(Uri remoteAddress, ICapabilities desiredCapabilities)
So from this I can see the newSessionWaitTimeout works fine if it's limit is actually reached.
Running the same test with the newSessionWaitTimeout either below 0 (indefinite wait) or anything above the driver command timeout (default 60 seconds) never hits the above error because the request times out first. Instead is gives the error I listed in the Overview section above.
Now, if I increase the timeout of the RemoteWebDriver, I can see that this is the timeout that I'm hitting when waiting for my new sessions. For example, if I initialize my RemoteWebDriver with
new RemoteWebDriver(new Uri($"http://{myHubIP}:4444/wd/hub"), capabilities, TimeSpan.FromMinutes(3)); then I get the error:
OneTimeSetUp: OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL http://{myHubIP}:4444/wd/hub/session timed out after 180 seconds. ----> System.Net.WebException : The operation has timed out Exception doesn't have a stacktrace
Again, if I bump that value up enough (could need to be very large if I have lots of tests to run) it solves my waiting sessions problem but now all driver commands run the risk of being hung for a very long time. Any ideas on how to get around this correctly (this seems incorrect)?
I'm using Selenium WebDriver and server standalone 3.0.1
newSessionWaitTimeoutso that it actually behaves like it should. Luckily the command timeout doesn't seem to be used for things like finding elements on a page or whatnot (my tests do timeout appropriately when elements are missing) so I haven't seen any side-effects in the tests. I still wonder how other people address this though because I've never liked this solution. – Maxinemaxiskirt