Download a file using HTTP in WebDriver

Question

I am trying to use the WebDriver to navigate through a https site and download a file using WebDriver. When I did it like this, the file download dialog popped up.

WebDriver driver = new ChromeDriver();
driver.get("http://xxx/file1.txt");

I am wondering is there any way just call a method in WebDriver to download the file using regular https request without simulating the click?

Thanks in advance.

Danica

Answer 1

There is not, at least not any way that works in all browsers. You might be able to configure some browsers (Firefox and Chrome) to download files to a specified location without prompting. However, for something like what you're asking about, you don't need Selenium at all. Any programmatic HTTP client will do. In Java, I'd recommend looking at HttpClient from Apache; in .NET using an HttpWebRequest will get the job done. Note that if the site you're downloading the file from requires authentication, you may need to specify custom headers in your HTTP request.

As a side note, you might want to reevaluate why you think you need to test downloading a file, if it's not as simple as executing an HTTP request outside the browser. This article discusses the issue in great detail, and provides a very well-reasoned argument why testing downloading a file is problematic, and often unnecessary.

Wu

Thanks, but I need to log in first and then get to the download page and crawl the URL. Hit directly to the URL won't work due to the authorization process. – Danica

To get Socratic for a moment, how does the browser download the file using only HTTP? In many cases, the authentication process sets a cookie containing authentication information, which is then passed in the headers of the HTTP request. Fortunately, HTTP client libraries usually allow you a measure of control over the HTTP request headers, so all you need is the cookie information. I'll edit my answer to address the issue, which wasn't mentioned in the original question. – Wu

I'm curious about the downvote to this answer. It's not incorrect on a technical level. Its tone is not inflammatory. It might not be what you want to hear, but it doesn't diminish the usefulness or correctness of the answer. If downvoters would leave comments as to where they think the deficiencies are in the answer, it could be improved. Simply downvoting without any other feedback is not helpful, either to the answerer or to those coming afterwards to find an answer to the same or a similar question. – Wu

Answer 2

Yes you can. You need to setup a custom Chromedriver profile:

profile = Selenium::WebDriver::Chrome::Profile.new
profile['download.prompt_for_download'] = false
profile['download.default_directory'] = download_directory

It will not prompt any dialogs. I had a more detailed answer on how to setup the download directory and validate that the file is of any given size here.

Additional chromedriver switches can be found here: http://peter.sh/experiments/chromium-command-line-switches/

Estrogen