Equivalent of cURL for Ruby?

Question

Is there a cURL library for Ruby?

Eucaine

Answer 1

Use OpenURI and

  open("http://...", :http_basic_authentication=>[user, password])

accessing sites/pages/resources that require HTTP authentication.

Fern

Honestly, if I see that a library uses open-uri internally, I don't use that library. It's a deeply flawed library built on top of a deeply flawed URI parser. It's fine for usage in IRB and that's it. – Demodulate

@Sporkmonger: Well, that's what we have— what do you suggest, raw Net::HTTP? – Fern

Use Net::HTTP or Patron or Curb or any of the other libraries mentioned in the other answers. How this answer got voted up or accepted is beyond me. – Demodulate

Well maybe because open-uri is part of standard library? – Soak

Net::HTTP and open-uri are both notoriously bad in how they do http. – Refugio

What if you need to save a session in cookie you receive in respond to login request and you need to pass this cookie every time you fetch data ? – Shana

Answer 2

Curb and Curl::Multi provide cURL bindings for Ruby.

Guenna

Answer 3

If you like it less low-level, there is also Typhoeus, which is built on top of Curl::Multi.

Brooder

Answer 4

Curb-fu is a wrapper around Curb which in turn uses libcurl. What does Curb-fu offer over Curb? Just a lot of syntactic sugar - but that can be often what you need.

Preconcerted

Answer 5

HTTP clients is a good page to help you make decisions about the various clients.

Venu

Answer 6

You might also have a look at Rest-Client

Orozco

Answer 7

the eat gem is a "replacement" for OpenURI, so you need to install the gem eat in the first place

$ gem install eat

Now you can use it

require 'eat'
eat('http://yahoo.com')                 #=> String
eat('/home/seamus/foo.txt')             #=> String
eat('file:///home/seamus/foo.txt')      #=> String

It uses HTTPClient under the hood. It also has some options:

eat('http://yahoo.com', :timeout => 10)                   # timeout after 10 seconds
eat('http://yahoo.com', :limit => 1024)                   # only read the first 1024 chars
eat('https://yahoo.com', :openssl_verify_mode => 'none')  # don't bother verifying SSL certificate

Lodged

I can't get this to work. `require': cannot load such file -- eat (LoadError) – Elstan

Answer 8

If you know how to write your request as a curl command, there is an online tool that can turn it into ruby (2.0+) code: curl-to-ruby

Currently, it knows the following options: -d/--data, -H/--header, -I/--head, -u/--user, --url, and -X/--request. It is open to contributions.

Downcome

Answer 9

There's also Mechanize, which is a very high-level web scraping client that uses Nokogiri for HTML parsing.

Hydrostat

Answer 10

Here's a little program I wrote to get some files with.

base = "http://media.pragprog.com/titles/ruby3/code/samples/tutthreads_"

for i in 1..50

  url = "#{ base }#{ i }.rb"
  file = "tutthreads_#{i}.rb"

  File.open(file, 'w') do |f|   
    system "curl -o #{f.path} #{url}"
  end

end

I know it could be a little more eloquent but it serves it purpose. Check it out. I just cobbled it together today because I got tired of going to each URL to get the code for the book that was not included in the source download.

Amniocentesis

Answer 11

Adding a more recent answer, HTTPClient is another Ruby library that uses libcurl, supports parallel threads and lots of the curl goodies. I use HTTPClient and Typhoeus for any non-trivial apps.

Anatase

Answer 12

To state the maybe-too-obvious, tick marks execute shell code in Ruby as well. Provided your Ruby code is running in a shell that has curl:

puts `curl http://www.google.com?q=hello`

or

result = `
  curl -X POST https://www.myurl.com/users \
  -d "name=pat" \
  -d "age=21"
` 
puts result

Sound

Answer 13

A nice minimal reproducible example to copy/paste into your rails console:

require 'open-uri'
require 'nokogiri'

url = "https://www.example.com"
html_file = URI.open(url)
doc = Nokogiri::HTML(html_file)
doc.css("h1").text
# => "Example Domain"

Hepplewhite