How to set a proxy in rubys net/http?

Question

I'm trying to set a proxy and to use it in a simple get request like in the documentation. But I always receive an error! The adress and port are right with open-uri it worked.. it's http://proxy:8080 .

proxy_addr = 'proxy'
proxy_port = 8080

Net::HTTP.new('google.de', nil, proxy_addr, proxy_port).start { |http|
  # always proxy via your.proxy.addr:8080
  Net::HTTP.get('google.de', '')
}

What am I doing wrong? Thanks for all answers!

Multiped

Answer 1

Net::HTTP.new('google.de', nil, proxy_addr, proxy_port).start { |http|

This will create an http object for you to use in the block. Use that rather than generating new ones each time, here Net::HTTP.get('google.de', '')

proxy_addr = 'proxy'
proxy_port = 8080

Net::HTTP.new('google.de', nil, proxy_addr, proxy_port).start { |http|
  # always proxy via your.proxy.addr:8080
  http.get('google.de', '')
}

Oersted

What if you need authentication? – Regicide

Answer 2

There is another option:

Net::HTTP will automatically create a proxy from the http_proxy environment variable if it is present.

So you can use

ENV['http_proxy'] = 'http://172.16.3.160:4226' # your http://address:port here

and Net::HTTP will use it for all requests by default.

It can be helpful for net_http requests in third-party libraries (for example it works for gem gibbon for MailChimp).

Pass nil for the proxy address to disable default http_proxy.

Walston

A little clarification: Net::HTTP will automatically use http_proxy environment variable only when called through Net::HTTP.new. Using Net::HTTP.start will not use http_proxyenvironment variable because it passes nil for the proxy address if not specified (instead of passing :ENV). – Stewardson

@Stewardson Good point. Though, you can still use auto-discovery of proxy in NET::HTTP.start by using NET::HTTP.start(url.hostname, url.port, :p_port => :ENV). – Harlamert

@Harlamert using :p_port => :ENV works, but I do not like its readibility :). Another thing to notice: username and password from http_proxy are not read (like in http://username:password@address:port). – Stewardson

Answer 3

Here is the code that works if you are making a REST api call behind a proxy:

require "uri"
require 'net/http'
    
proxy_host = '<proxy addr>'
proxy_port = '<proxy_port'
proxy_user = '<username>'
proxy_pass = '<password>'
    
uri   = URI.parse("https://saucelabs.com:80/rest/v1/users/<username>")
proxy = Net::HTTP::Proxy(proxy_host,
                         proxy_port,
                         proxy_user,
                         proxy_pass)
    
req = Net::HTTP::Get.new(uri.path)

req.basic_auth(<sauce_username>,
               <sauce_password>)
    
result = proxy.start(uri.host,uri.port) do |http|
  http.request(req)
end
    
puts result.body

Quartic

Answer 4

i tried all above but not working.

my final scripts is like below (work for me):

url = "https://host.com/url"
proxy_url = "http://proxy.com:port"

# split important values
uri = URI(url)
proxy_array = proxy_url.split(":")
proxy_protocol = proxy_array[0]
proxy_addr = proxy_array[1]
proxy_addr = proxy_addr[2 .. proxy_addr.length - 1]
proxy_port = proxy_array[2].to_i

# new objects
proxy = Net::HTTP::Proxy(proxy_addr, proxy_port)
result = proxy.get(uri);

Desireedesiri

Answer 5

#!/usr/bin/env ruby
#

require 'rubygems' 
require 'net/http' 
require 'open-uri' 
require 'timeout' 


mimvp_url = "http://proxy.mimvp.com/exist.php"
mimvp_url2 = "https://proxy.mimvp.com/exist.php"
mimvp_url3 = "https://apps.bdimg.com/libs/jquery-i18n/1.1.1/jquery.i18n.min.js"

$proxy = '183.222.102.95:8080'

$proxy_addr = $proxy.split(':')[0].strip
$proxy_port = $proxy.split(':')[1].strip

puts $proxy_addr
puts $proxy_port


begin 
  Timeout.timeout(30) {

    # mimvp_url = http://proxy.mimvp.com/exist.php
    # uri.host = proxy.mimvp.com
    # uri.port = 80
    # uri.path = /exist.php
    uri = URI.parse(mimvp_url)
    result = Net::HTTP.new(uri.host, nil, $proxy_addr, $proxy_port).start { |http|
      http.get(uri.path)
    }
    puts result.body


    # mimvp_url = http://proxy.mimvp.com/exist.php
    # uri.host = proxy.mimvp.com
    # uri.port = 80
    # uri.path = /exist.php
    # req = #<Net::HTTP::Get:0x007fafa594ff78>
    uri = URI.parse(mimvp_url)
    req = Net::HTTP::Get.new(uri.path)
    result = Net::HTTP::Proxy($proxy_addr, $proxy_port).start(uri.host, uri.port) {|http| 
      http.request(req)
    } 
    puts result.body


    # proxy auth （NO TEST）
    $proxy_addr = '<proxy addr>'
    $proxy_port = '<proxy_port'
    $proxy_user = '<username>'                # 代理用户名
    $proxy_pass = '<password>'                # 代理密码

    $website_username = '<website_username>'  # 目标网站登录用户名
    $website_password = '<website_password>'  # 目标网站登录密码

    uri = URI.parse(mimvp_url)
    req = Net::HTTP::Get.new(uri.path)
    req.basic_auth($website_username, $website_password)

    result = Net::HTTP::Proxy($proxy_addr, $proxy_port, $proxy_user, $proxy_pass).start(uri.host, uri.port) {|http| 
      http.request(req)
    } 
    puts result.body

}
rescue => e  
    puts e.inspect  
    exit  
end

Mireillemireles

More than posting some code, explain what your code does. The SO focus is more than giving the solution, but to teach who is asking for help how things work. – Lynch

you can directly run the code, use $browser->proxy(%proxy); to set proxy ip:port address – Mireillemireles