How to use sbt from behind proxy?

Question

How do I configure sbt to use a proxy?

For example, my build definition needs to connect to GitHub, specifying connection parameters for http.proxy, http.proxyPort, user, and password.

How would I pass in these settings to sbt?

Is there an easy way to switch between proxy/no-proxy settings for when I work from home?

Tomkins

Answer 1

sbt respects the usual environment variables for http proxy settings:

export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=yourserver -Dhttp.proxyPort=8080 -Dhttp.proxyUser=username -Dhttp.proxyPassword=password"

(That's of course, assuming Unix (Linux/OSX etc). On windows you could just set the same environment variable (%JAVA_OPTS%) as usual in the Windows way.)

Then run sbt as usual:

sbt

Switching between proxy/no-proxy should be a matter of setting up a little script that you can 'slurp' in whenever you need it.

Gotchas

• Don't include "http://" in the yourserver value
• Don't include the port in the yourserver value
• You probably also want to include https.proxyHost and https.proxyPort since a lot of stuff works over https
• If your proxy requires authentication, don't even bother trying unless it just uses Basic Authentication as SBT doesn't support anything else. Also always beware clear texting credentials into environment variables! Be sure to remove the commands from your .bash_history using a text editing method that won't create trace files (technically you should shred or srm the entire file). If you are on Windows, don't worry about it, your security is already messed up you can't do any more harm.

Singh

Thanks, this is the correct answer. Turns out my real problem is that my proxy blocks the ports used by GIT, so I have to use the http protocol. – Tomkins

I am trying to do the same thing but it isn't working, any tips on how to debug? I am using Paul's sbt-extra script but it should work just the same. github.com/paulp/sbt-extras – Gaff

What if you mess around with all the options under # jvm options and output control? – Singh

@chriswynnyk How did you configure SBT to use Http protocol? – Darell

+1 However, on Windows 7, from the Command Prompt I had to use set JAVA_OPTS=-Dhttp.proxyHost=yourserver -Dhttp.proxyPort=8080 -Dhttp.proxyUser=username -Dhttp.proxyPassword=password to get this to work. – Witted

Doesn't work for me, I keep getting "Your proxy requires authentication" and it fails to dl deps. I'm behind a legacy NTLM :( – Healion

After I figure it out - i realized that in my case I MUST use the HTTPS java system parameter -Dhttps.proxyHost and -Dhttps.proxyPort! – Hoard

In case you are behind a legacy NTLM, I recommand using CNTLM, it works for me – Kafir

“If you are on Windows, don’t worry about it, your security is already messed up and you can’t do any more harm.” - I lol’d irl, +1. – Grumpy

Answer 2

sbt works in a fairly standard way comparing to the way other JVM-based projects are usually configured.

sbt is in fact two "subsystems" - the launcher and the core. It's usually xsbt.boot.Boot that gets executed before the core starts up with the features we all know (and some even like).

It's therefore a matter of how you execute sbt that says how you could set up a proxy for HTTP, HTTPS and FTP network traffic.

The following is the entire list of the available properties that can be set for any Java application, sbt including, that instruct the Java API to route communication through a proxy:

• http_proxy
• http_proxy_user
• http_proxy_pass
• http.proxyHost
• http.proxyPort
• http.proxyUser
• http.proxyPassword

Replace http above with https and ftp to get the list of the properties for the services.

Some sbt scripts use JAVA_OPTS to set up the proxy settings with -Dhttp.proxyHost and -Dhttp.proxyPort amongst the others (listed above). See Java Networking and Proxies.

Some scripts come with their own way of setting up proxy configuration using the SBT_OPTS property, .sbtopts or (only on Windows) %SBT_HOME%\conf\sbtconfig.txt. You can use them to specifically set sbt to use proxies while the other JVM-based applications are not affected at all.

From the sbt command line tool:

# jvm options and output control
JAVA_OPTS          environment variable, if unset uses "$java_opts"
SBT_OPTS           environment variable, if unset uses "$default_sbt_opts"
.sbtopts           if this file exists in the current directory, it is
                   prepended to the runner args
/etc/sbt/sbtopts   if this file exists, it is prepended to the runner args
-Dkey=val          pass -Dkey=val directly to the java runtime
-J-X               pass option -X directly to the java runtime 
                   (-J is stripped)
-S-X               add -X to sbt's scalacOptions (-S is stripped)

And here comes an excerpt from sbt.bat:

@REM Envioronment:
@REM JAVA_HOME - location of a JDK home dir (mandatory)
@REM SBT_OPTS  - JVM options (optional)
@REM Configuration:
@REM sbtconfig.txt found in the SBT_HOME.

Be careful with sbtconfig.txt that just works on Windows only. When you use cygwin the file is not consulted and you will have to resort to using the other approaches.

I'm using sbt with the following script:

$JAVA_HOME/bin/java $SBT_OPTS -jar /Users/jacek/.ivy2/local/org.scala-sbt/sbt-launch/$SBT_LAUNCHER_VERSION-SNAPSHOT/jars/sbt-launch.jar "$@"

The point of the script is to use the latest version of sbt built from the sources (that's why I'm using /Users/jacek/.ivy2/local/org.scala-sbt/sbt-launch/$SBT_LAUNCHER_VERSION-SNAPSHOT/jars/sbt-launch.jar) with $SBT_OPTS property as a means of passing JVM properties to the JVM sbt uses.

The script above lets me to set proxy on command line on MacOS X as follows:

SBT_OPTS="-Dhttp.proxyHost=proxyhost -Dhttp.proxyPort=9999" sbt

As you can see, there are many approaches to set proxy for sbt that all pretty much boil down to set a proxy for the JVM sbt uses.

Incommunicative

Answer 3

In windows environment simply add following line in the sbt/sbtconfig.txt

-Dhttp.proxyHost=PROXYHOST 
-Dhttp.proxyPort=PROXYPORT 
-Dhttp.proxyUser=USERNAME 
-Dhttp.proxyPassword=XXXX

or the Https equivalent (thanks to comments)

-Dhttps.proxyHost=PROXYHOST 
-Dhttps.proxyPort=PROXYPORT 
-Dhttps.proxyUser=USERNAME 
-Dhttps.proxyPassword=XXXX

Caliban

This worked for me as long as I also added another line in sbtconfig.txt with the https equivalent properties. I used exactly the same values as for the corresponding http properties. Thus: -Dhttps.proxyHost=PROXYHOST -Dhttps.proxyPort=PROXYPORT -Dhttps.proxyUser=USERNAME -Dhttps.proxyPassword=XXXX – Pickaxe

IMPORTANT NOTE: does not work anymore! You have to use the sbt/conf/sbtopts file instead. – Mastership

@antoine-sac where can I find sbt/conf/sbtopts file to set proxy? – Thumbsdown

@Thumbsdown in the sbt installation directory - mine was under C:\Program Files (x86)\sbt\conf\sbtopts - on windows obviously. – Mastership

@antoine-sac it seems to work for me without touching the sbtopts file – Holbrook

@ antoine-sac I am running Windows 10 and sbt 1.2.8 and I actually tried to do it with sbtopts as suggested but and it did not work for me. The solution involving sbtconfig.txt did. – Garrott

Answer 4

I used (this is a unix environment) :

export SBT_OPTS="$SBT_OPTS -Dhttp.proxyHost=myproxy-Dhttp.proxyPort=myport"

This did not work for my setup :

export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=myproxy-Dhttp.proxyPort=myport"

In sbt.sh file :

JAVA_OPTS          environment variable, if unset uses "$java_opts"
SBT_OPTS           environment variable, if unset uses "$default_sbt_opts"

But apparently SBT_OPTS is used instead of JAVA_OPTS

Galata

I cannot find in the Internet where should I put the sbt.sh file in my project! – Copacetic

The SBT_OPTS ENV variable was the only thing that made it work for me on CentOS – Adriaadriaens

Answer 5

For Windows users, enter the following command :

set JAVA_OPTS=-Dhttp.proxySet=true -Dhttp.proxyHost=[Your Proxy server] -Dhttp.proxyPort=8080

Dewhurst

In my case i had to add settings for https as well... code set JAVA_OPTS=-Dhttp.proxySet=true -Dhttp.proxyHost=<Proxy Server> -Dhttp.proxyPort=<Port> -Dhttp.proxyUser=<Username> -Dht tp.proxyPassword=<Password> -Dhttps.proxySet=true -Dhttps.proxyHost=<Proxy Server> -Dhttps.proxyPort=<Port> -Dhttps.proxyUser=<Username> -Dhttp s.proxyPassword=<Password> code – Elbow

Answer 6

To provide one answer that will work for all Windows-users:

Add the following to your sbtconfig.txt (C:\Program Files (x86)\sbt\conf)

-Dhttp.proxyHost=XXXXXXX -Dhttp.proxyPort=YYYY -Dhttp.proxySet=true -Dhttps.proxyHost=XXXXXXX -Dhttps.proxyPort=YYYY -Dhttps.proxySet=true

Replace both XXXXXXX with your proxyHost, and both YYYY with your proxyPort.

If you get the error "Could not find or load main class" you need to set your JAVA_HOME:

set JAVA_HOME=C:\Progra~1\Java\jdkxxxxxx

When on 64-bit windows, use:

Progra~1 = 'Program Files'

Progra~2 = 'Program Files(x86)'

Cur

Answer 7

Add both http and https configuration:

export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=yourserver -Dhttp.proxyPort=8080 -Dhttp.proxyUser=username -Dhttp.proxyPassword=password"

export JAVA_OPTS="$JAVA_OPTS -Dhttps.proxyHost=yourserver -Dhttps.proxyPort=8080 -Dhttps.proxyUser=username -Dhttps.proxyPassword=password"

(https config is must, since many urls referred by the sbt libraries are https)

In fact, I even had an extra setting 'http.proxySet' to 'true' in both configuration entries.

Schizophrenia

Answer 8

When I added the proxy info to the %JAVA_OPTS%, I got an error "-Dhttp.proxyHost=yourserver was unexpected at this time". I put the proxy info in %SBT_OPTS% and it worked.

Bedell

On windows I had the same error but setting SBT_OPTS corrected the error but didn't set the proxy. The solution was to set JAVA_OPTS or SBT_OPTS but without the quotes around the value. – Schnorr

Answer 9

Using

sbt -Dhttp.proxyHost=yourServer-Dhttps.proxyHost=yourServer -Dhttp.proxyPort=yourPort -Dhttps.proxyPort=yourPort

works in Ubuntu 15.10 x86_64 x86_64 GNU/Linux.

Replace yourServer by the proper address without the http:// nor https:// prefixes in Dhttp and Dhttps, respectively. Remember to avoid the quotation marks. No usr/pass included in the code-line, to include that just add -Dhttp.proxyUser=usr -Dhttp.proxyPassword=pass with the same typing criteria. Thanks @Jacek Laskowski!. Cheers

Entozoic

nowhere it's the clean command line to quick copy and paste, just that @mad guy! – Entozoic

Answer 10

I found an item on the FAQ section of Lightbend Activator useful. I am using Activator, which in turn uses SBT, so not sure if this helps users with just SBT, but if you use Activator, like me, and are behind a proxy, follow the instructions in the "Behind A Proxy" section of the FAQ:

https://www.lightbend.com/activator/docs

Just in case the content disappears, here's a copy-paste:

When running activator behind a proxy, some additional configuration is needed. First, open the activator configuration file, found in your user home directory under ~/.activator/activatorconfig.txt. Note that this file may not exist. Add the following lines (one option per line):

-Dhttp.proxyHost=PUT YOUR PROXY HOST HERE
-Dhttp.proxyPort=PUT YOUR PROXY PORT HERE
-Dhttp.nonProxyHosts="localhost|127.0.0.1"
-Dhttps.proxyHost=PUT YOUR HTTPS PROXY HOST HERE
-Dhttps.proxyPort=PUT YOUR HTTPS PROXY PORT HERE
-Dhttps.nonProxyHosts="localhost|127.0.0.1"

Loughlin

Answer 11

SBT use both HTTP/HTTPS/SFTP/SSH and other kind of connections to a repository. so when behind a proxy, these protocols should be available.

In most simple cases on Windows, you just need to pass proxy parameters options to JVM, like:

java -Dhttp.proxyHost=myproxy -Dhttp.proxyPort=8080

That will do.

But if not, there are few things you should be aware of:

1. whether if you are making a HTTPS connection to the repository.
2. whether sever certificates been imported to jre cacerts
3. whether your proxy would replace your server certificates

to solve first, you should pass https proxy parameter to jvm, like:

java -Dhttps.proxyHost=myproxy -Dhttps.proxyPort=8080 -Djavax.net.ssl.trustStore=${TRUST_STORE_PATH}

to solve the second, you should import the ca. there are a lot of tips.

to solve the third, you maybe could considering using a authentication proxy.

to Simplify the config of SBT, it provide sbtconfig.txt and sbtops in the conf directory, look into it.

Reference:
http://www.scala-sbt.org/0.13/docs/Setup-Notes.html
http://www.scala-sbt.org/1.0/docs/Publishing.html

Elenore

what kind of error message do you get when it's a certificate issue ? When I use http properties, I get a clear message saying I'm probably behind a proxy, etc. When I switch to httpS properties, I don't have the message anymore, so I assume it's better... but I get issues as described here : #43465085 – Lilas

Answer 12

On Mac OS X / El Capitan you can set java environment variables:

$launchctl setenv _JAVA_OPTIONS "-Dhttp.proxyHost=192.168.1.54 -Dhttp.proxyPort=9999"

Defenestration

Answer 13

I found that starting IntelliJ IDEA from terminal let me connect and download over the internet. To start from terminal, type in:

$ idea

Thumbsdown

Answer 14

Try providing the proxy details as parameters

sbt compile -Dhttps.proxyHost=localhost -Dhttps.proxyPort=port -Dhttp.proxyHost=localhost -Dhttp.proxyPort=port

If that is not working then try with JAVA_OPTS (non windows)

export JAVA_OPTS = "-Dhttps.proxyHost=localhost -Dhttps.proxyPort=port -Dhttp.proxyHost=localhost -Dhttp.proxyPort=port"

sbt compile

or (windows)

set JAVA_OPTS = "-Dhttps.proxyHost=localhost -Dhttps.proxyPort=port -Dhttp.proxyHost=localhost -Dhttp.proxyPort=port"
sbt compile

if nothing works then set SBT_OPTS

(non windows)

export SBT_OPTS = "-Dhttps.proxyHost=localhost -Dhttps.proxyPort=port -Dhttp.proxyHost=localhost -Dhttp.proxyPort=port"'
sbt compile

or (windows)

set SBT_OPTS = "-Dhttps.proxyHost=localhost -Dhttps.proxyPort=port -Dhttp.proxyHost=localhost -Dhttp.proxyPort=port"
sbt compile

Pinafore

Answer 15

If you are using a Proxy which requires authentication, I have a solution for you :)

As @Faiz explained above, SBT has a very hard time handling proxy requiring authentication. The solution is to bypass this authentication, if you cannot turn off your proxy on demand (corporate proxy for example). To do so, I suggest you use a squid proxy, and configure it with your username and password to access your corporate proxy. See : https://doc.ubuntu-fr.org/squid Then, you can set JAVA_OPTS or SBT_OPTS environment variables so that SBT connects to your own local squid proxy instead of your corporate proxy :

export JAVA_OPTS = "-Dhttps.proxyHost=localhost -Dhttps.proxyPort=3128 -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128"

(just c/c this in your bashrc without modifying anything and it should work fine).

The trick is that Squid Proxy does not require any authentication, and acts as an intermediate between SBT and your other proxy.

If you have troubles in applying this advise, please let me know.

Regards,

Edgar

Beanfeast

Answer 16

For those still landing on this thread trying to find where/how to configure HTTP proxy in IntelliJ, here's how I managed to get it to work for me. I hope this helps!

(Note: specify your network username and password in the corresponding boxes):-

Budgerigar