HTTP 407 Proxy Authentication Required while accessing Amazon S3

Question

I have tried everything but I cant seem to fix this issue that is happening for only one client behind a corporate proxy/firewall. Our Silverlight application connects to Amazon S3 for downloading/Uploading some documents. On one client and one client only it returns a 407 error and after that the application fails to save anything.

Inner Exception:
 System.ServiceModel.ProtocolException: [UnexpectedHttpResponseCode]
Arguments: 407,Proxy Authentication Required

We had something similar at a different client but there was more of a CORS issue. to resolve this I used cloud-front to fake a sub-domain that then accesses the S3 bucket and it solved the issue. I was hoping it would fix it with this client as well but it didnt.

I have tried adding this code to web.config as suggested by a lot of answers

 <system.net>
      <defaultProxy useDefaultCredentials="true" >
      </defaultProxy>
   </system.net>

I have read articles about passing a proxy headers with basis authentication using username and password but I am not sure how this would help us. The Proxy server is used by client and any authentication it requires is outside our domain.

**Additional Information**

The Silverlight code references 2 services. One is our wcf service that retrieves all the data for the application. One is The Amazon S3 service that uses the amazon Soap api, the endpoint for which is at http://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl?

If I go into our app and only use part of the system that dont make any calls to the Amazon S3 api the application works fine. As soon as I go to a part of the system that makes a call to the S3, the problem starts. funny enough the call to S3 goes fine and I can retrieve the doc fine but then any calls to our wcf service return 407.

Any ideas?

**Update 2**

Based on comments from Elliot Nelson I check the stack we were using for making http requests in our application. Turns out we are using client http for both http and https requests by default. Here is the code we have in the App.xaml constructor

  public App()
        {
            Startup += Application_Startup;
            UnhandledException += Application_UnhandledException;

            InitializeComponent();

            WebRequest.RegisterPrefix("http://", WebRequestCreator.ClientHttp);
            WebRequest.RegisterPrefix("https://", WebRequestCreator.ClientHttp);

        }

Now, to understand the differences between clienthttp and browserhttp and when to use them. Also, the potential impacts/issues of switching to browserhttp.

**Update 3**

Is there a way to request browsers to run your in-browser Silverlight application in trusted mode and would it help bypass this issue?

Retract

Do you have logging turned on for your S3 bucket? Do requests make it from this client to S3, or are they dying on the proxy? – Muckraker

Answer 1

(Answer #2)

So, most likely (for corporate environments like this network), almost nothing can be done without whatever custom proxy settings are set in IE, usually pushed by corporate policy. To take advantage of these proxy settings, you want to use WebRequestCreator.BrowserHttp, which automatically uses the browser's default settings when making requests.

There's a table of the differences between these two clients available in the Microsoft docs. I'm guessing you were using something (maybe setting custom headers or reading the raw response body) that wasn't supported in BrowserHttp.

For security reasons, you can't "ask" the browser what its proxy settings are and use them, so this is a tricky situation. You can specify Browser vs Client handling by domain, or even for a specific request (the same page above describes how); you may be able in this case to get away with just using ClientHttp for your service calls and BrowserHttp for your S3 calls, and avoid the problem altogether!

For next steps, I'd try that approach; if it doesn't work, I'd try switching wholesale to BrowserHttp just to see if it bypasses the proxy issue (there's almost no chance the application will actually work, since you're probably using ClientHttp-only options).

Long term, you may want to consider making changes to your services so they are usable by a BrowserHttp-only application (this would require you to be pretty basic in your requests/responses, but using only BrowserHttp would be a guarantee you'd work in pretty much any corp network).

Reflexion

I am 99% sure that the issue is with the Client's network as it is only happening with them currently. We had this with another client but we requested them to white list our domain and that solved our issue. We cant get our domain white listed with this client. Going forward I believe this is going to be a bigger issue as more and more organisations are tightening their network by using third part firewalls and proxies so I am looking for a solution that we can implement on our end. – Retract

Ah that's good info, so today you aren't explicitly handling proxies. Silverlight should automatically use IE proxy settings unless you turned it off in your code by switching to WebRequestCreator.ClientHttp (instead of BrowserHttp). Make sure you haven't done that, just in case. – Reflexion

It would make sense to test if you can hit all of your service endpoints in IE directly, if you can, then in theory your application should be able to as well. – Reflexion

Wow, just realized that we are in fact using WebRequestCreator.ClientHttp as default for app by registering it in constructor. I will update my question based on this as it is more visible and easy to read. – Retract

Great! I updated my answer as well to suggest some next steps (in particular, try using BrowserHttp for your S3 calls and ClientHttp for your service calls). – Reflexion

Tried using the Browser http just for the S3 calls. No luck. – Retract

That's a bummer. Using BrowserHttp for everything should eliminate your proxy woes completely, but might hamstring features of your application (especially socket connections). Long term, perhaps you could modify the app to work, basically, with BrowserHttp only (probably requires server code changes as well), and inform users they must whitelist your app to unblock features requiring ClientHttp. – Reflexion

Answer 2

Running in trusted mode is probably a group policy thing which would require their AD admins to approve / whitelist your app.

I think the underlying issue you are facing is that the proxy requires NTLM authentication and for whatever reason the browser declines to provide your app with that context.

One way to prove that it's an NTLM auth issue is to test with curl - get it to make a req through the proxy, then it should be a bit easier to code to. EG the following curl will get you through 99% of Windows corporate proxies (assuming the proxy is at proxy-host.corp:3128):

C:\> curl.exe -v --proxy proxy-host:3128 --proxy-user : --proxy-ntlm https://www.google.com

NOTE The --proxy-user : tells curl to use the current user session to perform the NTLM challenge.

So if you can get the client to run that, you can at least identify that NTLM works, then it's a just a matter of getting the app to perform the NTLM challenge using the default credentials (which may or may not be provided by the browser session)

Shellback

Answer 3

Since you described this as a silverlight application, I'm going to assume you can't use classic browser-proxy troubleshooting like "move browser to public network" or "try a different browser", to isolate the problem.

You should try to isolate the proxy server, and have the customer use the required proxy-auth.

The application is making request, but it might be intercepted by a transparent proxy, or the result might be coming from what you consider a web server.

In the early days, the 401 error was pretty strictly associated with web-auth, and 407 was for proxy-auth.

Architecturally, the separation is a convenience, a web server can have both web server, proxy, and reverse-proxy behaviors.

What happens is your customer's environment is making a web connection to the destination, but it receives a HTTP 407 status from some host, probably their network, or sometimes the provider. Almost certainly the request is received not forwarded. The HTTP client your application lives in needs to provide the credentials that host requires. Companies have environments that are complex enough where often your customer will say this is the first time they have heard of this (some proxy-auth is also dynamic or destination specific).

Also, in some corporate environments, the operator will allow temporary or permanent white-listing from the proxy-auth service. You should see if they can do this, even temporarily, to confirm there aren't going to be other problems.

In the end, it sounds like your application might not robustly support proxy-auth, or the proxy-auth type they use in their environment.

Trovillion

That is definitely the case but what I am looking here is for what we can do on our end to solve this. White listing has worked in the past but sure there is a way to modify our application to handle this. – Retract

See if you can find out what kind of proxy-auth is used. This could be a config problem, or it could be that you need to extend your app with support for newer types of credentials carried by proxy-auth. – Trovillion

What do you mean by what kind of proxy-auth. Do you mean if the client proxy is NTLM based or Kerberos? or are your referring to something else. – Retract

Yes. When I worked on this for Mozilla, there was basic-auth, and NTLM... but I'm sure that the technology has moved forward... – Trovillion