Why our server receives CONNECT method requests from our android app while we don't use it in our code?

Question

Some users complained about network issues. Our android app communicates to our server through https.

Our Apache logs showed responses with the status; "405 Method not allowed (CONNECT)", this problem was only reproduced on specific IP addresses.

I don't understand why the android app is trying to reach the server with a CONNECT method, I never use this method in the app, I use only GET, POST or PUT.

It seems a proxy may be involved in that problem, but I have no idea how to solve it. Does anyone know ?

Kerrin

If you use IIS : support.microsoft.com/en-us/kb/216493 Some more info : checkupdown.com/status/E405.html – Kamala

maybe you are being hacked :P – Beaut

I don't use IIS, I mentionned Apache, and the version is 2.4 – Kerrin

How can you be sure that these requests is sent from your android app? – Adamson

how many users complained about this network issue? are IP addresses served by the same ISP? are you using Volley Network Library (by Google) or standard Android API network calls or some sort of custom network library? are you totally sure that network errors comes from your Android App? – Diaphaneity

Those CONNECT immediatly follows a standard request from the android app. – Kerrin

Only a very small amount of users are complaining, and it doesn't occurs every time. For now I use Spring rest template under the hood. I'm pretty sure they come from the android app because they immediatly follows a standard request from the same device. – Kerrin

Also the first standard request and the second request (with CONNECT method) share the same TCP flux. – Kerrin

did you be able to reproduce the connect using the Android App? Have you got a firewall and/or a reverse proxy in front of your production server? – Diaphaneity

I didn't manage to reproduce by myself. I can mitigate the problem by returning Connection:close header on each response, but it's bad. I got a firewall but no reverse proxy, it's a load balancer. – Kerrin

Maybe the server is being port scanned? What is the standard request that follows this connect request? Is it possible that this "standard request" be so generic that any other could be doing it? Something like: "connect and then get index.html". That's something the google does regularly to scan internet... – Stephine

It's not generic at all, there is a specific base64 header computed from the request parameters and a secret key. – Kerrin

If you have a new question, please ask it by clicking the Ask Question button. Include a link to this question if it helps provide context. – Lauzon

I am not sure if this is possible but could the device be issuing the first call over a local network behind a proxy, when that fails the device switches to use the phones cellular network? – Garry

Hey what's happened with the answers, I see only 2 answers but yesterday there was more ?? @Diaphaneity did you delete your answer ? – Kerrin

The CONNECT request happens between the client and the https proxy server as a way to tell the proxy server the destination desired by the client as the later data(the https request and response including the HOST header) will be encrypted, the final destination server doesn't need the CONNECT method, check this. you have to inspect the client who made this CONNECT request in a deeper way to know why did he do it. – Achorn

Answer 1

Looking at the wiki for http connect

A variation of HTTP tunneling when behind an HTTP Proxy Server is to use the "CONNECT" HTTP method.[1][2]

Not to point out the obvious, but enabling the connect method in Apache seams like the answer. Easier than asking the customers to remove their proxy servers.

Garry

Answer 2

You need to provide support for the CONNECT HTTP method, which is commonly used to tunnel SSL requests through proxy servers. Thats why you are receiving them, some users are behind a proxy.

In Apache, to enable handling CONNECT requests, mod_proxy and mod_proxy_connect have to be present in the server.

The problem is that you need to secure your server, which will probably defeat the purpose of you app.

DON'T enable mod_proxy and mod_proxy_connect if you have not secured your server.

Heracles

I know that option but the server team told me they don't want to enable the mod_proxy, because of unknow side effects. – Kerrin

Sad because without it Apache wont be able to handle the CONNECT request. You should consider using http instead of https to prevent the CONNECT request and encrypt the data youself or change to a hosting provider that allows you to enable what you need. – Melano

but this isn't a solution of the problem! if a piece of code was designed to handle GET and POST method against a server. If log puts in evidence that some specific IP uses a CONNECT method there are problem on user-side or into the network architecture (firewall/reverse-proxy). Enabling apache module to handle a specific uses-case/malfunctions isn't a proper solution. – Diaphaneity

@Diaphaneity This is not an issue with the users side. It is a standard networking practice for a proxy server sending a request on behalf of the client behind it. – Garry

He is not running a https proxy, why do you need him to turn his Apache into a proxy server ? CONNECT method needs to be enabled for https proxies servers, not for the final destination server (the website server ). 1) client ----connect---->proxy-----tcp connection--->server , 2) client <--------->proxy<-------->server check this source on how https proxy servers work – Achorn

Answer 3

I cannot provide a turn-key solution, as half the battle here is in the source tree for your supported Android application, as well as a combination of variables pertaining to your employer's infrastructure policies. You have three big questions to answer, and should conceptually do this for every problem brought to you as the Apache administrator:

1. When were the "problem" clients last able to connect without issue?
2. What changed between then and now, and when did it change?
3. Do CONNECT messages correlate 1:1 with clients reporting errors?

Questions one and two are priority, but should not be discussed in depth on a public forum like this. Changes made to your public or private configurations, applications, etc., are often considered the intellectual property of your employer. Use caution if you discuss that here or anywhere. If you find that changes were made, even "harmless" changes, discover their correlation to the customer issue and implement regression testing where applicable.

Question number three is what I will discuss. Based on the messages I've read above, it is not confirmed that CONNECT correlates to every customer issue. It seems as though some customers reported issues, and you looked at logs for symptoms of a problem. The CONNECT errors look like a problem, and based on some of the Android app spec you've shared, they might be the problem. However, they might also be "log noise" generated by someone scanning your server for vulnerable modules.

If you have not yet proven the correlation of CONNECT to customer error, try using the <If> directive and logging additional data about clients who issue CONNECT statements. As a generic example:

<If "%{REQUEST_METHOD} == CONNECT">
    ... some extra log format fields to get ALL of the data ...
    ... maybe a special log file just for CONNECTers?
</If>

Use the gathered data to understand a trend. It might be that only specific versions of Android with your app are behaving this way. You can branch <If> to change the way those users receive content, or you can work with the developer of your Android app (the current one, or the next one you hire ;) ) to develop a list of web server requirements based on the app, itself.

Better still, a well-constructed block can enable you capture debug data for specific clients without disrupting those whose apps work. As always, I recommend building and testing in a lab first; never deploy brand new ideas to production, and most certainly never enable modules because the Internet told you to, even if they were right in naming the module.

Here are links to Apache's documentation for the <If> directive:

http://httpd.apache.org/docs/2.4/mod/core.html#if
http://httpd.apache.org/docs/2.4/expr.html

Good luck!

Marks