Is it acceptable for a server to send a HTTP response before the entire request has been received?

Question

Consider a large HTTP request:

POST /upload HTTP/1.1
Content-Type: multipart/form-data
Content-Length: 1048576

...

The client now begins uploading a megabyte of data, which may take a while. However, the server determines that HTTP authorization is needed, so it decides it will respond with HTTP 401 Unauthorized.

MUST the server wait until it has received the entire request (IE, headers + CRLF CRLF + Content-Length bytes) before it can respond?

In practical terms, will such behavior break any browsers? Do browsers continue uploading the file anyway, or will they stop transmitting if they receive a 'premature' response?

More importantly, in this scenario, will they be able to successfully authenticate and begin the upload again (with credentials), or is it unreliable to cut off the upload like this?

Mixedup

So did you find out the answer? – Southeaster

@DonghwanKim: Yes, it is valid for a HTTP server to send a response before the entire request has been received. Unfortunately, no browser will see the early response and stop sending the request, which itself is probably in violation of RFC 2616 § 8.2.2. – Mixedup

Answer 1

Looking at RFC 2616 which defines the protocol, in Section 8.2.2 Monitoring Connections for Error Status Messages, it states

An HTTP/1.1 (or later) client sending a message-body SHOULD monitor the network connection for an error status while it is transmitting the request. If the client sees an error status, it SHOULD immediately cease transmitting the body.

So I would say use you can jump in a send a 401 error. And then looking at 10.4.2 401 Unauthorized

The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field

States that the client can retry with suitable credentials.

I haven't performed any experiments to see how browsers actually performed however.

Bluff

I ended up testing this behavior for a slightly different problem, and the answer isn't pretty. All browsers won't process an early response until after they've finished sending the request. – Mixedup

Okay, so the RFC states the client SHOULD "cease transmitting the body". But the server can't know where the next request begins, so the client "ceasing transmission" would mean disconnecting, because keep-alive requires knowing the end of the request. – Electrodialysis

@Electrodialysis That answer brings up another question. Couldn't the browser truncate and replace by an early request termination? – Lalalalage

It seems that Apache HttpClient for Java doesn't survive the server not reading the whole request body, which is sad :( We've run into this issue, when not using pre-emptive HTTP Basic authentication and server sees already from the headers that the request cannot be allowed and send immediately HTTP 401 back. I know that curl and wget can survive this case. My opinion is, that all clients really should be able to handle this as it's important to make it possible for servers to protect against Denial-of-Service attacks made possible by malicious clients sending big requests. – Gentleness

I'm seeing the same issue. All browsers do not see the actual response from the server. They transmit the entire thing and and see the connection as closed and retry. If the browser is NOT fixed, then the server has to compensate and is impacted in terms of resources in receiving all the payloads sent. If this were done, it would create the potential of DDOS attacks waiting for data to upload before responding. Not good. The browsers need to support the spec. – Estis

Better solution is to perform a quick pre-flight check before attempting to send a significant payload. If the permissions fail, don't send the payload. – Estis

I in turn experienced that Async version of Apache HttpClient for Java does indeed stop sending at early response when request is still been sent over but this version does it even when there is no error status present. – Sorites

Regarding the comments about DoS above, for server operators: Do not ask for permission to throttle or reset the TCP stream from the attacker (i.e. client), this makes no sense. Just do it. Changing the behavior of all legitimate client implementations in the world (early interruption or preflight requests) would achieve exactly nothing against a DoS attacker who will simply.. not do that, however nicely you ask no one in particular on the Internet. – Redmond