How to stream an HTTP file upload without the Content-Length header?

Question

Is it possible to upload a file to an apache php server without including the content-length header ?

I am trying to stream a file that I am creating on the fly as a file upload. When I don't use the content-length header I got the apache "501 Method Not Implemented".

$sock = fsockopen($host,80,$errno, $error);
fwrite($sock, "POST $resource HTTP/1.1\r\n" .
                     "Host: $host\r\n\r\n");
fwrite($sock,fread($readHandle,filesize($file)));

If I include the content-length it works fine.

The server is reading from php://input

Stretch

What happens if the length header is missing? Is there an error message? – Hydantoin

unrelated: sweet john snow gravatar -- asoiaf FTW. – Damselfish

Answer 1

According to the HTTP spec you aren't technically required to specify the Content-Length header. From RFC 2616 14.13:

Applications SHOULD use this field to indicate the transfer-length of the message-body, unless this is prohibited by the rules in section 4.4.

However, this is a pretty standard requirement for most servers, and they'll generally send back an error response if the Content-Length header is missing or incorrectly specified. For all intents and purposes, SHOULD in this case equates to MUST.

The problem is that (especially with keep-alive connections), the server doesn't know when your request message actually ends without the Content-Length header. The other option if you're streaming a request entity body is to send a Transfer-Encoding: chunked header and manually send one chunk of the entity body at a time.

So in summary, if you want to send an entity body with your message but don't want to send a Content-Length header, your only real option is to send a chunked HTTP message. This is basically required if you want to stream that entity body and don't know its length ahead of time.

How to chunk-encode an HTTP entity body for streaming ...

Transfer-Encoding: chunked means that you're encoding the entity body of the HTTP message according to the constraints laid out in RFC2616 Sec3.6.1. This encoding format can be applied to either requests or responses (duh, they're both HTTP messages). This format is extremely useful because it allows you to start sending an HTTP message right away before you know the size of the entity body or even exactly what that entity body is going to be. In fact, this is exactly what PHP does transparently for you when you echo any output without having sent a length header like header('Content-Length: 42').

I'm not going to get into details of chunked encoding -- that's what the HTTP spec is for -- but if you want to stream a request entity body you need to do something like this:

<?php

$sock = fsockopen($host,80,$errno, $error);
$readStream = fopen('/some/file/path.txt', 'r+');

fwrite($sock, "POST /somePath HTTP/1.1\r\n" .
              "Host: www.somehost.com\r\n" .
              "Transfer-Encoding: chunked\r\n\r\n");

while (!feof($readStream)) {
    $chunkData = fread($readStream, $chunkSize);
    $chunkLength = strlen($chunkData);
    $chunkData = dechex($chunkLength) . "\r\n$chunkData\r\n";

    fwrite($sock, $chunkData);
}

fwrite($sock, "0\r\n\r\n");

Damselfish

I would clean up the header code like this: $headers = array(); $headers[] = 'POST /somePath HTTP/1.1'; $headers[] = 'Host: example.com'; $headers[] = 'Transfer-Encoding: chunked'; fwrite($sock, implode(PHP_EOL, $headers) . PHP_EOL); – Okhotsk

@ShaquinTrifonoff Unfortunately, your code won't work in non-windows environments. Using PHP_EOL in this context breaks things because the HTTP protocol specifically requires CRLF line breaks between headers. – Damselfish

@rdlowrey Interesting, I didn't know that. Anyway, my point was putting the headers into an array, then imploding it (it looks nicer that way). :-) – Okhotsk