POST request to PHP7 with chunked encoding does not properly return result
Asked Answered
U

2

27

I'm sending a POST request from a client (tested with curl and custom nodejs script) and don't get the response properly back. The whole thing works fine with PHP 5.6.

Environment

The whole thing is reduced as much as possible:

  • everything running inside Vagrant VM Ubuntu 14.04 LTS
  • nginx 1.9.7 from http://nginx.org/packages/ubuntu/
  • PHP7 FPM compiled from official sources with --disable-all --enable-fpm

The minimal nginx site config I'm using:

server {
  listen 80;
  server_name localhost;
  location / {
    fastcgi_param REQUEST_METHOD $request_method;
    fastcgi_pass  unix:/var/run/php/php7.0-fpm-api.sock;
    fastcgi_param SCRIPT_FILENAME /vagrant/index.php;
  }
}

Example PHP script from /vagrant/index.php:

<?php
echo str_repeat('.', 512);
flush(); // not necessary, only due testing

curl call I'm using: curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''

NodeJS script I'm using:

'use strict';

var http = require('http');
var url = require('url');

var uri = url.parse(process.env.URL);
var options = {
  method: 'POST', protocol: uri.protocol, hostname: uri.hostname,
  port: uri.port, path: uri.path,
};
var data = '';

var httpRequest = http.request(options, function(res) {
    res.on('data', function(chunk) {
      console.log('received data', chunk.length);
      data += chunk;
    });
    res.on('end', function() { console.log('final size', data.length); });
  })
  .on('error', function(err) { console.log(err); });

httpRequest.write('');
httpRequest.end();

Sending my test requests to PHP 5.6

$ curl http://localhost/
..........[cut off]

$ curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''
..........[cut off]

$ URL=http://localhost/ node php7test.js
received data 512
final size 512

Sending my test requests to PHP 7.0

$ curl http://localhost/
..........[cut off]

$ URL=http://localhost/ node php7test.js
final size 0

$ curl -XPOST http://localhost/ -H "Transfer-Encoding: chunked" -d ''
curl: (18) transfer closed with outstanding read data remaining

Why am I mucking around with chunked encoding?

There's no business reason to do so, however I was using a very similar NodeJS code which defaults to chunked encoding which suddenly stopped working when switching to PHP7.

I've found the following to work from the nodejs side: explicitly setting a Content-Length header removes the implicit Transfer-Encoding: chunked header sent by NodeJS and thus works with both PHP versions.

However I'd like to understand why PHP7 behaves differently here and whether I'm in error or what's really going on here.

Update 1:

  • I compared the sapi/fpm/ sources between 5.6 and 7.0 and there's almost no difference I could spot except changes due PHP internal changes
  • The built-in server (php -S) is not affected, all tests

Update 2:

I bisected the PHP sources and was able to pinpoint when the behavior changed:

In between, output from git bisect, commits I couldn't compile:

$ git bisect skip
There are only 'skip'ped commits left to test.
The first bad commit could be any of:
ba5ecf355fe792a5a2a8e6582d5e081d02b16fbf
e383cb4493031a7cd952cfcaed3297e583149c07
fef18f4bea1980a59a9283c2197bd090aaf500cb
18cf4e0a8a574034f60f4d123407c173e57e54ec
We cannot bisect more!

Having a feeling this could be a bug, I wrote this to internals, maybe they have some insights: https://marc.info/?l=php-internals&m=145090900217798&w=2

Ubiquitous answered 23/12, 2015 at 16:46 Comment(5)
Are you using the latest version of curl? Have you tried with Apache? I ran it with PHP 7.0.0 (although 7.0.1 just came out), curl (7.43.0) and Apache (2.4.16). No issues.Pivotal
I don't have apache and no experience, so no, no apache. I bissected the git sources and could pinpoint when the behaviour changed. Will update the post shortly. I did not bother another curl version because the NodeJS scripts behaviour changed too, leading me to the conclusion it's not a client side issue per se.Ubiquitous
Have you filed a bug on bugs.php.net yet? If not, I'd do that now. This definitely seems to be a bug, and I didn't get any further than you did digging through the Git history. First, make sure it's still broken in the latest commits.Runck
I sent to the mailing list but since I didn't get a reply yet there (bad idea during holidays I guess) I will ultimately do that later this week. Thanks for your time and encouragement!Ubiquitous
This bug is still here on PHP7.3 FPM versions: #54444720Adventurous
U
0

It is/was a bug in PHP7 which was fixed very recently with https://github.com/php/php-src/pull/1745 . It is not yet within any official release but eventually will end up there some time.

Before the above PR, a bug was also reported describing a similar issue: https://bugs.php.net/bug.php?id=71466

Ubiquitous answered 5/2, 2016 at 18:10 Comment(0)
F
2

From your post I'm guessing that you're using PHP7.0.0. If my guess is (bool) TRUE then I'll suggest you migrate to PHP7.0.1.

PHP7.0.0 has approximately 27 bugs that were squashed in PHP7.0.1; among other fixed items.
Source: PHP.net changelog.


I also looked at your PHP code above (with my google glasses) but it's ridiculously simple. I doubt that anything can be wrong with it. Although I'm guessing it has to do with the way PHP7 handles flush() and outputs.


Furthermore:

Noting your updates (specifically Update 1, and Update 2); I must really commend you there! THAT'S VERY IMPRESSIVE. Be sure to check out this fixed bug #61751. If I was right about your PHP version, then this fixed bug might have solved your issue; you'll just need to migrate to PHP7.0.1.



NOTE: I know that I should check out the internals of the fixed bug but ...

Footstep answered 4/1, 2016 at 23:16 Comment(0)
U
0

It is/was a bug in PHP7 which was fixed very recently with https://github.com/php/php-src/pull/1745 . It is not yet within any official release but eventually will end up there some time.

Before the above PR, a bug was also reported describing a similar issue: https://bugs.php.net/bug.php?id=71466

Ubiquitous answered 5/2, 2016 at 18:10 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.