Send HTTP request from PHP without waiting for response?

Question

I want to have an HTTP GET request sent from PHP. Example:

http://tracker.example.com?product_number=5230&price=123.52

The idea is to do server-side web-analytics: Instead of sending tracking information from JavaScript to a server, the server sends tracking information directly to another server.

Requirements:

• The request should take as little time as possible, in order to not noticeably delay processing of the PHP page.

• The response from the tracker.example.com does not need to be checked. As examples, some possible responses from tracker.example.com:

  • 200: That's fine, but no need to check that.

  • 404: Bad luck, but - again - no need to check that.

  • 301: Although a redirect would be appropriate, it would delay processing of the PHP page, so don't do that.

  In short: All responses can be discarded.

Ideas for solutions:

• In a now deleted answer, someone suggested calling command line curl from PHP in a shell process. This seems like a good idea, only that I don't know if forking a lot of shell processes under heavy load is a wise thing to do.

• I found php-ga, a package for doing server-side Google Analytics from PHP. On the project's page, it is mentioned: "Can be configured to [...] use non-blocking requests." So far I haven't found the time to investigate what method php-ga uses internally, but this method could be it!

In a nutshell: What is the best solution to do generic server-side tracking/analytics from PHP.

Afternoon

madflow: Because in tracking should not slow down page load time. If the tracker fails, then bad luck - but no need to check that every time. – Afternoon

@Will: Note that I am asking for a method, and not necessarily for a package. For example, the method that php-ga uses for doing non-blocking requests could be of interest also for tracking solutions that are - like in my case - not based on Google Analytics. – Afternoon

@Will: There was one answer that could be definite, but that requires more discussion concerning possible performance issues. Unfortunately it has been deleted. The idea was calling command line curl from PHP. If there was a link, I believe it can be removed. curl is part of the standard UNIX web server tool chain (curllib is accessible from PHP, by the way). In addition, once I find the time, I would like to look into php-ga and investigate the method that is uses for non-blocking requests. But maybe someone on Stack Overflow knows this off the top of his head? – Afternoon

@Will: Then the question would become too specific, already related to one specific way to send a request to the tracker. Only if absolutely necessary, I want fork a shell process and run curl in it. But if I do so, then the method in the linked question could be one part of the solution. – Afternoon

Alright. Please edit the question to add all the relevant details from your comments. Hopefully we won't get anymore link-only answers. – Snowwhite

@Khez If you have found a solution in the meantime, it would be great if you could post it! – Afternoon

@Afternoon wasn't this question closed? Sure, there's a couple of techniques for low latency "pinging" of pages, I'll make a response by the end of the day. – Kirk

@Khez: It was reopened. :-) – Afternoon

@Afternoon Sorry for the long wait, made an answer (not very proud of the content). If you want I might edit and offer a little bit more information (maybe some code examples). I started Friday night to make an answer but ended up giving way to much information (in a not very structured fashion). – Kirk

Possibly useful: segment.io/blog/how-to-make-async-requests-in-php and Asynchronous PHP calls?/ (maybe the second one is even a duplicate? will look more later.) – Sardis

@JeremyBanks The segment.io article looks very interesting - thanks! Note that I don't need a response, though. – Afternoon

@Kirk Thanks. Now it took me long to reply, sorry about that. I highly appreciate your answer. Perhaps one day I will find the time to look into php-ga and how it does non blocking tracking requests. – Afternoon

it uses stream_set_blocking, but that affects reading and writing to the stream not "http-ping-ing" – Kirk

@Khez: Still interesting. I didn't know about stream_set_blocking‌​. Perhaps you add it to your answer? – Afternoon

Answer 1

Unfortunately PHP by definition is blocking. While this holds true for the majority of functions and operations you will normally be handling, the current scenario is different.

The process which I like to call HTTP-Ping, requires that you only touch a specific URI, forcing the specific server to boot-strap it's internal logic. Some functions allow you to achieve something very similar to this HTTP-ping, by not waiting for a response.

Take note that the process of pinging an url, is a two step process:

1. Resolve the DNS
2. Making the request

While making the request should be rather fast once the DNS is resolved and the connection is made, there aren't many ways of making the DNS resolve faster.

Some ways of doing an http-ping are:

1. cURL, by setting CONNECTION_TIMEOUT to a low value
2. fsockopen by closing immediately after writing
3. stream_socket_client (same as fsockopen) and also adding STREAM_CLIENT_ASYNC_CONNECT

While both cURL and fsockopen are both blocking while the DNS is being resolved. I have noticed that fsockopen is significantly faster, even in worst case scenarios.

stream_socket_client on the other hand should fix the problem regarding DNS resolving and should be the optimal solution in this scenario, but I have not managed to get it to work.

One final solution is to start another thread/process that does this for you. Making a system call for this should work, but also forking the current process should do that also. Unfortunately both are not really safe in applications where you can't control the environment on which PHP is running.

System calls are more often than not blocked and pcntl is not enabled by default.

Kirk

Actually, I could avoid the DNS look up by specifying an IP. It just didn't occur to me yet. – Afternoon

@Afternoon wanted to mention it in the post, but near the end I was already wondering what access rights you have on the server. Careful for DNS changes though! – Kirk

I don't fully understand what access rights have to do with specifying an IP (other than that I could specify it also in /etc/hosts), but I do have full access rights (root). In any case, a generic answer obviously suites Stack Overflow best. – Afternoon

Khez I have to say that is a beautiful solution. It may even have potential for very basic asynchrony in php without a plugin – Oracle

@DaMaxContent well, async is already part of the hack spec and I wouldn't be surprised if PHP 7.x will pickup on it... but I'm glad you found the solution "beautiful" :D – Kirk

Hi @Khez, mind to share which solution work great nowadays without using any third party plugins? – Simulcast

Answer 2

I would call tracker.example.com this way:

get_headers('http://tracker.example.com?product_number=5230&price=123.52');

and in the tracker script:

ob_end_clean();
ignore_user_abort(true);
ob_start();
header("Connection: close");
header("Content-Length: " . ob_get_length());
ob_end_flush();
flush();

// from here the response has been sent. you can now wait as long as you want and do some tracking stuff 

sleep(5); //wait 5 seconds
do_some_stuff();
exit;

Aftercare

ignore_user_abort() needs a value (true), otherwise it will only return its current value. – Yongyoni

it works fine on my local wamp server, but not on my server. i think my server needs configuration – Banter

Answer 3

I implemented function for fast GET request to url without waiting for response:

function fast_request($url)
{
    $parts=parse_url($url);
    $fp = fsockopen($parts['host'],isset($parts['port'])?$parts['port']:80,$errno, $errstr, 30);
    $out = "GET ".$parts['path']." HTTP/1.1\r\n";
    $out.= "Host: ".$parts['host']."\r\n";
    $out.= "Content-Length: 0"."\r\n";
    $out.= "Connection: Close\r\n\r\n";

    fwrite($fp, $out);
    fclose($fp);
}

Caravan

if you want to add query parameters to your request change the "GET" line to this: $out = "GET ".$parts['path'] . "?" . $parts['query'] . " HTTP/1.1\r\n"; – Quick

Simple and elegant solution. Thanks – Sommer

Answer 4

We were using fsockopen and fwrite combo, then it up and stopped working one day. Or it was kind of intermittent. After a little research and testing, and if you have fopen wrappers enabled, I ended up using file_get_contents and stream_context_create functions with a timeout that is set to 100th of second. The timeout parameter can receive floating values (https://www.php.net/manual/en/context.http.php). I wrapped it in a try...catch block so it would fail silently. It works beautifully for our purposes. You can do logging stuff in the catch if needed. The timeout is the key if you don't want the function to block runtime.

function fetchWithoutResponseURL( $url )
{

    $context = stream_context_create([
        "http" => [
            "method"=>"GET",
            "timeout" => .01
            ]
        ]
    );

    try {
        file_get_contents($url, 0, $context);
    }catch( Exception $e ){
        // Fail silently
    }
}

Odonnell

If SSL handshake and other connection preparation steps are not completed in 10ms this would fail to initiate a connection. Actually, all solutions that depends on connection timeout would fail if you are targeting a piece of code in the target script and it fails to run in this limited time span. To make sure the target script runs fully, you have to wait until the connection closes. – Danzig

Answer 5

For those of you working with wordrpess as a backend - it is as simple as:

wp_remote_get( $url, array(blocking=>false) );

Cerebritis

Thanks, that's a really good tip, and exactly what I was looking for! Submitting an email for sending via Autopilot via CURL was taking anywhere from 2 to 12 seconds, but using this method got the time down to around 2 seconds consistently. Of course, this means that we can't see the results of the POST request – but I was never doing anything with the response from Autopilot anyway, so that doesn't matter in my case. – Regulator

Answer 6

You can use shell_exec, and command line curl.

For an example, see this question

Jackelynjackeroo

This looks like a decent solution. Only I wonder whether it could be problematic to spawn lots of shell processes when many users are accessing the page. – Afternoon

Can you flesh this answer out a bit? (see edits made to question) – Baird

Answer 7

Came here whilst researching a similar problem. If you have a database connection handy, one other possibility is to quickly stuff the request details into a table, and then have a seperate cron-based process that periodically scans that table for new records to process, and makes the tracking request, freeing up your web application from having to make the HTTP request itself.

Ronnieronny

Actually, in the app in question there is something like that. It's not Cron based, however (I try to avoid Cron). It's a daemon that downloads from URLs read from a database, in an infinite loop. For the given use case, though, utilizing that daemon and the associated database feels like overkill. – Afternoon

Answer 8

You can actually do this using CURL directly.

I have both implemented it using a very short timeout (CURLOPT_TIMEOUT_MS) and/or using curl_multi_exec.

Be advised: eventually i quit this method because not every request was correctly made. This could have been caused by my own server though i haven't been able to rule out the option of curl failing.

Gnarl

Isn't it likely that the requests failed because in fact a connection has to be established before sending the request? AFAICS, for the problem at hand, it is unwise to set CURLOPT_TIMEOUT_MS to an extremely low value. – Afternoon

Answer 9

I needed to do something similar, just ping a url and discard all responses. I used the proc_open command which lets you end the process right away using proc_close. I'm assuming you have lynx installed on your server:

<?php    
function ping($url) {
      $proc = proc_open("lynx $url",[],$pipes);
      proc_close($proc);
    }
?>

Privative

It waits until the command is fully executed – Danzig

Answer 10

<?php
// Create a stream
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"Accept-language: en" 
  )
);

 $context = stream_context_create($opts);

// Open the file using the HTTP headers set above
$file = file_get_contents('http://tracker.example.com?product_number=5230&price=123.52', false, $context);
?>

Herrah

Perhaps offer an explanation for how your code works and why it fulfills the needs of the questioner? – Kantian

I'm pretty sure this will do the whole request and wait for a response before proceeding won't it ? – Fantoccini