Should I close cURL or not?

Question

I have a function that calls 3 different APIs using cURL multiple times. Each API's result is passed to the next API called in nested loops, so cURL is currently opened and closed over 500 times.

Should I leave cURL open for the entire function or is it OK to open and close it so many times in one function?

Arnaldo

Pretty vague question without seeing the usage and how the code is being handled. – Adit

I tend to err on the reliability side, and fresh handles seem less problematic because leftover state from a request seems less likely to pollute future requests if you make a fresh handle. With that said, I have a process that regularly runs for weeks, making near a million http requests on the same curl handle. They are very plain http requests to a single api on a single domain. I've experienced no issues. – Bouilli

Possible duplicate of When to use cURLs function curl_close? – Huonghupeh

Be carefull on using this for changing data by curl (post, put etc), for it can reuse old data in the background, see my answer: https://mcmap.net/q/396103/-reusing-the-same-curl-handle-big-performance-increase – Inpatient

Does this answer your question? Reusing the same curl handle. Big performance increase? – Inpatient

Answer 1

There's a performance increase to reusing the same handle. See: Reusing the same curl handle. Big performance increase?

If you don't need the requests to be synchronous, consider using the curl_multi_* functions (e.g. curl_multi_init, curl_multi_exec, etc.) which also provide a big performance boost.

UPDATE:

I tried benching curl with using a new handle for each request and using the same handle with the following code:

ob_start(); //Trying to avoid setting as many curl options as possible
$start_time = microtime(true);
for ($i = 0; $i < 100; ++$i) {
    $rand = rand();
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, "http://www.google.com/?rand=" . $rand);
    curl_exec($ch);
    curl_close($ch);
}
$end_time = microtime(true);
ob_end_clean();
echo 'Curl without handle reuse: ' . ($end_time - $start_time) . '<br>';

ob_start(); //Trying to avoid setting as many curl options as possible
$start_time = microtime(true);
$ch = curl_init();
for ($i = 0; $i < 100; ++$i) {
    $rand = rand();
    curl_setopt($ch, CURLOPT_URL, "http://www.google.com/?rand=" . $rand);
    curl_exec($ch);
}
curl_close($ch);
$end_time = microtime(true);
ob_end_clean();
echo 'Curl with handle reuse: ' . ($end_time - $start_time) . '<br>';

and got the following results:

Curl without handle reuse: 8.5690529346466
Curl with handle reuse: 5.3703031539917

So reusing the same handle actually provides a substantial performance increase when connecting to the same server multiple times. I tried connecting to different servers:

$url_arr = array(
    'http://www.google.com/',
    'http://www.bing.com/',
    'http://www.yahoo.com/',
    'http://www.slashdot.org/',
    'http://www.stackoverflow.com/',
    'http://github.com/',
    'http://www.harvard.edu/',
    'http://www.gamefaqs.com/',
    'http://www.mangaupdates.com/',
    'http://www.cnn.com/'
);
ob_start(); //Trying to avoid setting as many curl options as possible
$start_time = microtime(true);
foreach ($url_arr as $url) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_exec($ch);
    curl_close($ch);
}
$end_time = microtime(true);
ob_end_clean();
echo 'Curl without handle reuse: ' . ($end_time - $start_time) . '<br>';

ob_start(); //Trying to avoid setting as many curl options as possible
$start_time = microtime(true);
$ch = curl_init();
foreach ($url_arr as $url) {
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_exec($ch);
}
curl_close($ch);
$end_time = microtime(true);
ob_end_clean();
echo 'Curl with handle reuse: ' . ($end_time - $start_time) . '<br>';

And got the following result:

Curl without handle reuse: 3.7672290802002
Curl with handle reuse: 3.0146431922913

Still quite a substantial performance increase.

Pena

I wonder if curl is using keep-alive connections. That alone could account for most of the performance boost. – Bouilli

I believe cURL uses keep-alive, but each call to curl_exec() initiates a fresh request (because options may have changed, etc.) Especially when connecting to a different server, this would have to be the case. – Pena

Thanks everyone. While I am connecting to the same server but different url, Im amazed by the benchmarks Otome posted. However I really like the reliability point that Chris Posted..... – Arnaldo

While I realize this is an old post, one thing to keep in mind with the different-servers benchmark is DNS looking has to occur in the first execution's timeframe, poisoning the results. If you run a simple duplication of the test in the same file (just copy and paste a couple of times) you'll notice the performance ends up being roughly on par. – Allaallah

i love this answere - and it is on par with my assumption, namely that somebody once upon a time wrote a curl tutorial where they closed the handle after each operation and this, over time, led to so many copy-paste tutorials keeping to this strange "convention". KUDOS – Erubescence