PHP file_get_contents() returns "failed to open stream: HTTP request failed!"

Question

I am having problems calling a URL from PHP code. I need to call a service using a query string from my PHP code. If I type the URL into a browser, it works ok, but if I use file-get-contents() to make the call, I get:

Warning: file-get-contents(http://.... ) failed to open stream: HTTP request failed! HTTP/1.1 202 Accepted in ...

The code I am using is:

$query=file_get_contents('http://###.##.##.##/mp/get?mpsrc=http://mybucket.s3.amazonaws.com/11111.mpg&mpaction=convert format=flv');
echo($query);

Like I said - call from the browser and it works fine. Any suggestions?

I have also tried with another URL such as:

$query=file_get_contents('http://www.youtube.com/watch?v=XiFrfeJ8dKM');

This works fine... could it be that the URL I need to call has a second http:// in it?

Varuna

Answer 1

Try using cURL.

<?php

$curl_handle=curl_init();
curl_setopt($curl_handle, CURLOPT_URL,'http://###.##.##.##/mp/get?mpsrc=http://mybucket.s3.amazonaws.com/11111.mpg&mpaction=convert format=flv');
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_USERAGENT, 'Your application name');
$query = curl_exec($curl_handle);
curl_close($curl_handle);

?>

Spindell

This is way too complicated when the real problem lies within ampersands. – Mertiemerton

@Christian, can you elaborate? – Eisteddfod

Not everyone has (but should though) cURL installed. cURL sure is many times faster, but file_get_contents isn't that slow either, and doesn't require that you remember all the options whenever you use it. – Mertiemerton

This CURLOPT_USERAGENT was very important in my case, thanks! – Expiate

I believe my problem was with it timing out and this fixed it. Thank you! – Newborn

is there some cheap and dirty way to download a file using curl ? – Testify

Answer 2

Could this be your problem?

Note: If you're opening a URI with special characters, such as spaces, you need to encode the URI with urlencode().

Smail

I had the same error the OP had, and this was my problem - spaces in the arguments. urlencode() on the GET parameters solved the issue. – Rechaba

If you do not want to use the CURL solution, this one works! Use the URLecnode on the parameters. – Splenius

This is the actual solution to this problem. – Trichinize

Answer 3

<?php

$lurl=get_fcontent("http://ip2.cc/?api=cname&ip=84.228.229.81");
echo"cid:".$lurl[0]."<BR>";


function get_fcontent( $url,  $javascript_loop = 0, $timeout = 5 ) {
    $url = str_replace( "&amp;", "&", urldecode(trim($url)) );

    $cookie = tempnam ("/tmp", "CURLCOOKIE");
    $ch = curl_init();
    curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1" );
    curl_setopt( $ch, CURLOPT_URL, $url );
    curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie );
    curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
    curl_setopt( $ch, CURLOPT_ENCODING, "" );
    curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
    curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
    curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );    # required for https urls
    curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, $timeout );
    curl_setopt( $ch, CURLOPT_TIMEOUT, $timeout );
    curl_setopt( $ch, CURLOPT_MAXREDIRS, 10 );
    $content = curl_exec( $ch );
    $response = curl_getinfo( $ch );
    curl_close ( $ch );

    if ($response['http_code'] == 301 || $response['http_code'] == 302) {
        ini_set("user_agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1");

        if ( $headers = get_headers($response['url']) ) {
            foreach( $headers as $value ) {
                if ( substr( strtolower($value), 0, 9 ) == "location:" )
                    return get_url( trim( substr( $value, 9, strlen($value) ) ) );
            }
        }
    }

    if (    ( preg_match("/>[[:space:]]+window\.location\.replace\('(.*)'\)/i", $content, $value) || preg_match("/>[[:space:]]+window\.location\=\"(.*)\"/i", $content, $value) ) && $javascript_loop < 5) {
        return get_url( $value[1], $javascript_loop+1 );
    } else {
        return array( $content, $response );
    }
}


?>

Quentin

Thanks for the code, looks like you got it from: php.net/manual/en/ref.curl.php The main problem is the function call get_url should actually be get_fcontent since you changed the name of the function itself. This is actually a recursive function call that re-attempts to get the URL contents by changing some parameters. – Epicalyx

you got it! was trying to https and was getting turned down. you nailed it. UPVOTED ;) – Chiropodist

Answer 4

file_get_contents() utilizes the fopen() wrappers, therefore it is restricted from accessing URLs through the allow_url_fopen option within php.ini.

You will either need to alter your php.ini to turn this option on or use an alternative method, namely cURL - by far the most popular and, to be honest, standard way to accomplish what you are trying to do.

Tetanic

Nevermind, just noticed he said file_get_contents() worked on a different URL. Nonetheless, this is still a good tip for others having this issue. – Tetanic

yes, Ive looked at other options and see that cURL is the standard way to go but I tried this as it was easier and worked with other urls, I think i have to restart apache to intsall cURL? and cant figure out how to do this (another question)... thanks for your quick response – Varuna

Answer 5

You basically are required to send some information with the request.

Try this,

$opts = array('http'=>array('header' => "User-Agent:MyAgent/1.0\r\n")); 
//Basically adding headers to the request
$context = stream_context_create($opts);
$html = file_get_contents($url,false,$context);
$html = htmlspecialchars($html);

This worked out for me

Overskirt

This worked for me too! Looks like it needed a user agent – Koester

If the url seems properly encoded, this could very well be the issue. Some site will restrict upon 'User-Agent', perhaps 'Accept' as well. – Versicular

super answer. vote for promote this one to the top. all in php and worked for https request too. just two extra lines code. what if shared server do provide curl? this is very helpful. – Olvera

Answer 6

I notice that your URL has spaces in it. I think that usually is a bad thing. Try encoding the URL with

$my_url = urlencode("my url");

and then calling

file_get_contents($my_url);

and see if you have better luck.

Depressor

Answer 7

I got a similar problem.

Due to timeout !

Timeout can be indicated like this :

$options = array(
    'http' => array(
        'header'  => "Content-type: application/x-www-form-urlencoded\r\n",
        'method'  => "POST",
        'content' => http_build_query($data2),
        'timeout' => 30,
    ),
);
$context = stream_context_create($options);
$retour = @file_get_contents("http://xxxxx.xxx/xxxx", false, $context);

Kinship

What is the content inside $data2 ? – Bibliopole

hi ! $data2 is an array used to pass variables to the distant script. – Kinship

Answer 8

I got a similar problem , I parsed the youtube url. The code is;

$json_is = "http://gdata.youtube.com/feeds/api/videos?q=".$this->video_url."&max-results=1&alt=json";
$video_info = json_decode ( file_get_contents ( $json_is ), true );     
$video_title = is_array ( $video_info ) ? $video_info ['feed'] ['entry'] [0] ['title'] ['$t'] : '';

Then I realise that $this->video_url include the whitespace. I solved that using trim($this->video_url).

Maybe it will help you . Good Luck

Holliehollifield

Answer 9

This function solved my problem

function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
curl_setopt($ch, CURLOPT_URL, $url);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

Sealey

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Este

Thank you, it worked just fine for me.:) – Sheepdog

Answer 10

I'm not sure about the parameters(mpaction, format), if they are specified for the amazonaws page or ##.##.

Try to urlencode() the url.

Appointor

thanks - these are params for the mediaplug instance. If I urlencode the url it still does not work - I get a very garbled url in the error... ?? – Varuna

You should be encoding only the parameter string: "convert format" should be "convert%20format" (or alternatively "convert+format"). – Uppercase

Answer 11

$query=file_get_contents('http://###.##.##.##/mp/get?' . http_build_query(array('mpsrc' => 'http://mybucket.s3.amazonaws.com/11111.mpg&mpaction=convert format=flv')));

Heidt

Answer 12

This wasn't working for me and I was getting a null value for the result query. So I checked on postman to see if the api was actually returning values and it was. Post man has this tab on the right where you can get sample code that was used to get the results and after trying that it finally worked, but initially you might get a weird error 411 saying the post length needs to be specified I added the fix in my code below. To bypass the 411 POST length error create an empty array and use the http_build_query function. Then set that variable in the CURLOPT_POSTFIELDS curl option.

$data_string = array();
$curl = curl_init();
$test = http_build_query($data_string);
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://www.googleapis.com/geolocation/v1/geolocate?key=APIKEY',
CURLOPT_POSTFIELDS => $test,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'POST',
));

$response = curl_exec($curl);

curl_close($curl);
echo $response;

Abner

This worked for me, thanks! I have no idea why this worked but the others not. – Keelby

Answer 13

This is what worked for me... I did not use curl.

I was able to access a particular API url via browser, but when used in file_get_contents, it gave the error " failed to open stream".

Then, I modified the API URL that I wanted to call by encoding all double quotes with urlencoding and kept everything else untouched.

Sample format is given below:

$url = 'https://stackoverflow.com/questions'.urlencode('"'.$variable1.'"');

Then use

file_get_contents($url);

Mutiny

Answer 14

Had same issue but it was a firewall issue... once I had the API server whitelisted, it worked fine, using both get_file_contents($url) and the curl method above...

hours wasted before discovering the firewall rule issue.

Kazimir

Answer 15

For me the problem was an incorrect value in the Content-Length header. The value was too great, so nginx kept waiting for the rest of the content that never arrived.

Ingenious

Answer 16

In my case I had to specify multiple headers:

$headers = [
    'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36', // User-Agent header
    'Accept-Language: en-US,en;q=0.5',
    'Cache-Control: no-cache',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
    'Connection: keep-alive',
    'Accept-Encoding: gzip, deflate, br',
];

$options = [
    'http' => [
        'method' => 'HEAD',
        'timeout' => 3, // Timeout in seconds
        'header' => implode("\r\n", $headers)
    ],
];

set_error_handler("exception_error_handler");
$context = stream_context_create($options);
    
$headers = file_get_contents($url, false, $context);

I recommend to try the URL in like postman or your web browser and check which headers are used and mimic that.

Insouciant

Answer 17

Use this

file_get_contents($my_url,null,null);

Expectorate