php file_get_contents and &

Question

I'm trying to use php's file_get_content('a url');

The thing is if the url has '&' in it, for example

file_get_contents('http://www.google.com/?var1=1&var2=2')

it automatically make a requests to www.google.com/?var1=1&var2=2

How do I prevent that from happening?

Zwickau

Could you show a little more code around the file_get_content() line, I suspect the error may be in a previous line. – Zephaniah

This is happening to me as well!! is this a cronjob on godaddy by any chance? – Leaseholder

same thing happening with me, out of 3 urls only one is producing & so no way to say use curl, I want solution within this function. file_get_contents().. thank you – Aspirin

Answer 1

I agree with the original poster of the question. To be very specific:

http://maps.googleapis.com/maps/api/geocode/json?sensor=false&address=301+E.+Linwood+Avenue++Turlock%2C+CA

This requires the sensor=false variable to be passed, or the query will return BAD results from Google. If I pass this STRING through file_get_contents, it (PHP file_get_contents) replaces the "&" with "&" so Google doesn't like me:

Array
(
    [type] => 2
    [message] => file_get_contents(http://maps.googleapis.com/maps/api/geocode/json?address=301 E. Linwood Avenue  Turlock, CA&amp;sensor=false) [<a href='function.file-get-contents'>function.file-get-contents</a>]: failed to open stream: HTTP request failed! HTTP/1.0 400 Bad Request
)

So this is the solution I came up with, using http_build_query

$myURL = 'http://maps.googleapis.com/maps/api/geocode/json?';   
        $options = array("address"=>$myAddress,"sensor"=>"false");
    $myURL .= http_build_query($options,'','&');

    $myData = file_get_contents($myURL) or die(print_r(error_get_last()));

I also include the code (thanks Marco K.) I found on the PHP website to use a custom function for PHP < 5:

if (!function_exists('http_build_query')) { 
    function http_build_query($data, $prefix='', $sep='', $key='') { 
        $ret = array(); 
        foreach ((array)$data as $k => $v) { 
            if (is_int($k) && $prefix != null) { 
                $k = urlencode($prefix . $k); 
            } 
            if ((!empty($key)) || ($key === 0))  $k = $key.'['.urlencode($k).']'; 
            if (is_array($v) || is_object($v)) { 
                array_push($ret, http_build_query($v, '', $sep, $k)); 
            } else { 
                array_push($ret, $k.'='.urlencode($v)); 
            } 
        } 
        if (empty($sep)) $sep = ini_get('arg_separator.output'); 
        return implode($sep, $ret); 
    }// http_build_query 
}//if

Comprehend

Answer 2

You should try looking at the CURL libraries in PHP that would allow you to do something like:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://mysite.com/file.php?blah=yar&test=blah");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch);

You can then get the results from $data.

Southeaster

Why use cURL (which the user may not have installed on their PHP installation, as it's an extra extension) if file_get_contents should work properly? In this case it just looks like something odd is happening with URL Encoding. – Gyronny

cURL is a fairly common extension... and it is specifically designed for URL requests. File_get_contents is not "designed" for URLs. – Southeaster

I don't agree that file_get_contents was not designed for URLs. Check the manual for the function on www.php.net and you'll see that the first example actually demonstrates reading a file over HTTP. But I agree that it's purpose ends with the most simple GET requests. Anything more complex (POST, manipulating headers, uploading files, etc) is not supported. However, I wouldn't recommend using cURL as it's fairly old extension. Instead, how about looking into Zend_Http which is part of the Zend Framework and is fully OO: framework.zend.com/manual/en/zend.http.client.html – Bentley

@Janci, Yea you could use Zend. I know one of the Zend_Http adapters is Curl, and the other uses straight sockets. I've always been really happy with the curl libraries – Southeaster

Careful! Before PHP 5.6, file_get_contents did not verify the peer when attempting to connect over HTTPS, so it can open your app up to security vulns. – Question

Answer 3

Try replace simple quote ' for double quote ":

file_get_contents("http://www.google.com/?var1=1&var2=2")

it seems odd, but works!

Uveitis

This doesn't work if your query string is set with vars: $var1=1; file_get_contents("google.com/?var1=$var1&var2=2") will cause the string to become html special char encoded. – Bolus

Stay the same for me. Still getting the &amp; – Rumilly

Answer 4

It looks like you in fact have a string with "&amp ;". You probably can't see it because you just show the output in the browser. You should use the browsers source view to make html elements visible.

Crosspurpose

Answer 5

The problem is most likely part of your PHP code. I suspect that what you think is happening is not actually happening.

I suspect that rather than the code you posted, you are probably grabbing part of that URL from a variable, such as:

$data = file_get_contents("http://example.com/?$myvariable");

Echoing that variable to the browser may be hiding the fact that you've inadvertently encoded special characters in a previous step, because in the browser an "&amp;" is going to look like "&".

It's also possible that your file_get_contents() request is working correctly, and what you think is a problem with encoding the & sign may be a problem somewhere else - posting actual source code might help determining the problem.

Saker

Answer 6

Actually, there's nothing wrong with using & as field separator in query string. It is perfectly valid and even recommended by w3c. The problem with using & is that it interacts with character entity references (which has form &xxx;).

Also, see the note in PHP manual for url_encode() function:

Be careful about variables that may match HTML entities. Things like &amp, &copy and &pound are parsed by the browser and the actual entity is used instead of the desired variable name. This is an obvious hassle that the W3C has been telling people about for years. PHP supports changing the argument separator to the W3C-suggested semi-colon through the arg_separator .ini directive. Unfortunately most user agents do not send form data in this semi-colon separated format. A more portable way around this is to use & instead of & as the separator.

If you really don't like the & in your URLs, I'd suggest to check the setting of arg_separator directive in your php.ini file, although I'm not 100% sure it influences the way how file_get_contents modifies the URLs

Bentley

whoever down-voted this, could you please comment on your concerns? – Bentley

The problem with this post is that an ampersand is represented by &amp; in HTML. It is used only in a HTML-encoded string, e.g. in the <a href=" ... "> but never in a URL. – Contort

Answer 7

try this config in php.ini

allow_url_fopen = On

if use https url, must config

extension=php_openssl.dll

Teerell