PHP: read a remote file (ideally using fopen)

Question

I'd like to read a remote text file (ideally using fopen) using PHP. My script works using fopen when I'm using this function on a local file.

I've tried:

$file = fopen ("http://abc.abc.abc", "r");
if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
}

and I got:

Warning: fopen(http://abc.abc.abc): failed to open stream: No connection could be made because the target machine actively refused it. in C:\xampp\htdocs\NMR\nmrTest5.php on line 2 Unable to open remote file.

I've read that phpseclib could be a good option and since I can access to my files using WinSCP (SFTP) or by using Puttyfor I tried this (after copying all the files from phpseclib to my directory) hoping that I could copy locally the file and then read it with fopen (not the best thing for met but I could live with that):

include('Net/SFTP.php');

$sftp = new Net_SFTP('abc.abc.abc');
if (!$sftp->login('username', 'pass')) {
    exit('Login Failed');
} 

and I got:

Notice: No compatible server to client encryption algorithms found in C:\xampp\htdocs\NMR\Net\SSH2.php on line 1561 Login Failed

Interstingly, I got a different message if I was connected to the server (using WinSCP):

Notice: Error reading from socket in C:\xampp\htdocs\NMR\Net\SSH2.php on line 3362

Notice: Connection closed by server in C:\xampp\htdocs\NMR\Net\SSH2.php on line 1471 Login Failed

Any idea on how I could get it to work? Ideally I would use fopen but I'm open to other solution.

Uriniferous

To use fopen or file or file_get_contents for a URL, you need to enable allow_url_fopen in the php.ini or using ini_set. Otherwise as the answer given says, use cURL to grab the contents of the URL – Anarthrous

Have you tried to understand the error messages? The one produced by fopen() is very clear: "No connection could be made because the target machine actively refused it". It is not a problem on your code, the remote computer doesn't accept the connection. Can you open the URL (http://abc.abc.abc) in browser? If you can, then fopen() should also be able to open it. If you cannot then maybe the URL is incorrect and it won't open no matter how you try. – Sisk

@axiac: Yes I tried to understand my error message. And I think you're right, I just can't open the URL in a browser. I posted what I did the give as much info as I could and to show that I tried something on my own. That's also the reason I tried it with phpseclib since I know that I can get to the file using a SFTP protocol (using WinSCP). I'm still investigating the answer given by Tanvir Ahmed but I'm not sure if it will work if I need to use SFTP. – Uriniferous

For phpseclib you're probably not setting the include_path. You can do that by doing something like this: set_include_path(get_include_path() . PATH_SEPARATOR . 'phpseclib'); – Schiro

@neubert: thanks for the comment. I've set the include_path and now when I run my script I get nothing which tells me it works. (I also added Else echo "it works!"; to my code and I get: it works). Now I need to get to my file... I think I'm moving in the right direction. Thanks. – Uriniferous

@Schiro I can now get my file as a string by adding $text = $sftp->get('/path/to/file'); in my else statement. I think I'm downloading the file when I do that (which is not ideal) but I can live with that now. Thanks for your help. – Uriniferous

You can't really read a file without downloading it lol. But if you only want to read a portion of it or something check out phpseclib.sourceforge.net/sftp/examples.html#get . It talks about the parameters that get takes. – Schiro

Answer 1

This is how I ended fixing my problem usng phpseclib as suggested by @neubert in the comments of my question.

I first added the phpseclib folder on my server. Then I used this code in my PHP file to get access to my file on a remote server:

//needed for phpseclib
set_include_path(get_include_path() . PATH_SEPARATOR . 'phpseclib');
include_once('Net/SFTP.php');

//connection to the server
$sftp = new Net_SFTP('abc.abc.abc');
if (!$sftp->login('my_login', 'my_password')) {
  exit('Login Failed');
}

//dump entire file in a string, convert to an array and check number of lines

else {
  $text = $sftp->get('full_path_to_my_file');
}
$myArray = explode("\n", $text);
$nbline = count($myArray);

Uriniferous

This is the basic method for getting a file (downloading it). It should work great for most use cases but it's not ideal if you need to do lots of updates on a remote file due to bandwidth usage and file synchronisation issues. – Dozy

Answer 2

I've just been working through this exact problem myself and couldn't find any good documentation in any one single place for how to accomplish this.

I have just made a logging service that uses Monolog and basically makes a custom stream handler based on the log files that are being written to/created. As such it requires a resource (such as one created by fopen) in order to write the log files to an SFTP server.

I had it working using the ssh2 library like this:

$connection = ssh2_connect($this->host, 22);
ssh2_auth_password($connection, $this->user, $this->password);

$sftp = ssh2_sftp($connection);

//some stuff to do with whether the file already exists or not

$fh=fopen("ssh2.sftp://$sftp".ssh2_sftp_realpath($sftp,".")."/$this->logName/$this->fileName", 'a+');

return new StreamHandler($fh);

Everything was working beautifully until I went to integrate the service into a different project and realised this was only working on my development machine because it has the libssh2 library installed as outlined in this question.

Unfortunately, the production server is not so easy to add libraries to. I therefore found myself looking for a different solution.

I have used phpseclib in other projects but only for basic get(), put() and some nlist() calls.

In order to get this working I had to use a Stream object. Not very well documented but there is a good discussion here.

Based on the info there, plus some digging around in the SFTP class, particularly the get() function, this is how I managed to achieve the same functionality using phpseclib

SFTP\Stream::register();
$sftpFileSystem = new SFTP($this->host);
if (!$sftpFileSystem->login($this->user, $this->password)) {
    throw new Exception("Error logging in to central logging system. Please check the local logs and email for details", 1);
}

$context = [
    'sftp' => [
        'sftp' => $sftpFileSystem
    ],
];

//some stuff to do with whether the file already exists or not
$remote_file = $sftpFileSystem->realpath('test.txt');

$sftpStream = fopen("sftp://.{$remote_file}", 'a+', null, stream_context_create($context));
if (!$sftpStream) {
    exit(1);
}

return new StreamHandler($sftpStream);

note the dot (.) after the sftp:// in the call to fopen(). It wasted me a good half an hour!

Dozy

Thanks for the answer @DazBaldwin. I just posted below what I finally ended up using for my small app. It works for me. I'm not sure I understand everything in your answer but it looks like a more sophisticated version of what I've been using. Thanks for posting your answer! – Uriniferous

Thanks for the feedback. If there's anything in particular that you'd like me to expand on, I'd happily edit the answer to cover it. gotta admit though the SFTP\Stream::register() call is just downright weird =0] – Dozy

Answer 3

I have faced similiar issues with fopen. Curl is useful for these purposes.

Please check with the following basic example function(if the url is https, please uncomment the CURLOPT_SSL_VERIFYPEER = FALSE line).

$url = '***THE URL***';
$result = get_web_page_by_curl($url);

if ($result['errno'] != 0) echo 'error: bad url, timeout, redirect loop ...';
if ($result['http_code'] != 200) echo 'error: no page, no permissions, no service ...';
else {
    $page = $result['content'];
    echo $page;
}

function get_web_page_by_curl($url) {
    $agent = "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)";

    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => $agent,   // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
        //CURLOPT_SSL_VERIFYPEER => FALSE   // this line makes it work under https 
    );

    $ch = curl_init($url);
    curl_setopt_array($ch, $options);
    $content = curl_exec($ch);
    $err     = curl_errno($ch);
    $errmsg  = curl_error($ch);
    $header  = curl_getinfo($ch);
    curl_close($ch);

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

Sublieutenant

Thanks for your help. It doesn't work for me (so far). Is it possible that my server isn't accepting the http or https protocol. How can I test that? I know it accepts SFTP using port 22 (I have to use my username and password to do that). Also if it helps, I'm trying to access a Linux server, not a web server (I think these two things are different but I'm not sure) – Uriniferous

Actually a webserver is configured to run in a linux, unix, windows or other machine. When you hit the url: http : //abc.abc.abc, you are trying to access a webserver using http protocol(which might be in a linux server). Have you checked the debug info of the curl call? Can you please check if curl is enabled in your server? You can do so using phpinfo(). – Sublieutenant

I ran phpinfo() and I saw that curl was enabled. But as others mentionned in the above comments I think one of my problem is that my server is not accepting http protocol. It seems to accept SFTP. Maybe I can make it work somehow with curl using SFTP. I will investigate more. – Uriniferous

Is the url accessible publicly? Can you get the page when you hit the url in browser? If yes, then curl should be enough(you will need to check with the curl options). If not and the content is in a secured place on the Linux server then you can consider ftp or sftp connect. – Sublieutenant

My url is not publicly accessible. I've been able to get my file using phpseclib (by setting the include_path as proposed by @neubert. Now I'm able to download my file. It would be better if I would'nt have to download it (just read it like I use to do with fopen() would be better) but I can live with this now. Thanks for your help. – Uriniferous