How can I tail a remote file?

Question

I am trying to find a good way to tail a file on a remote host. This is on an internal network of Linux machines. The requirements are:

1. Must be well behaved (no extra process laying around, or continuing output)

2. Cannot require someone's pet Perl module.

3. Can be invoked through Perl.

4. If possible, doesn't require a custom built script or utility on the remote machine (regular linux utilities are fine)

The solutions I have tried are generally of this sort

ssh remotemachine -f <some command>

"some command" has been:

tail -f logfile

Basic tail doesn't work because the remote process continues to write output to the terminal after the local ssh process dies.

$socket = IO:Socket::INET->new(...);
$pid = fork();
if(!$pid)
{
  exec("ssh $host -f '<script which connects to socket and writes>'");
  exit;
}

$client = $socket->accept;
while(<$client>)
{
  print $_;
}

This works better because there is no output to the screen after the local process exits but the remote process doesn't figure out that its socket is down and it lives on indefinitely.

Asthmatic

The code example you posted makes absolutely no sense at all. Can you post the real thing? – Beghtol

What do you mean by "the remote process continues to spew"? When either side of the ssh connection dies, the other should die as well... puzzled – Haplo

Yeah - I've seen ssh sessions die, and whatever was running through them croak too, unless they were running in detached screen sessions or something – Housemother

@Aaron: from a shell try: ssh host -f "tail -f <somefile>" and then give Ctrl-C. On my RedHat machine I continue to get the tail of the file in the terminal and the remote SSH+tail remains very much alive. The use of -t instead of -f fixes this. – Asthmatic

Answer 1

Have you tried

ssh -t remotemachine <some command>

-t option from the ssh man page:

 -t      Force pseudo-tty allocation. This can be used to execute 
         arbitrary screen-based programs on a remote machine, which
         can be very useful, e.g. when implementing menu services.
         Multiple -t options force tty allocation, even if ssh has no local tty.

instead of

 -f      Requests ssh to go to background just before command execution.  
         This is useful if ssh is going to ask for passwords or passphrases, 
         but the user wants it in the background.
         This implies -n.  The recommended way to start X11 programs at a remote
         site is with something like ssh -f host xterm.

Lolland

So much win! I've been looking everywhere for a solution. -t is so elegant, and works with things like top too! Thanks :) – Nicolette

From the ssh man page. "-t Force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine" – Thaumaturge

Answer 2

Some ideas:

• You could mount it over NFS or CIFS, and then use File::Tail.
• You could use one of Perl's SSH modules (there are a number of them), combined with tail -f.

Beghtol

Answer 3

You could try Survlog Its OS X only though.

Administrator

I just tried it and it didn't seem to work at all; not maintained since 2015. Waste of $3. – So

Hi @stephan.com. I made the app survlog. An update is coming soon that won't cost you anymore money. – Administrator

Answer 4

netcat should do it for you.

Chantal

I had a netcat based solution where forked and ran netcat on the remote machine and exec'ed a nc listener on the local machine which worked great but the ssh -t solution by Manni is even better. – Asthmatic

nc doesn't work, it exits for me – Sferics

Answer 5

You can Tail files remotely using bash and rsync. The following script is taken from this tutorial: Tail files remotely using bash and rsync

#!/bin/bash
#Code Snippet from and copyright by sshadmincontrol.com
#You may use this code freely as long as you keep this notice.

PIDHOME=/a_place/to/store/flag/file
FILE=`echo ${0} | sed 's:.*/::'`
RUNFILEFLAG=${PIDHOME}/${FILE}.running

if [ -e $RUNFILEFLAG ]; then
   echo "Already running ${RUNFILEFLAG}"
   exit 1
else
   touch ${RUNFILEFLAG}
fi

hostname=$1 #host name to remotlely access
log_dir=$2  #log directory on the remotehost
log_file=$3 #remote log file name
username=$3 #username to use to access remote host
log_base=$4 #where to save the log locally

ORIGLOG="$log_base/$hostname/${log_file}.orig"
INTERLOG="$log_base/$hostname/${log_file}.inter"
FINALLOG="$log_base/$hostname/${log_file}.log"

rsync -q -e ssh $username@$hostname:$log_dir/$log_file ${ORIGLOG}
grep -Ev ".ico|.jpg|.gif|.png|.css" > ${INTERLOG}  

if [ ! -e $FINALLOG ]; then
   cp  ${INTERLOG} ${FINALLOG}
else
   LINE=`tail -1 ${FINALLOG}`
   grep -F "$LINE" -A 999999999 ${INTERLOG} \
      | grep -Fv "$LINE" >> ${FINALLOG}
fi

rm ${RUNFILEFLAG}
exit 0

Weitman

Guys my team mate is the king of bash and we resolved that issue by using bash script and rsync. it is working in prod like a dream. – Weitman

Answer 6

rsync://[USER@]HOST[:PORT]/SRC... [DEST] | tail [DEST] ?

Albemarle

The number and size of the log files makes rsync-ing them to the local machine impractical. – Asthmatic

rsync can be quite efficient as it only transfers the deltas. Additionally, since it can compress the data, and log files are usually quite compressible, it might work quite well. Yes, rsync does take a while if you have millions of files, but for thousands of large files it works quite well. – Organzine

Answer 7

Someone suggested using nc (netcat). This solution does work but is less ideal than just using ssh -t. The biggest problem is that you have to use nc on both sides of the connection and need to do some port discovery on the local machine to find a suitable port over which to connect. Here is the adaptation of the above code to use netcat:

$pid = fork();
if(!$pid)
{
  exec("ssh $host -f 'tail -f $filename |nc $localhost $port'");
  exit;
}

exec("nc -l -p $port");

Asthmatic

Answer 8

There is File::Tail. Don't know if it helps?

Sarcoma

That doesn't do remote files by itself – Beghtol