Verify LFTP transfer success

Question

I am using lftp to transfer files from local to a remote server which only allows SFTP access. I am using the following syntax to transfer the files :

lftp -c "open -u $UN,$Pass sftp://$Server ; mirror -R $Directory"

And this is working all fine when it comes to transffering file. Since I am using this as a cron embedded in a .sh file and sometime the lftp fails (for unknown reason).

How can I verify the transfer using some exit code or something which can be used to ensure the correctness of tranfer?

Banderilla

Anyone? Any idea? This transfer is still failing intermittently without anything in the logs. – Banderilla

Answer 1

You'll have to wrap it inside a shellscript, I think.

The shellscript must generate an lftp script like, for instance:

set xfer:log true
set xfer:log-file "/path/to/somewhere/$DATESTAMP.log"
lcd "/path/of/source"
open -u $UN,$Pass sftp://$Server
mirror -R $Directory
quit

Execute by lftp -f $ScriptName.

Afterwards, parse the generated logfile; AFAIK it contains only successful transfers.

awk '{print $3,$6}' $DATESTAMP.log | sed -r -e 's/ [0-9]+-/ /'

The above will generate a list of files and bytes transferred for each file. You can loop over the result and compare with actual file size, for example

# WARNING: Script assumes that filename contains no space.
while read fname fsize; do
  # The following line used if the lftp runs from e.g. the source folder
  actualname=$Prefix/$fname
  actualsize=$(stat -c "%s" "$actualname")
  if (( actualsize != fsize )); then
    # Some error handling code here
  fi
done <<< "$(awk '{print $3,$6}' $DATESTAMP.log | sed -r -e 's/ [0-9]+-/ /')"

Note: I am using a "herestring" there instead of piping, to enable manipulation of variables within the while loop.

Etienne

Answer 2

It's because cron doesn't use the same Environment as the user it runs the script as. All those things with $ don't exist in the cron environment. When you run the .sh file to test it, it uses the shell environment to fill in those things. When cron runs it, it needs each setting taken from env to be created in the shell script.

Hetti