Download files from SFTP server that are older than 5 days using Python

Question

I got a Python script on this site that downloads files from the directory from SFTP server. Now I need help to modify this code so that it only downloads the files that older than 5 days from the day the code is used.

Code to download files (based on Python pysftp get_r from Linux works fine on Linux but not on Windows):

import os
import pysftp
from stat import S_IMODE, S_ISDIR, S_ISREG

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None    
sftp=pysftp.Connection('192.168.X.X', username='username',password='password',cnopts=cnopts)

def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir(remotedir):
        remotepath = remotedir + "/" + entry
        localpath = os.path.join(localdir, entry)
        mode = sftp.stat(remotepath).st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath,mode=777)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

remote_path=input("enter the remote_path: ")
local_path=input("enter the local_path: ")

get_r_portable(sftp, remote_path, local_path, preserve_mtime=False)

Please help me to modify the code so that it only download files 5 days older from the present day.

Fare

Answer 1

Use the pysftp.Connection.listdir_attr to get file listing with attributes (including the file timestamp).

Then, iterate the list and pick only the files you want.

import time
from stat import S_ISDIR, S_ISREG

def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir_attr(remotedir):
        remotepath = remotedir + "/" + entry.filename
        localpath = os.path.join(localdir, entry.filename)
        mode = entry.st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            if (time.time() - entry.st_mtime) // (24 * 3600) >= 5:
                sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Though the code can be much simpler, if you do not need a recursive download:

for entry in sftp.listdir_attr(remotedir):
    mode = entry.st_mode
    if S_ISREG(mode) and ((time.time() - entry.st_mtime) // (24 * 3600) >= 5):
       remotepath = remotedir + "/" + entry.filename
       localpath = os.path.join(localdir, entry.filename)
       sftp.get(remotepath, localpath, preserve_mtime=True)

Though note that pysftp seems dead. Consider using Paramiko instead. It has pretty much same API, so the above code will work almost as it is (except that Paramiko does not have the preserve_mtime parameter). See also pysftp vs. Paramiko.

---

Based on:

• Python pysftp get_r from Linux works fine on Linux but not on Windows
  (I have updated this source of your code to use listdir_attr as it's more effective)
• How to sync only the changed files from the remote directory using pysftp?
• Delete files that are older than 7 days

Unveiling