Finding file size and last modified of SFTP oldest file using Java

Question

I'm using JSch to get files from an SFTP server, but I'm trying to figure out a way to only get the oldest file, and to make sure that it is not currently being written to. The way I imagine myself doing this is first finding which file in the specified remote folder is oldest. I would then check the file size, wait x seconds (probably about 10, just to be safe) and then check it again. If the file size has not changed, I download the file and process it. However, I have no idea how to do this! If anybody knows how to do this, or knows of something else that supports SFTP that has this built-in (I know Apache Commons does, but only does FTPS), it would be greatly appreciated.

Thanks in advance.

Ddt

Answer 1

Turns out that this is entirely possible in JSch, the hardest part is simply finding the documentation. Code I used is below, hopefully somebody else will find it helpful! (I'm sure there are optimizations to be made, I know, I know. There are also variables that are defined elsewhere, but hopefully anybody that needs this will be able to figure them out!)

public static String oldestFile() {
    Vector list = null;
    int currentOldestTime;
    int nextTime = 2140000000; //Made very big for future-proofing
    ChannelSftp.LsEntry lsEntry = null;
    SftpATTRS attrs = null;
    String nextName = null;
    try {
        list = Main.chanSftp.ls("*.xml");
        if (list.isEmpty()) {
            fileFound = false;
        }
        else {
            lsEntry = (ChannelSftp.LsEntry) list.firstElement();
            oldestFile = lsEntry.getFilename();
            attrs = lsEntry.getAttrs();
            currentOldestTime = attrs.getMTime();
            for (Object sftpFile : list) {
                lsEntry = (ChannelSftp.LsEntry) sftpFile;
                nextName = lsEntry.getFilename();
                attrs = lsEntry.getAttrs();
                nextTime = attrs.getMTime();
                if (nextTime < currentOldestTime) {
                    oldestFile = nextName;
                    currentOldestTime = nextTime;
                }
            }
            attrs = chanSftp.lstat(Main.oldestFile);
            long size1 = attrs.getSize();
            System.out.println("-Ensuring file is not being written to (waiting 1 minute)");
            Thread.sleep(60000); //Wait a minute to make sure the file size isn't changing
            attrs = chanSftp.lstat(Main.oldestFile);
            long size2 = attrs.getSize();
            if (size1 == size2) {
                System.out.println("-It isn't.");
                fileFound = true;
            }
            else {
                System.out.println("-It is.");
                fileFound = false;
            }
        }
    } catch (Exception ex) {ex.printStackTrace();}
    return Main.oldestFile;
}

Ddt

Just a note: I wrote some JavaDoc for JSch. – Gagliardi

i am not able to get the part through which i can retrieve File size from remote Unix. SftpATTRS is not avaibale in my Jsch jar file – Vain

@Ddt what is Main in this code – Cockade

@Cockade You'd have to ask 13 years ago I'm afraid 😁 No memory of what this was at this point. – Ddt

@Ddt sad i am 13 years late :D – Cockade

Answer 2

I don't have a direct answer to your question, but it sounds like you want to do something similar to reliable file transfer. This is part of a larger project in Grid computing that is now apparently organized here. I don't know if it includes security features or if you can add them on, but it is an open source project.

Merritt

"I don't have a direct answer to your question" goes on to post a direct answer to his question – Dexamethasone

Answer 3

You can easily do this using edtFTPj/PRO, which supports SFTP.

Simply get a directory listing, and sort the listing by date. If the oldest date isn't in the last few minutes, you can download.

Piaffe

I forgot the most important part: This needs to be free! Thanks for the tip, though :) – Ddt

Answer 4

calculate the folder size in remote server just call the ftpFolderSize(ftpFolderSize,client) directory path, and pass the object FTPClient as a parameter. It will return the size of folder.

Works only for FTP.

import org.apache.commons.net.ftp.FTPClient;

import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.io.FileUtils;

long dirSize = 0L; //global variable

private long ftpFolderSize(String directoryPath,FTPClient client){

    try{
        client.changeWorkingDirectory(directoryPath); 
        FTPFile[] ftpFiles = client.listFiles();
        if(client.changeWorkingDirectory(directoryPath))
        {
            for (FTPFile ftpFile : ftpFiles) {
                if(ftpFile.isFile()){
                    dirSize = dirSize+ftpFile.getSize();// file size is calculated
                }
                else if(ftpFile.isDirectory())
                {
                    dirSize = dirSize + 4096;//folder minimum size is 4kb
                    ftpFolderSize(directoryPath+"/"+ftpFile.getName(),client);
                }

            }
        }
    }catch (Exception e) {
        e.printStackTrace();
    }
    return dirSize;
}

Hesychast

This will not work for SFTP (which is a different protocol than FTP). – Gagliardi