Download file help for AWS S3 Java SDK

Question

The code below only works for downloading text files from a bucket in S3. This does not work for an image. Is there an easier way to manage downloads/types using the AWS SDK? The example included in the documentation does not make it apparent.

AWSCredentials myCredentials = new BasicAWSCredentials(
       String.valueOf(Constants.act), String.valueOf(Constants.sk)); 
AmazonS3Client s3Client = new AmazonS3Client(myCredentials);        
S3Object object = s3Client.getObject(new GetObjectRequest("bucket", "file"));
    
BufferedReader reader = new BufferedReader(new InputStreamReader(
       object.getObjectContent()));
File file = new File("localFilename");      
Writer writer = new OutputStreamWriter(new FileOutputStream(file));
    
while (true) {          
     String line = reader.readLine();           
     if (line == null)
          break;            
     
     writer.write(line + "\n");
}
    
writer.close();

Guillermo

i have to do thing and try your code but jar not found can you please suggest which jar to use.Currently i am using aws-android-sdk-2.1.5-s3.jar jar but not found all th classes. – Pantile

do you have any idea how to upload file to amazon.I am using your code to download file and it is working perfectly. – Pantile

Answer 1

Instead of Reader and Writer classes you should be using InputStream and OutputStream classes:

InputStream reader = new BufferedInputStream(
   object.getObjectContent());
File file = new File("localFilename");      
OutputStream writer = new BufferedOutputStream(new FileOutputStream(file));

int read = -1;

while ( ( read = reader.read() ) != -1 ) {
    writer.write(read);
}

writer.flush();
writer.close();
reader.close();

Act

This works.. Programs listed by aws documentation does not work.This worked very well. Thank you for posting – Salable

Answer 2

Though the code written in Mauricio's answer will work - and his point about streams is of course correct - Amazon offers a quicker way to save files in their SDK. I don't know if it wasn't available in 2011 or not, but it is now.

AmazonS3Client s3Client = new AmazonS3Client(myCredentials);

File localFile = new File("localFilename");

ObjectMetadata object = s3Client.getObject(new GetObjectRequest("bucket", "s3FileName"), localFile);

Kitchenette

Needs a minor edit: The object contents are saved to the file, but the returned object is ObjectMetadata, not S3Object. See docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/com/… – Koto

@azdev Actually, the code sample uses a different version of getObject which does return a S3Object. – Kitchenette

How come I cannot see the S3Client.getObject method in my code?..I want to achieve the same but there is no such method associated to my s3Client object.Am i missing something? – Phoebephoebus

@ShikhaShah The getObject method which returns a S3Object is from the AmazonS3 interface which is implemented by AmazonS3Client. Perhaps your IDE doesn't show the methods inherited from the interface? – Kitchenette

But I am able to download the file and save it via AmazonS3Client only...but cant see this particular method which expedite the saving process..m still confused!! – Phoebephoebus

What version of the SDK are you using? I have a Maven dependency to aws-java-sdk, version 1.3.12. – Kitchenette

Answer 3

Eyals answer gets you half way there but its not all that clear so I will clarify.

AmazonS3Client s3Client = new AmazonS3Client(myCredentials);

//This is where the downloaded file will be saved
File localFile = new File("localFilename");

//This returns an ObjectMetadata file but you don't have to use this if you don't want 
s3Client.getObject(new GetObjectRequest(bucketName, id.getId()), localFile);

//Now your file will have your image saved 
boolean success = localFile.exists() && localFile.canRead(); 

Cleary

what you have used for id.getId()) ? – Lovable

The id.getId() is the key you used to store the object in the bucket. I just use my id for the key as I know its unique. To summarise, you create an empty file, do a getRequest telling the aws api where the new empty file is and the aws api handles writing the object to the file you specified. So convenient. – Cleary

@ShawnVader : Please can you clear why you are validating file. I though S3 api does a hash check. Thank you. – Tushy

@Tushy You are right the above example of checking the file may return incorrect results as it is acceptable for the localFile in the above case to exist. Perhaps a better way is to ensure that the ObjectMetadata which returns is not null. Docs say > Returns All S3 object metadata for the specified object. Returns null if constraints were specified but not met. So if it has metadata then you can be confident that it's valid content in the file because looking through the AWS source I can see it validates the file integrity and will retry once before failing. – Cleary

@ShawnVader: Edited the answer, please can you review it once it is peer reviewed. – Tushy

Answer 4

There is even much simpler way to get this. I used below snippet. Got reference from http://docs.ceph.com/docs/mimic/radosgw/s3/java/

AmazonS3 s3client = AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion(Regions.US_EAST_1).build();

s3client.getObject(
                new GetObjectRequest("nomad-prop-pics", "Documents/1.pdf"),
                new File("D:\\Eka-Contract-Physicals-Dev\\Contracts-Physicals\\utility-service\\downlods\\1.pdf")
    );

Jaclin

Answer 5

Use java.nio.file.Files to copy S3Object to local file.

public File getFile(String fileName) throws Exception {
    if (StringUtils.isEmpty(fileName)) {
        throw new Exception("file name can not be empty");
    }
    S3Object s3Object = amazonS3.getObject("bucketname", fileName);
    if (s3Object == null) {
        throw new Exception("Object not found");
    }
    File file = new File("you file path");
    Files.copy(s3Object.getObjectContent(), file.toPath());
    return file;
}

Unaesthetic

Where does inputStream come from? I think you mean either s3Object.close or s3Object.getObjectContent.close(). Regardless, nio is a good suggestion. – Cowfish

Answer 6

Dependencies AWS S3 bucket

implementation "commons-logging:commons-logging-api:1.1"
implementation platform('com.amazonaws:aws-java-sdk-bom:1.11.1000')
implementation 'com.amazonaws:aws-android-sdk-core:2.6.0'
implementation 'com.amazonaws:aws-android-sdk-cognito:2.2.0'
implementation 'com.amazonaws:aws-android-sdk-s3:2.6.0'

Download object from S3 bucket and store in local storage.

try {
                //Creating credentials
                AWSCredentials awsCredentials = new BasicAWSCredentials(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY);

                //Creating S3
                AmazonS3Client s3Client = new AmazonS3Client(awsCredentials);

                //Creating file path
                File dir = new File(this.getExternalFilesDir("AWS"),"FolderName");
                if(!dir.exists()){
                    dir.mkdir();
                }

                //Object Key
                String bucketName = "*** Bucket Name ***";
                String objKey = "*** Object Key ***";
                
                //Get File Name from Object key
                String  name = objKey.substring(objKey.lastIndexOf('/') + 1);

                //Storing file S3 object in file path
                InputStream in = s3Client.getObject(new GetObjectRequest(bucketName, objKey)).getObjectContent();
                Files.copy(in,Paths.get(dir.getAbsolutePath()+"/"+name));
                in.close();

            } catch (Exception e) {
                Log.e("TAG", "onCreate: " + e);
            }

Get List of Object from S3 Bucket

public void getListOfObject()
    {
        ListObjectsV2Result result ;
        AWSCredentials awsCredentials = new BasicAWSCredentials(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY);
        AmazonS3Client s3Client = new AmazonS3Client(awsCredentials);
        result = s3Client.listObjectsV2(AWS_BUCKET);
        for( S3ObjectSummary s3ObjectSummary : result.getObjectSummaries())
        {
            Log.e("TAG", "onCreate: "+s3ObjectSummary.getKey() );
        }
    }

Check if any object exists in bucket or not

public String isObjectAvailable(String object_key)
    {

        try {
            AWSCredentials awsCredentials = new BasicAWSCredentials(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY);
            AmazonS3 s3 = new AmazonS3Client(awsCredentials);
            String bucketName = AWS_BUCKET;
            S3Object object = s3.getObject(bucketName, object_key);
            Log.e("TAG", "isObjectAvailable: "+object.getKey()+","+object.getBucketName() );
        } catch (AmazonServiceException e) {
            String errorCode = e.getErrorCode();
            if (!errorCode.equals("NoSuchKey")) {
               // throw e;
                Log.e("TAG", "isObjectAvailable: "+e );
            }

            return "no such key";
        }
        return "null";
    }

Launch