Copy between two AWS buckets - Stream returned from AWS: HashStream does not support seeking

Question

I am trying to re-upload a stream that I just retrieved. It shouldn't really matter that I am using AWS I believe... Maybe my understanding of working with streams is just too limited? :-)

I am using the following method straight from the AWS documentation to download and upload streams:

File upload:

public bool UploadFile(string keyName, Stream stream)
{
using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast1))
{
    try
    {
        TransferUtility fileTransferUtility = new TransferUtility(new AmazonS3Client(Amazon.RegionEndpoint.USEast1));

        fileTransferUtility.Upload(stream, bucketName, keyName);

        return true;
    }
    catch (AmazonS3Exception amazonS3Exception)
    {
        [...]
    }
}
}

Getting the file:

public Stream GetFile(string keyName)
{    
using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
{
    try
    {
        GetObjectRequest request = new GetObjectRequest
        {
            BucketName = bucketName,
            Key = keyName
        };

        GetObjectResponse response = client.GetObject(request);
        responseStream = response.ResponseStream;

        return responseStream;
    }
    catch (AmazonS3Exception amazonS3Exception)
    {
        [...]
    }
}
}

Now, I am trying to combine the two methods: I am getting a stream and immediately want to upload it again. However, I am getting the following error message: System.NotSupportedException : HashStream does not support seeking

I am guessing it has something to do that I am getting a stream which is somehow now immediately ready to be uploaded again?

This is how I am trying to get the stream of an existing file (.jpg) and immediately try to upload it with a different filename:

newAWS.UploadFile(newFileName, oldAWS.GetFile(oldFile));

Where newAWS and oldAWS are instances of the AWS class, and newFileName is a string :-) This is the line I am getting the error message pasted above.

Please let me know if I am missing something obvious here why I would not be able to re-upload a fetched stream. Or could it be that it is related to something else I am not aware of and I am on the wrong track trying to troubleshoot the returned stream?

What I am basically trying to do is to copy one file from an AWS bucket to another using streams. But for some reason I get the error message trying to upload the stream that I just downloaded.

Thank you very much for taking your time digging through my code :-)

Strum

Answer 1

As the error indicates, the Hashstream object that Amazon returns does not support seeking.

Something in your code, or a method you're calling, is trying to do seeking on that stream. Most likely it's trying to seek to the beginning of the stream.

So you need to convert Hashstream to a different stream that supports seeking:

using (GetObjectResponse response = client.GetObject(request))
using (Stream responseStream = response.ResponseStream)
using (MemoryStream memStream = new MemoryStream())
{
    responseStream.CopyTo(memStream);
    memStream.Seek(0, SeekOrigin.Begin);

    // now use memStream wherever you were previously using responseStream
}

Blida

Answer 2

It looks like that my approach won't work and I am not really sure why it doesn't. The good news is, that you can simply use the following code to copy from one AWS bucket to another:

static IAmazonS3 client;
client = new AmazonS3Client(Amazon.RegionEndpoint.USEast1);

CopyObjectRequest request = new CopyObjectRequest()
{
    SourceBucket      = bucketName,
    SourceKey         = objectKey,
    DestinationBucket = bucketName,
    DestinationKey    = destObjectKey
};
CopyObjectResponse response = client.CopyObject(request);

So I didn't have to try to recreate this functionality using the upload and download methods I wrote but simply use the copy method provided.

HOWEVER, I still would like to know why my approach was failing? I really think I am missing something regarding the streams. Any feedback would be greatly appreciated :-)

I will leave this question as unanswered since I would really like to know what I am missing regarding streams. I would be happy to mark an answer as accepted that explains my missing knowledge :-)

Strum