S3fs throws error as Transport endpoint is not connected

Question

I am doing a large amount of file upload and download between two s3fs mounted directories within two EC2 machines. If the transaction is high, automatically mounted s3fs throws an error Transport endpoint is not connected. To resolve this issue, I have to forcefully unmount and remount.I would like to know what is the cause for the issue?

Note: I am using s3 as a mounted drive by use of s3fs, but I heard from another developer at some point of time, which not advisable. He said about use s3 as SDK means by use API.

Bianco

Answer 1

I've used s3fs and although it seems fairly reliable (in my limited experience) if there is a problem with networking or permissions at the AWS layer then the error you report is shown

I agree with the other developer that if you are having problems under these high load conditions then using direct access to s3 with the API for at least part of your process could be a good idea

Locative