how to upload image through python-eve to some external storage server e.g. S3?

Question

I know we now can upload/post a media type field to the python-eve REST-API framework and it will be stored in mongodb. Is there a way we can change the storage of those media? E.g. a remote storage server e.g. amazon S3? So that we only store the URL of the image in mongodb instead of the whole image?

Careycarfare

Answer 1

While the standard behaviour is to store on GridFS, you can also provide your own MediaStorage subclass which can store wherever you want to (file system, S3, etc.)

Something like this would do the trick:

from eve.io.media import MediaStorage

class AmazonS3MediaStorage(MediaStorage):
    """ Your actual implementation """
    pass

app = Eve(media=AmazonS3MediaStorage)

if __name__ == '__main__':
    app.run()

Check out the actual MediaStorage class for implementation details and/or see the actual GridFSMediaStorage class for reference.

Casady

@JohnZ if you come up with a S3 MediaStorage class and are interested in sharing it please get in touch with me, we could add it to the Eve repo or, at the very least, make an extension out of it. – Casady

I am trying to implement an FS Media storage (just for studying) and two things are hapenning: 1- The get is being called after the put always , 2- i am receiving "resourceWarning: unclosed file" on creation. should i return a stream or something like that? – Gagliardi

I've managed to make a MediaStorage using S3. Here's my code: github.com/gwainer/eve/commit/… – Pb

@Pb nice, how about you refactor it into a stand-alone package? It could then join the official extensions list. – Casady

sure, let's do this. I will check how to package it correctly. – Pb

@Pb Has the s3 mediastorage implementation in the github link you have shared been integrated with eve?? if yes How do I use it with existing eve I have? Should I just update eve? I'm asking without having to copy your code to my project,etc. Is there a way to just import that implementation of your? – Mysticism

@Pb The github code you shared has not worked for me. It kept throwing "fileobj must implement read". – Mysticism

@Mysticism what python version are you using? – Pb

@Pb I got it this uploading to s3 using boto3. I'm using python 3.6.5 version. thank you. – Mysticism