Upload ndarray(image in OpenCV) on to google cloud storage as a .jpg or .png

Question

I have a similar issues like How to upload a bytes image on Google Cloud Storage from a Python script.

I tried this

from google.cloud import storage
import cv2
from tempfile import TemporaryFile
import google.auth
credentials, project = google.auth.default()
client = storage.Client()
# https://console.cloud.google.com/storage/browser/[bucket-id]/
bucket = client.get_bucket('document')
# Then do other things...
image=cv2.imread('/Users/santhoshdc/Documents/Realtest/15.jpg')
with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    blob = bucket.get_blob(gcs_image)
    print(blob.download_as_string())
    blob.upload_from_string('New contents!')
    blob2 = bucket.blob('document/operations/15.png')

    blob2.upload_from_filename(filename='gcs_image')

This is the error that's posing up

> Traceback (most recent call last):   File
> "/Users/santhoshdc/Documents/ImageShapeSize/imageGcloudStorageUpload.py",
> line 13, in <module>
>     blob = bucket.get_blob(gcs_image)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/bucket.py",
> line 388, in get_blob
>     **kwargs)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/blob.py",
> line 151, in __init__
>     name = _bytes_to_unicode(name)   File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/_helpers.py",
> line 377, in _bytes_to_unicode
>     raise ValueError('%r could not be converted to unicode' % (value,)) ValueError: <_io.BufferedRandom name=7> could not be
> converted to unicode

Can anyone guide me what's going wrong or what I'm doing incorrectly?

Malicious

@A.Queue Please look into this – Pyramid

Answer 1

As suggested by @A.Queue in(gets deleted after 29 days)

from google.cloud import storage
import cv2
from tempfile import TemporaryFile

client = storage.Client()

bucket = client.get_bucket('test-bucket')
image=cv2.imread('example.jpg')
with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    gcs_image.seek(0)
    blob = bucket.blob('example.jpg')
    blob.upload_from_file(gcs_image)

The file got uploaded,but uploading a numpy ndarray doesn't get saved as an image file on the google-cloud-storage

PS:

numpy array has to be convert into any image format before saving.

This is fairly simple, use the tempfile created to store the image, here's the code.

with NamedTemporaryFile() as temp:

    #Extract name to the temp file
    iName = "".join([str(temp.name),".jpg"])

    #Save image to temp file
    cv2.imwrite(iName,duplicate_image)

    #Storing the image temp file inside the bucket
    blob = bucket.blob('ImageTest/Example1.jpg')
    blob.upload_from_filename(iName,content_type='image/jpeg')

    #Get the public_url of the saved image 
    url = blob.public_url

Pyramid

was the issue solved in the end? I am asking because in the other question you have wrote that it was a issue with the path, but here you say it is unreadable. – Donitadonjon

@Donitadonjon Yes the issue was solved. numpy array direct upload will not be stored in image format hence unreadable. I have updated the code for numpy array first converted and stored in tempfile then uploaded – Pyramid

but you need to delete the file after you uplaoded. Otherwise the disk can be overfilled – Subsoil

Answer 2

You are calling blob = bucket.get_blob(gcs_image) which makes no sense. get_blob() is supposed to get a string argument, namely the name of the blob you want to get. A name. But you pass a file object.

I propose this code:

with TemporaryFile() as gcs_image:
    image.tofile(gcs_image)
    gcs_image.seek(0)
    blob = bucket.blob('documentation-screenshots/operations/15.png')
    blob.upload_from_file(gcs_image)

Scissile

Traceback (most recent call last): File "/Users/santhoshdc/Documents/ImageShapeSize/imageGcloudStorageUpload.py", line 34, in <module> blob2.upload_from_filename(filename='gcs_image') File "/Users/santhoshdc/.virtualenvs/test/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 1021, in upload_from_filename with open(filename, 'rb') as file_obj: FileNotFoundError: [Errno 2] No such file or directory: 'gcs_image'. This is the error I'm getting on removing those three lines – Malicious

Urgs. filename='gcs_image' makes no sense. I'll add code I think might work to my answer. Hang on. – Scissile

@PrashantPanwar I added example code which should do the trick. – Scissile

The program seems to work fine no errors, but there isn't any file being uploaded into the bucket. Trying to upload an image through gsutil in the terminal seems to upload the image. Have no idea where it is going wrong? – Malicious

Maybe you need to close() the blob? I didn't look that up. – Scissile

Getting AttributeError: ‘Blob’ object has no attribute ‘close’. – Malicious

Maybe slashes are not allowed in the blob names? As I said, I'm not familiar with the Google Cloud Services. – Scissile

Scanning through googlecloudplatform.github.io/google-cloud-python/latest/… I found examples which inspired this: from google.cloud.storage import Blob — blob = Blob('15.png', bucket) — blob.upload_from_file(gcs_image). Could you test this? – Scissile

Tried with this code as well but still same issue. Getting confused here – Malicious

Try running this code pastebin.com/Jr6k3SW2. It's mostly the same but without google.auth which is unnecessary here as google.cloud is handling everything in the background. It will be a good point of reference as I have tried it and it works. – Donitadonjon

Where are you executing the script? Have you set all the necessary environment variables and checked that the bucket exists? I know this questions are to obvious but it never hurts to check. – Donitadonjon

I'm explaining my work in another post. You might get a clear picture there. @Donitadonjon – Pyramid

link @Donitadonjon – Pyramid

pastebin.com/Jr6k3SW2 Worked as expected Thank You @Donitadonjon but please check the above link. – Pyramid

Tried to upload a PIL JpegImageFile. I am getting AttributeError: 'JpegImageFile' object has no attribute 'read' @Donitadonjon – Pyramid

@Donitadonjon Could you please look into the link here – Pyramid

.tofile in image.tofile(gcs_image) is saving an ndarray into your blob. So blob2 = bucket.blob('document/operations/15.png') isn't really a png file. If you want to write it as an image instead of ndarray maybe you can use imwrite. – Donitadonjon

Answer 3

None of the above answers seemes to uplaod a correctly formatted file. Converting the image to bytes and uploading as a string worked for me:

def upload_image(self, directory_name: str, image_name: str, image: cv2.Mat):
    image_bytes = cv2.imencode(".jpg", image)[1].tobytes()
    blob = self._bucket.blob("{}/{}.jpg".format(directory_name, image_name))
    blob.upload_from_string(image_bytes, content_type="image/jpeg")

Andreandrea