Decode an App Engine Blobkey to a Google Cloud Storage Filename

Question

I've got a database full of BlobKeys that were previously uploaded through the standard Google App Engine create_upload_url() process, and each of the uploads went to the same Google Cloud Storage bucket by setting the gs_bucket_name argument.

What I'd like to do is be able to decode the existing blobkeys so I can get their Google Cloud Storage filenames. I understand that I could have been using the gs_object_name property from the FileInfo class, except:

You must save the gs_object_name yourself in your upload handler or this data will be lost. (The other metadata for the object in GCS is stored in GCS automatically, so you don't need to save that in your upload handler.

Meaning gs_object_name property is only available in the upload handler, and if I haven't been saving it at that time then its lost.

Also, create_gs_key() doesn't do the trick because it instead takes a google storage filename and creates a blobkey.

So, how can I take a blobkey that was previously uploaded to a Google Cloud Storage bucket through app engine, and get it's Google Cloud Storage filename? (python)

Gonnella

To my knowledge you can only rewrite the blob, that means reading it byte by byte and writing it again to the cloudstorage similar to renaming a cloudstorage file, but this time saving the cloudstorage path instead of the blobkey. BlobKeys saving cloudstorage objects seem to be in "monkeypatch" state. #18697918 – Pettitoes

Answer 1

You can get the cloudstorage filename only in the upload handler (fileInfo.gs_object_name) and store it in your database. After that it is lost and it seems not to be preserved in BlobInfo or other metadata structures.

Google says: Unlike BlobInfo metadata FileInfo metadata is not persisted to datastore. (There is no blob key either, but you can create one later if needed by calling create_gs_key.) You must save the gs_object_name yourself in your upload handler or this data will be lost.

https://developers.google.com/appengine/docs/python/blobstore/fileinfoclass

Update: I was able to decode a SDK-BlobKey in Blobstore-Viewer: "encoded_gs_file:base64-encoded-filename-here". However the real thing is not base64 encoded.

create_gs_key(filename, rpc=None) ... Google says: "Returns an encrypted blob key as a string." Does anyone have a guess why this is encrypted?

Pettitoes

Probably encrypted because it includes the access token. – Tessitura

Answer 2

From the statement in the docs, it looks like the generated GCS filenames are lost. You'll have to use gsutil to manually browse your bucket.

https://developers.google.com/storage/docs/gsutil/commands/ls

Lavenialaver

Answer 3

If you have blobKeys you can use: ImagesServiceFactory.makeImageFromBlob

Dereliction