Uploading files from Firebase Cloud Functions to Cloud Storage

Question

The documentation is too complex for me to understand. It shows how to download a file from Cloud Storage to Cloud Functions, manipulate the file, and then upload the new file to Cloud Storage. I just want to see the basic, minimum instructions for uploading a file from Cloud Functions to Cloud Storage. Why doesn't this work:

const functions = require('firebase-functions');
const admin = require('firebase-admin');

admin.initializeApp();

exports.storage = functions.firestore.document('Test_Value').onUpdate((change, context) => {

  var metadata = {
    contentType: 'text',
  };

  admin.storage().ref().put( {'test': 'test'}, metadata)
  .then(function() {
    console.log("Document written.");
  })
  .catch(function(error) {
    console.error(error);
  })

});

The error message is admin.storage(...).ref is not a function. I'm guessing that firebase-admin includes Firestore but not Storage? Instead of firebase-admin should I use @google-cloud/storage? Why doesn't this work:

const functions = require('firebase-functions');
const admin = require('firebase-admin');

const {Storage} = require('@google-cloud/storage')();
const storage = new Storage();

admin.initializeApp();

exports.storage = functions.firestore.document('Test_Value').onUpdate((change, context) => {

  storage.bucket().upload( {'test': 'test'} , {
    metadata: {
      contentType: 'text'
    }
  })

});

I can't even deploy this code, the error message is

Error parsing triggers: Cannot find module './clone.js'

Apparently a npm module dependency is missing? But the module isn't called clone.js? I tried requiring child-process-promise, path, os, and fs; none fixed the missing clone.js error.

Why does admin.initializeApp(); lack parameters, when in my index.html file I have:

firebase.initializeApp({
    apiKey: 'swordfish',
    authDomain: 'myapp.firebaseapp.com',
    databaseURL: "https://myapp.firebaseio.com",
    projectId: 'myapp',
    storageBucket: "myapp.appspot.com"
  });

Another issue I'm seeing:

npm list -g --depth=0       

/Users/TDK/.nvm/versions/node/v6.11.2/lib
├── [email protected]
├── UNMET PEER DEPENDENCY  error: ENOENT: no such file or directory, open '/Users/TDK/.nvm/versions/node/v6.11.2/lib/node_modules/firebase-admin/package.json
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

In other words, there's something wrong with firebase-admin, or with Node 6.11.2. Should I use a Node Version Manager to revert to an older version of Node?

Maley

Same pain here, I'm guessing we should use this API reference.. googleapis.dev/nodejs/storage/latest/File.html#save – Abeyant

Answer 1

I uploaded a file from my hard drive to Firebase Cloud Storage via Google Cloud Functions. First, I found the documentation for Google Cloud Functions bucket.upload.

    const functions = require('firebase-functions');
    const admin = require('firebase-admin');
    admin.initializeApp();
    
    exports.Storage = functions.firestore.document('Storage_Value').onUpdate((change, context) => {
    
      const {Storage} = require('@google-cloud/storage');
      const storage = new Storage();
      const bucket = storage.bucket('myapp.appspot.com');
    
      const options = {
        destination: 'Test_Folder/hello_world.dog'
      };
    
      bucket.upload('hello_world.ogg', options).then(function(data) {
        const file = data[0];
      });
    
      return 0;
    });

The first three lines are Cloud Functions boilerplate. The next line

    exports.Storage = functions.firestore.document('Storage_Value').onUpdate((change, context) => {

creates the Cloud Function and sets the trigger. The next three lines are more Google Cloud boilerplate.

The rest of the code locates the file hello_world.ogg on my computer's hard drive in the functions folder of my project directory and uploads it to the directory Test_Folder and changes the name of the file to hello_world.dog in my Firebase Cloud Storage. This returns a promise, and the next line const file = data[0]; is unnecessary unless you want to do something else with the file.

Lastly we return 0;. This line does nothing except prevent the error message

Function returned undefined, expected Promise or Value

Maley

Answer 2

1. Go to https://console.cloud.google.com/iam-admin/iam
2. Click the pencil icon next to your App Engine default service account
3. + ADD ANOTHER ROLE
4. Add Cloud Functions Service Agent

In my specific use case, I needed to decode a base64 string into a byte array and then use that to save the image.

    var serviceAccount = require("./../serviceAccountKey.json");

    import * as functions from 'firebase-functions';
    import * as admin from 'firebase-admin';    

    admin.initializeApp({
        projectId: serviceAccount.project_id, 
        credential: admin.credential.cert(serviceAccount),
        databaseURL: "https://your_project_id_here.firebaseio.com", //update this
        storageBucket: "your_bucket_name_here.appspot.com" //update this
      });

    function uploadProfileImage(imageBytes64Str: string): Promise<any> {

        const bucket = admin.storage().bucket()
        const imageBuffer = Buffer.from(imageBytes64Str, 'base64')
        const imageByteArray = new Uint8Array(imageBuffer);
        const file = bucket.file(`images/profile_photo.png`);
        const options = { resumable: false, metadata: { contentType: "image/jpg" } }

        //options may not be necessary
        return file.save(imageByteArray, options)
        .then(stuff => {
            return file.getSignedUrl({
                action: 'read',
                expires: '03-09-2500'
              })
        })
        .then(urls => {
            const url = urls[0];
            console.log(`Image url = ${url}`)
            return url
        })
        .catch(err => {
            console.log(`Unable to upload image ${err}`)
        })
    }

---

Then you can call the method like this and chain the calls.

    uploadProfileImage(image_bytes_here)
    .then(url => {
        //Do stuff with the url here        
    })

---

Note: You must initialize admin with a service account and specify the default bucket. If you simply do admin.initializeApp() then your image urls will expire in 10 days.

Steps to properly use a service account.

1. Go to Service Accounts and generate a private key
2. Put the JSON file in your functions folder (next to src and node_modules)
3. Go to Storage and copy the URL not including the "gs://" in the front. Use this for the storage bucket url when initializing admin.
4. Use your project ID above for the database URL.

Nickelsen

thanks. I'm a little bit surprised all of this is needed. Firestore (database) works out of the box? – Tucson

Answer 3

See Introduction to the Admin Cloud Storage API for further details on how to use the Cloud Storage service in Firebase Admin SDK.

var admin = require("firebase-admin");

var serviceAccount = require("path/to/serviceAccountKey.json");

admin.initializeApp({
    credential: admin.credential.cert(serviceAccount),
    storageBucket: "<BUCKET_NAME>.appspot.com"
});

var bucket = admin.storage().bucket();

// 'bucket' is an object defined in the @google-cloud/storage library.
// See https://googlecloudplatform.github.io/google-cloud-node/#/docs/storage/latest/storage/bucket
// for more details.

Regarding uploading objects, see Cloud Storage Documentation Uploading Objects sample code:

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const bucketName = 'Name of a bucket, e.g. my-bucket';
// const filename = 'Local file to upload, e.g. ./local/path/to/file.txt';

// Uploads a local file to the bucket
await storage.bucket(bucketName).upload(filename, {
  // Support for HTTP requests made with `Accept-Encoding: gzip`
  gzip: true,
  metadata: {
    // Enable long-lived HTTP caching headers
    // Use only if the contents of the file will never change
    // (If the contents will change, use cacheControl: 'no-cache')
    cacheControl: 'public, max-age=31536000',
  },
});

console.log(`${filename} uploaded to ${bucketName}.`);

Alkaloid

That code creates a bucket but doesn't upload a file to Storage. – Maley

For 2nd code block, adding field/value destination : 'test/coffeeLogo.png' to the .upload() options JSON completes the file upload - stackoverflow.com/a/54227249, and in this thread stackoverflow.com/a/53181557 – Lactose

Answer 4

For people coming here, wondering why they don't get an error but files do not appear in the Online console while debugging locally.

Solution: Don't start the storage emulator locally.

I.e., when you just want to debug functions: firebase emulators:start --inspect-functions --only functions

Tucson

Answer 5

if (req.rawBody) {
    busboy.end(req.rawBody);
}
else {
    req.pipe(busboy);
}

As described in this issue: https://github.com/GoogleCloudPlatform/cloud-functions-emulator/issues/161#issuecomment-376563784

That