How to parse multipart/form-data on firebase cloud functions?

Question

I've been trying to post a multipart/form-data object with text and an image file to one of my cloud functions, according to the documents here:

https://cloud.google.com/functions/docs/writing/http#multipart_data_and_file_uploads

I've kept my cloud function almost entirely the same as the example, except I've wrapped it in a CORS response. It seems though, that no matter what, busboy's 'field' and 'file' events never fire, and when I print the request body's toString method, I get some of the data, before it goes to gibberish.

Is it possible I'm setting something up incorrectly when sending the FormData?

Here's the code containing my XMLHttpRequest():

var formData = new FormData(document.getElementById("ticketForm"));
return new Promise(function (resolve, reject) {
      var xmlhttp = new XMLHttpRequest();
      xmlhttp.open("POST", "https://us-central1-XXXXXXX.cloudfunctions.net/ticketFunction");
      var boundary = Math.random().toString().substr(8) + "--";
      xmlhttp.setRequestHeader('Content-Type', 'multipart/form-data;charset=utf-8; boundary=' + boundary);
      // xmlhttp.setRequestHeader('Content-Type', undefined);

      xmlhttp.onload = function () {
        if (this.status >= 200 && this.status < 300) {
          resolve(xmlhttp.response);
        } else {
          reject({
            status: this.status,
            statusText: xmlhttp.statusText
          });
        }
      };
      xmlhttp.onerror = function () {
        reject({
          status: this.status,
          statusText: xmlhttp.statusText
        });
      };
      xmlhttp.send(formData);
    });

Here's my cloud function:

exports.newTicketWithPhoto = functions.https.onRequest((req, res) => { cors(req, res, () => {

if (req.method === 'POST') {

  const busboy = new Busboy({ headers: req.headers });
  const tmpdir = os.tmpdir();
  console.log("Length: " + req.headers['content-length']);
  console.log(req.body.toString());

  // This object will accumulate all the fields, keyed by their name
  const fields = {};

  // This object will accumulate all the uploaded files, keyed by their name.
  const uploads = {};

  // This code will process each non-file field in the form.
  busboy.on('field', (fieldname, val) => {
    // TODO(developer): Process submitted field values here
    console.log(`Processed field ${fieldname}: ${val}.`);
    fields[fieldname] = val;
  });

  busboy.on('error', function(err){
    console.log("Error: " + err);
  });

  // This code will process each file uploaded.
  busboy.on('file', (fieldname, file, filename) => {
    // Note: os.tmpdir() points to an in-memory file system on GCF
    // Thus, any files in it must fit in the instance's memory.
    console.log(`Processed file ${filename}`);
    const filepath = path.join(tmpdir, filename);
    uploads[fieldname] = filepath;
    file.pipe(fs.createWriteStream(filepath));
  });

  // This event will be triggered after all uploaded files are saved.
  busboy.on('finish', () => {
    // TODO(developer): Process uploaded files here
    console.log(fields);
    console.log("Uploads: " + JSON.stringify(uploads));
    for (const name in uploads) {
      console.log(name);
      const file = uploads[name];
      fs.unlinkSync(file);
    }
    res.send();
  });

  req.pipe(busboy);
} else {
  // Return a "method not allowed" error
  res.status(405).send("Something weird happened");
}

}) });

A couple of things I notice is this: Printing the content-length value of the header always seems to return undefined.

When I print the req.body.toString() method, I get this:

 ------WebKitFormBoundaryeYZHuHsOLlohyekc
Content-Disposition: form-data; name="description"

testing description
------WebKitFormBoundaryeYZHuHsOLlohyekc
Content-Disposition: form-data; name="priority"

Low
------WebKitFormBoundaryeYZHuHsOLlohyekc
Content-Disposition: form-data; name="dueDate"

2018-07-27
------WebKitFormBoundaryeYZHuHsOLlohyekc
Content-Disposition: form-data; name="customer"

zavtra
------WebKitFormBoundaryeYZHuHsOLlohyekc
Content-Disposition: form-data; name="email"

[email protected]
------WebKitFormBoundarysseArmLvKhJY0TAm
Content-Disposition: form-data; name="photo"; filename="brighthabits1.png"
Content-Type: image/png

�PNG

IHRGB���@IDATx�}�ݴտ��I�$�V���*�EH ! �:(_7m)-ݻ�ί���{-dCaf��*�=!����N����ͽ�ږm�y�׶tt�OG�ʶ,6L���L*�ć[����V;�x�+[�c�/�0;@a�5��;��]]<x��\R�cqoG`rGƵ�t����O�y�J���"
����*�,�F,��.�ib�
                 ��I�.�SV�;��h�!v��~T�EY����(u\�4+&��I��9@~wP�`N��H�;�G"7.BI��h
                                                                               P��$R
                                                                                    �0pt,�[=��E��8����$^$��
"�,�,�4�>�Y�YY|�v3JSW��
                       )�q,���i>w��A��q\-
                                         �u���ՠ�hJW�oF������W7X��]��
                                                                    )#mx������&�њ�����iu���;D��ŗL��ޥh[F�8���D�^������IW��#��

                                �
                                 �
�TL�n���� {�l�`h����r   ��S>�[���&���_�%R8���W��mok�E����R���.]#@5������j���o���e����?Ӟ�u�Ţ�Y��5�N'�Nf��Թ#ߏ��E;�<�?^X��x�uπʭ�V??�� s�plzBǶ 

I'm not sure what causes all the gibberish at the end, but that's explicitly only when I upload an image. When there's no image in the form data, busboy's 'field' events still don't fire, leading me to believe something still isn't being parsed correctly.

It's frustrating because it otherwise seems like I'm following the documentation exactly correctly.

Minnow

The gibberish at the end is the actual contents of the file. If you do a toString() on that, it's expected that you get ASCII characters. Try opening any file with a texteditor and you will see the exact same. – Romina

Also, are there actually any lines printed by console.log('Processed field ${fieldname}: ${val}.'); ? – Romina

Answer 1

// Node.js doesn't have a built-in multipart/form-data parsing library.
// Instead, we can use the 'busboy' library from NPM to parse these requests.
const Busboy = require("busboy")
const busboy = new Busboy({ headers: request.headers })
let fields = []
busboy.on("field", (field, val) => {
    console.log(`Processed field ${field}: ${val}.`)
    fields[field] = val
})
busboy.end(request.rawBody)

Belcher

Have you actually tested that this works now with Firebase? – Bagger

This worked for me with Typescript functions! thanks a lot – Bornu

Answer 2

First install a busboy dependency

npm i busboy

After create a function to return data from FormData

const getFieldsFromFormData = (headers: any, body: any) =>
new Promise(async (resolve, reject) => {
    const Busboy = require('busboy');
    const busboy = new Busboy({ headers });
    let fields: any = {};
    
    busboy.on("field", (field: string, val: any) => fields[field] = val)
    busboy.on('finish',() => resolve(fields));
    busboy.end(body)
});

In controler

expressInstance.post('/upload', async (req, res, next) => {

    const body = await getFieldsFromFormData(req.headers, req.body)
    console.log('MyFormData:>> ', body);

    res.send({});
})

Khamsin

Answer 3

I am following this repository Image Upload with Busboy and worked very well on Firebase Cloud.

Keenankeene

Answer 4

The official google docs suggest the use of the Busboy npm package as the accepted answer suggests. Just posting these here in case they prove useful to others:

https://cloud.google.com/functions/docs/writing/http#multipart_data

Hemianopsia