Firebase Cloud Functions and Busboy not parsing fields or files

Question

I've been doing some experiments with Firebase Cloud Functions and Express, and I am stuck with a problem when I try to process a FormData with Busboy. It seems that I only get one big malformed text field with all the data in it, including also any binary data of files I try to upload (i.e. gibberish ascii characters).

I've tried the different solutions found online, even here on SO, and I see that all of them are built around the example provided by Google about Multipart Data: https://cloud.google.com/functions/docs/writing/http

This is my server-side code:

// index.js
const functions = require('firebase-functions');
const express = require('express');
const Busboy = require('busboy');

app = express();

app.post('/upload', (req, res) => {
  const busboy = new Busboy({
    headers: req.headers,
    limits: {
      // Cloud functions impose this restriction anyway
      fileSize: 10 * 1024 * 1024,
    }
  });

  busboy.on('field', (key, value) => {
    console.log(`Busboy field ${key}: ${value}`);
  });

  busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
    console.log(`Busboy file ${fieldname}: ${filename}`);
  });

  busboy.on('finish', () => {
    console.log('Busboy finish');
    return res.send({
      status: 'Success',
      text: 'Great job!'
    });  
  });

  busboy.end(req.rawBody);
});

exports.api = functions.https.onRequest(app);

And this is the client in Node JS:

// index.js
import axios from 'axios';
import FormData from 'form-data';

const ENDPOINT_URL = XXXXXXXXXXXXXXXXX;

const postFile = async () => {

    try {
        const form_data = new FormData();
        form_data.append('userName', 'Fred');
        form_data.append('password', 'Flintstone');
        const response = await axios.post(`${ENDPOINT_URL}/upload`, form_data);
        console.log(response.data);
    } catch (error) {
        console.error(`Error: ${error}`);
    }
}

postFile();

On the client log everything is as expected, and I get the 'Great job' response back. However, this is what I get on the Firebase Cloud Functions log:

Busboy field ----------------------------047691570534364316647196
Content-Disposition: form-data; name: "userName"

Fred
----------------------------047691570534364316647196
Content-Disposition: form-data; name="password"

Flintstone
----------------------------047691570534364316647196--
)

Note that it's just a single output line in the log, meaning that Busboy called onField only once. As said above, if I add to the FormData a file, the output is very messy and I still get only ONE call to onField and none to onFile.

Midkiff

Answer 1

After further investigations, I found out that the server is working properly, while on client I need to change this line:

const response = await axios.post(`${ENDPOINT_URL}/upload`, form_data);

to:

const config = { headers: { 'content-type': `multipart/form-data; boundary=${form_data._boundary}` }};
const response = await axios.post(`${ENDPOINT_URL}/upload`, form_data, config);

Apparently, despite what stated in other posts here on SO, not specifying the multipart header doesn't cause it to be determined automatically. Note also that if you omit the boundary setting, you will get a Boundary not found error from Busboy on the server.

Midkiff

Can anyone confirm this? Although I think this may be because you are using axios while using something like fetch will automatically handle it. – Eer

Unfortunately I no longer have this project available, and can't try to swap axios for fetch. But thinking twice: if that worked, does it mean that fetch is automatically adding the header? – Midkiff

I don't have ._boundary on formData though. Where you getting that from? – Crowboot

Answer 2

For me, I had to write bb.end(req.body) before req.pipe(bb)

Subfamily