POST multipart/form-data to Serverless Next.js API (running on Vercel / Now.sh)
Asked Answered
N

8

7

I'm using a Vercel Serverless function to handle a file upload to Digital Ocean Spaces (identical API to AWS S3). However, I am running into issues handling multipart/form-data in the request handler.

On the front-end I'm using fetch to post a FormData() object with files and a couple of text fields. When logging the body and servers on the header, I can see everything there as expected, however when handling the multipart with Multer (I have also tried a couple of other similar packages) I am not able to retrieve any of the posted fields or files.

Also worth noting is that when using Postman to test the POST requests I'm running into the exact same issue, so I'm confident the issue lies in the serverless function.

Front-end:

const handleSubmit = async (values) => {
    const formData = new FormData();

    // build my Form Data from state.
    Object.keys(values).forEach(key => {
      formData.append(key, values[key]);
    });

    const response = await fetch("/api/post-submission", {
      method: "POST",
      headers: {
        Accept: "application/json",
      },
      body: formData,
    });
    const json = await response.json();
  };

Serverless handler:

const util = require("util");
const multer = require("multer");

module.exports = async (req, res) => {
  await util.promisify(multer().any())(req, res);
  console.log("req.body", req.body); // >> req.body [Object: null prototype] {}
  console.log("req.files", req.files); // >> req.files []

  // Do the file upload to S3...

  res.status(200).json({ uploadData });
};

Expected behavior:

req.body and req.files should be populated with my submitted data.

Nammu answered 16/6, 2020 at 14:54 Comment(3)
You could try generating a Presigned URL and give that to the client to upload files directly to S3 thereby skipping the Serverless FunctionDrapery
@Jack-wild have you solved it?Issy
@T.Vojtech I'm afraid not — the Vercel help team asked for a reproduction which I provided, but I'm yet to hear back from them. It does seem like it's a bug when using next.js on Vercel (serverless functions work fine if you run them directly on Vercel, but when you're also using Next.js they dont even if code is the same)Nammu
D
12

2022 Update

You can use the multiparty package to parse multipart/form-data. The key is to also export a config object turning the bodyParser off. This will allow multiparty to work as designed and prevent the dreaded stream ended unexpectedly error.

The below code is a fully working example of an upload api page.

import { NextApiRequest, NextApiResponse } from "next";
import multiparty from "multiparty";

const uploadImage = async (req: NextApiRequest, res: NextApiResponse) => {
  const form = new multiparty.Form();
  const data = await new Promise((resolve, reject) => {
    form.parse(req, function (err, fields, files) {
      if (err) reject({ err });
      resolve({ fields, files });
    });
  });
  console.log(`data: `, JSON.stringify(data));

  res.status(200).json({ success: true });
};

export default uploadImage;
export const config = {
  api: {
    bodyParser: false,
  },
};
Dol answered 8/4, 2022 at 15:12 Comment(3)
Hello, thanks for the answer, it also helped me out! Quick question about the answer though, I've never seen the: export const config = { api: { bodyParser: false, }, }; How does this work and what does it do exactly?Grannias
@Grannias its part of the Next middleware nextjs.org/docs/api-routes/api-middlewares. It outputs a config object that overrides the default.Dol
Fantastic solution. Finally worked after hours of searching!Paranoiac
D
5

I'm not exactly sure about the Multer package, but there aren't any inherent limitations that prevent multipart/form-data from being handled by Serverless Functions on Vercel (AWS Lambda under the hood).

let multiparty = require('multiparty')
let http = require('http')
let util = require('util')

module.exports = (req, res) => {
    if (req.method === "POST") {
        let form = new multiparty.Form();
        form.parse(req, (err, fields, files) => {
            res.writeHead(200, { 'content-type': 'text/plain' });
            res.write('received upload: \n\n');
            res.end(util.inspect({ fields: fields, files: files }));
        });
        return;
    } else {
        res.writeHead(405, { 'content-type': 'text/plain' });
        res.end("Method not allowed. Send a POST request.");
        return;
    }
}

I created a demo repository with a deployed URL here

Decrescendo answered 28/7, 2020 at 18:13 Comment(3)
Thanks — this is really helpful I'll try it out. Maybe you're right and it is supported, but when I asked the Vercel support person said multipart wasn't supported (they may have just been misinformed). Could be that it's not supported on hobby accounts perhaps.Nammu
Oh, just seen you are the Vercel Support Team! 😅 I guess the person I spoke to was mistaken, or maybe outdated info.Nammu
— I have done some debugging and tests, and it seems that multi-part form data is not supported when running API functions within the next.js framework — only when deployed as stand-alone serverless functions in Vercel. Is there a reason why this is the case, and is there any way to change this so they will run in next.jsNammu
A
3

I was able to get multipart/form-data working by using busboy.

const Busboy = require('busboy');

module.exports = (req, res) => {
    const busboy = new Busboy({ headers: req.headers });

    busboy.on('file', function(fieldname, file, filename, encoding, mimetype) {
      console.log('File [' + fieldname + ']: filename: ' + filename);

      file.on('data', function(data) {
        console.log('File [' + fieldname + '] got ' + data.length + ' bytes');
      });

      file.on('end', function() {
        console.log('File [' + fieldname + '] Finished');
      });
    });

    busboy.on('field', function(fieldname, val, fieldnameTruncated, valTruncated) {
      console.log('Field [' + fieldname + ']: value: ' + val);
    });

    busboy.on('finish', function() {
      console.log('Done parsing form!');
      res.writeHead(303, { Connection: 'close', Location: '/' });
      res.end();
    });

    req.pipe(busboy);
}
Apthorp answered 30/7, 2020 at 13:59 Comment(0)
S
3

Hi this can be achieved without any libraries, Next.js 13 already supports FormData!

This is the code I used in my project, it will definitely help you.

For context: the feature I implemented is allowing users to update their profile image.

// In your React client component

const [file, setFile] = useState<File | null>(null)

const handleSubmit = async (e: FormEvent<HTMLFormElement>) => {
  e.preventDefault()

  if (!file) throw new Error('No file selected')

  const { imageUrl } = await uploadProfileImage(file)

  console.log('New image URL', imageUrl)
}

const uploadProfileImage = async (file: File) => {
  const body = new FormData()

  body.set('image', file)

  const response = await fetch('/api/upload/profile-image', {
    method: 'POST',
    body,
  })

  if (!response.ok) {
    throw new Error('Error uploading profile image')
  }

  const result: UploadProfileImageResponse = await response.json()
  if (!result) throw new Error('Error uploading profile image')
  return result
}
// In `src/app/api/upload/profile-image/route.ts`

import sharp from 'sharp'
import { uploadToS3 } from '~/server/aws'
import { db } from '~/server/db/db'
import { eq } from 'drizzle-orm'
import { users } from '~/server/db/schema'
import { NextRequest, NextResponse } from 'next/server'
import { getServerSession } from 'next-auth'
import { authOptions } from '~/server/auth'

export async function POST(request: NextRequest) {
  // Step 1: Check if user is authenticated (With NextAuth)
  const session = await getServerSession(authOptions)
  if (!session) {
    return NextResponse.json(null, { status: 401 })
  }

  // Step 2: Get image from request (With Next.js API Routes)
  const formData = await request.formData()
  const imageFile = formData.get('image') as unknown as File | null
  if (!imageFile) {
    return NextResponse.json(null, { status: 400 })
  }
  const imageBuffer = Buffer.from(await imageFile.arrayBuffer())

  // Step 3: Resize image (With Sharp)
  const editedImageBuffer = await sharp(imageBuffer)
    .resize({ height: 256, width: 256, fit: 'cover' })
    .toBuffer()

  // Step 4: Upload image (With AWS SDK)
  const imageUrl = await uploadToS3({
    buffer: editedImageBuffer,
    key: `profile-images/${session.user.id}`,
    contentType: imageFile.type,
  })

  // Step 5: Update user in database (With Drizzle ORM)
  await db
    .update(users)
    .set({
      image: imageUrl,
    })
    .where(eq(users.id, session.user.id))

  // Step 6: Return new image URL
  return NextResponse.json({ imageUrl })
}

// Export types for API Routes
export type UploadProfileImageResponse = ExtractGenericFromNextResponse<
  Awaited<ReturnType<typeof POST>>
>
type ExtractGenericFromNextResponse<Type> = Type extends NextResponse<infer X>
  ? X
  : never

// In `src/server/aws.ts`

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3'

export const BUCKET_REGION = 'my-bucket-region'
export const BUCKET_NAME = 'my-bucket-name'

export const s3 = new S3Client({
  region: BUCKET_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID ?? 'missing-env',
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY ?? 'missing-env',
  },
})

export async function uploadToS3<K extends string>({
  buffer,
  key,
  contentType,
}: {
  buffer: Buffer
  key: K
  contentType: string
}) {
  await s3.send(
    new PutObjectCommand({
      Bucket: BUCKET_NAME,
      Key: key,
      Body: buffer,
      ContentType: contentType,
    })
  )
  return `https://${BUCKET_NAME}.s3.${BUCKET_REGION}.amazonaws.com/${key}` as const
}
Schwejda answered 24/10, 2023 at 15:30 Comment(0)
V
0

i see you use form-data, you can set header content type into multipart/form-data;boundary=----WebKitFormBoundaryyrV7KO0BoCBuDbTL

Vocational answered 1/8, 2021 at 3:43 Comment(0)
B
0

My function is like this , and I just find formdata this func. I wish a good doc.

import { getBody, result } from "../../lib/quickapi";
export async function POST(req:Request) {
    const obj = await req.formData()
    const res = {} as any
    obj.forEach((item,key) => {
        res[key] = item
        if (item instanceof Blob) {
            res[key] = item.size
        }
    })  
    // const form = new multiparty.Form();
    // form.parse(req)
    // const obj = await req.text()
    return result(200, res)
}
Bartie answered 21/5, 2024 at 16:11 Comment(1)
As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.Boggess
B
0

Here's one more good solution w/o using any external packages. https://vancelucas.com/blog/how-to-access-raw-body-data-with-next-js/

That works for me.

Client-side:

const formData = new FormData();
const { file, ...restValues } = values;
formData.append('Document', file);

const response = await fetch(
  '/api/upload/',
  {
    method: 'POST',
    body: formData,
  },
);

if (response.ok) {
  console.log('success!');
}

Server-side:

export const config = {
  api: {
    bodyParser: false,
  },
};

async function getRawBody(readable) {
  const chunks = [];
  for await (const chunk of readable) {
    chunks.push(typeof chunk === 'string' ? Buffer.from(chunk) : chunk);
  }
  return Buffer.concat(chunks);
}

export default async function handler(req, res) {
  const { headers, method } = req;

  switch (method) {
  case 'POST':

    const rawBody = await getRawBody(req);
    const data = Buffer.from(rawBody).toString('utf8');

    const formData = new FormData();
    formData.append('file', data);

    const response = await fetch(
      `https://your-API.com`, {
        method: 'POST',
        headers,
        body: formData,
      },
    );

    response.ok
      ? res.status(201).json({ message: 'Uploaded!' })
      : res.status(response.status).json({ message: response.statusText });
    break;
  default:
    res.setHeader('Allow', ['POST']);
    res.status(405).end(`Method ${method} Not Allowed`);
  }
}
Bassist answered 20/6, 2024 at 8:32 Comment(0)
D
-3

Serverless function servers on the vercel service need a configuration file (vercel.json) with routes to be able to work, if you need to see how routes works : https://vercel.com/docs/configuration#project/routes

The config file will help you redirect your incoming post request to your js file. ex:

{
  "routes" : [
    {"src":"/api/post-submission", "methods": ["POST"], "dest": "<js file>"}
  ]
}
Deloisedelong answered 22/7, 2020 at 1:17 Comment(2)
I think this advice might be outdated, as the routes work automatically if you name them like this. The issue was in fact that Vercel serverless functions don't support multi-part dataNammu
@Jack, so how to post to serverless fns in vercel? How even this serverless business is viable if you can't post form data to the server?Thecla

© 2022 - 2025 — McMap. All rights reserved.