NodeJS, Axios - post file from local server to another server

Question

I have an API endpoint that lets the client post their csv to our server then post it to someone else server. I have done our server part which save uploaded file to our server, but I can't get the other part done. I keep getting error { message: 'File not found', code: 400 } which may mean the file never reach the server. I'm using axios as an agent, does anyone know how to get this done? Thanks.

// file = uploaded file
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
    method: "post",
    url: url,
    headers: {
        "Authorization": "Bearer " + access_token,
        "Content-Type": "multipart/form-data"
    },
    data: form_data
};
return axios(request_config);

Update

As axios doc states as below and the API I'm trying to call requires a file

// data is the data to be sent as the request body // Only applicable for request methods 'PUT', 'POST', and 'PATCH' // When no transformRequest is set, must be of one of the following types: // - string, plain object, ArrayBuffer, ArrayBufferView, URLSearchParams // - Browser only: FormData, File, Blob // - Node only: Stream, Buffer

Is there any way to make axios send a file as a whole? Thanks.

Pamulapan

npmjs.com/package/axios#using-multipartform-data-format – Nazi

Answer 1

I'm thinking the createReadStream is your issue because its async. try this. Since createReadStream extends the event emitter, we can "listen" for when it finishes/ends.

var newFile = fs.createReadStream(file.path);

// personally I'd function out the inner body here and just call 
// to the function and pass in the newFile
newFile.on('end', function() {
  const form_data = new FormData();
  form_data.append("file", newFile, "filename.ext");
  const request_config = {
    method: "post",
    url: url,
    headers: {
        "Authorization": "Bearer " + access_token,
        "Content-Type": "multipart/form-data"
    },
    data: form_data
  };
  return axios(request_config);
});

Kappenne

Hi James. Your solution does work. However, does createReadStream read the entire file into memory, if yes, I might encounter memory problem for massive files. – Pamulapan

So, with the Request module (import it), you could do that easily like so: newFile.pipe(request(request_config)) . I think axios might be able to do it too. But the the idea is, stream in the file, and pipe it to the request/axios to transfer. – Kappenne

@jamesemanon have you any code snippet to pipe the file stream for axios upload? if so, please post here – Steelman

How to do it in case one wants to upload multiple files ? @jamesemanon – Thirtythree

what's the solution if we have used multer – Wincer

Hi! Thanks for the answer! In typescript I get an error: Argument of type 'ReadStream' is not assignable to parameter of type 'string | Blob'. Type 'ReadStream' is missing the following properties from type 'Blob': size, type, arrayBuffer, slice, and 2 more. Any ideas? – Solder

For typescript add import FormData from "form-data"; – Auriga

Answer 2

The 2 oldest answers did not work for me. This, however, did the trick:

const FormData = require('form-data'); // npm install --save form-data

const form = new FormData();
form.append('file', fs.createReadStream(file.path));

const request_config = {
  headers: {
    'Authorization': `Bearer ${access_token}`,
    ...form.getHeaders()
  }
};

return axios.post(url, form, request_config);

form.getHeaders() returns an Object with the content-type as well as the boundary.
For example:

{ "content-type": "multipart/form-data; boundary=-------------------0123456789" }

Pigfish

This was very helpful. Thanks a lot! Here is another useful example showing how to mimic file upload without using a filesystem: github.com/axios/axios/issues/1006#issuecomment-320165427. – Uncomfortable

Works like a charm. This is very helpful. Non of above solutions worked. – Diderot

Hi! Thanks for the answer! In typescript I get an error: Argument of type 'ReadStream' is not assignable to parameter of type 'string | Blob'. Type 'ReadStream' is missing the following properties from type 'Blob': size, type, arrayBuffer, slice, and 2 more. Any ideas? – Solder

What if I have a file buffer instead of a file path? – Nucleo

@GabrielAugusto use import FormData from "form-data"; – Utas

this answer helped in in 2023 – Whatley

Instead of a file from the filesystem, I wanted to send the multer uploaded file right away. What worked for me was to convert my buffer into a Blob like this: form.append("file", new Blob([file.buffer], { type: file.mimetype }));. Took me a while to figure this one out, hope this helps someone else. – Indignant

Answer 3

This is what you really need:

const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));

const request_config = {
  headers: {
    "Authorization": "Bearer " + access_token,
    "Content-Type": "multipart/form-data"
  },
  data: form_data
};

return axios
  .post(url, form_data, request_config);

Armipotent

It seems there is a little mistake in your sample: you miss the filename... The right code is ` form_data.append("file", fs.createReadStream(file.path), "filename.ext"); – Martinson

Answer 4

In my case, fs.createReadStream(file.path) did not work.
I had to use buffer instead.

const form = new FormData();
form.append('file', fs.readFileSync(filePath), fileName);

const config = {
  headers: {
    Authorization: `Bearer ${auth.access_token}`,
    ...form.getHeaders(),
  },
};

axios.post(api, form.getBuffer(), config);

Madagascar

Answer 5

I have made an interceptor you can connect to axios to handle this case in node: axios-form-data. Any feedback would be welcome.

• npm i axios-form-data
• example:

import axiosFormData from 'axios-form-data';
import axios from 'axios';

// connect axiosFormData interceptor to axios
axios.interceptors.request.use(axiosFormData);

// send request with a file in it, it automatically becomes form-data
const response = await axios.request({
  method: 'POST',
  url: 'http://httpbin.org/post',
  data: {
    nonfile: 'Non-file value',
    // if there is at least one streamable value, the interceptor wraps the data into FormData
    file: createReadStream('somefile'),
  },
});

// response should show "files" with file content, "form" with other values
// and multipart/form-data with random boundary as request header
console.log(response.data);

Bucharest

Answer 6

For anyone who wants to upload files from their local filesystem (actually from anywhere with the right streams architecture) with axios and doesn't want to use any external packages (like form-data).

Just create a readable stream and plug it right into axios request function like so:

await axios.put(
  url,
  fs.createReadStream(path_to_file)
) 

Axios accepts data argument of type Stream in node context.

Works fine for me at least in Node v.16.13.1 and with axios v.0.27.2

Wryneck

Answer 7

I had a same issue, I had a "pdf-creator-service" for generate PDF document from html.

I use mustache template engine for create HTML document - https://www.npmjs.com/package/mustache Mustache.render function returns html as a string what do I need to do to pass it to the pdf-generator-service ? So lets see my suggestion bellow

//...

async function getPdfDoc(props: {foo: string, bar: string}): Promise<Buffer> {
    const temlateFile = readFileSync(joinPath(process.cwd(), 'file.html'))

    mustache.render(temlateFile, props)
    const readableStream = this.getReadableStreamFromString(htmlString)

    const formData = new FormData() // from 'form-data'
    formData.append('file', options.file, { filename: options.fileName })
    const formHeaders = formData.getHeaders()

    return await axios.send<Buffer>(
      {
        method: 'POST',
        url: 'https://pdf-generator-service-url/pdf',
        data: formData,
        headers: {
          ...formHeaders,
        },
        responseType: 'arraybuffer', // ! important
      },
    )

}

getReadableStreamFromString(str: string): Readable {
    const bufferHtmlString = Buffer.from(str)
    const readableStream = new Readable() // from 'stream'

    readableStream._read = () => null // workaround error
    readableStream.push(bufferHtmlString)
    readableStream.push(null) // mark end of stream

    return readableStream
}

Wagoner

Answer 8

 const file = new Blob([await fs.readFile(filepath)], { type: 'application/pdf' });

        const formData: FormData = new FormData();

        formData.append('param1', '0');
        formData.append('param2', '10');
        formData.append('param3', file, 'fileName');

Haddington