Fetch blob from URL and write to file

Question

I'm trying to fetch some binary data (MP3) from a server and then store it in a file:

var fs = require ('fs');
var fetch = require ('node-fetch');

fetch (
   audioUrl,
   {
      method: 'GET',
      headers: { 'Accept': '*/*' }
   }
)
.then ((res) => res.blob())
.then ((blob) => {
    console.log ('Blob size: ', blob.size);

    fs.writeFile (
        `filename.mp3`,
        blob,  // BUG HERE
        (err) => {
            if (err) {
                console.log (`Error writing audio ${audioIdx}`, err);
            }
        }
    );
})

The problem is marked at BUG HERE. I'm passing a blob to fs, which doesn't understand blobs, and simply writes [object Blob]. I also tried the following:

blob                                // writes [object Blob]
new Uint8Array (blob)               // writes empty file
Buffer.from (new Uint8Array (blob)) // writes empty file
new Buffer (blob, 'binary')   // Error: argument is not a Buffer

None of the above work. How to do this correctly?

Note that I'm logging blob.size before calling writeFile. blob.size shows the correct size, so fetch seems to be successful.

Vassalize

which fetch library are you using? – Hypocycloid

node-fetch. I'll update the question. – Vassalize

Answer 1

Ok, so from the MDN Docs a Blob is not a buffer, which is what you need for writing to the file system.

I think you might be better off just getting the buffer from the response, like:

.then(res => res.buffer())
.then(/* write your file here */)

though depending on how big the files are, your best option might be to skip the .writeFile function altogether and pipe streams instead.

Hypocycloid

I don't get it. There is no buffer() method for HTTP responses. Do you mean arrayBuffer()? – Leveret

Answer 2

You can stream the response to a file using fs.createWriteStream.

fetch(
    audioUrl,
    {
        method: 'GET',
        headers: { 'Accept': '*/*' }
    }
).then(res => {
    new Promise((resolve, reject) => {
        const dest = fs.createWriteStream(`filename.mp3`);
        res.body.pipe(dest);
        res.body.on('end', () => resolve());
        dest.on('error', reject);
    })
});

Brandnew

This is better than .buffer() or .arrayBuffer() (at least for large files) – Provender

Answer 3

Simple and working ( With the use Buffer API in nodejs )

const fetch = require('node-fetch');
const Buffer = require('buffer').Buffer 
const fs = require('fs');
fetch("https://example.com/file.mp3", {
  "headers": {},
  "method": "GET"
}).then(function(resp) {
    return resp.blob();
}).then(async function(blob) {
    var buffer = await blob.arrayBuffer();
    buffer = Buffer.from(buffer)
    fs.createWriteStream('file.mp3').write(buffer);
})

Lanthanide