How can you resize an image in NodeJS using Sharp having only a URL, using async/await, and without a local copy being created?

Question

I'm working in an environment where the available image processing library is NodeJS's Sharp for scaling images. It has been stable as it is being pipe based, but I'm tasked with converting it into TypeScript, and set it up using Async/Await when possible. I have most of the pieces ready, but the issue I am facing lies in the fact that all I have is a URL of an image and Sharp either expects a string URI (local file only) or a Buffer.

Currently, I am using the package Axios in order to fetch the image as a string retrievable by the data property on the response. I've been feeding a buffer created from the string by Buffer.from(response.data) into Sharp, and it doesn't have any issues until I try and "work" with the image by attempting to gather the metadata. At this point it throws an error: [Error: Input buffer contains unsupported image format]. But I know that the image is valid, as it worked in the old system, and I didn't change any dependencies.

I use QuokkaJS to test, and the following PoC fails, and I need to get it to functioning order.

import axios from 'axios';
import Sharp from 'sharp';
const url = 'https://dqktdb1dhykn6.cloudfront.net/357882-TLRKytH3h.jpg';

const imageResponse = await axios({url: url, responseType: 'stream'});
const buffer = Buffer.from(imageResponse.data);
let src = new Sharp(buffer);
const src2 = src.clone();//this is simply because it will end up being a loop, if this is the issue let me know.
try {
    await src2.jpeg();
    await src2.resize(null, 1920);
    await src2.resize(1080, null);
    const metadata = await src2.clone().metadata();//this is where it fails
    console.log(metadata);
} catch(e) {
    console.log(e);//logs the mentioned error
}

If anybody has any idea what I am doing incorrectly, or has anything specific information that they would like me to add, please let me know! If I need to pipe the image data, let me know. I've tried to directly pipe it getting a pipe is not a function on the string (which makes sense).

Update #1:

A big thank you to @Thee_Sritabtim for the comment, which solved the issue. Basically, I had been trying to convert a Stream based String into a Buffer. I needed to instead declare that the request was for an ArrayBuffer, and then feed it into Sharp while declaring its type of binary. The working example of the PoC is below!

import axios from 'axios';
import Sharp from 'sharp';
const url = 'https://dqktdb1dhykn6.cloudfront.net/357882-TLRKytH3h.jpg';

const imageResponse = await axios({url: url, responseType: 'arraybuffer'});
const buffer = Buffer.from(imageResponse.data, 'binary');
let src = new Sharp(buffer);
try {
    await src.jpeg();
    await src.resize(null, 1920);
    await src.resize(1080, null);
    const metadata = await src.metadata();//this was where it failed, but now it prints an object of metadata
    console.log(metadata);
} catch(e) {
    console.log(e);//Doesn't catch anything any more!
}

Jerryjerrybuild

sharp also accepts stream. And I don't think you can get a buffer from stream like this Buffer.from(respose.data). Try responseType: 'arraybuffer' and Buffer.from(imageResponse.data, 'binary') instead – Manos

This was exactly the issue. It is fixed by those 2 small changes! Feel free to add an answer so that I can select it, but either way, thank you, @TheeSritabtim! – Jerryjerrybuild

I've added an answer with an alternative method using stream – Manos

Answer 1

To get a buffer from a axios response, you'll have to set responseType to 'arraybuffer'.

const imageResponse = await axios({url: url, responseType: 'arraybuffer'})
const buffer = Buffer.from(imageResponse.data, 'binary')

Alternatively,

You could also use stream as input for sharp(), so you could keep the responseType to 'stream'

const imageResponse = await axios({url: url, responseType: 'stream'})

const src = imageResponse.data.pipe(sharp())
//...
const metadata = await src.metadata()

Manos

I went the 'arraybuffer` route. I was hoping to avoid piping already, so unless there's a noticeable performance penalty, I'm going to stick to the first option. Thank you again, @TheeSritabtim! – Jerryjerrybuild