Node.js - Pipe a file stream from one server to another

Question

I'm trying to use Node.js to get a file from a remote URL then send it to another server (using an API provided by each of the two websites). I already managed to successfully upload a local file to the remote server using fs.createReadStream("file.png"). Remote files seem to be a different story however: I can't simply put "https://website.com/file.png" in there, I need an equivalent for createReadStream for remote files.

Obviously I could use a separate command to download the file locally and upload it using createReadStream then delete the local file, but I want my code to be efficient and not rely on manually downloading temporary files, plus this is a good learning experience. I'd thus like to know the simplest way to pipe files as streams between two different servers.

Also I would like to avoid using extra dependencies if possible, as I'm writing a simple script which I'd rather not make reliant on too many npm packages. I rely on require("https") and require("fs") primarily. I'm curious if this can be achieved through a simple https.get() call.

Stepha

Why do you think you need an createReadStream? Your HTTP body should already be a stream which can be used the same way as a stream created from createReadStream. – Alejandrinaalejandro

I didn't quite understand how that works. So if I use https.get am I able to pipe in the response.on("data") object? I thought I need to use a special function to download and format it as stream. If it's as simple as that then it's great! I'll test this soon. – Stepha

I can't currently test it and I haven't worked with node's http(s) packages but I'm pretty sure the response object you get passed in your callback is a stream (or behaves like one). – Alejandrinaalejandro

I found a way to use https.get to receive the file data into a local buffer. The problem I'm facing now is that I can't convert it to a ReadStream, which is what the API of the target expects to receive. Using console.log on the result of createReadStream with the local file, I found that the object needs to have the following format: ReadStream { _readableState: ReadableState { ... } } Like I said, fs.createReadStream generates exactly that kind of object. But it does it from a local file. What I need now is a way to do it from a buffer object or binary string in a variable. – Stepha

See this post #32139248 – Jillian