Is there a way to pipe ReadableStream<Uint8Array> into NextApiResponse?

Question

I'd like to display Google Place Photos in my NextJS application. To GET these images from their specified URL an API key is needed, but at the same time, I do not want to expose this API key to the public.

My goal is to implement a NextJS API route that fetches and returns a specified image from Google Places Photos, while also being able to be accessed directly from an image tag like so:

<img src={`/api/photos/${place?.photos[0].photo_reference}`} alt='' />

I found a few different sources online that recommended I pipe the response stream from my google request directly to the response stream of the outgoing response like so:

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  const response = await fetch(
    `https://maps.googleapis.com/maps/api/place/photo
  &photo_reference=${id}
  &key=${process.env.GOOGLE_PLACE_API_KEY}`,
  );

  if (!response.ok) {
    console.log(response);
    res.status(500).end();
    return;
  }

  response.body.pipe(res);
}

however as response.body is a ReadableStream, it does not have a .pipe() function. Instead it has .pipeTo() and .pipeThrough().

I then tried

response.body.pipeTo(res);

however, this also fails to work because res is a NextApiResponse rather than a WritableStream. Although I searched online, I haven't found out how to write to NextApiResponse similarly to WritableStreams.

Finally, I tried manually converting the response into a buffer and writing it to the NextApiResponse like so:

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  const response = await fetch(
    `https://maps.googleapis.com/maps/api/place/photo
  &photo_reference=${id}
  &key=${process.env.GOOGLE_PLACE_API_KEY}`,
  );

  if (!response.ok) {
    console.log(response);
    res.status(500).end();
    return;
  }

  const resBlob = await response.blob();
  const resBufferArray = await resBlob.arrayBuffer();
  const resBuffer = Buffer.from(resBufferArray);

  const fileType = await fileTypeFromBuffer(resBuffer);
  res.setHeader('Content-Type', fileType?.mime ?? 'application/octet-stream');
  res.setHeader('Content-Length', resBuffer.length);
  res.write(resBuffer, 'binary');
  res.end();
}

and while this completes the response, no image is displayed.

How can I directly pass the retrieved google places image from the server to the frontend so that it can be used by a tag?

Analemma

Answer 1

What worked for me was to simply cast the response.body of type ReadableStream<Uint8Array> to a NodeJS.ReadableStream and then use the pipe function.

const readableStream = response.body as unknown as NodeJS.ReadableStream;
readableStream.pipe(res);

Claudiaclaudian

Only thing that worked for me. – Roose

I get "TypeError: readableStream.pipe is not a function". Are you using node-fetch? I'm using Node.js 18's built-in fetch, so perhaps that's the issue. – Wilbertwilborn

Answer 2

One of the comments to the accepted answer points out that it doesn't work for Node.js's built-in fetch (available in Node.js >= 17.5). To make it work:

Using Next.js's "nodejs" runtime

import {Readable} from 'stream';

// omitting handler code for readability

const nodeReadableStream = Readable.from(response.body);
nodeReadableStream.pipe(res);

Using Next.js's "edge" runtime

You can simply do:

return Response(response.body)

Baseborn

I get Argument of type 'ReadableStream<Uint8Array>' is not assignable to parameter of type 'Iterable<any> | AsyncIterable<any>'.ts(2345) when trying this – Formosa

Answer 3

Simply iterating over values of the reader will do the job. This doesn't require any hacks or whatsoever, and is as performant as other solutions.

import { NextApiRequest, NextApiResponse } from "next";

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  const image = await fetch("https://domain-to-your-cool-image")
  res.setHeader("Content-Type", "image/png");

  if (!image.body) {
    res.end()
    return
  }

  const reader = image.body.getReader()
  while (true) {
    const result = await reader.read()
    if (result.done) {
      res.end()
      return
    }

    res.write(result.value)
  }
}

Pindus

Answer 4

It's a little late, but although the typings on response.body are not quite a Node.js stream, it's still compatible enough for use in stream.pipeline which is likely to be a common use case:

import { pipeline } from 'streams/promises';

const res = await fetch('...');
await pipeline(res.body, yourTargetStream);

Any

Answer 5

1. If you're using Vercel you can stream directly from the Edge. You can convert API route to use experimental-edge runtume

// pages/api/png.ts 

import { NextResponse, type NextRequest } from 'next/server'

export const config = {
  runtime: 'experimental-edge',
}

// Streamable content
const RESOURCE_URL =
  'https://mdn.github.io/dom-examples/streams/png-transform-stream/png-logo.png'

export default async function handler(request: NextRequest) {
  const r = await fetch(RESOURCE_URL)

  const reader = r.body.getReader()

  const response = new ReadableStream({
    async start(controller) {
      while (true) {
        const { done, value } = await reader.read()

        // When no more data needs to be consumed, break the reading
        if (done) {
          break
        }

        // Enqueue the next data chunk into our target stream
        controller.enqueue(value)
      }

      // Close the stream
      controller.close()
      reader.releaseLock()
    },
  })

  return new Response(response)
}

2. Use next.js rewrites in the config (by paths or regex to pass only that id)
3. Use middleware to match those url to handle rewrite, no need to transfer and stream

Abject