Is there a way to stream OpenAI (chatGPT) responsse when using firebase cloud functions as a backend? [duplicate]

Question

I'm currently building a chatbot using OpenAI's ChatGPT and Firebase Cloud Functions as the backend. I want to create a real-time chat experience where the responses from ChatGPT are streamed back to the client as they are generated. However, I'm facing some challenges in achieving this.

I've successfully integrated ChatGPT with Firebase Cloud Functions and can make API calls to generate responses. But the problem is that the responses are returned only when the entire response is generated, resulting in a delay before the user receives any output.

Is there a way to stream the responses from ChatGPT in real-time as they are generated, rather than waiting for the complete response? I want the user to see each partial response as soon as it's available.

Here's a simplified version of my current code:

// Firebase Cloud Functions endpoint
exports.chat = functions.https.onRequest(async (req, res) => {
  const { message } = req.body;

  // Make API call to OpenAI ChatGPT
  const response = await openai.complete({
    model: 'gpt-3.5-turbo',
    stream: true,
    messages: [{ role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: message }],
  });

  // Process the response and send it back to the client
  const output = response.data.choices[0].message.content;
  res.send({ message: output });
});

Is there a way to modify this code or use a different approach to achieve the desired real-time streaming of ChatGPT responses?

Any suggestions or insights would be greatly appreciated. Thank you!

Intake

Cloud Functions does not support streaming or chunked responses. The response must be sent in its entirety, then the function terminates. If you want streaming responses, you will need a different backend product. – Guidance

Having the same issue. The only alternative is to stream into a file on some bucket storage and redirect the client to read from there. But the additional I/O step feels like adding extra overhead / more delays. – Sophistry

[Here][1] you can see examples of "onCall" type response and [here][2] another one with "fetch" and "stream". [1]: github.com/klich3/firebase-gpt-chat-completion [2]: github.com/klich3/firebase-gpt-chat-completion-stream – Hennessey