How to achieve text streaming in react native using openai api?

Question

I am working on a react native chat app, and I would like to use openai api to implement auto replying.
There is stream available from openai's api, but it uses SSE, which seems not working well in react native.
Is there any way that I can use text streaming in my project?

I have tried most of the advices mentioned here and other solutions, but they were not working as expected.

1. External libraries: I have successfully implemented streaming in React (using Next.js) with libraries like openai-streams, react-chat-stream, openai-streaming-hooks .etc . However, these libraries do not work in react native.
2. Fetch for Streaming: someone mentioned fetch is available for streaming, I tried it, along with libraries like @async-util/fetch, react-native-fetch-api too. But they weren't working too.
3. React Native WebView: it can make requests and send the response back to the app, inspired by this repo, but I think it is not feasible for long term usage as there might be more and more functions coming to the app in the future, I think it will be difficult to maintain as using this method requires injecting js code.
4. SSE Packages: i tried packages such as react-native-event-source, react-native-sse. However, I'm still encountering issues where the functionality is not working as expected. I'm unsure if the problem lies with my implementation or with the packages.

I am so confused right now. Is there any guidance or working examples to achieve the streaming response?
Here are two proposed solutions that I think is the most appropriate and will keep trying, please tell me if there are any alternative approaches that I should consider or improvements to those methods, I would greatly appreciate the advice.

• implement SSE, but I am relatively new to it, so it would be great if there are some specific boilerplate code or examples provided.
• use a server (like axios?) to receive the stream response and return back to the app, but I am not sure whether the server should be written in the app itself or outside the app. Also, as I am using Expo, I noticed the upcoming ExpoSDK include api routes, does that mean I can write the request in the app later?

Thank you for your time.

---

Respond to zoran404:
Yes, I have tried. The react-native-polyfill-globals/auto is added because without it will raise error Property 'TextEncoder' doesn't exist. Here are my approach and I am really not sure about it.

import 'react-native-polyfill-globals/auto'
import { fetch } from 'react-native-fetch-api'

async function fetchStreamMessage() {
    const url = 'https://api.openai.com/v1/chat/completions'
    fetch(url, {
      reactNative: { textStreaming: true },
      method: 'POST',
      headers: {
        Authorization: `Bearer ${OPENAI_KEY}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        messages: [{ role: 'user', content: 'Tell me a joke' }],
        model: 'gpt-4',
        stream: true
      })
    })
      .then((response: Response) => response.body)
      .then((body: ReadableStream) => {
        const reader = body.getReader()
        console.log(reader)
      })
  }

Sunbow

It can work with react-native. Have you tried the code from this answer? https://mcmap.net/q/1355838/-is-there-any-way-to-stream-response-word-by-word-of-chatgpt-api-directly-in-react-native-with-javascript – Honey

The comment is too long so I placed it in the question. Please give me some insights, thank you! – Sunbow

Hi @WanDur, I am encountering the same issue, did you ever find a way to do streaming in RN? – Yuri

@Yuri check my answer – Sunbow

Answer 1

Just an update on my question, I have found the way to get the stream message and it seems to be the most stable for now, just by using react-native-sse. I am using expo sdk 51 for your reference.

import EventSource from 'react-native-sse'

const [messages, setMessages] = useState<Message[]>([])

const getCompletion = async (message: string) => {
    const newMessages: Message[] = [
      ...messages,
      {
        role: Role.User,
        content: message
      },
      {
        role: Role.Bot,
        content: ''
      }
    ]

    setMessages(newMessages)
    
    // send request here
    const payload = {
      model: 'gpt-3.5-turbo',
      stream: true,
      messages: [{ role: 'user', content: message }]
    }

    const es = new EventSource(API_URL, {
      headers: {
        Authorization: `Bearer ${API_KEY}`,
        'Content-Type': 'application/json'
      },
      method: 'POST',
      body: JSON.stringify(payload),
      pollingInterval: 25000
    })

    const listener = (event: any) => {
      if (event.type === 'open') {
        console.log('Open SSE connection.')
      } else if (event.type === 'message') {
        if (event.data !== '[DONE]') {
          const data = JSON.parse(event.data)
          const delta = data.choices[0].delta
          const finishReason = data.choices[0].finish_reason

          if (finishReason === 'stop') {
            es.close()
          } else {
            if (delta?.content) {
              setMessages((previousMessages) => {
                const updatedMessages = [...previousMessages]
                // here is where the stream result is appended to messages
                updatedMessages[updatedMessages.length - 1].content += delta.content 

                return updatedMessages
              })
            }
          }
        } else {
          console.log('Done. SSE connection closed.')
          es.close()
        }
      } else if (event.type === 'error') {
        console.error('Connection error:', event.message)
      } else if (event.type === 'exception') {
        console.error('Error:', event.message, event.error)
      }
    }

    es.addEventListener('open', listener)
    es.addEventListener('message', listener)
    es.addEventListener('error', listener)

    return () => {
      es.removeAllEventListeners()
      es.close()
    }
  }

Then just use messages to render the text.
You can try it out first, but there's a small problem when a lot of messages are streamed from chatgpt, like generating a story, it seems the UI is blocked so you couldn't press when generating response, it will back to normal when the stream is done.
I am currently working on it, and may update later. Anyone with a solution is welcomed!!!

Sunbow

Answer 2

Have you tried looking into ai SDK by Vercel? It has complete support for handling all kinds of streams from different sources. Apart from this, you can follow this discussion for clarity on how to use this in react native.

Micrometeorite

Answer 3

So as of Fall 2024, ReactNative doesn't support ReadableStream. So when we are making a request where the response is ReadableStream we encounter errors such as

• ReferenceError: Property 'ReadableStream' doesn't exist
• Reference Error: Property 'Text Encoder' doesn't exist
• ...

The solution is to polyfill globally all the missing functionality. As stated in other answers you may use react-native-polyfill-globals/auto

Note that it requires web-streams-polyfill v3.3.3 because in version v4, web-streams-polyfill changed the structure of the module.

Note that for polyfills to correctly work you need to add to the correct place. In bare ReactNative projects, it seems to be index.js just before the AppRegistry call.

For Expo you need to:

Create an index.js file and make your polyfill the first import:
import 'polyfill'
import 'expo-router/entry'

Then change the main field in the package.json to point to the "main": "./index" as stated here

Also here is an another example on how I polyfilled fetch and ReadableStream https://mcmap.net/q/1190371/-stream-api-with-fetch-in-a-react-native-app

If you receive Text Encoder doesn't exist

Try to also polyfill TextEncoder. Or use directly. I used import encoding from "text-encoding"; And then const decoder = new encoding.TextDecoder("utf-8"); to decode values from my response.body.getReader().read()

Matrona

Answer 4

Once you have the reader from body.getReader(), you need a loop to read the stream chunks. After const reader = body.getReader()

            const reader = body.getReader();
            let done, value;
            while (!done) {
                ({ done, value } = await reader.read());
                if (!done) {
                    const chunk = new TextDecoder().decode(value);
                    console.log(chunk);
                }
            }
            console.log('Streaming complete.');

Phobos