Pulling real time data and update in Streamlit and Asyncio

Question

The goal is to pulling real time data in the background (say every 5 seconds) and pull into the dashboard when needed. Here is my code. It kinda works but two issues I am seeing: 1. if I move st.write("TESTING!") to the end, it will never get executed because of the while loop. Is there a way to improve? I can imagine as the dashboard grows, there will be multiple pages/tables etc.. This won't give much flexibility. 2. The return px line in the async function, I am not very comfortable with it because I got it right via trial and error. Sorry for being such a newbie, but if there are better ways to do it, I would really appreciate.

Thank you!

import asyncio
import streamlit as st
import numpy as np

st.set_page_config(layout="wide")

async def data_generator(test):
    while True:
        with test:
            px = np.random.randn(5, 1)
        await asyncio.sleep(1)
        return px

test = st.empty()
st.write("TESTING!")

with test:
    while True:
        px = asyncio.run(data_generator(test))
        st.write(px[0])

Orange

Have your tried the method here?. – Oecd

ferdy, unfortunately the code in your link only updates in a for loop, using a sleep statement to wait/block the main thread. This method doesn't allow interaction. This implementation allows interactions, but it doesn't address all of @Orange concerns, unfortunately. – Autocatalysis

Answer 1

From my experience, the trick to using asyncio is to create your layout ahead of time, using empty widgets where you need to display async info. The async coroutine would take in these empty slots and fill them out. This should help you create a more complex application.

Then the asyncio.run command can become the last streamlit action taken. Any streamlit commands after this wouldn't be processed, as you have observed.

I would also recommend to arrange any input widgets outside of the async function, during the initial layout, and then send in the widget output for processing. Of course you could draw your input widgets inside the function, but the layout then might become tricky.

If you still want to have your input widgets inside your async function, you'd definitely have to put them outside of the while loop, otherwise you would get duplicated widget error. (You might try to overcome this by creating new widgets all the time, but then the input widgets would be "reset" and interaction isn't achieved, let alone possible memory issue.)

Here's a complete example of what I mean:

import asyncio
import pandas as pd
import plotly.express as px

import streamlit as st
from datetime import datetime


CHOICES = [1, 2, 3]


def main():
    print('\nmain...')

    # layout your app beforehand, with st.empty
    # for the widgets that the async function would populate
    graph = st.empty()
    radio = st.radio('Choose', CHOICES, horizontal=True)
    table = st.empty()
    
    try:
        # async run the draw function, sending in all the
        # widgets it needs to use/populate
        asyncio.run(draw_async(radio, graph, table))
    except Exception as e:
        print(f'error...{type(e)}')
        raise
    finally:    
        # some additional code to handle user clicking stop
        print('finally')
        # this doesn't actually get called, I think :(
        table.write('User clicked stop!')
    
    
async def draw_async(choice, graph, table):
    # must send in all the streamlit widgets that
    # this fn would interact with...

    # this could possibly work, but layout is tricky
    # choice2 = st.radio('Choose 2', CHOICES)

    while True:
        # this would not work because you'd be creating duplicated
        # radio widgets
        # choice3 = st.radio('Choose 3', CHOICES)

        timestamp = datetime.now()
        sec = timestamp.second

        graph_df = pd.DataFrame({
            'x': [0, 1, 2],
            'y': [max(CHOICES), choice, choice*sec/60.0],
            'color': ['max', 'current', 'ticking']
        })

        df = pd.DataFrame({
            'choice': CHOICES,
            'current_choice': len(CHOICES)*[choice],
            'time': len(CHOICES)*[timestamp]
        })

        graph.plotly_chart(px.bar(graph_df, x='x', y='y', color='color'))
        table.dataframe(df)

        _ = await asyncio.sleep(1)


if __name__ == '__main__':
    main()

Autocatalysis

Answer 2

Would something like this work?

import asyncio

import streamlit as st


async def tick(placeholder):
    tick = 0
    while True:
        with placeholder:
            tick += 1
            st.write(tick)
        await asyncio.sleep(1)


async def main():
    st.header("Async")
    placeholder = st.empty()
    await tick(placeholder)


asyncio.run(main())

Mutineer

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Pinder