OSError: [Errno 28] inotify watch limit reached

Question

I am making a python-based web app using Streamlit. After deploying it in Heroku, the build succeeds but there is an application error. I don't have any idea where in the source code this error is being generated. Please help me! The error :

2022-07-18T18:55:37.985429+00:00 app[web.1]:     Inotify._raise_error()
2022-07-18T18:55:37.985439+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/watchdog/observers/inotify_c.py", line 398, in _raise_error
2022-07-18T18:55:37.985636+00:00 app[web.1]:     raise OSError(errno.ENOSPC, "inotify watch limit reached")
2022-07-18T18:55:37.985677+00:00 app[web.1]: OSError: [Errno 28] inotify watch limit reached
2022-07-18T18:55:38.387667+00:00 heroku[web.1]: Process exited with status 1
2022-07-18T18:55:38.510041+00:00 heroku[web.1]: State changed from starting to crashed
2022-07-18T18:55:48.000000+00:00 app[api]: Build succeeded
2022-07-18T18:57:33.589417+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=invezto.herokuapp.com request_id=bc8f4556-852e-4dad-8b67-71e49ffaaf23 fwd="49.37.45.19" dyno= connect= service= status=503 bytes= protocol=https
2022-07-18T18:57:33.917128+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=invezto.herokuapp.com request_id=46e2e615-17dc-42f6-a86d-4dfc5fd5ecfc fwd="49.37.45.19" dyno= connect= service= status=503 bytes= protocol=https ``` 

 

Yokefellow

Answer 1

Adding option --server.fileWatcherType none at the command line helped me to resolve similar issue. A full example would look like this:

streamlit run app.py --server.fileWatcherType none

More solutions here

Fluorite

Note that my answer, although not the popular vote, keeps file watching turned on, a nice feature for debugging live. – Pless

Answer 2

Streamlit recursively monitors changes in the parent folder of the application script, e.g. app.py [source].

The idea is that if source files update, the app is 'rerun', as a convenience for app development. However, if this folder contains too many subfolders and subfiles, apparently watchdog reaches a limit and raises an error. This easily happens if you run straight from your home directory, which contains tons of configuration files, etc. Or maybe when you run from a git repository with numerous files in .git (just speculating).

You can resolve this by either placing the app.py script in a (sub)folder than only contains dependencies that you'll edit during development, or by --server.fileWatcherType none and accept the nuisance of manually restarting.

streamlit run app.py --server.fileWatcherType none

(as suggested by @swwt)

Pless

+1 for pointing out the consequences of setting this to none. When deploying a Streamlit app, it's perfectly fine to disable it then. – Soapberry

In my case is a 260KB git repository, and my code has 100 lines max. I highly doubt is a size issue right? Unless importing libraries like pandas or numpy creates the issue. – Lynnett

The issue is in the number of files, not their size, since inotify needs to watch them all. A .git folder can easily contain many files. – Pless

Answer 3

This problem can be solved permanently by increasing the limit by modifying the fs.inotify.max_user_watches kernel parameter.

1. Open a terminal.
2. Check the current limit with the following command:

cat /proc/sys/fs/inotify/max_user_watches

3. If you want to increase the limit to, say, 524288, you can do so with the following command:

echo 524288 | sudo tee /proc/sys/fs/inotify/max_user_watches

4. To make the change permanent, you can add the following line to your /etc/sysctl.conf file:

fs.inotify.max_user_watches=524288

5. Then load the new configuration with the command:

sudo sysctl -p

Please replace 524288 with a number that suits your needs. Be aware that increasing the limit will consume more kernel memory. So, choose a number that is suitable for your system resources.

Palmetto

Answer 4

This actually happened to me due to "opening folders" in Visual studio Code. Running on linux. When I started vscode with other streamlit python files open. Then started/Opened a folder project Python/Django from another directory within a larger code directory/repository. However it started adding all the files in the root folder to the watch list.

There are thousands of files in the root from other projects.

~notebooks/work/dev ~notebooks/personal/dev

Each with different projects in them.

The new django projects was ~notebooks/work/dev/djangoproject

Vscode was watching everything all the way down to ~notebooks

Streamlit would then complain. Something akin to too many open file handles.

It took a day or so to realize the culprit. I closed out the folder, and just opened the files from the other streamlit projects and it was fine from then on.

Resolve