Compiled slug size: 946.2M is too large (max is 500M) with spacy library

Question

Hi I am trying to deploy flask application to heroku and I am getting error saying slug size is too large .

catalogue==1.0.0
certifi==2019.11.28
chardet==3.0.4
Click==7.0
cymem==2.0.3
Flask==1.1.1
gunicorn==20.0.4
idna==2.8
importlib-metadata==1.4.0
itsdangerous==1.1.0
Jinja2==2.10.3
MarkupSafe==1.1.1
more-itertools==8.1.0
murmurhash==1.0.2
nltk==3.4.5
numpy==1.18.1
plac==1.1.3
preshed==3.0.2
requests==2.22.0
six==1.14.0
spacy==2.0.18
https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.0.0/en_core_web_lg-2.0.0.tar.gz#egg=en_core_web_lg==2.0.0
srsly==1.0.1
textblob==0.15.3
thinc==7.3.1
tqdm==4.41.1
urllib3==1.25.8
wasabi==0.6.0
Werkzeug==0.16.0
zipp==2.0.1

this is my requirement.txt I want spacy's en_core_web_lg model in my app

Robinette

Answer 1

The spacy model documentation shows the en_core_web_lg model is 560MB on its own. Based on what you have shared I would recommend using the smallest spacy model: en_core_web_sm

Bly

Yes, it was the solution I found. But it decreased the quality of the results. The Large model is really more precise. I would recommend, for this case moving from Heroku to Amazon; it's possible to have this running without problems with the same money but better resources. – Pectize