OSError: [E050] Can't find model 'en_core_web_trf'. It doesn't seem to be a Python package or a valid path to a data directory

Question

I am trying to deploy an app in heroku, and it completes successfully, but when I click to view the app it shows this error in a red box!

OSError: [E050] Can't find model 'en_core_web_trf'. It doesn't seem to be a Python package or a valid path to a data directory.

Here is my code:

import spacy_streamlit
import streamlit as st
import pandas as pd
from spacy_transformers import Transformer
from spacy_transformers.pipeline_component import DEFAULT_CONFIG

DEFAULT_TEXT = """Google was founded in September 1998 by Larry Page and Sergey Brin while they were Ph.D. students at Stanford University in California. Together they own about 14 percent of its shares and control 56 percent of the stockholder voting power through supervoting stock. They incorporated Google as a California privately held company on September 4, 1998, in California. Google was then reincorporated in Delaware on October 22, 2002."""

spacy_model = "en_core_web_trf"

st.title("Identifyer")
text = st.text_area("Text to analyze", DEFAULT_TEXT, height=200)
doc = spacy_streamlit.process_text(spacy_model, text)

spacy_streamlit.visualize_ner(
    doc,
    labels=["CARDINAL", "DATE", "EVENT", "FAC", "GPE", "LANGUAGE", "LAW", "LOC", "MONEY", "NORP", "ORDINAL", "ORG", "PERCENT", "PERSON", "PRODUCT", "QUANTITY", "TIME", "WORK_OF_ART"],
    show_table=False,
    title="filter",
)

df = pd.DataFrame(pd.DataFrame({
'type': ["ORG", "DATE", "EVENT", "FAC", "GPE", "LOC", "MONEY", "NORP", "PERCENT", "PERSON", "PRODUCT", "QUANTITY", "TIME", "WORK_OF_ART", "LANGUAGE", "LAW", "ORDINAL", "CARDINAL"],
'meaning': ["Companies, agencies, institutions, etc.", "Absolute or relative dates or periods", "Named hurricanes, battles, wars, sports events, etc.", "Buildings, airports, highways, bridges, etc.", "Countries, cities, states", "Non-GPE locations, mountain ranges, bodies of water", "Monetary values, including unit", "Nationalities or religious or political groups", "Percentage (including “%”)", "People, including fictional", "Vehicles, weapons, foods, etc. (Not services)", "Measurements, as of weight or distance", "Times smaller than a day", "Titles of books, songs, etc.", "Any named language", "Named documents made into laws", "first”, “second”, ...", "Numerals that do not fall under another type"],
}))

df.index = [""] * len(df)
st.table(df)

Eruptive

And have you verified that the model file (or directory) is available to your heroku instance? – Panslavism

thans for the reply! but how i see that? – Eruptive

There are instruction on the Spacy website that say how to install the models. spacy.io/models/en – Panslavism

i've installed the models! i tried everything! – Eruptive

Can you paste the output of python -m spacy validate, run in the exact same environment that the app is running in? – Sexagesimal

didn´t understand very well! can you detail more pls? – Eruptive

Answer 1

Run this command from your IDE:

!python -m spacy download en_core_web_trf

Then do restart runtime from the menu bar if available:

Now, it will execute successfully:

Chappy

Answer 2

I created several virtual enviroments and I noticed, that in last two of them model 'en_core_web_trf' was not installed properly with this command: python3 -m spacy download en_core_web_trf , so I got the same error. I don't know why, but when i tried python instead of python3, it worked: python -m spacy download en_core_web_trf. If it is ok, you will see this text:

✔ Download and installation successful You can now load the package via spacy.load('en_core_web_trf')

Another interesting point is that I was unable to install model in one virtual enviroment while my application was running in another virtual enviroment.

I would be very grateful if someone could explain to me why is it happening, and why python installs model for spacy properly, but python3 doesn't. (I use Kubuntu linux)

Upd:
It looks like you have no model, or it installed incorrectly. Try to check your model's version with pip show en_core_web_trf if you are using virtual environment, or reinstall spacy and model.

Cati

thanks for the suggestion, but didn´t work though... Is it necessary to do spacy.load('en_core_web_trf')? my code doesn´t have that part – Eruptive

On many Debian-like systems (which I guess would include Kubuntu), python is python2 and is completely different from python3. You can check with python --version. – Backed

@Backed I checked, python and python3 on my system is equal to python version 3.8.5 – Cati