Error using Using DocArrayInMemorySearch in Langchain: Could not import docarray python package

Question

Here is the full code. It runs perfectly fine on https://learn.deeplearning.ai/ notebook. But when I run it on my local machine, I get an error about

ImportError: Could not import docarray python package

I have tried reinstalling/force installing langchain and lanchain[docarray] (both pip and pip3). I use mini conda virtual environment. python version 3.11.4

from langchain.vectorstores import DocArrayInMemorySearch
from langchain.schema import Document
from langchain.indexes import VectorstoreIndexCreator
import openai
import os

os.environ['OPENAI_API_KEY'] =  "xxxxxx" #not needed in DLAI

docs = [
    Document(
        page_content="""[{"API_Name":"get_invoice_transactions","API_Description":"This API when called will provide the list of transactions","API_Inputs":[],"API_Outputs":[]}]"""
    ),
    Document(
        page_content="""[{"API_Name":"get_invoice_summary_year","API_Description":"this api summarizes the invoices by vendor, product and year","API_Inputs":[{"API_Input":"Year","API_Input_Type":"Text"}],"API_Outputs":[{"API_Output":"Purchase Volume","API_Output_Type":"Float"},{"API_Output":"Vendor Name","API_Output_Type":"Text"},{"API_Output":"Year","API_Output_Type":"Text"},{"API_Output":"Item","API_Output_Type":"Text"}]}]"""
    ),
    Document(
        page_content="""[{"API_Name":"loan_payment","API_Description":"This API calculates the monthly payment for a loan","API_Inputs":[{"API_Input":"Loan_Amount","API_Input_Type":"Float"},{"API_Input":"Interest_Rate","API_Input_Type":"Float"},{"API_Input":"Loan_Term","API_Input_Type":"Integer"}],"API_Outputs":[{"API_Output":"Monthly_Payment","API_Output_Type":"Float"},{"API_Output":"Total_Interest","API_Output_Type":"Float"}]}]"""
    ),
    Document(
        page_content="""[{"API_Name":"image_processing","API_Description":"This API processes an image and applies specified filters","API_Inputs":[{"API_Input":"Image_URL","API_Input_Type":"URL"},{"API_Input":"Filters","API_Input_Type":"List"}],"API_Outputs":[{"API_Output":"Processed_Image_URL","API_Output_Type":"URL"}]}]"""
    ),
    Document(
        page_content="""[{"API_Name":"movies_catalog","API_Description":"This API provides a catalog of movies based on user preferences","API_Inputs":[{"API_Input":"Genre","API_Input_Type":"Text"},{"API_Input":"Release_Year","API_Input_Type":"Integer"}],"API_Outputs":[{"API_Output":"Movie_Title","API_Output_Type":"Text"},{"API_Output":"Genre","API_Output_Type":"Text"},{"API_Output":"Release_Year","API_Output_Type":"Integer"},{"API_Output":"Rating","API_Output_Type":"Float"}]}]"""
    ),
    # Add more documents here   
]

index = VectorstoreIndexCreator(
        vectorstore_cls=DocArrayInMemorySearch
        ).from_documents(docs)

api_desc = "do analytics about movies"
query = f"Search for related APIs based on following API Description: {api_desc}\
        Return list of API page_contents as JSON objects."


print(index.query(query))
 

Here is the error:

(streamlit) C02Z8202LVDQ:sage_response praneeth.gadam$ /Users/praneeth.gadam/opt/miniconda3/envs/streamlit/bin/python /Users/praneeth.gadam/sage_response/docsearch_copy.py Traceback (most recent call last):   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/vectorstores/docarray/base.py", line 19, in _check_docarray_import
    import docarray ModuleNotFoundError: No module named 'docarray'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):   File "/Users/praneeth.gadam/sage_response/docsearch_copy.py", line 30, in <module>
    ).from_documents(docs)
      ^^^^^^^^^^^^^^^^^^^^   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/indexes/vectorstore.py", line 88, in from_documents
    vectorstore = self.vectorstore_cls.from_documents(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/vectorstores/base.py", line 420, in from_documents
    return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/vectorstores/docarray/in_memory.py", line 67, in from_texts
    store = cls.from_params(embedding, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/vectorstores/docarray/in_memory.py", line 38, in from_params
    _check_docarray_import()   File "/Users/praneeth.gadam/opt/miniconda3/envs/streamlit/lib/python3.11/site-packages/langchain/vectorstores/docarray/base.py", line 29, in _check_docarray_import
    raise ImportError( ImportError: Could not import docarray python package. Please install it with `pip install "langchain[docarray]"`.

Nonu

Answer 1

Try to install this way: pip install docarray

Morrissey

I could have sweared I tried this a 100 times before. Not sure why, it worked this time, but I used pip3 instead of pip. Other commands I tried conda install -c conda-forge "langchain[docarray]" export HNSWLIB_NO_NATIVE=1 pip3 install HNSWLIB – Nonu

Answer 2

I run:

  pip3  install -U docarray

then I got:

    ValidationError: 2 validation errors for DocArrayDoc
    text
      Field required [type=missing, input_value={'embedding': [-0.0044996... -0.040091351353726185]}, input_type=dict]

then I run

pip3  install pydantic==1.10.9

problem solved

Unassuming

pip install "langchain[docarray]" and pydantic==1.10.9 worked for me – Planter

Answer 3

I had the same issue, try to upgrade docarray using pip install -U docarray.

Nyeman

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Manvel

ZJ Q's answer solved the problem for me. – Zielinski

Answer 4

I've also had an issue with DocArray:

When debugging I realized import docarray did not work, and an error related to pydantic was the issue. In the end, solved it by downgrading pydantic to version 1.10.9 (I had a much more recent version of pydantic initially)
pip install pydantic==1.10.9

More about pydantic and Langchain compatibility: https://python.langchain.com/docs/guides/pydantic_compatibility

Mudguard

This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review – Disenthrone

Answer 5

This is an compatibility issue between docarray and pydantic libraries, when utilized by langchain.

Here some related resource from the official langchain docs.

For me to make all three libraries to work together i had to use the following libraries versions in a Python 3.10 venv:

langchain==0.2.3
langchain-community==0.2.4
langchain-core==0.2.5
docarray==0.40.0
pydantic==2.7.4
pydantic_core==2.18.4

Ideally langchain locks compatible libraries to prevent any issues on future releases.

Congratulant