Llama_index unexpected keyword argument error on ChatGPT Model Python

Question

I'm testing a couple of the widely published GPT models just trying to get my feet wet and I am running into an error that I cannot solve.

I am running this code:

from llama_index import SimpleDirectoryReader, GPTListIndex, GPTSimpleVectorIndex, LLMPredictor, PromptHelper
from langchain import OpenAI
import gradio as gr
import sys
import os

os.environ["OPENAI_API_KEY"] = 'MYKEY'

def construct_index(directory_path):
    max_input_size = 4096
    num_outputs = 512
    max_chunk_overlap = 20
    chunk_size_limit = 600

    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    llm_predictor_gpt = LLMPredictor(llm=OpenAI(temperature=0.7, model_name="text-davinci-003", max_tokens=num_outputs))

    documents = SimpleDirectoryReader(directory_path).load_data()

    index = GPTSimpleVectorIndex(documents, llm_predictor=llm_predictor_gpt, prompt_helper=prompt_helper)

    index.save_to_disk('index.json')

    return index

def chatbot(input_text):
    index = GPTSimpleVectorIndex.load_from_disk('index.json')
    response = index.query(input_text, response_mode="compact")
    return response.response




iface = gr.Interface(fn=chatbot,
                     inputs=gr.inputs.Textbox(lines=7, label="Enter your text"),
                     outputs="text",
                     title="Custom-trained AI Chatbot")

index = construct_index("salesdocs")
iface.launch(share=False)

And I keep getting this error

  File "C:\Users\Anonymous\anaconda3\lib\site-packages\llama_index\indices\vector_store\base.py", line 58, in __init__
    super().__init__(
TypeError: __init__() got an unexpected keyword argument 'llm_predictor'

Having a hard time finding much documentation on llamma index errors, hoping someone can point me in the right direction.

Hygienist

Where did you learn that you can pass a llm_predictor keyword argument to GPTSimpleVectorIndex? – Inkster

You can solve the error by deleting this: llm_predictor=llm_predictor_gpt, . – Inkster

I didn't this was taken from several different online tutorials - but if you remove that section, then the llm_predictor function is not used in any capacity, and I would think need this.... no? – Hygienist

It's also not used if you write it like you have now, plus you get an error. – Inkster

How would I set the model and temperature that gets passed to openai then? It looks like LM predictor is just for estimating costs (?). Doesn't seem overly useful outside of that function. – Hygienist

From what I'm seeing, that does look like a valid input forGPTSimpleVectorIndex - that, or a whole host of online resources are wrong. – Hygienist

Answer 1

You need to change the code according to this example: LlamaIndex usage pattern

Basically, you need to send that information as a ServiceContext:

from llama_index import ServiceContext

service_context = ServiceContext.from_defaults(
  llm_predictor=llm_predictor, 
  prompt_helper=prompt_helper,
  embed_model=embedding_llm,
  )
    
index = GPTSimpleVectorIndex(nodes, service_context=service_context)

But the majority of the tutorials online are older versions. So, you were misguided, and me as well.

To complete even more the answer, if you later need to load the created index you must send the service_context as well:

index = GPTSimpleVectorIndex.load_from_disk(
           filename, service_context=service_context
        )

Otherwise, the code will break while loading the index file.

Carola

Even the llama_index README is outdated in that regard – Steenbok

Answer 2

I got the same error When tried that example too. A simplest fix is to use from_document using defaults:

    index = GPTSimpleVectorIndex.from_documents(documents)

But if you want to set your own prompt context and llm_predictor, then create a context and pass it to GPTSimpleVectorIndex.from_document() call:

def construct_index(directory_path):
    max_input_size = 4096
    num_outputs = 512
    max_chunk_overlap = 20
    chunk_size_limit = 600

    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

    llm_predictor_gpt = LLMPredictor(llm=OpenAI(temperature=0.7, model_name="text-davinci-003", max_tokens=num_outputs))

    documents = SimpleDirectoryReader(directory_path).load_data()

    # index = GPTSimpleVectorIndex(documents, llm_predictor=llm_predictor_gpt, prompt_helper=prompt_helper)

    service_context = ServiceContext.from_defaults(
        llm_predictor=llm_predictor, prompt_helper=prompt_helper
    )

    index = GPTSimpleVectorIndex.from_documents(documents,service_context=service_context)

    index.save_to_disk('index.json')

    return index

Consider using model_name="gpt-3.5-turbo" that is faster and costs less.

llm_predictor_gpt = LLMPredictor(llm=ChatOpenAI(temperature=0.7,
   model_name="gpt-3.5-turbo", max_tokens=num_outputs))

Softa