langchain: logprobs, best_of and echo parameters are not available on gpt-35-turbo model

Question

I am trying to use langchain with gpt-35-turbo. It seems that the new gpt3.5 turbo is not using certain parameters anymore as per the link Learn how to work with the ChatGPT and GPT-4 models (preview)

The following parameters aren't available with the new ChatGPT and GPT-4 models: logprobs, best_of, and echo. If you set any of these parameters, you'll get an error.

Now each time I initialise an LLM model through langchain with gpt-3.5-turbo, it gives me this error:

InvalidRequestError: logprobs, best_of and echo parameters are not available on gpt-35-turbo model. Please remove the parameter and try again. For more details, see https://go.microsoft.com/fwlink/?linkid=2227346.

I don't know how to 'unset' these parameters in langchain.

This is my code:

from langchain.chains.llm import LLMChain
from langchain.llms.openai import OpenAI
from langchain.prompts.prompt import PromptTemplate

llm = OpenAI(temperature=0, engine=deployment_name)

template = """
You are a helpful assistant that translates English to French. Translate this sentence from English to French: {text}
"""
prompt = PromptTemplate(input_variables=["text"], template=template)
llm_chain = LLMChain(llm=llm, prompt=prompt)
response = llm_chain.generate([
        {"text": "I love AI"},
        {"text": "I love the ocean"},
    ])

for g in response.generations:
    print(g[0].text)

Note that I am using openAI in Azure, I also tried this code and it's still giving me the same error

deployment_name = "my-deployment-name"
from langchain.llms import AzureOpenAI
llm = AzureOpenAI(deployment_name=deployment_name )
print(llm)
llm("Tell me a joke")

Graphomotor

Answer 1

So, I finally was able to fix it by creating an extension of AzureOpenai class and nullifying those arguments. Code that works is below:

from langchain.llms import AzureOpenAI
from typing import List
class NewAzureOpenAI(AzureOpenAI):
    stop: List[str] = None
    @property
    def _invocation_params(self):
        params = super()._invocation_params
        # fix InvalidRequestError: logprobs, best_of and echo parameters are not available on gpt-35-turbo model.
        params.pop('logprobs', None)
        params.pop('best_of', None)
        params.pop('echo', None)
        #params['stop'] = self.stop
        return params
    
llm = NewAzureOpenAI(deployment_name=deployment_name,temperature=0.9)
llm("Tell me a joke")

The answer was actually found in this link and it worked for me.

Graphomotor

Answer 2

Using ChatOpenAI instead of OpenAI (and model_name instead of engine) works for me on langchain==0.0.127. There is this warning on langchain's OpenAI code

        if model_name.startswith("gpt-3.5-turbo") or model_name.startswith("gpt-4"):
            warnings.warn(
                "You are trying to use a chat model. This way of initializing it is "
                "no longer supported. Instead, please use: "
                "`from langchain.chat_models import ChatOpenAI`"
            )

This code is working for me:

import os
from langchain.chains.llm import LLMChain
from langchain.chat_models import ChatOpenAI
from langchain.prompts.prompt import PromptTemplate

llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", openai_api_key=os.environ['OPENAI_KEY'])

template = """
You are a helpful assistant that translates English to French. Translate this sentence from English to French: {text}
"""
prompt = PromptTemplate(input_variables=["text"], template=template)
llm_chain = LLMChain(llm=llm, prompt=prompt)
response = llm_chain.generate([
        {"text": "I love AI"},
        {"text": "I love the ocean"},
    ])

for g in response.generations:
    print(g[0].text)

Fafnir

Tried that and it gave me this error: InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>.. – Graphomotor

Your error message mentions ChatCompletion as opposed to ChatOpenAI, could you be using the wrong class? If not, what version of langchain are you using? Edit: I didn't realize the ChatCompletion came from the openai package. Maybe try pip install -U openai – Fafnir

If you're using azure, this might also be happening to you github.com/openai/openai-python/issues/318 – Fafnir

I am using Azure. is there anyway to make langchain work with azure? – Graphomotor

I tried AzureOpenAI as well and it's giving me the same error (edited the post above with that code too) – Graphomotor

The comments on the GitHub issue I linked have a couple of suggestions. Unfortunately I don't have experience with Azure models. – Fafnir