Logging in the Open AI python library

Question

I am using the open-ai python library. How can I enable logging of HTTP request/response/headers for all calls made to open-ai?

I am not able to find anything specific in the API docs.

Allanallana

Answering my own question. You can set openai.log='debug' and that will log the details. – Allanallana

What if you want to turn OFF logging? openai.log="error" doesn't seem to eliminate INFO logging statements from OpenAI. – Parasitism

To turn off INFO, instead of enabling INFO for everything, enable info only for the module you care about, i.e., don't do logging.basicConfig(level=logging.INFO, ...). Instead, omit that level param and set the level for your module by writing logging.getLogger(...).setLevel(logging.INFO) for each of module you care about. – Andiron

Answer 1

You can enable debug logging through an environment variable:

export OPENAI_LOG=debug

Note that this also enables httpx logging. If you want to log response data, you'll need to customize the http_client (example here)

Centerboard

Answer 2

Here's another way to set logging levels for specific loggers.

First, find the OpenAI loggers (more info on getting loggers):

import logging

loggers = [logging.getLogger(name) for name in logging.root.manager.loggerDict]
openai_loggers = [logger for logger in loggers if logger.name.startswith("openai")]
# [<Logger openai._legacy_response (WARNING)>, <Logger openai (WARNING)>, <Logger openai._response (WARNING)>, <Logger openai._base_client (DEBUG)>]

Once you know their name, you can set a custom logging level for each logger. For example:

logging.getLogger("openai._base_client").setLevel(logging.DEBUG)

Divisible

Amazing coverage for multiple cases! – Commination

Answer 3

Per the comment from the OP, you can change the log level (at least to debug) by setting the log attribute:

import openai

openai.log = "debug"

Knownothing

Answer 4

It might be overkill, but this provided me excessive and detailed information in logging. Not just openai calls though.

import logging
logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s - %(levelname)s - %(filename)s:%(funcName)s:%(lineno)d - %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')

Afore