How can I use batch embeddings using OpenAI's API?

Question

I am using the OpenAI API to get embeddings for a bunch of sentences. And by a bunch of sentences, I mean a bunch of sentences, like thousands. Is there a way to make it faster or make it do the embeddings concurrently or something?

I tried Looping through and sending a request for each sentence but that was super slow, but so is sending a list of the sentences. For both cases, I used this code: '''

response = requests.post(
    "https://api.openai.com/v1/embeddings",
    json={
        "model": "text-embedding-ada-002",
        "input": ["text:This is a test", "text:This is another test", "text:This is a third test", "text:This is a fourth test", "text:This is a fifth test", "text:This is a sixth test", "text:This is a seventh test", "text:This is a eighth test", "text:This is a ninth test", "text:This is a tenth test", "text:This is a eleventh test", "text:This is a twelfth test", "text:This is a thirteenth test", "text:This is a fourteenth test", "text:This is a fifteenth test", "text:This is a sixteenth test", "text:This is a seventeenth test", "text:This is a eighteenth test", "text:This is a nineteenth test", "text:This is a twentieth test", "text:This is a twenty first test", "text:This is a twenty second test", "text:This is a twenty third test", "text:This is a twenty fourth test", "text:This is a twenty fifth test", "text:This is a twenty sixth test", "text:This is a twenty seventh test", "text:This is a twenty eighth test", "text:This is a twenty ninth test", "text:This is a thirtieth test", "text:This is a thirty first test", "text:This is a thirty second test", "text:This is a thirty third test", "text:This is a thirty fourth test", "text:This is a thirty fifth test", "text:This is a thirty sixth test", "text:This is a thirty seventh test", "text:This is a thirty eighth test", "text:This is a thirty ninth test", "text:This is a fourtieth test", "text:This is a forty first test", "text:This is a forty second test", "text:This is a forty third test", "text:This is a forty fourth test", "text:This is a forty fifth test", "text:This is a forty sixth test", "text:This is a forty seventh test", "text:This is a forty eighth test", "text:This is a forty ninth test", "text:This is a fiftieth test", "text:This is a fifty first test", "text:This is a fifty second test", "text:This is a fifty third test"],
    },
    headers={
        "Authorization": f"Bearer {key}"
    }
    )

For the first test, I did a bunch of those requests one by one, and for the second one I sent a list. Should I send individual requests in parallel? Would that help? Thanks!

Paratroops

Answer 1

According to OpenAi's Create Embeddings API, you should be able to do this:

To get embeddings for multiple inputs in a single request, pass an array of strings or array of token arrays. Each input must not exceed 8192 tokens in length.

https://beta.openai.com/docs/api-reference/embeddings/create

Geosphere

Answer 2

Sounds like you are referring to the OpenAI Batch API. You can create a jsonl file where each line represents a request. E.g.

{"custom_id": "AN_ID_THAT_HELPS_YOU_IDENTIFY_IT_LATER", "method": "POST", "url": "/v1/embeddings", "body": {"model": "text-embedding-3-large", "input": "Howdy cowboy", "encoding_format": "float"}}
{"custom_id": "ANOTHER_ID_THAT_HELPS_YOU_IDENTIFY_IT_LATER", "method": "POST", "url": "/v1/embeddings", "body": {"model": "text-embedding-3-large", "input": "Python and the Pandas", "encoding_format": "float"}}

As a next step you need to upload the file, create the batch, wait until it is ready and then download the results as outlined in the OpenAI Batch docs

Paulsen