(Java) How to use the Assistants API

[CJCrafter]

Hello everyone,

This is a quick tutorial on how to create and use assistants in the Java API. It is expected that you already have this library installed, and you have an OpenAI instance.

Contents

1. Creating an assistant
2. Creating a thread
3. Adding messages to a thread
4. Running the thread
5. Handling run termination
6. Putting it all together

Creating an Assistant

Now, we need to create an assistant. We only need to do this step once, then we can reuse the
assitant by retrieving it from the OpenAI API. More on that later.

Creating an assistant programmatically

We can use the Java API to create an assistant:

CreateAssistantRequest request = CreateAssistantRequest.builder()
        .model(Models.Chat.GPT_3_5_TURBO)
        .name("My Helper")
        .description("Solves Calculus Problems")
        .instructions("Provide step-by-step solutions to math problems, and cite the formulas you use")
        .build();

// Using .assistants() returns the handler for interacting with assistants
Assistant assistant = openai.assistants().create(request);

// You should save this result somewhere so you can use it in the future
System.out.println("You can now use the assistant by id: " + assistant.getId());

Creating an assistant using playground

If you would like to skip the code, you can easily create, list, and modify your assistants on
OpenAI's playground.

For the rest of this tutorial, we will be reusing this assistant we just created. Your id
will be unique, but we will say that my assistant has an id asst-abc123 (All assistant ids
start with asst-).

Creating a Thread

If you are unfamiliar, the assistants API is structed like a forums page. The general process
is outlined below:

1. Create a Thread
2. Add a user's message to that Thread
3. Run the Thread so the Assistant can generate a response
4. Wait for the Run to terminate, and show the outputs to the user

Luckily, creating a new thread is simple:

Thread thread = openai.threads.create();

Note that this is a com.cjcrafter.openai.threads.Thread, not a java.lang.Thread, which is
an entirely different concept altogether. Remember that a Thread, in this context, is like a forum thread.

Adding Messages to a Thread

Now let the user add a message. For simplicity, we'll make this a console application.

In this example, we'll add 1 message to the thread. Typically speaking, you might do this
in a loop so that the user can have a full conversation with the bot. You'll see this later
in the complete example.

Thread thread = /* from previous steps */;

Scanner scan = new Scanner(System.in);
System.out.println("Please type your input below: ");
String input = scan.nextLine();

openai.threads().messages(thread).create(CreateThreadMessageRequest.builder()
        .role(ThreadUser.USER) // this message comes from the user, not the assistant
        .content(input) // what the user input
        .build());

Running the Thread

So now we have a thread with 1 user message on it. Now we need the assistant to respond.

At the time of writing, OpenAI does not have a great method for this. Basically, once
we create a run, the Assistant will get to work; great! Except, how do we know when it
is done? Short answer: We don't.

So in order to detect when the run has completed, we have to busy-wait.

Thread thread = /* from previous steps */;

// We are getting the assistant that we have already created. You should
// only need to create a few assistants. 1 assistant can be a part of
// any number of threads, so reuse! Your id will be different than this example:
Assistant assistant = openai.assisants().retrieve("asst-abc123");

// Calling this "create" method will cause the assistant to start working.
// Some amount of time later, the run will be completed.
Run run = openai.threads().runs(thread).create(CreateRunRequest.builder()
        .assistant(assistant)
        .build());
        
// busy-wait until the run is complete
while (!run.getStatus.isTerminal()) {
    java.lang.Thread.sleep(1000);
    run = openai.threads().runs(thread).retrieve(run);
}

Note that busy waiting is a bad practice. Consider using a CompleteableFuture<Run> instead.
Unfortunately, async calls are a bit outside the scope of this tutorial.

Handling Run Termination

At this point, the run variable stores a complete run, but what does complete mean?.

In most cases, this statement will be true: run.getStatus() == RunStatus.COMPLETED

But you should handle cases like RunStatus.REQUIRED_ACTION for function calls, RunStatus.CANCELLED
if you cancelled a run before it could be completed, or RunStatus.FAILED if you got rate limited (or other errors).

But now how do get the message from the assistant? Since an assistant can make multiple
messages, make function calls, and output files and other things, we need to carefully filter
out extra data:

Thread thread = /* from previous steps */;
Run run = /* from previous steps */

List<RunStep> steps = openai.threads().runs(thread).steps(run).list().getData();
for (RunStep step : steps) {
    // Type can be either MESSAGE_CREATION or TOOL_CALLS (for 
    // functions, retrieval, and code interpeter). We're going
    // to assume the assistant does not use functions, so we can
    // ignore this extra data.
    if (step.getType() != RunStep.Type.MESSAGE_CREATION) {
        System.out.println("Assistant made step: " + step.getType());
        continue;
    }
    
    // Since we already checked the type above, this is a safe cast
    MessageCreationDetails details = (MessageCreationDetails) step.getStepDetails();
    String messageId = details.getMessageCreation().getMessageId();
    ThreadMessage message = openai.threads().messages(thread).retrieve(messageId);
    
    // Look for contents with text contents
    for (ThreadMessageContent content : message.getContent()) {
        if (content.getType() != ThreadMessageContent.Type.TEXT) {
            System.err.println("Unhandled message content type: " + content.getType());
            System.err.println("This will never occur since this Assistant doesn't use images");
            System.exit(-1);
        }
        
        System.out.println(((TextContent) content).getText().getValue());
    }
}

Putting it all together

Putting it altogether, here is what we get:

import com.cjcrafter.openai.OpenAI;
import com.cjcrafter.openai.assistants.Assistant;
import com.cjcrafter.openai.assistants.ListAssistantResponse;
import com.cjcrafter.openai.threads.Thread;
import com.cjcrafter.openai.threads.message.*;
import com.cjcrafter.openai.threads.runs.CreateRunRequest;
import com.cjcrafter.openai.threads.runs.MessageCreationDetails;
import com.cjcrafter.openai.threads.runs.Run;
import com.cjcrafter.openai.threads.runs.RunStep;

import java.util.Scanner;

public class ThreadExample {

    public static void main(String[] args) throws InterruptedException {

        // This is the "main openai instance" that we will use to interact with
        // the OpenAI API. 
        OpenAI openai = OpenAI.builder()
                .apiKey("sk-abc123)
                .build();

        // Retrieve the assistant we already created
        Assistant assistant = openai.assistants().retrieve("asst-abc123");

        // We have to create a new thread.  We'll save this thread, so we can
        // add user messages and get responses later.
        Thread thread = openai.threads().create();

        while (true) {

            // Handle user input
            System.out.println("Type your input below: ");
            String input = scan.nextLine();
            openai.threads().messages(thread).create(CreateThreadMessageRequest.builder()
                    .role(ThreadUser.USER)
                    .content(input)
                    .build());

            // After adding a message to the thread, we have to run the thread
            Run run = openai.threads().runs(thread).create(CreateRunRequest.builder()
                    .assistant(assistant)
                    .build());

            // busy-wait until the run is complete
            while (!run.getStatus().isTerminal()) {
                java.lang.Thread.sleep(1000);
                run = openai.threads().runs(thread).retrieve(run);
            }

            // Once the run stops, we want to retrieve the steps of the run.
            // this includes message outputs, function calls, code
            // interpreters, etc.
            for (RunStep step : openai.threads().runs(thread).steps(run).list().getData()) {
                if (step.getType() != RunStep.Type.MESSAGE_CREATION) {
                    System.out.println("Assistant made step: " + step.getType());
                    continue;
                }

                // This cast is safe since we checked the type above
                MessageCreationDetails details = (MessageCreationDetails) step.getStepDetails();
                ThreadMessage message = openai.threads().messages(thread).retrieve(details.getMessageCreation().getMessageId());
                for (ThreadMessageContent content : message.getContent()) {
                    if (content.getType() != ThreadMessageContent.Type.TEXT) {
                        System.err.println("Unhandled message content type: " + content.getType());
                        System.err.println("This will never occur since this Assistant doesn't use images.");
                        System.exit(-1);
                    }

                    System.out.println(((TextContent) content).getText().getValue());
                }
            }
        }
    }
}