How to make AI even smarter.

AI agents aren't a new concept. LangChain has been around for quite some time - but I hear a lot of people have pain with it in production environments and the AI landscape is moving on so quickly with OpenAI can I find a better way to create chained AI agents?

Turns out I can. What follows is an application that can work across multiple contexts whilst maintaining a consistent chat history.

The concept

The concept of this approach is as follows:

Whenever a message is sent to the application, a proxy router agent analyses the message and determines which 'journey' the user should be on. Depending on the journey, the router will pass the message to one of many AI agents who each specialise in completing a different job.

The router keeps a context of the user message stream so it can understand if the user is in an existing conversation with an agent, or it needs to pick an agent to start this conversation.

So the proxy router' response looks like this: { agent: 'ONBOARD' }

While all the other AI agents respond with JSON in a format that looks more like this: { content: 'This is the message', completed: true | false }

When an AI agent deems a conversation as completed, it will set the completed flag to true and the proxy router will then know to stop directing messages to that agent.

Like my previous article on AI Assistants with WhatsApp, for the demo data will be stored against an object locally. Obviously, in production you'd want an encrypted database.

The proxy

The proxy agent evaluates each request made to the system and works out the agent that should be used. The prompt would look something like the below, with the keywords matching the problem your AI assistant is trying to solve:

You are an operator managing conversations with AI agents. You will be given one or more sentences that form part of a dialogue of a conversation and you need to respond with the relevant keyword.

The keywords are:
* "GREETING" - if the user says hello or hi
* "CONTINUE" - continue talking about the same subject. This should be used if the latest sentence is the same topic as the previous sentence.
* "HISTORY" - the user is referring to something they have spoke about before
* "PROGRESS" - the user wants to discuss their progress, understand what they are learning or change their focus
* "PROFILE" - the user wants to change or view their profile

You should respond with the JSON:

{ "agent": "GREETING" | "CONTINUE" | "HISTORY" | "PROGRESS" | "PROFILE" }

The proxy service uses this prompt to decide where to send the user request. We simply pass in the user request in to

import openai from "../lib/openai";

const prompt = 'OUR PROMPT';

type Message = {
  content: string;
  role: "system" | "user" | "assistant";
  type?: string;
};

async function main(request: string) {
  if (!request) {
    return { agent: "ERROR", message: "Invalid request" };
  }

  if (request.toUpperCase() === "HELP") {
    return { agent: "HELP" };
  }

We have a shortcut method of 'HELP' for if a user gets stuck in a path.

  try {
    const chatCompletion = await openai.chat.completions.create({
      messages: [
        { role: "system", content: prompt },
        { role: "user", content: request },
      ],
      model: "gpt-4o", // you could use a different model here
      response_format: { type: "json_object" },
    });

    const response = chatCompletion.choices[0].message.content;

    if (response) {
      try {
        return JSON.parse(response);
      } catch (error) {
        return { agent: "ERROR", message: error.message };
      }
    }

    return { agent: "ERROR", message: "Invalid response" };
  } catch (error) {
    return {
      agent: "ERROR",
      message:
        "Unable to contact the server at this time. Please try again later.",
    };
  }
}

The try/catch around the response parsing is to ensure we are receiving JSON from the AI response. Normally setting the type as json_object is enough, but it's worth having a safety check.

async function routeChat(content: string, id: string): Promise<RouteResponses> {
  const userData = getUserData(id);

  let route: Route = { agent: Agent.ONBOARDING };

  if (userData.state?.isOnboarded || false) {
    if (userData.state?.journey) {
      route = { agent: userData.state?.journey };
    } else {
      route = await proxy(content);
    }
  }

  let threadId = userData.threadId;

  if (!threadId) {
    const thread = await openai.beta.threads.create();
    threadId = thread.id;
    setThreadId(id, threadId);
  }

  let responseList: Array<any> = [];

  if (route.agent === Agent.ONBOARDING) {
    const onboardingResponse = parseResponse(
      await onboarding(content, threadId, id)
    );
    responseList.push({
      ...onboardingResponse,
    });

    if (onboardingResponse.completed === true) {
      setUserData(id, { state: { isOnboarded: true, journey: Agent.TRIAGE } });
      const coachingResponse = parseResponse(await triage(content, threadId));
      responseList.push({
        ...coachingResponse,
        agent: Agent.TRIAGE,
      });
    }
  }

  if (route.agent === Agent.TRIAGE) {
    responseList.push(parseResponse(await triage(content, threadId)));
  }

  if (route.agent === Agent.COACH) {
    if (userData.state?.journey !== Agent.COACH) {
      setUserData(id, { state: { ...userData.state, journey: Agent.COACH } });
    }
    responseList.push(parseResponse(await coach(content, threadId)));
  }

  if (route.agent === Agent.GREETING) {
    responseList.push(greeting(userData));
  }

  if (route.agent === "HELP") {
    responseList.push({
      content: `I can help you with X.

      What would you like to talk about?`,
    });
  }
}

As you can see, each 'agent' uses the same consistent OpenAI thread, but has a different context.