Connecting to the LLM API

Maegan Morrison
Maegan Morrison
  • Updated

In order for Max to leverage a large language model (LLM), it must be connected to a large language model. There are two types of connections that you will need to have in place in AnswerRocket: Chat Completion and Embeddings.

The chat completion LLM connection contains options for "Chat" and "Narrative." Chat is the one that handles Max's replies and conversations. Narrative is the one responsible for the insights blocks that are sometimes returned with analysis. Both of these are completion API endpoints, meaning they take a prompt and complete it.

Embeddings are used when AnswerRocket tries to find semantic matches. This occurs if someone misspells a term (a dimension value for example) we would do a semantic search for close matches. Embeddings are also used for semantic searches for Hints, only Hints that are a semantic match to the question asked will be presented.


For your chat completion connection, we suggest using a GPT 4 model.

For your embeddings connection, we suggest using text-embedding-3-small.


To connect to your LLM AnswerRocket will need the following from your Microsoft Azure Open AI account:

Fill in the fields for your connection.jpg

The Endpoint URL for the resource

This can be found by opening the resource.*

The API Key for the resource

Both the API Key and the Endpoint URL can be found within “Keys & Endpoints” in your resource.*

The Deployment Name

This is the name you created for your deployment

The Model Name

This is the model that was chosen. Choose GPT-4 when setting up the deployment
Choose 1106-Preview for the Model Version


Give each of these deployment models a unique and recognizable name. These names will need to be provided to AnswerRocket and need to be distinguishable from each other.


To navigate to the above screen:

  1. From your AnswerRocket instance use the main navigation and select Skill Studio.
    Skill Studio.jpg
  2. Choose the left navigation option for API Config.
    Select API Config.jpg
  3. Create a new Chat Completion API Configuration if you are connecting to your LLM for the first time.
    Add New API Config.jpg


Was this article helpful?



Article is closed for comments.