Introduction
The Question Browser feature allows users to review all questions asked within a particular Max tenant. It displays questions in a tabular view and includes metadata about each question, such as:
- Time and Date
- Question Type
- Copilot Name
- User Name
- Skill Used
- Time to Answer
- Software Version
- User Feedback
Users can drill into a particular question to see a static version of each question and the preceding questions in the thread. By drilling into a question, users also gain access to all diagnostic and evaluation information, enabling easy troubleshooting and review of a user's experience in Max.
Access and Availability
- Permissions: Users must have query browser permission.
- Navigation: Access via the question bubble with a down arrow in the left navigation chat bar or through direct links to specific answers.
- Prerequisites: No additional prerequisites or special settings required.
Purpose
The Question Browser serves as a single source of truth for user experiences in Max. It helps Admins and AnswerRocket internal users review chats and identify issues and their causes. Additionally, it provides insights into how users leverage the copilots available to them.
Users
- Customer Admins
- AnswerRocket's Internal Engineering, Product, Services, and Sales Teams
- Anyone needing to view the history of a particular tenant's chats
Key Components
Table View
A grid displaying all questions asked within a specific Max tenant, including:
- Time and Date
- Question Type
- Copilot Name
- User Name
- Skill Used
- Time to Answer
- Software Version
- User Feedback
Focused Answer View
- Content Tab: Displays all content available in the chat thread at the time the user asked their question, including charts, narratives, messages, and feedback.
- Evaluations Tab: Allows running evaluations to determine how well a question was answered, including assessments on Skill Choice, Parameter Choice, Relevance, Faithfulness, and Context Adequacy.
- Diagnostics Tab: Provides detailed information for diagnosing issues, including LLM Chat Response Payload, Chat Pipeline Profile, token consumption, hints used, LLM model details, skill run content, and available reports. Users can copy a link to the question or copy content from this tab.
Functionality and Usage
Users can interact with the Question Browser by:
- Clicking answer links shared with them.
- Navigating to the Question Browser from the left navigation menu in Max chat (symbol with a download arrow over a speech bubble).
In the browser, users can filter and rearrange the table, and any actions taken will be recalled on their next visit. In the Focused Answer View, users can run evaluations, inspect diagnostics and feedback, and review chat components.
Detailed Features
- All Questions: Review all questions asked within a tenant.
- Answer Pipeline Understanding: Explore records of the answer pipeline and make evaluations on its efficacy. Inspect diagnostic information for each pipeline step.
- Insights: Provides detailed insights into the source copilot, cost, model, and token usage.
- Diagnostic Tools: Includes a comprehensive diagnostics tab with playground access for each entry's last question and conversation state.
User Scenarios
- Issue Identification and Troubleshooting: Identify examples of an issue and gather metadata for troubleshooting and problem-solving.
- Auditing Customer Interactions: Audit interactions customers have had with a specific copilot and explore the overall quality alongside user feedback.
- Pattern Identification: Identify patterns in questions and answers within a copilot.
Updated