How to Create a New Dialogue Assessment?

Creating a dialogue assessment allows launching the quality check process. You can create an assessment with dialogue search or quickly from a specific chat in the OperatorLine module.

When It Will Be Useful

You need to create a new assessment for dialogue quality check.
You need to quickly create an assessment from the current chat in OperatorLine.
You need to launch automatic AI assessment of the dialogue.

What's Important to Know

Assessment — quality check of a dialogue using a checklist with criteria.
Automatic assessment — AI automatically assesses the dialogue by checklist criteria.
Manual assessment — QA specialist manually assesses the dialogue after creating the assessment.
Checklist — a set of criteria for assessing dialogue quality.

Before You Start

You are logged in with QA Lead or QA Specialist rights.
Checklists for assessment are configured in the system.

Step-by-Step Instructions

Option 1: Creating Assessment with Dialogue Search

In the Quality Assurance module, go to the "Assessments" section.
Click the "Create Assessment" (Launch Assessment) button on the Assessments page.
In the "Create Assessment" modal:
- In the "Dialogue Search" field, enter the dialogue ID or dialogue name.
- The system automatically performs search and displays results.
- Select the needed dialogue from the search results list.
The system automatically determines the operator from the dialogue and displays dialogue information (ID, name, operator, channel, topic).
- Note: The system automatically excludes fastLine AI agents when determining the operator. If a fastLine agent participated in the dialogue, the system will identify the real human operator instead.
In the "Checklist" field, select a checklist from the list of available checklists.
(Optional) Enable the "Automatic Assessment" (Auto Evaluate) option if you need AI to automatically assess the dialogue.
Click the "Create Assessment" button.
If automatic assessment is used, the system shows a message about AI analysis progress.

Option 2: Creating Assessment from Specific Chat (OperatorLine)

In the OperatorLine module, open the chat that needs to be assessed.
In the active chat menu or chat context, find the "Create Assessment" button or similar.
In the "Create Assessment" modal, dialogue fields will be automatically filled:
- Dialogue — dialogue ID and name.
- Operator — operator handling the dialogue.
- Channel — communication channel.
- Topic — request topic.
In the "Checklist" field, select a checklist from the list of available checklists.
(Optional) Enable the "Automatic Assessment" option for AI assessment.
Click the "Create Assessment" button.
The assessment is created with automatically filled dialogue data.

Checklist Suggested by the Auto-Assessment Settings

If the auto-assessment settings are on for your instance, both assessment windows suggest the checklist by the topic route of the selected chat (for a chat without a topic — by "Chat without a topic") and show "Checklist suggested by the auto-assessment settings" under the "Checklist" field. You can change the suggested checklist; a checklist you already chose yourself is never overwritten. If the topic is set to "Do not assess", the window shows "This topic is not assessed automatically" — you can still launch the assessment. If the setting of the chat's bot has skip rules on for single assessment, the window warns "Skip rules apply to this chat: if a rule matches, the assessment will not be created." and, when a rule matched, shows "The chat was not assessed: …" with the reason and creates no assessment.

What Happens After

The assessment is created and added to the system:

The assessment appears in the Assessments list.
If automatic assessment is used, AI analyzes the dialogue and assesses criteria.
After AI analysis completion, the assessment receives "evaluated" status and is ready for QA specialist review.
If automatic assessment is not used, the assessment is ready for manual review.

How to Verify Everything Worked

Check that the new assessment appeared in the Assessments list.
If automatic assessment is used, wait for AI analysis completion and check the assessment status.
Open the assessment and verify that all data is filled correctly.

Important Notes

🤖 Automatic assessment: AI analysis may take some time — the system shows a progress message.
📋 Checklist: Be sure to select a checklist before creating the assessment — without a checklist, the assessment cannot be created.
🔍 Dialogue search: You can search dialogues by ID or name — the system automatically performs search while typing.
👤 Operator detection: The system automatically identifies the real human operator, excluding fastLine AI agents. If no real operator is found, you will see an error message.