How to Configure AI QA Auto-Assessment?

The auto-assessment settings collect in one place of the Quality Assurance module everything that decides which dialogs the AI assesses and with which checklist: skip rules, topic routes, separate bot settings and the list of those who are not auto-assessed. The settings live in the module; bot scenarios do not need to be rewritten for them. But auto-assessment on chat close works only for bots whose scenario has the AI QA assessment block on chat close — ConnectiveOne adds it.

When It Will Be Useful

• Dialogs of some topics must not be assessed, or different topics need different checklists.
• Dialogs that are not worth assessing must be skipped: the client did not reply, blocked the bot, the dialog was too short or it was transferred to another operator.
• One bot (for example, an email bot) must work by other rules than the rest.
• Dialogs with admins or trainees must not get into the automatic assessment.

What's Important to Know

• The main "All bots" setting applies to every bot that has no separate setting.
• A separate setting for a bot is a copy of the main setting that you can change. It fully replaces the main setting for this bot; later changes of the main setting do not apply to it.
• A topic route is a row "topic → what to do": assess with the selected checklist or do not assess. A row for a topic family applies to all its topics; a row for a specific topic wins over the family row. Topics without their own row follow the "All other topics" row.
• Three kinds of assessment. Bulk — many chats in one launch: from the table, from a file, by pasting numbers or by a link. Single — one chat from the card in the operator panel or from the module. Automatic — the system runs it by itself when a chat is closed; its rules always come from the "Skip rules" switches. Skip rules apply to bulk and single assessment only if the matching checkbox is on in the setting — separately for each kind.
• ConnectiveOne turns the auto-assessment settings on for your instance. Until then there is no "Auto-assessment settings" tab and assessments work as before.

Before You Start

• ConnectiveOne turned the auto-assessment settings on for your instance.
• To see the settings, your role can view the "Assessments" section of the Quality Assurance module.
• To change the settings, your role has the "AI QA auto-assessment settings" permission (Settings → Roles → "Quality Assurance" module, "Save" column). Without it all fields are read-only.
• The module has at least one active checklist.

How to Create the Main Setting

This step is done once — right after the auto-assessment settings were turned on.

1. Open Quality Assurance → Auto-assessment settings.
2. On the "Auto-assessment is not set up yet" card, select the Checklist for "All other topics" — the checklist you assess dialogs with today.
3. Read the warning: once the main setting exists, the checklist parameters in the scenario assessment block stop applying — the checklist comes from this screen.
4. Click "Create main setting".

After creation the skip rules are off, the short dialog threshold is 4 messages, and "Chat without a topic" and "All other topics" are assessed with the selected checklist.

How to Configure Skip Rules

1. Open Quality Assurance → Auto-assessment settings, the "Settings" tab.
2. In the list on the left select "All bots" or a bot setting.
3. In the "Skip rules" block turn on the rules you need:
   • "Client did not reply" — the operator wrote, the client never replied;
   • "Client blocked the channel" — works for Telegram; in the widget and email the client cannot block the bot;
   • "Dialog was transferred to another operator" — transfer to an operator or a skill group, or two or more operators wrote in the dialog;
   • "Short dialog" — the client and the operator together wrote fewer messages than set in "Minimum messages" (minimum 3).
4. In the "Chat without a topic" block choose "Assess with checklist" (and the checklist) or "Do not assess".
5. Click "Save". The button is active only when there are changes and all fields are valid.

Admins and trainees are set not here but on the "Who is not auto-assessed" tab (see below).

How to Choose Which Topics to Assess and With Which Checklist

1. In the settings form find the "Topic routes" block.
2. Click "Add topics".
3. In the topic tree click one or several topics or topic families. A click on a family selects only the family, not all its topics; topics that already have a row cannot be selected. Use the search field above the tree — it helps to pick an exception topic that repeats in several families.
4. Choose what to do: "Assess with checklist" (and the checklist) or "Do not assess".
5. Click "Add (N)" — every selected topic gets its own row.
6. Check the "All other topics" row: it applies to all topics without their own row, including new ones.
7. Click "Save".

Rows with the same action are shown as a group, for example "Do not assess — topics: 12 ▸"; a click expands the group. To change the action of a row, remove it and add the topic again.

If a topic of a row was deleted, the screen shows a warning and the row gets the "Topic deleted" tag: such a row does not apply, its chats follow the family row or "All other topics". Click "Remove inactive rows" and save.

If a checklist used in the setting became unavailable, the screen shows a warning and a ⚠ sign where it is used. Chats with this route are not assessed until you choose another checklist.

How to Make a Separate Setting for a Bot

1. Open Quality Assurance → Auto-assessment settings.
2. Below the settings list click "Separate setting for a bot".
3. Select the bot (by default — the bot the module is open for) and click "Create". The setting is created as a copy of the main one.
4. Change what you need — rules, "Chat without a topic", topic routes, the bulk and single assessment checkboxes — and click "Save".

Above the tabs the screen shows which setting the bot of the module uses: "All bots" or its own.

• If the main setting was changed after you set up the bot, the bot form shows "The main setting was changed after you set up this bot (date)". To make the bot use the main setting again, click "Revert to main" — the separate setting is deleted.
• If the bot was deleted, its setting is marked "Bot deleted" in the list and is read-only. You can delete it with "Delete setting"; if the bot is restored, the setting applies again.

How to Mark a Role as Admin or Trainee

1. Open Quality Assurance → Auto-assessment settings → Who is not auto-assessed.
2. In the "Roles" table select "Admin — do not assess" or "Trainee — do not assess" for the role. The change is saved at once.

An automatic assessment on chat close is not created if an operator with such a profile took part in the dialog. Mark trainees here — by role or by an operator exception; the word "trainee" in an operator's name changes nothing.

The role card (Settings → Roles) shows a read-only row, for example "AI QA: Admin (managed in the AI QA module)", with a link to this screen.

How to Set an Exception for One Operator

1. Open Quality Assurance → Auto-assessment settings → Who is not auto-assessed.
2. In the "Operators" table find the person by name. The "Only with an exception" checkbox shows operators that already have an exception.
3. In the "Exception" column select "Inherit from role", "Regular", "Admin" or "Trainee". The change is saved at once.

The "Applies" and "Source" columns show the actual profile of the operator and where it comes from (role, exception or default). The ⚠ sign means the exception differs from the role profile. Deactivated operators are not listed, but their exception is kept and applies again when the operator is back at work.

How to Check Why an Automatic Assessment Was Skipped

If the automatic assessment skipped a chat, the dialog card in the operator panel shows the "AI QA skip reason" field. Only the automatic assessment on chat close fills it.

If needed, assess such a chat manually — see the next section.

How to Assess a Skipped Chat Manually

1. Open the dialog in the operator panel and choose "Send for evaluation" (or open the "Assess Dialog" window in the Quality Assurance module and find the chat).
2. Check the checklist: it is suggested by the chat topic route, and for a chat without a topic — by "Chat without a topic". The hint "Checklist suggested by the auto-assessment settings" is shown under the field. Choose another one if needed.
3. If the window shows "This topic is not assessed automatically", it is only a reminder: choose a checklist and launch the assessment.
4. Click "Launch Assessment".
5. If the window shows "The chat was not assessed: …", a skip rule stopped the chat: the bot setting has skip rules on for single assessment. The reason follows the colon; no assessment is created and the window stays open. The hint "Skip rules apply to this chat: if a rule matches, the assessment will not be created." warns about it in advance.

How to Apply Skip Rules to Bulk and Single Assessment

1. Open the setting you need (all bots or a bot).
2. In the "Bulk and single assessment" block turn on the checkboxes you need — they are independent:
   • "Apply skip rules to bulk assessment" — for launches on many chats;
   • "Apply skip rules to single assessment" — for assessing one chat from the card in the operator panel or from the module (off by default).
3. Click "Save".

With the bulk checkbox on, the bulk assessment window shows the "Apply skip rules" switch, on by default; it can be turned off for one launch. Topics set to "Do not assess" are not assessed in bulk regardless of the checkbox: this is a topic setting, not a skip rule.

With the single checkbox on, the single assessment window warns that the rules apply and, if a rule matched, shows "The chat was not assessed: " and creates no assessment. A "Do not assess" topic does not stop a single assessment — the window only reminds about it. The checkboxes do not change automatic assessment on chat close: its rules always come from the "Skip rules" switches.

What Happens After

• A saved setting applies at once: to the checklist suggestion in manual assessment windows, to bulk and (with the checkbox) to single assessment.
• Auto-assessment on chat close works only for bots whose scenario has the AI QA assessment block on chat close — ConnectiveOne adds it; a line above the screen tabs reminds about it. For such bots the assessment takes the checklist and rules from this screen; the assessment block in the scenario builder shows "Managed in the AI QA module".
• If the settings change log is kept on the instance, every change of a setting, routes, a role profile or an operator exception is written to it.

How to Verify Everything Worked

• After "Save" the "Settings saved" message appeared.
• Open the assessment window for a chat of a routed topic — the checklist is suggested; for a "Do not assess" topic — "This topic is not assessed automatically" is shown.
• Above the tabs you can see which setting the bot uses.

Why Can't I Delete or Deactivate a Checklist?

The checklist is used in the auto-assessment settings — in a topic route, in "All other topics" or in "Chat without a topic". The message on deletion lists where exactly, for example "All bots: topic routes (3)". Replace the checklist in the setting first.

Related Materials

• How to create a new dialogue assessment
• Configure roles