How to Adjust AI Assessment Criteria?

Adjusting AI assessments allows correcting AI errors and adding comments to criteria for providing clear feedback to operators.

When It Will Be Useful

  • AI incorrectly assessed a criterion (for example, mistakenly set "failed" instead of "passed").
  • You need to change criterion scores after QA specialist review.
  • You need to add a comment explaining the assessment for the operator.

What's Important to Know

  • AI assessment — automatic criterion assessment performed by AI based on dialogue analysis.
  • Adjustment — changing criterion assessment by QA specialist with mandatory comment.
  • Changed criteria — criteria that were changed by QA specialist are highlighted in a special color.
  • Final score — automatically recalculated after changing any criterion.

Before You Start

  • You are logged in with QA Lead or QA Specialist rights.
  • You have an assessment open with "evaluated" status (AI assessment completed).

Step-by-Step Instructions

1. Open Assessment Details

  1. In the Quality Assurance module, go to the "Assessments" section.
  2. Find the needed assessment in the list and click on it.
  3. The "Assessment Details" page opens with dialogue transcript and interactive checklist.

2. Review AI Assessment Criteria

  1. In the right part of the page, review the interactive checklist with criteria.
  2. Each criterion shows:
    • AI Result — passed/failed, scores, AI comment.
    • Criticality — whether the criterion is critical.
  3. Check if all AI assessments are correct.

3. Identify AI Error

  1. Review the dialogue transcript and compare with AI criterion assessment.
  2. If AI incorrectly assessed a criterion (for example, set "failed" instead of "passed"), it needs to be corrected.

4. Adjust Criterion Assessment

  1. Find the criterion that needs to be corrected in the checklist.
  2. Click the "Cancel Score" button (if AI granted score but shouldn't have) or "Grant Score" (if AI didn't grant score but should have).
  3. In the form that appears:
    • Change the result (passed/failed) if needed.
    • Enter a new score (if you need to change scores).
    • Mandatory enter a comment explaining the change (for example, "AI incorrectly assessed — operator correctly performed the action").
  4. Click the "Confirm Changes" or "Save" button.

5. Verify Changes

  1. After saving, the criterion is marked as changed (highlighted in a special color).
  2. The final assessment score is automatically recalculated based on all criterion assessments (including changes).
  3. The "Approve" button displays the number of changed criteria.

6. Revert All Changes (if needed)

  1. If you made several changes but decided to return to the original AI assessment, click the "Revert All" button.
  2. All changes are reverted, criteria return to AI assessments.
  3. The final score is recalculated to the AI score.

What Happens After

Changes are applied to the assessment:

  • The criterion is updated with new values and comment.
  • The final score is automatically recalculated.
  • Changed criteria are highlighted in the interface.
  • After approval, the operator will see both AI comments and QA specialist comments.

How to Verify Everything Worked

  • Check that the changed criterion is highlighted in a special color.
  • Check that the final score is updated according to changes.
  • Check that the comment is displayed in the criterion.
  • Check the changed criteria counter in the "Approve" button.

Important Notes

  • 📝 Mandatory Comment: When changing criterion assessment, you must add a comment explaining — without a comment, changes are not saved.
  • 🔄 Automatic Recalculation: The final score is automatically recalculated after changing any criterion — no need to recalculate manually.
  • 🎨 Change Highlighting: Changed criteria are highlighted in a special color — this helps quickly find all adjustments.
  • ↩️ Reverting Changes: You can revert all changes and return to the original AI assessment — this is convenient if you made a mistake.