Solutions
Whoever runs the interview, the criteria stay the same
Interview scores vary with the interviewer. Decide the questions and evaluation criteria up front, and apply the same conditions to every candidate.
Challenge
Score variation is not about interviewer ability
When the criteria are vague, scores drift from interviewer to interviewer. That has less to do with how good the interviewers are than with the interviews themselves not being alike.
- Different questions in each interview mean different information collected
- The same answer is weighted differently depending on the interviewer’s experience
- The bar shifts depending on who was interviewed just before
01
Design the criteria
Decide up front what to ask and what to measure
02
Applied to every candidate
The same conditions in every interview
03
AI interview
Asks the preset questions and follows up
04
Score and reasoning
Generated for each criterion
05
Final decision
Your hiring team
The three shaded steps are automated. The pass-or-fail decision is made by a person.
Solution
Decide the criteria first, then interview
Rather than putting an impression into words after the interview, you decide what to measure before it. A score and its reasoning are generated for each criterion and kept on record.
- Every candidate is asked the same questions
- A score and its reasoning are kept for each criterion
- The AI evaluation and a human one can be recorded side by side
Verification
You can check afterwards whether anything is skewed
Looking at the score distribution per criterion shows which criteria the AI keeps scoring the same way. You can then work out whether the criterion is vaguely worded or the questions fail to probe it, and fix the design accordingly.
- Review the score distribution per criterion
- Identify criteria with little variation
- Feed the findings back into the question and criteria design
Show the numbers
| Criterion | 1 pt | 2 pt | 3 pt | 4 pt | 5 pt | Std. dev. | Modal share |
|---|---|---|---|---|---|---|---|
| Intent to stay | 12 | 63 | 68 | 30 | 2 | 0.87 | 39% |
| Communication | 4 | 19 | 64 | 64 | 24 | 0.94 | 37% |
| Adaptability | 7 | 36 | 67 | 49 | 16 | 0.99 | 38% |
| Logical thinking | 9 | 33 | 79 | 37 | 17 | 0.99 | 45% |
| Initiative | 13 | 29 | 62 | 48 | 23 | 1.10 | 35% |
| Domain knowledge | 19 | 44 | 54 | 36 | 22 | 1.18 | 31% |
The figure on each bar is the share taken by the most common score. Criteria are sorted by standard deviation, smallest first.
Related pages
More detail, by topic
Even with shared criteria, scores can diverge depending on how the criteria are interpreted and how many there are. These articles look at both. How AI scoring changes variation between interviewers, and the cycle for refining criteria, are covered in another article.
Same answer, different score: when interviewers read the criteria differently →
Use gaps between human and AI scores to find where interpretations diverge, and fix the criteria.
Choose an AI interview tool that can explain its scores →
How to check what the AI reads and the standard it scores against.
More criteria do not mean more evidence →
Why scores bunch up, checked against public civil-service rating data.
AI interview services compared →
The leading services, sorted by what they optimise for and compared on questions, criteria and verification, using public information.
Can AI evaluate candidates correctly? HR writes the criteria →
AI scoring removes the differences between interviewers. How to prepare criteria and refine them by comparing human and AI scores.
Frequently asked questions
What information is the evaluation based on?
Evaluation is based on what the candidate says. Non-verbal signals such as facial expression, tone of voice and speaking pace are not used. This is in line with the EU AI Act, which bans emotion recognition in the workplace in most cases.
Does the AI decide who is hired?
No. The AI does not decide. Your hiring team makes the pass-or-fail decision, and the score is one input to it.
If a candidate asks why they were rated that way, can we answer?
The score for each criterion is kept along with written reasoning. The interview recording and transcript are stored too, so you can go back later and check how the rating was reached.
Can it be compared with a human interviewer’s evaluation?
Yes. The AI evaluation and a human reviewer’s can be recorded side by side. The overall score and the gap on each criterion are shown, so you can see where the two diverged.
Start the conversation with the criteria themselves
We will show how your current standards translate into questions and evaluation criteria. Free, about 30 minutes.