Column
Choose an AI interview by whether it can explain its scoring
When an AI interview goes through internal approval, the last question is usually “What did the AI look at to give this score?” What to check when choosing is whether the service can explain how it scores. Below are four questions to ask, and Proomi’s answers.
Why scoring
Can you explain the AI’s scores inside your company?
Roll it out without being able to say why a score is what it is, and nobody knows how to use the scores. People end up redoing the interviews themselves.
Japan’s AI Guidelines for Business (version 1.2, March 2026), issued by the Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry, ask organisations not to leave judgements to AI alone but to bring in human judgement at the right moments, and list telling the people involved that AI is being used. The Personal Information Protection Commission likewise asks that, when an organisation analyses a person’s behaviour or interests (so-called profiling), the purpose of use be specified to include that analysis. The same thinking is a useful reference when AI evaluates interview records.
Both assume that the organisation itself can explain what the AI is doing. Some AI interview vendors overseas publish documents explaining how their scoring works. It is fair to ask the same when choosing a service in Japan.
Sources: Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry, AI Guidelines for Business (version 1.2), March 2026 (in Japanese) https://www.soumu.go.jp/main_content/001064279.pdf; Personal Information Protection Commission, Q&A on the Guidelines for the Act on the Protection of Personal Information, Q2-1 (in Japanese) https://www.ppc.go.jp/personalinfo/faq/APPI_QA/
Question 1
What does it read to score?
The first question is what the AI uses as material. Whether it is only what the candidate says, or also facial expressions and tone of voice, makes a large difference to whether the reason for a score can be put into words.
Proomi transcribes what the candidate says and scores by reading that transcript. It does not summarise the interview first; scoring works directly from the transcript. Facial expressions, tone of voice and speaking speed are not used for scoring.
Research has raised doubts about reading emotions from faces. In a 2019 paper drawing on a large body of studies, Barrett and colleagues showed that people express the same emotion with different faces, and that the same expression can mean different things in different situations. The EU AI Act prohibits AI that infers people’s emotions in the workplace, and guidelines the EU published in 2025 include candidates in a selection process within that workplace.
If only what was said is used, the reason for a score can be explained in the candidate’s own words.
Sources: Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930; Regulation (EU) 2024/1689 (AI Act), Article 5(1)(f) https://eur-lex.europa.eu/eli/reg/2024/1689/oj; European Commission, Guidelines on prohibited artificial intelligence practices established by Regulation (EU) 2024/1689 (AI Act), February 2025, paragraph 254 https://digital-strategy.ec.europa.eu/en/library/commission-publishes-guidelines-prohibited-artificial-intelligence-ai-practices-defined-ai-act
Question 2
What standard does it score against?
Next is the standard behind the scores. If the service supplies the standard, it is hard for the organisation to describe it in its own words.
In Proomi, you write the evaluation criteria in your own words rather than choosing them from a list. Beyond the name of each criterion, you can define what an answer at each score level looks like.
Level definitions are easier for both the AI and human interviewers to apply when they pair what people who can do it say and do with what people who cannot do it say and do. Here is an example.
How levels might be written for “spotting problems” (an illustrative example)
- 5 pointsCan describe, in a concrete situation, noticing a problem nobody had pointed out and checking its cause before acting
- 3 pointsCan describe working on a given problem by listing its possible causes
- 1 pointWhen asked about a problem, only describes what happened and cannot say what the problem was
Criteria written as impressions, such as “shows initiative”, push both the AI and interviewers towards safe middle scores. In another article, where we simulated interviews with personas, rewriting the criteria with level definitions widened the gap between candidates for the same answers. See “Rewrite your criteria, then test them on past interviews”.
Question 3
What happens when there is no evidence?
Third is how it handles a criterion the candidate never spoke to. Guessing a score there leaves nobody sure what the score means.
For each criterion, Proomi writes a score and a reason and quotes the statements it relied on. A criterion with no supporting statement can be recorded as “no evidence” instead of being given a guessed score.
“No evidence” does not mean the candidate scored low. It signals that the interview did not include enough questions to test that criterion. If the same criterion keeps coming back with no evidence, add questions or rewrite the criterion. How to add questions is covered in “Your performance review criteria can be used for hiring as they are”, and how to read skewed scores in “More criteria do not mean more evidence”.
Tying the evidence to statements also helps when explaining things to candidates. Proomi can send candidates feedback written for them; by default it leaves out pass/fail and scores and points to where their strengths showed in what they said. The company decides whether to send feedback and what it contains.
Question 4
Does it present scores in a form people can act on?
Last is whether the AI’s scores arrive in a form people can use as they are. If only a number comes back, the person in charge has to recheck from scratch whether that number is enough to move someone forward.
Proomi gives an overall score and a score for each criterion, each with its reason and quotes of the statements behind it, and the recording and transcript sit on the same screen. Whether you move high-scoring candidates straight to the next interview or reread the records only for those you are unsure about, you can show the reason for the decision in the candidate’s own words.
Hiring staff record the pass/fail decision, and where to draw the line for moving candidates on is set to fit each company’s approach to hiring.
Using the records
With records kept, you can check your standard later
When the scoring can be seen, the records feed straight into the next improvement.
When you rewrite your criteria, you can rescore past interviews against the new criteria and see how the scores change. We can do this work with you.
Set against pass/fail decisions and retention after hiring, the records also show which criteria were linked to real decisions and performance. The approach is set out in “Whether an interview evaluation was right only shows after the hire”.
What to compare
Four questions to ask when choosing a service
A service that can answer these four lets you explain what the scores mean inside your company after you adopt it.
01
What do you read to score?
Check whether it uses faces and voices, or only what was said.
02
Can we write the criteria and level definitions in our own words?
If you can only choose from a list, you cannot describe the standard yourselves.
03
What happens to the score when there is no evidence?
If guessed scores creep in, you cannot see which criteria lack questions.
04
Do scores come with reasons and evidence?
With the reasons and the statements behind them, the scores can be used as they are.
Q&A
Frequently asked questions
Do you use facial expressions or tone of voice?
No. Scoring uses the transcript of what the candidate said. The recording is kept, so hiring staff can watch it again later.
How far can we write the criteria and level definitions ourselves?
Both the names of the criteria and the definition of each level can be written freely, and you can use the items and definitions from your performance reviews as they are. If you have not settled them yet, we can build them with you from your current evaluation sheet.
Can we use the AI’s scores to decide who moves to the next stage?
Yes. Each overall and per-criterion score comes with its reason and the statements behind it, so you can decide from the scores who moves to the next interview. Hiring staff record the pass/fail decision.
Related pages
How it applies, by situation
Once you have checked how the scoring works, these pages cover choosing a service and putting it to use.
How to choose an AI interview tool →
Who owns the criteria decides which service fits, set out in five layers and six questions.
Set the criteria first and narrow the spread between interviewers.
Show us your evaluation criteria as they are
We will show how your evaluation sheet could be written as Proomi criteria. Free, about 30 minutes.