Insights

Getting Approval for AI Interviews: Likely Questions, Evidence and Sample Requests

When you seek internal approval to adopt AI interviews, questions come from HR, interviewers, management, legal and IT. This guide shows which evidence answers each of them, so your request is less likely to be sent back. The samples are written so they can be copied straight into your own approval request.

Published

Introduction

What this article covers

This article sets out which evidence answers which stakeholder’s questions, and gives samples you can copy into the approval request and the trial report. It also covers the process when a product is selected through a request for proposal (RFP).

The approach shown here is one example, and you can drop items to suit your company’s approval practice. Even a simplified request can be built around these four points.

  • Set the conditions for moving to a full contract in advance, and decide on the basis of the trial results
  • During the trial, measure the time spent on first-round interviews, on reading AI interview records, and on designing the scoring guide and questions
  • Complete the data agreement and security review before interviewing real applicants
  • State when renewal will be decided and the conditions for not renewing in the approval request

Summary

Key points

  • A list of the questions raised by HR, interviewers, management, legal and IT, with the evidence that answers each
  • Cost-effectiveness is shown as the payback period and the impact on operating profit, based on time you measure yourself
  • The trial shows whether AI scores can inform decisions, by setting the gap between the AI and your interviewers against the variation among the interviewers themselves

Overview

List the questions in advance and your request is less likely to be sent back

Most questions raised during approval can be answered from vendor materials or an estimate. Only a few need trial results. Identify which ones those are for each stakeholder before you start.

A trial of about one month measures the gap between AI scores and your interviewers’ scores, and the interviewer time each interview takes. From the time per interview you can also estimate how many interviews the same number of interviewers could handle. Base the decision to adopt on short-term figures, namely score agreement, time and cost, and add the interview capacity as a projection. The link with performance and retention after hiring is a long-term effect, checked as live use continues.

Comparison

AI Interview Services: 16 Compared, Nov 2026 →

Major services grouped and compared by what they optimise for.

HR and interviewers

Answer “Can AI scores inform decisions?” with trial results

Even when interviewers sit in the same interview, their ratings still vary. A meta-analysis of studies on interviewer agreement found that deciding the questions and scoring method in advance raises agreement, but some variation remains. In Japan, too, research on graduate interviews has reported that interviewers differ in the kind of candidate they rate highly. Another study, in which two people rated the same candidates, found that agreement varied widely by criterion (rank correlations of 0.04 to 0.41). Because human interviewers disagree this much, agreement between AI and humans should be judged against the variation among your own interviewers.

Source: Huffcutt, A. I., Culbertson, S. S., & Weyhrauch, W. S. (2013). Employment interview reliability: New meta-analytic estimates by structure and format. International Journal of Selection and Assessment, 21(3), 264–276. https://doi.org/10.1111/ijsa.12036

Source: Imajo, S. (2005). Empirical study on employment interviews: Multilevel analyses of applicant’s, interviewer’s, and organization’s effect on interview ratings [in Japanese]. Japanese Association of Industrial/Organizational Psychology Journal, 19(1), 3–16. https://doi.org/10.32222/jaiop.19.1_3

Source: Suzuki, T. (2016). Inter-rater reliability of recruiting interviews: Experimental study on evaluation components [in Japanese]. Japan Journal of Human Resource Management, 17(1), 69–91. https://doi.org/10.24592/jshrm.17.1_69

In the trial, two of your interviewers and the AI score the same interview transcripts independently. Separately from the score, interviewers can add a note for each criterion that it is “hard to judge from the transcript alone”. For each criterion, HR then sets side by side the number of interviews where the two interviewers’ scores differed by 2 or more points on the 5-point scale, and the number where the AI and each interviewer differed by 2 or more points.

Once the figures are in, HR assigns each criterion to a category (use the AI score / check in a human interview / re-test). A criterion is placed in “use the AI score” when it meets all of the following conditions.

  • The interviewers agree with each other
  • The gap between the AI and each interviewer is no larger than the variation between the interviewers
  • Scores are not bunched on a few values
  • Notes saying “hard to judge from the transcript alone” stay within the threshold set in advance

The AI score is one input among several. People still make the pass/fail decision, as before.

Criteria on which the interviewers disagreed are re-tested after the scoring guide is revised. Criteria where the gap between the AI and the interviewers exceeded the variation between interviewers, criteria you want to probe in live conversation, and criteria with many notes are checked in a human interview. Where the AI gave almost every interview the same score, the gap may only look small, so leave that criterion uncategorised and check it in human interviews until there are more cases. Add to the scoring guide the points interviewers weighed when scoring, and add real answers as examples for each level.

When interviewers ask whether it is fair to compare on transcripts alone, explain that the purpose of this scoring is to compare score gaps using the same material the AI sees. Criteria that are hard to judge from text are noted, and those criteria are checked in a human interview.

When interviewers ask whether this will be used to assess them personally, you can answer that what HR examines in the trial is the variation and gap for each criterion, not the performance of individual interviewers.

Note: Figures for each interviewer are seen only by HR and the interviewer concerned, and the approval request records only the category for each criterion. If you agree before the trial that interviewers’ names will not appear in internal reports and that the results will not be used in performance reviews, interviewers can score as they normally would, without worrying about their reviews, and the gap with AI scores is measured accurately. The same applies to the scoring of sampled interviews in live use.

A separate article covers how to run the scoring, categorise criteria by gap and estimate interviewer workload.

Method

Can You Rely on AI Interview Scores? How to Test Them →

Two interviewers and the AI score the same interviews, and each criterion is assigned to a category.

Tell interviewers up front what changes for them

Besides how ratings are used, interviewers worry about the burden of new procedures. To secure their cooperation in the trial, explain beforehand which of their tasks will go and which will stay.

ItemWhat changes for interviewers
Work that goes away
  • First-round interviews that repeat the same questions
  • Scheduling first-round interviews
Use of the time freed up
  • More interviews
  • Engaging with candidates and offer holders
  • Less overtime
  • Their core work (including preparing for second-round interviews)
Each department decides in advance how the time will be used
Work done differently
  • Read the AI interview record before the second-round interview
  • Start with the points that still need checking
What stays with interviewers
  • Checking the criteria assigned to “check in a human interview”
  • Pass/fail decisions from the second round onwards (HR decides who passes the first round after reading the AI scores and records)
  • Recording what they look for in the scoring guide and as example answers
Extra work at the start
  • Helping to write the scoring guide
  • Scoring in the trial
  • After finishing their own scoring, comparing the AI record with their impressions

Existing first-round interviews continue during the trial, so workload does not fall during this period; it rises by the extra work

Measure the time the added work actually takes in the trial, and share it together with how much of it will continue in live use. If you start without mentioning the added work, interviewers tend to feel they have simply been given more to do.

Management

State the purpose, the return and when renewal will be reviewed

Choose a purpose that matches the problem you want to solve

State a purpose that fits your company’s issues, such as reducing workload, making additional hires or making evaluation consistent. If there is more than one purpose, decide which one the trial results will be judged against. Under benefits, list both quantitative and qualitative benefits, and treat the benefit that matches the purpose as the main one.

Selection

How to Choose an AI Interview Tool: What to Compare →

Choose according to whether you want more capacity or better evaluation.

Show cost-effectiveness as the payback period and the impact on operating profit

HR measures the time saved on first-round interviews and the time spent designing the scoring guide and questions. Management, however, decides on the basis of the payback period, the impact on operating profit and the return on investment (ROI). In the approval request, convert the measured time into money using hourly rates, and translate it into these three measures by following the steps in the table below. Design work is needed not only at the outset but each time a role is added or the scoring guide is revised, so include it in the annual running cost as well. Use the hourly rate of whoever does each task.

Once live, criteria assigned to “check in a human interview” are checked in the second round. If this makes second-round interviews longer, deduct that time as well. If managers or executives conduct the second round, use a higher hourly rate than for the first round.

Cutting interview time changes neither labour costs nor profit unless you decide how the freed time will be used. Management often asks about this, so state the use of the freed time in the approval request too. Once you choose between more interviews, engaging with candidates and offer holders, less overtime, or core work, the first three can be expressed in revenue or cost terms. If overtime is reduced, the benefit includes better work-life balance for interviewers as well as lower labour costs. Record this as a qualitative benefit without converting it into money.

1

Measure time

Time saved on first-round interviews and time added to second-round interviews

2

Annual benefit

Multiply each by the hourly rate and take the difference

3

Net annual benefit

Deduct the annual running cost from the annual benefit

4

Measures for management

Payback period, impact on operating profit, return on investment (ROI)

ItemFormula
Time saved on first-round interviews
  • Time saved per interview × annual number of first-round interviews
  • Time saved per interview = time to prepare, conduct and write up a first-round interview − time for HR to read one AI interview record
Time added to second-round interviews
  • Time added per interview × annual number of candidates reaching the second round

Include the extra length of second-round interviews and the time second-round interviewers spend reading AI interview records beforehand. This cannot be measured in the trial, so use an estimate and replace it with measured figures in live use

Annual benefitTime saved on first-round interviews × first-round interviewer’s hourly rate − time added to second-round interviews × second-round interviewer’s hourly rate
Design time
  • At the outset: time to design the scoring guide and questions (included in the payback period calculation)
  • Annual: time per role addition or scoring guide revision × number of times per year (included in the annual running cost)
Net annual benefit
  • Net annual benefit = annual benefit − annual running cost
  • Annual running cost = annual fee + internal time (design, comparing human and AI scores) × hourly rate
Payback period
  • Total upfront cost ÷ net annual benefit

Upfront cost is the setup fee plus the internal time spent at the outset (design, trial scoring, preparing records), converted into money

Impact on operating profit
  • Impact on operating profit = overtime pay saved − annual fee
  • Overtime pay saved = time saved on first-round interviews × share used to cut overtime × hourly overtime rate (including premium)

Internal time is not a cash outlay, so only the annual fee is deducted here. If added second-round time increases overtime, deduct that overtime pay as well. Any setup fee is deducted separately for the first year. If freed time goes to more interviews or to engaging with candidates and offer holders, estimate it under “Additional hires” below; if it goes back to core work, record it as a qualitative benefit

Return on investment (ROI)
  • (Total benefit over the contract term − total cost over the contract term) ÷ total cost over the contract term
  • Total cost over the contract term = upfront cost + annual running cost × number of contract years

For time per interview, use your own figures measured in the trial, not the vendor’s. Measure the time to prepare, conduct and write up first-round interviews, the time to read AI interview records, and the time spent designing the scoring guide and questions during the trial, and report the measured values in the trial report.

Also show the net benefit under the three scenarios below, in case things do not go as projected. This lets the approver check whether the benefit still exceeds the cost under worse conditions. The annual fee is based on the interview volume assumed at contract, so it may not fall even if the number of interviews does.

  • Applications fall and the number of interviews is below projection
  • More candidates leave the AI interview part-way through or withdraw after it (assume those who leave part-way through return to a human first-round interview)
  • Second-round interviews take longer than projected

Estimate additional hires from the extra first-round interviews you can hold

Because AI interviews do not depend on interviewers’ schedules, you can offer a first-round interview to candidates previously screened out at CV screening for lack of interviewers, and to those who withdrew because no slot suited them. Record additional hires as a projection separate from the workload reduction, and decide on the full contract on the basis of whether the workload benefit alone exceeds the cost.

ItemFormula
Additional first-round interviews made possibleCandidates screened out at CV screening for lack of interviewers + candidates who withdrew because of scheduling (both annual actuals)
Additional hiresAdditional first-round interviews made possible × first-round-to-offer conversion rate × offer acceptance rate
Benefit of additional hires
  • Additional hires × cost of vacancy per unfilled position

Estimate the loss from revenue forgone while the position is vacant and the overtime cost of colleagues covering it

Effect on cost
  • Volume for estimating the annual fee = current first-round interviews + additional first-round interviews made possible
  • Second-round interviews = current second-round interviews + additional first-round interviews made possible × first-round pass rate

For the decision on the full contract, use the annual fee estimated from current volume. Do not include the additional first-round interviews in the calculation of time saved on first-round interviews

If you have not recorded why candidates were screened out at CV screening, do not project additional hires until you have actual records. Estimate the benefit only for roles where hiring is below target. Candidates who used to be screened out may pass at a lower rate than past results suggest, so measure the pass rate in the first few months of live use and update the projection. More withdrawals because of the AI interview would offset the gain, so reflect the incomplete interviews and withdrawals recorded in the trial in the projection.

Record qualitative benefits with the facts that support them

If the purpose is consistent evaluation, the main benefits are qualitative. Do not convert them into money; record the facts that support each one.

Qualitative benefitSupporting facts to record
Consistent first-round evaluation
  • Criteria assigned to “use the AI score”
  • Number of interviews where the interviewers’ scores differed
  • Number of interviews where the AI and each interviewer differed

Record the counts from the trial

What interviewers look for, captured in the scoring guide
  • Points added to the scoring guide
  • Example answers added to each level
Candidates do not have to repeat themselves
  • Human interviewers read the AI interview record before meeting the candidate
Visible grounds for evaluation
  • Scores used in the decision
  • Record of the statements the AI based its scores on
  • Interviewers and HR can decide from the same record
Better work-life balance for interviewers
  • Monthly overtime hours per interviewer
  • Number of interviews held outside working hours or on days off

Record the change before and after adoption in each case

Record better retention as a projected benefit, and say how it will be verified

When first-round evaluation follows one scoring guide and the grounds are kept on record, you can revise the scoring guide against the traits of employees who stay with your company. Over time, this should improve retention. Confirming the effect on retention needs at least a year of tenure data after hiring, and comparing with post-hire performance reviews takes several years. So record it as a projected benefit in the approval request, and set out when and how AI interview scores will be compared with your tenure data under “When and how results will be verified” in both the approval request and the trial report.

Split costs into upfront and annual running costs

TimingCost items
Upfront cost
  • Setup fee
  • Time to design the scoring guide and questions
  • Trial scoring time
  • Time to prepare records for scoring
Annual running cost
  • Annual fee (assumed volume, overage rate, treatment of unused interviews)
  • Design time for added roles or scoring guide revisions
  • Time spent continuing to compare human and AI scores

Estimate comparison time as time per interview × annual number of interviews, and design time as time per occasion × annual number of occasions.

Workload

Cut Interview Workload with AI Interviews →

Hand scheduling, preparation and write-ups to AI interviews.

Will the initial set-up add to HR’s workload?

The scoring guide can start from the criteria you already use, a standard template (such as the Ministry of Economy, Trade and Industry’s Fundamental Competencies for Working Persons), or a draft the AI creates from the experience level and job title. You then put it into your own words over the trial and live use.

Can we cancel if it does not suit us?

Assess whether the service suits you before signing and again when renewal is decided. Some annual contracts require you to pay for the rest of the term even if you cancel part-way, so check the cancellation terms before signing.

Fix when renewal will be decided at the start of the contract as well, working back from the notice deadline before expiry. If you wait until enough human-AI score comparisons have accumulated, the deadline may pass.

With Proomi, you can also start small with prepaid interview credits (Flex) and move to an annual contract later. No second setup fee is charged when you sign the annual contract. Contract terms and cancellation conditions are on the pricing page.

Pricing

Pricing →

Estimate from your annual interview volume; contract terms and cancellation conditions are included.

Could the service be discontinued?

Besides the vendor’s business continuity, check how many days’ notice is given before discontinuation, in what format and by when records and scoring guides can be exported, and how long it takes before data is deleted.

What you keep if you switch servicesWhat you would have to measure again
  • Scoring guide written in your own words
  • Records exported before the contract ends
  • Trial verification procedure
  • Results on agreement with the AI

Legal and IT

In the trial, settings are tested in role-plays with employees acting as candidates before any real applicants are interviewed. Video and audio are passed to the vendor for processing even at this stage, so explain the purpose of use to the employees taking part. Before interviewing applicants, complete the agreement on handling personal data and the security review, and prepare the notice to candidates and an alternative for those who would find an AI interview difficult.

Items to confirm with the vendor

ItemWhat to confirm
Storage and processing location
  • Countries where data is stored and processed
  • Companies handling it there (including AI model providers)
Subcontracting
  • List of subcontractors (including AI model providers)
  • Notice when subcontractors change
Use for training
  • Whether interview data is used to train AI
  • What the contract with AI model providers says about use for training
Retention and deletion
  • Retention period
  • Deletion method
How pass or fail is decided
  • Whether the product is designed so that the AI never makes the pass/fail decision on its own
Information used in evaluation
  • Whether facial expressions or tone of voice are used in evaluation
Bias checks
  • Whether the vendor regularly checks for score bias and for bias arising from differences in speaking style or culture

Items to ask your legal department to review

ItemWhat to ask them to review
Stating the purpose of use
  • Purpose of use shown to candidates and to employees taking part in role-plays (including comparison with interviewer scores and checks for evaluation bias during the trial)
  • How the purpose of use is shown
Information collected
  • Scope of information collected
  • Topics that must not be asked about in selection (health, family, place of origin, beliefs and so on)
Notice to candidates and alternatives
  • Notice that an AI conducts the interview
  • Alternative for candidates who would find an AI interview difficult
Oversight of the vendor
  • Agreement with the vendor on handling personal data
  • Supervision of the vendor as a contractor
Storage and processing overseas
  • How overseas storage or processing is handled

Ask your legal department which of these are legal obligations and how they are treated under current laws and guidelines.

Vendor and legal checklists (Excel)

The tables show only the main items. We can send checklists covering every item, including security (authentication, access control and so on) and integration, with space for answers. The same file contains the sample approval request and trial report. The workbook is in Japanese; we can prepare an English version on request.

  • Sample approval request (all items)
  • How to apply the decision criteria
  • Sample trial report (all items)
  • Vendor checklist (with space for answers)
  • Legal checklist (with space for findings)

A separate article explains how facial expressions and tone of voice are handled; the product page FAQ covers integration with applicant tracking systems.

Scoring

Choose an AI Interview Tool That Explains Its Scores →

Check what it reads and what it scores against.

Product

Proomi’s AI interviewer →

AI interview features and the path to adoption.

We provide our standard written answers and explanatory materials from the enquiry stage. Please ask if you need answers in your own format or a response to an RFP.

Templates

Get the trial plan and the full-contract conditions approved in one request

The samples assume that the trial and the full contract are approved together in one request. The conditions for moving to a full contract are fixed at approval, so after the trial you only need to check whether the figures from your own interviews meet them.

1

Obtain approval

The trial plan is approved together with the conditions, cost and term of the full contract

2

Trial (about one month)

In your own interviews, record the gap between the AI and the interviewers, time per interview, incomplete interviews and withdrawals

3

Check the conditions

The heads of HR and the business unit check against measured figures whether the conditions for a full contract are met

4

Report to the approver

Report the results. Proceed to a full contract if the conditions are met; otherwise do not

If your practice is to approve the trial and the full contract separately, split the same items across two approval requests.

A trial shows which criteria have large gaps, where operational problems arise, and the extent of incomplete interviews and withdrawals. All of this feeds into the provisional categories and the decision on the full contract. Whether the gap patterns persist should be checked in live use as cases build up for each role, and the link with retention after hiring takes years longer to confirm. So say in the trial report that the categories are provisional, and set out the conditions for not renewing in the approval request.

When selecting a product through an RFP

When a product is selected through a request for proposal (RFP), add a selection stage before the approval request. First, draw up a long list of candidate services. Next, assess them on paper using public information and enquiries to vendors, and narrow them to a short list. Send the RFP to the short-listed services, compare the responses and decide which to trial. Record the selection process in the approval request, run the trial once approved, and make the decision on the full contract from its results. Interviewer scoring work grows with every service trialled, so trial only as many services as you can realistically assess.

For the long list, see AI Interview Services: 16 Compared, Nov 2026. For the RFP questions, use the evaluation items in How to Choose an AI Interview Tool: What to Compare together with “Items to confirm with the vendor”. Set the evaluation criteria and their weights before issuing the RFP; if you set them after reading the responses, they end up tailored to a particular service.

Whether AI scores agree with your interviewers’ scores is compared using trial results, not RFP responses. So the RFP should also ask about the trial period, how records for scoring will be provided, and what the vendor will be responsible for. Record the items compared in the RFP in the selection process section of the approval request, and the trial results in the trial report. If you do not run an RFP, describe how you narrowed the field using public information and comparison tables.

How AI and human interviews fit together once live

When the head of HR or recruitment marketing asks whether candidates will drop out, start by showing how AI and human interviews share the work once live. Candidates take the AI interview first, and those who pass move on to a human interview. Human interviewers read the AI interview record before meeting the candidate, so candidates do not have to repeat themselves and interviewers can pick up where the AI interview left off. Interviewers decide what to ask again, but anything that bears on pass or fail should be confirmed with the candidate rather than judged from the AI record alone. Time in the human interview goes to the criteria assigned to “check in a human interview” and to explaining the company to the candidate.

If you record during the trial how many candidates left the AI interview part-way through and how many withdrew after it, the trial report can answer “Will candidates drop out?” with figures from your own candidates. In the trial, the AI interview is added on top of existing interviews, so withdrawals may be higher than when the AI interview replaces the first round in live use; note this alongside the figures. How to review candidates’ reactions from interview records is covered in “Question Design and AI Interviewer Behaviour“.

During the trial, AI interview records are not passed to the next interviewer, and human interviews continue independently as before. Records are passed on to the next interviewer only once live, after the trial records have been checked for transcription accuracy and the quality of the follow-up questions.

Sample approval request

The main items relevant to the decision are shown here. The company in this example treats consistent evaluation as the main benefit and judges whether the cost is covered by the workload benefit. Rows such as the selection process, use of freed time, and when and how results will be verified, along with example counts for the decision criteria and how to apply them, are in the Excel workbook we send on request.

ItemExample
PurposeAdopt AI interviews to make first-round evaluation consistent. Run a trial before the full contract, and decide whether to proceed on figures from our own interviews
Scope and volume
  • Mid-career sales hires
  • 5 employee role-plays, then 20 applicant interviews
Cost
  • Trial: no charge (vendor’s free trial)
  • Full contract: setup fee ¥X, annual fee ¥X, 1-year term
Decision criteria
  • Two interviewers and the AI each score the same interviews, and score gaps are compared for each criterion
  • Criteria where the two interviewers’ scores align and the gap between the AI and each interviewer is no larger than the gap between the two: use the AI score
  • Criteria where the two interviewers’ scores diverge: revise the level definitions in the scoring guide and re-test
  • Criteria where the gap between the AI and each interviewer exceeds the gap between the two, criteria with many interviews noted as “hard to judge from the transcript alone”, and criteria to be probed in live conversation (e.g. motivation for the work): check in a human interview
  • Criteria where the AI gave almost every interview the same score: leave it uncategorised and check in human interviews until there are more cases

The thresholds for “align”, “no larger” and “many” are set as counts before the trial

Conditions for a full contract
  • At least one criterion is assigned to “use the AI score”
  • The criteria to check in a human interview have been settled
  • Once live, the time spent comparing human and AI scores plus the time added to second-round interviews is no more than X hours per interviewer per month
  • The annual benefit estimated from measured values exceeds the annual running cost (additional hires are excluded as they are projections)
  • The upfront cost is paid back within X years
  • At the time of the decision, candidates who left the AI interview part-way through or withdrew after it (anyone in both groups counted once) are no more than X% of those who started the AI interview
  • The annual fee is within budget

Sign the full contract if all conditions are met; otherwise do not. Record separately the number of candidates who chose a human interview only

Decision on the full contractWithin two weeks of the end of the trial, the heads of HR and Sales decide whether the conditions for a full contract are met and report the result to the approver
Renewal review date and conditions for not renewing

Decide on renewal in the tenth month of the contract. If any of the following applies, give written notice of non-renewal by the deadline set in the contract

  • No criteria remain in “use the AI score”
  • Once live, the time spent comparing human and AI scores plus the time added to second-round interviews exceeded X hours per interviewer per month
  • The annual benefit estimated from live use is below the next year’s annual running cost

The categories for each pattern of gaps are shown in a diagram in “Can You Rely on AI Interview Scores? How to Test Them“.

Proomi’s free trial lasts about one month, and extensions can be discussed. During the trial, we hold a meeting to revise the draft scoring guide and provide a scoring sheet template for two interviewers and the AI. At the end, we hold a meeting to review the category for each criterion together. All of this is free of charge. The roles at each stage of adoption are set out in Onboarding and Support.

Sample trial report (before the full contract)

The report refers to the approved request and states whether the conditions for a full contract have been met. The main items are shown here. The purpose and selection process are already in the approval request and need not be repeated.

ItemExample
Trial results

Two interviewers and the AI scored 20 applicant interviews (5-point scale)

  • “Drawing out customer issues” and “Structuring proposals”: the interviewers agreed, the AI–interviewer gaps and notes were within the thresholds set before the trial, and scores were not bunched, so these were provisionally assigned to “use the AI score”
  • “Mobilising others”: the interviewers’ scores varied, so the level definitions were expanded and the criterion will be re-tested in the next hiring round. Until then it is checked in the second round
  • “Motivation for the work”: to be probed in live conversation, so checked in the second round as stated in the approval request
  • Transcription errors and the AI’s follow-up questions were checked in the records. Counts by error type are in a separate table
Check against the conditions for a full contract
  • Criteria assigned to “use the AI score”: X, X
  • Criteria to check in a human interview: X, X
  • Time spent comparing human and AI scores plus time added to second-round interviews in live use (projected) is X hours per interviewer per month, within the cap of X hours
  • Annual benefit of ¥X exceeds the annual running cost of ¥X (still exceeds it by ¥X if volume is X% of projection)
  • Payback period is X years, within the X-year limit
  • Of X candidates who started the AI interview, X left part-way or withdrew after it (at the time of the decision), within X%. X candidates chose a human interview only
  • Annual fee of ¥X is within the budget of ¥X
Quantitative benefits (workload, measured)
  • Annual benefit: time saved on first-round interviews (X min per interview × X per year × hourly rate ¥X) − time added to second-round interviews (X min per interview × X per year × hourly rate ¥X) = ¥X per year
  • Net annual benefit: annual benefit ¥X − annual running cost (annual fee ¥X + ¥X for X hours of internal time) = ¥X per year
  • Payback period: upfront cost ¥X (setup fee ¥X + ¥X for X hours of internal time) ÷ net annual benefit ¥X = X years
  • Impact on operating profit: overtime pay saved ¥X − annual fee ¥X = ¥X per year
  • Return on investment (ROI): (annual benefit ¥X × X years − total cost ¥X) ÷ total cost ¥X = X% (total cost is the upfront cost plus X years of annual running cost)
  • Net annual benefit if things do not go as projected
    • Interview volume at X% of projection: ¥X per year
    • Candidates leaving the AI interview part-way through or withdrawing after it up by X%: ¥X per year
    • Second-round interviews X min longer each: ¥X per year

All use measured trial values (time added to second-round interviews is projected)

What the trial did not establish
  • With 20 interviews, the results cannot yet be distinguished from chance
  • The results reflect the particular pair of interviewers who scored
  • The relationship with post-hire performance reviews is unknown
  • The pass rate for additional hires is a projection, as there are no results yet for previously screened-out candidates
When and how results will be verified
  • Once live, sample 10 interviews a month. Two interviewers who did not conduct those candidates’ human interviews score them from transcripts that exclude the AI evaluation
  • At 50 and 100 interviews in total, check whether any criterion exceeds the decision criteria, reset for the larger number of cases
  • In each month’s results, read the statements the AI relied on in interviews with gaps to find the cause
  • In the first X months of live use, measure the pass rate of candidates newly given a first-round interview and revise the projection of additional hires
  • Map the selection criteria to the items in post-hire performance reviews
  • Starting with each hiring year once one-year tenure is known, compare AI interview scores with post-hire reviews and tenure data, and add the results to the renewal decision

Post-hire verification takes years to produce results, so set out the when and how results will be verified in the approval request and update them in the trial report in light of the trial results.

Templates and checklists (Excel, 5 sheets)

We can send an Excel workbook with every item of the samples shown here in shortened form, plus checklists for the review items. It also covers selecting the scope, internal time, notice to candidates, steps to complete in advance, and risks and countermeasures. The workbook is in Japanese; we can prepare an English version on request.

  • Sample approval request (all items)
  • How to apply the decision criteria
  • Sample trial report (all items)
  • Vendor checklist (with space for answers)
  • Legal checklist (with space for findings)
After hiring

Was the Interview Evaluation Right? →

Compare selection ratings with post-hire performance.

Related pages

Further reading

The following articles cover how to verify scores in a trial and revise the scoring guide.

Method

Can You Rely on AI Interview Scores? How to Test Them →

Two interviewers and the AI score the same interviews, and each criterion is assigned to a category.

Scoring guide

Can AI evaluate candidates correctly? HR writes the criteria →

Prepare a scoring guide and refine it by comparing human and AI scores.

Candidates

Question Design and AI Interviewer Behaviour →

Improve how candidates experience the interview, using the interview records.

Onboarding

Onboarding and Support →

The stages from first enquiry to live use, and who does what.

Comparison

AI Interview Services: 16 Compared, Nov 2026 →

Major services grouped and compared by what they optimise for.

Company

Company profile →

About the company behind Proomi.

Send us the questions raised in your approval process

Send us the questions raised internally or a draft of your approval request. You can leave out company names and amounts. We will sort the questions into those answered by our materials and those to be checked by estimate or trial, and reply within about five working days. If you include the names of your criteria, we will also send a scoring sheet template with your criteria filled in.