How to build an interview evaluation sheet: axes and questions that put evidence on record

Introduction

What this article covers

An interview evaluation sheet is a record that lets every interviewer assess candidates against the same standard and leaves the reasons for each judgement on file. This article works through how to choose evaluation axes, how to write level definitions, how to decide the questions for each axis, and how to write up the sheet straight after the interview.

At the end of the article, you can download an Excel evaluation sheet for mid-career sales. Level definitions and questions for six roles are on our interview rubric templates page.

In brief

Key points

• An evaluation sheet has three parts: evaluation axes, a five-level definition for each axis, and the questions that go with them
• Levels are distinguished by behaviour you can confirm from what the candidate says in the interview, not by words such as “excellent”
• Straight after the interview, write down what the candidate said and, separately, why you chose the level

Definition

An interview evaluation sheet records judgements against the same standard, whoever is interviewing

If each interviewer looks for something different, the same candidate can pass with one interviewer and fail with another. Settling in advance what to look at and how to assign levels keeps the standard the same when the interviewer changes.

The problem is likely to be especially sharp in Japan’s graduate recruitment, where companies hire students each year before they graduate, often without assigning them to a particular job. With no work experience to assess, interviewers tend to focus on personal qualities such as motivation and cooperativeness, and on whether the candidate will fit the organisation.

A 2005 study by Imashiro, which analysed interview data from graduate recruitment at Japanese companies, suggested that interviewers differ in the kind of applicant they rate highly. In interviews without set questions or criteria, applicants whose personality resembled the interviewer’s tended to be rated higher; in interviews with criteria fixed in advance, this tendency was weaker in some respects.

Source: Imashiro, S. (2005). Empirical study on employment interviews: Multilevel analyses of applicant’s, interviewer’s, and organization’s effect on interview ratings [in Japanese]. Japanese Association of Industrial/Organizational Psychology Journal, 19(1), 3–16. https://doi.org/10.32222/jaiop.19.1_3

The sheet’s other job is to keep the reasons for each judgement. A second-round interviewer can read the first-round record and start with the points nobody has checked yet. After hiring, the records let you look back at how people who passed at a given level on a given axis are performing in the job.

An interview with questions and a method of evaluation decided in advance is called a structured interview. Many studies outside Japan have found that structured interviews predict job performance better than unstructured ones. The evaluation sheet is the part that fixes the method of evaluation. We summarise the research in “What is a structured interview?”.

Sources: McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616. https://doi.org/10.1037/0021-9010.79.4.599 / Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994

Scope

Choose only the axes you need to check in hiring, and use the wording of your performance reviews

There is no single right number of evaluation axes. A useful test is whether you can put a question into the interview that checks the axis. If you cannot, the interview is unlikely to produce evidence for it, and interviewers will often end up unable to choose a level. For a mid-career sales role, for example, choose behaviours that lead to results in that role, such as problem framing, execution and getting others involved, action towards targets, and repeatability.

Taking the wording of your axes from your own performance review items is a shortcut. When hiring and reviews use the same words, you can set interview levels side by side with later reviews. In our simulation, the competency indicators from the Japanese civil service’s published criteria for junior staff worked as evaluation axes without rewording. The results are in “Use your performance review criteria in interviews, word for word”.

If you have no performance review items, you can start from a public framework. For graduates, the elements of METI’s “Fundamental Competencies for Working Persons” (initiative, ability to identify issues, listening and so on) can serve directly as axis names. For mid-career roles, the Ministry of Health, Labour and Welfare’s Vocational Ability Evaluation Standards break work down into units of duties and abilities and give examples of the behaviour expected in each.

Sources: Ministry of Economy, Trade and Industry. Fundamental Competencies for Working Persons in the age of the 100-year life (diagram of three abilities and twelve elements) [in Japanese]. https://www.meti.go.jp/policy/kisoryoku/pdf/kisoryoku_image.pdf / Ministry of Health, Labour and Welfare. Structure of the Vocational Ability Evaluation Standards [in Japanese]. https://www.mhlw.go.jp/content/11800000/001018939.pdf

L1 to L5

Define each level by behaviour you can confirm from what the candidate says

A common format asks interviewers to pick a rating for each axis from “5: outstanding” down to “1: poor”. The trouble is that what counts as “outstanding” exists only in each interviewer’s head, so the same answer can get a different level when the interviewer changes.

Instead, write each level definition in terms of the behaviour you would see from a candidate at that level, not an impression word such as “excellent”. In the research literature this format is called behaviourally anchored rating scales (BARS), and it has been in use since the 1960s. For the axis “Framing the problem” in a mid-career sales role, the levels might read as follows. L5 is the highest level and L1 the lowest.

Level	Definition
L5	Pointed out something the customer did not yet see as a problem, and set the agenda for the discussion
L4	Checked the cause behind the request and redefined the problem to be solved
L3	Made the request concrete and confirmed, in the customer’s own words, why it was needed
L2	Took down the request accurately and turned it into a proposal as it stood
L1	Can describe how the deal unfolded, but says nothing about what they saw as the problem

Sources: Nagai, T. (2007). Competency as criteria of performance appraisal (meeting report) [in Japanese]. Japanese Association of Industrial/Organizational Psychology Journal, 21(1), 81–85. https://doi.org/10.32222/jaiop.21.1_81 / Schwab, D. P., Heneman, H. G., & DeCotiis, T. A. (1975). Behaviorally anchored rating scales: A review of the literature. Academy of Management Proceedings, 1975(1), 222–224. https://doi.org/10.5465/ambpp.1975.4982613

Note: The axis definitions and example questions are our own examples. They were not defined by any public body.

The higher the level, the wider the scope the candidate took on and the more specific the behaviour. L1 does not mean “poor”; it describes a candidate who mentions relevant experience but does not say what they themselves did. Not every hire needs L5, so decide the level you will treat as the hiring benchmark for each role and grade.

We believe this format also narrows differences in how leniently or strictly interviewers rate. Because the level depends on which behaviour the candidate described, there is less room for each interviewer’s personal sense of the scale.

Keep impression words such as “proactive” or “composed” out of the definitions, because interviewers read them differently. What happens when they do is covered in “Same answer, different score: when interviewers read the criteria differently”.

Before the interview

Decide in advance the questions that draw out evidence for each axis

Even with axes and definitions in place, you cannot choose a level if the candidate never talks about that axis. General questions such as “tell us about your strengths” or reasons for applying leave some axes without evidence.

We checked whether general questions alone produce evidence, using simulated interviews. We created 30 graduate personas who differ in background and speaking style, and had an AI answer in line with each persona. As evaluation axes, we used the competency indicators in the Japanese civil service’s published criteria for junior staff, unchanged.

The questions were the ten that surveys of companies listed as commonly asked in interviews: a self-introduction, the candidate’s strengths, what they put most effort into as a student, reasons for applying, and others most interviewers will recognise. With only these ten, more than 40% of candidates (12 or more of 30) gave no evidence on four axes, including understanding instructions and accuracy. Questions like these do not naturally bring out stories about receiving instructions from a supervisor or noticing a mistake in one’s work.

Sources: National Personnel Authority and Cabinet Bureau of Personnel Affairs. Personnel evaluation guide: Procedures for evaluatees (April 2026 edition) [in Japanese] (axis names). https://www.jinji.go.jp/content/900006282.pdf / National Personnel Authority and Cabinet Bureau of Personnel Affairs. Personnel evaluation manual: Reference materials (September 2021 edition) [in Japanese] (axis descriptions). https://www.jinji.go.jp/content/900006260.pdf / Gakujo Co., Ltd. (2023, February). Survey of HR staff on interviews [in Japanese]. https://service.gakujo.ne.jp/wp-content/uploads/2023/10/230222-comenq.pdf / Rikunabi job-hunting guide. Common internship interview questions and tips for answering [in Japanese] (survey December 2024 to January 2025). https://job.rikunabi.com/contents/internmensetsu/3286/

Adding three questions matched to the axes sharply reduced the number of candidates with no evidence, to around one in ten on every axis (4 or fewer of 30).

One of the added questions was:

Before it was added, two in three candidates (20 of 30) gave no evidence for understanding instructions. Once it was added, none did. On the evaluation sheet, write one to three questions like this for each axis, chosen to draw out evidence. For “Framing the problem” in a mid-career sales role, a question might be:

The personas are not real students, so look at the pattern rather than the exact numbers: adding questions matched to the axes greatly reduces the number of candidates who give no evidence. The same principle of placing a question to check each axis applies to mid-career hiring. For details of the simulation, see “Use your performance review criteria in interviews, word for word”.

Gaps

When an axis produces no evidence, say so on the sheet

If you asked but the candidate did not describe their own behaviour for an axis, do not force a level. Record that no evidence came up. If you fill in a level by inference, anyone reading the record later cannot tell whether it was checked or guessed.

Sometimes an interview leaves a concern that does not fit any axis. Do not discard it; note it in a column for concerns and points to check in the next interview.

Once these are written down, the second-round interviewer knows what to check first.

If one axis keeps producing no evidence across candidates, the cause may lie with the questions. Check whether the interview actually includes a question that tests that axis.

Note-taking

Write facts and evaluation separately, straight after the interview

We recommend giving the sheet separate columns: one for what the candidate said and did, recorded as it was, and one for the reason you chose the level. Reading only the facts column, another interviewer can choose a level again from the same material.

To keep the writing light, note the gist of what the candidate says under each axis during the interview. If sorting by axis while listening is hard, notes in time order are fine. Those notes become the facts column. After the interview, all that is left is the level and a one-line reason. The reason should state one thing: which words in the definition the behaviour matched.

When you have time, or when the judgement may be contested, write the scene, the candidate’s behaviour, how it matches the definition, and what separates it from the next level up, in that order. That makes the reasoning easy to follow. Here is a fuller write-up for the same candidate.

Comments with no scene or behaviour, such as “cheerful and likeable” or “strong communication skills”, leave other interviewers unable to follow the reasoning.

The longer you wait, the more your overall impression replaces your memory of what was said. Fill in the sheet on the day of the interview, ideally straight afterwards.

Variations

Change the axes for graduates, mid-career hires, frontline roles and staffing registration interviews

Even within one company, what you can check depends on the type of hiring.

• Graduate hires on a general career track: they have no work experience, so look for initiative and the ability to identify issues in their part-time jobs and studies
• Mid-career roles: describe the behaviours that lead to results in the role’s own terms, such as framing the problem and repeatability for sales, or troubleshooting and design decisions for engineers
• Frontline roles such as manufacturing and logistics: look at understanding instructions and reporting, and at working to procedures and rules
• Registration interviews at staffing agencies (interviews with people who sign up to be placed with client companies): look at the range of work they can do and how they work in a new workplace (axes: experience in detail, instructions and reporting, adapting to a new workplace, and rule compliance)

Instructions and reporting, an axis shared by frontline roles and staffing registration interviews, might be defined like this.

Level	Definition
L5	Checked the purpose of the instruction before acting, and when something went differently from plan, sorted out the points that needed a decision and reported early
L4	Checked anything unclear on the spot before starting, and told the person in charge immediately on noticing a mistake or delay
L3	Worked as instructed and reported when finished
L2	Worked on instructions, but only checked or reported when asked
L1	Talks about the work, but says nothing about how they received instructions or reported

A question to check this axis:

The staffing example assumes a registration interview held by the agency with people registering with it. Except in temp-to-perm placements (where the worker is placed with a view to being hired directly by the client), it is not intended for interviews in which a client company selects the workers to be dispatched to it. Check practical conditions, such as the days and hours someone can work, separately rather than as evaluation axes.

The evaluation axes and level definitions for six roles are on the “interview rubric templates” page.

Calibration

Check differences between interviewers by having each assign levels to the same record separately

Even with written definitions, interviewers will still read them somewhat differently.

One example from Japan is a 2016 study by Suzuki of final-round graduate interviews at a software company. Two interviewers, a board director and a division head, rated the same candidates on five evaluation axes, including attitude as a working adult, personality fit and potential to deliver results. Their ratings agreed only weakly on most axes, and for potential to deliver results, which looks ahead to performance after joining, there was almost no relationship at all. Even two people this senior can read the axes that differently.

Source: Suzuki, T. (2016). Inter-rater reliability of recruiting interviews: Experimental study on evaluation components [in Japanese]. Japan Journal of Human Resource Management, 17(1), 69–91. https://doi.org/10.24592/jshrm.17.1_69

A simple way to check is to have two interviewers assign levels to the same candidate’s record separately, then discuss the axes where their levels differ. You do not need to do this every time; the first few uses of a new sheet are enough. If you have many interviewers, set aside one session before you start where everyone assigns levels to the same record and then goes through the differences together. If a difference comes from how a definition is read, fix the wording of the definition, not the interviewer. A phrase such as “a major customer”, for instance, means something different to a candidate from a company whose customers are a different size from yours. Add a benchmark to the definition, such as “a customer in the top 10% of our revenue”.

What this discussion fixes is the definition, not the candidate’s level. Combine the interviewers’ levels for each candidate using a method decided in advance. For example, average the interviewers’ levels on each axis, and set a minimum such as “L3 or above” for axes the candidate must meet. If only one interviewer recorded no evidence on an axis, combine the levels from the interviewers who did assign one. Keep the “no evidence” record as well, and carry it forward as a point to check in the next interview.

In 1995, Conway and colleagues pooled studies of how closely interviewers’ ratings agree. Ratings from several interviewers were reliable when combined by a method fixed in advance, but there was no such support when the panel reached a single rating through discussion. The paper does not say why. Our view is that discussion tends to drift towards the opinion of the most senior or most vocal person in the room.

Source: Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80(5), 565–579. https://doi.org/10.1037/0021-9010.80.5.565

You do not need to drop the discussion. Hold it after everyone has assigned their levels, and use it to talk through why levels differed and any concerns that did not fit the sheet. That way you keep what discussion is good for. Where the president or an executive makes the final call, give them the combined levels and the records as input, and if they change the outcome, record why. That way the decision can be reviewed in the next hiring round.

Validation

Review the sheet’s records against post-hire evaluations

An evaluation sheet is not finished once it is built. For the people you hire, set the interview levels next to their later performance reviews. That shows, axis by axis, whether the interview’s assessment matched how they were rated after joining.

The comparison can turn up surprises. In a 2013 study of graduate recruitment interviews at a Japanese company, Suzuki found that interview ratings did not predict post-hire evaluations: people rated highly at interview were not necessarily rated highly after joining. You cannot know how your own interviews perform until you put the records side by side.

Source: Suzuki, T. (2013). Examining the predictive validity of recruiting interview [in Japanese]. Japan Journal of Human Resource Management, 14(2), 4–26. https://doi.org/10.24592/jshrm.14.2_4

For a staffing agency’s registration interviews, the comparison is with whether people stayed in their placements and how client companies rated them. Comparing registration interview levels between people whose placements ended early and people who stayed longer shows which axes were linked to staying in the job.

If your hiring axes use the same words as your performance reviews, this comparison is straightforward. Which axes to review first is covered in “Was your interview evaluation right? You find out after the hire”.

Uses

The axes and definitions on your evaluation sheet also work as criteria for AI interviews

An evaluation sheet with evaluation axes, five-level definitions and questions can be used, unchanged, as the criteria for an AI interview. The AI reads answers using the written definitions as its guide, so the more specific the wording, the easier it should be to follow why a level was chosen.

In Proomi, our AI interview service, companies write the evaluation axes and the definition of each level freely. The AI reads the interview transcript, assigns a level for each axis and quotes the remarks it based that level on. Where nothing the candidate said supports an axis, Proomi can record “no evidence”. For how this works in staffing registration interviews, see the staffing registration interviews page.

How to write the criteria is covered in detail in “Can AI evaluate candidates correctly? HR writes the criteria”.

Q&A

Frequently asked questions

How many levels should an interview evaluation sheet have?

If you can write distinct definitions for them, we recommend five. With three, answers of very different quality end up at the same level and are hard to compare later. If your current sheet has three levels, you can move to five by placing your A, B and C at L4, L3 and L2 and adding definitions for L5 and L1.

How many evaluation axes should there be?

There is no set number; choosing only what you need to check in hiring is enough. Every axis you add needs a question to check it, which makes the interview longer. In our simulation, going from 10 to 13 questions increased the number of exchanges in the interview by about a quarter. How to check whether your axes actually separate candidates is covered in “More criteria do not mean more evidence”.

Should graduate and mid-career hiring use different sheets?

We recommend it. Graduates have little experience, so use axes that look at how they approach work, such as initiative and the ability to identify issues, with questions about part-time jobs and studies. For mid-career hires, use axes written in the role’s own terms, with questions about real work situations.

Can we manage the sheets in Excel?

With a few interviewers and up to a few dozen interviews a month, Excel is enough. When each interviewer keeps their own file, versions of the definitions drift apart, so replace everyone’s file whenever a definition changes. As volume grows, consider a system that keeps all records in one place, such as an applicant tracking system.

How should we write comments?

At a minimum, give the level and one line on which words in the definition the behaviour matched. For a fuller write-up, describe the scene, the candidate’s behaviour, how it matches the definition, and what separates it from the next level up, in that order.

How long does it take to write the axes and definitions?

Allow a few hours per role while you are getting used to it. Rather than leaving it to HR alone, bring in one person who actually interviews for that role, so the definitions use the language of the job.

Is comparing with post-hire results worthwhile if we hire only a few people?

With few hires, patterns by axis are hard to see. Even so, reading each hire’s interview record alongside how they are doing in the job shows you what the interview missed and where the wording of a definition falls short.

What should we watch for when adding our own questions?

Do not ask about matters the applicant is not responsible for, such as permanent domicile (honseki) or family members’ occupations, or about matters of belief and conviction, such as religion or the political party they support, because these have no bearing on aptitude or ability. Japan’s Ministry of Health, Labour and Welfare sets out the thinking in its guidance on fair recruitment and selection.

Source: Ministry of Health, Labour and Welfare. Basics of fair recruitment and selection [in Japanese]. https://kouseisaiyou.mhlw.go.jp/basic.html

Download

Evaluation sheet templates

We have turned the mid-career sales evaluation sheet into an Excel template, with definitions and example questions for its four evaluation axes and a sheet on how to use it. Download it and replace the axes and wording with your own.

Evaluation axes, level definitions and example questions for six roles (graduate hires on a general career track, mid-career sales, mid-career engineers, staffing agency registration interviews, store staff and customer support) are on our interview rubric templates page.

Related

Read next

For more on writing and reviewing evaluation criteria, see these articles.