What is a structured interview? What the research shows, and how to build your own

Introduction

What this article covers

A structured interview is one where the questions and the way answers are rated are decided before the interview, and every candidate goes through it in the same way.

This article sets out what is known about structured interviews, drawing on research from both Japan and elsewhere. It then walks through how to build one for your own organisation, and looks at two common concerns: the preparation it takes and how candidates experience it.

In brief

Key points

• Structured interviews predict performance after hiring more than twice as strongly as unstructured interviews, on average (correlation 0.42 vs 0.19, 2022 re-analysis).
• Deciding how answers are rated (the evaluation axes and level definitions), not just the questions, also brings interviewers’ ratings closer together.
• Japanese graduate hiring leans on interviews about personality and fit, and values building rapport, so the format that suits it keeps the conversation natural and standardises how answers are rated.

Definition

A structured interview fixes the questions and the rating method in advance

Interviews fall roughly into three types, depending on how much is decided beforehand.

Type	Questions	How answers are rated
Structured interview	The same questions, in the same order, for everyone	Evaluation axes and level definitions are set in advance, and each answer is given a level
Semi-structured interview	Set questions form the core, with follow-up depending on the answer	Evaluation axes are set, but part of how levels are given is left to the interviewer
Unstructured interview	The interviewer decides on the spot	Left to the interviewer’s judgement

If the broad questions are fixed but what to probe and how to rate differ from one interviewer to the next, the interview is closer to semi-structured.

The idea is not new. Hiring research has examined structured interviews extensively since the 1980s. The situational interview, which describes a scenario and asks “What would you do?”, was published by Latham and colleagues in 1980. For the behaviour description interview, which asks about past behaviour, Janz reported one of the earliest comparisons with unstructured interviews in 1982.

Sources: Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65(4), 422–427. https://doi.org/10.1037/0021-9010.65.4.422 / Janz, T. (1982). Initial comparisons of patterned behavior description interviews versus unstructured interviews. Journal of Applied Psychology, 67(5), 577–580. https://doi.org/10.1037/0021-9010.67.5.577

Effect

Structured interviews are better at predicting performance after hiring

How does research test whether an interview is doing its job? Most studies put two ratings side by side for the same people: the interview rating and the rating their manager gives them after they join. If the people the interview rated highly are also rated highly on the job, the interview is predicting performance.

The strength of that link is expressed as a correlation. At 0, interview ratings and later ratings are unrelated; at 1, they move in lockstep. Performance also depends on the role someone is placed in, their manager and their own development, so no selection method gets close to 1. Research compares methods by their relative strength.

A 1994 analysis by McDaniel and colleagues is a cornerstone of interview research. Combining the results of hundreds of studies, they found that structured interviews, with fixed questions and a fixed rating method, were better predictors of performance than unstructured ones. The same analysis found differences by question content too: questions that describe a scenario and ask what the candidate would do predicted better than questions about psychological traits.

Source: McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616. https://doi.org/10.1037/0021-9010.79.4.599

Among studies comparing selection methods, a 1998 paper by Schmidt and Hunter was the standard reference for many years. It estimated the predictive power of structured interviews at the same level as general mental ability tests.

Source: Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262

In 2022, Sackett and colleagues revisited these estimates. On-the-job ratings exist only for people who were hired, so earlier analyses applied a correction to estimate what the correlation would be across all applicants. Sackett and colleagues argued that this correction had been too strong. Once it was revised, the figures for every method went down, but structured interviews kept the highest average of the methods compared. The gap with unstructured interviews is wide (correlation 0.42 for structured interviews, 0.19 for unstructured).

Source: Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994

Reliability

Standardising the questions and the rating narrows the gap between interviewers

For an interview to predict anything, interviewers looking at the same candidate need to reach similar ratings. You cannot track your weight on scales that show a different number every time you step on. The same goes for interviews: if the rating depends on who is interviewing, it cannot predict performance after hiring.

In 1995, Conway and colleagues reviewed studies of how far interviewers agree with each other. Interviewer training played a part, but on the structure side three things mattered: whether the questions were standardised, whether the way answers were rated was standardised, and how several ratings were combined. When ratings from several interviewers were combined using a method decided in advance, they were stable; when the panel settled on a rating by discussion, the study found no support for the same benefit.

From how closely ratings agreed, Conway and colleagues also estimated the upper limit on how well an interview can predict performance. For highly structured interviews, that ceiling was roughly twice as high as for unstructured ones. An interview whose ratings do not agree cannot rise above its ceiling, however well its questions are designed.

Source: Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80(5), 565–579. https://doi.org/10.1037/0021-9010.80.5.565

What happens when interviewers read an evaluation axis differently is covered in our column “Same answer, different score: when interviewers read the criteria differently”.

Scope

Structure applies to both the questions and the rating

When people think of structured interviews, they tend to focus on fixing the questions.

In a 1997 paper, Campion and colleagues drew 15 components of structure from earlier research and divided them into those concerning the content of the interview and those concerning how it is evaluated. Fixing the questions is a content component; deciding which level an answer corresponds to is an evaluation component.

Source: Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702. https://doi.org/10.1111/j.1744-6570.1997.tb00709.x

A common way to fix the rating is to describe the behaviour expected at each level. Research calls this a behaviourally anchored rating scale (BARS), and it has been used since the 1960s. Instead of words like “excellent” or “average”, each level describes behaviour you would see at that level. For “Execution” in a new-graduate, non-specialist role (Japan’s graduate hires usually join without a fixed job), it might look like the table below. L5 is the highest level and L1 the lowest.

Level	Definition
L5	Decided for themselves when to check progress, changed their approach in light of the results, and finished by the deadline
L4	When things did not go to plan, looked into the cause, changed their approach once, and saw it through
L3	Planned the steps towards a set goal and kept at it to the end
L2	Kept at it to the end, but carried on the same way even when they hit problems
L1	Can name something they worked on, but does not say what the goal was or how they went about it

Sources: Nagai, T. (2007). Competency as criteria of performance appraisal (meeting report) [in Japanese]. Japanese Association of Industrial/Organizational Psychology Journal, 21(1), 81–85. https://doi.org/10.32222/jaiop.21.1_81 / Schwab, D. P., Heneman, H. G., & DeCotiis, T. A. (1975). Behaviorally anchored rating scales: A review of the literature. Academy of Management Proceedings, 1975(1), 222–224. https://doi.org/10.5465/ambpp.1975.4982613 / Ministry of Economy, Trade and Industry. Fundamental Competencies for Working Persons in the age of the 100-year life (diagram) [in Japanese]. https://www.meti.go.jp/policy/kisoryoku/pdf/kisoryoku_image.pdf

If you fix only the questions and leave the rating to each interviewer’s instinct, what you have is a semi-structured interview. Only once the rating is fixed too can different interviewers arrive at the same level.

Limits

More structure is not always better

The usual criticisms of structured interviews are that they feel mechanical and that they miss unexpected sides of a candidate. Research also suggests that more structure is not always better.

In 1994, Huffcutt and Arthur reviewed studies of interviews for entry-level jobs (jobs people take without prior experience) and compared several degrees of structure. Predictive power rose as structure increased, but beyond a certain point, adding more structure made little further difference.

Source: Huffcutt, A. I., & Arthur, W., Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology, 79(2), 184–190. https://doi.org/10.1037/0021-9010.79.2.184

Sackett and colleagues’ re-analysis also shows that results for structured interviews vary. The average is 0.42, but depending on the job or the company, it was estimated to fall below 0.2 in some cases. Interviews that all go by the name “structured” can produce different results depending on how they are built.

Source: Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994

What the research shows is that the benefit of structure levels off, and that results depend on how the interview is built. It does not give a single answer on how much to decide in advance. We think that if you decide three things (what to evaluate, how levels are given, and the questions for each evaluation axis), there is no need to give up follow-up questions or a natural conversation. The Japanese research described below points in a similar direction.

Domestic research

Adapting structure to Japanese hiring interviews

Most of the research so far was carried out in the United States. Hiring interviews in Japan rest on different assumptions.

The first is graduate recruitment. Japanese companies hire students in large annual intakes before they graduate, often without assigning a specific job, and the students have no work experience. That makes job-based questions hard to ask. Interviewers tend to look at personal qualities such as motivation, initiative and cooperativeness, and at fit with the organisation.

In a 2005 paper, Imashiro pointed out these differences between Japan and the US and then analysed interview data from new-graduate hiring at Japanese companies. The analysis suggested that different interviewers rated different kinds of applicants highly. Imashiro gave a hypothetical illustration: one interviewer might favour applicants who speak logically, another those who lead with their enthusiasm. In unstructured interviews, applicants with personalities similar to the interviewer’s also tended to be rated more highly. When the interview was structured, this tendency weakened for some personality traits.

Source: Imashiro, S. (2005). Empirical study on employment interviews: Multilevel analyses of applicant’s, interviewer’s, and organization’s effect on interview ratings [in Japanese]. Japanese Association of Industrial/Organizational Psychology Journal, 19(1), 3–16. https://doi.org/10.32222/jaiop.19.1_3

The second is the assumption of long-term employment. In 2004, Takeda reported on an attempt to build a structured interview for new-graduate hiring at a Japanese company that assumed lifetime employment. Because the company put weight on building trust with applicants through the selection process, it settled on an interview with a relatively low degree of structure, so that interviewers and applicants could talk naturally and in depth.

Source: Takeda, T. (2004). A study of structured selection interviews at a Japanese corporation [in Japanese]. The Japanese Journal of Psychology, 75(4), 339–346. https://doi.org/10.4992/jjpsy.75.339

Research has also tested whether interviews at Japanese companies actually predict performance after hiring. In 2013, Suzuki analysed new-graduate hiring interviews at one Japanese company and reported that interview ratings did not predict later performance ratings. Ratings of the same students in different interviews did not agree closely either.

Source: Suzuki, T. (2013). Examining the predictive validity of recruiting interview [in Japanese]. Japan Journal of Human Resource Management, 14(2), 4–26. https://doi.org/10.24592/jshrm.14.2_4

The format that suits Japanese interviews, then, keeps the natural conversation and the rapport, and standardises how answers are rated. When the aim is to judge personality and fit, it matters all the more to put into words what “fit” means. Otherwise, each interviewer’s preferences flow straight into the rating.

Say an evaluation sheet just reads “Fits our culture”. One interviewer may see a bright, articulate student as a fit; another may see a quiet but persistent student that way. Neither is wrong, but the same student gets a different rating depending on who interviews them.

The fix is to translate your own idea of “fit” into behaviour you can observe at work. At a company where people work through disagreements by talking them out, the evaluation axis could be written as “Works through differences of opinion by discussion”. The level definitions might look like this (for illustration only).

Level	Definition
L5	When opinions split, found out the reasons behind the other person’s view, proposed a way forward both sides could accept, and got both sides to agree
L4	Explained their own view and reasons, checked the other person’s reasons, and decided the way forward together
L3	When opinions split, explained their own view and reasons to the other person
L2	Mentions that there was a difference of opinion, but went along with what others decided
L1	Mentions team activities, but does not say how they dealt with differences of opinion

The question can come up naturally in conversation.

Even if you ask it in a relaxed, chatty way, a fixed definition to check the answer against lets different interviewers give the same level. What “fit” means varies from company to company, so write the definitions around how your own organisation works. In a workplace that values carrying out instructions precisely, the evaluation axis itself would be different.

Formats

Questions either ask about past behaviour or describe a scenario

Questions in a structured interview take two broad forms.

One asks what the candidate actually did in the past. To check “Execution” for a new-graduate, non-specialist role, for example, you might ask:

The other describes a situation that could come up at work and asks how the candidate would act. For an experienced sales hire, a question might be:

Questions about past behaviour tend to produce evidence when the candidate has relevant experience to talk about; scenario questions work for candidates with little experience. Rather than choosing one, it is better to use each according to your evaluation axes and the candidate’s experience.

Whichever form you use, it is essential to decide which questions check each evaluation axis.

We created 30 personas (character profiles) of new graduates with different backgrounds and speaking styles, had an AI answer in character, and simulated interviews. With only ten general questions, on some evaluation axes between four and nearly seven in ten personas (12 to 20 of 30) produced no evidence, that is, nothing a level could be based on. Adding three questions matched to the evaluation axes cut this to around one in ten (4 or fewer of 30). The personas are not real students, so take the trend rather than the exact numbers: adding questions sharply reduces the gaps. The simulation setup is in our column “Use your performance review criteria in interviews, word for word”.

In practice

Steps to build a structured interview for your organisation

Build a structured interview in this order: evaluation axes, level definitions, then questions.

How to build an evaluation sheet is covered in our column “How to build an interview evaluation sheet”. Example evaluation axes and level definitions for six roles, including new-graduate, non-specialist hires and staffing agency registration interviews (the interviews agencies hold with people who sign up to be placed with client companies), are on the “Interview rubric templates” page.

Practicalities

What it takes to prepare, and how candidates respond

Most of the work in a structured interview comes at the preparation stage. Writing the evaluation axes and level definitions, deciding the questions and going through them with interviewers will likely take several hours even for one role. Once it is built, interviewers spend less time working out what to ask before each interview.

There is research on how candidates experience structured interviews both in Japan and elsewhere. A study outside Japan found that candidates perceived structured interviews as more difficult, and reacted less favourably as a result. At the same time, candidates’ sense that the selection procedure was fair did not drop with structure (a survey of 812 candidates and 592 interviewers).

Source: Chapman, D. S., & Zweig, D. I. (2005). Developing a nomological network for interview structure: Antecedents and consequences of the structured selection interview. Personnel Psychology, 58(3), 673–702. https://doi.org/10.1111/j.1744-6570.2005.00516.x

In Japan, Hayashi surveyed new graduates in 2009 who had accepted job offers from a large manufacturer. The better their impression of the company information sessions and of the interviewers’ behaviour and attitude in interviews, the more positively they viewed the company. Hayashi concluded that it helps when interviewers tell candidates enough about the company and, through the conversation, check whether the candidate fits. We read this as meaning that even with fixed questions, how candidates feel about the interview depends on how the interviewer treats them.

Source: Hayashi, Y. (2009). How does the recruitment process affect attitudes of incoming employees? Quantitative analysis of data from a large manufacturing company [in Japanese]. Japanese Journal of Administrative Science, 22(2), 131–141. https://doi.org/10.5651/jaas.22.131

Uses

An AI interview asks every candidate the same questions and rates them by the same standard

An AI interview can ask every candidate the same questions and rate them against the same evaluation axes and level definitions, so it lends itself to the structured format.

In Proomi, our AI interview service, companies write the evaluation axes and the definitions for each level freely. The AI reads the interview transcript, assigns a level for each evaluation axis, and quotes the remarks it based that level on. Where nothing the candidate said supports an evaluation axis, Proomi can record “no evidence”.

Switching to AI interviews does not by itself make an interview structured, though. Deciding what to evaluate and how to write the levels is still HR’s job. How to write evaluation criteria is covered in our column “Can AI evaluate candidates correctly? HR writes the criteria”.

Q&A

Frequently asked questions

What is the difference between a structured and a semi-structured interview?

The dividing line is whether the rating method is decided too. In a semi-structured interview, the broad questions are set and the interviewer follows up depending on the answers, but part of how levels are given is left to the interviewer. In a structured interview, the evaluation axes and level definitions are also set, and everyone goes through the interview in the same way.

Do we have to ask every candidate exactly the same questions?

Keep the questions that check each evaluation axis the same for everyone. Beyond that, you can follow up depending on the answers. Anything that comes out of a follow-up is rated against the same level definitions.

Does it work for new-graduate hiring?

Yes. New graduates have no work experience, so questions that check an evaluation axis through part-time jobs or studies, or questions that describe a scenario, work well. The elements of the “Fundamental Competencies for Working Persons” framework from Japan’s Ministry of Economy, Trade and Industry can be used as evaluation axis names without changes.

How many interviewers do we need?

A structured interview can be run by one interviewer. With two or more, we recommend that each gives levels independently and the ratings are then combined by a method decided in advance. For example, average the interviewers’ levels for each evaluation axis, and set a minimum such as “L3 or above” for any evaluation axis a candidate must meet. In Conway and colleagues’ analysis, combined ratings were more stable when they were merged by a method decided in advance than when they were settled by discussion. A discussion can still be useful after the levels are on the table, to talk through why they differed and any concerns.

Is there anything to watch for when we add our own questions?

Do not ask about matters the candidate is not responsible for, such as their permanent domicile (honseki, the family register’s place of origin) or family members’ occupations, or about matters of belief and conscience, such as religion or which political party they support. These have nothing to do with the candidate’s aptitude or ability. Japan’s Ministry of Health, Labour and Welfare sets out the principles in its “Basics of fair recruitment and selection”.

Source: Ministry of Health, Labour and Welfare. Basics of fair recruitment and selection [in Japanese]. https://kouseisaiyou.mhlw.go.jp/basic.html

Related

Read next

For more on evaluation sheets and criteria, see these articles.