4. Assessing and comparing candidates

When the interview is over, you are left with a pile of quotes and a half-formed impression. The route from that raw material to a well-founded decision runs through one document and past a number of thinking errors that everyone makes. This chapter is about that final stretch: how to substantiate a score, how to compare candidates side by side and how to prevent the most likeable candidate winning instead of the best one.

Scoring with evidence

An evaluation form is the document on which you record, for each competency, the score you give and the evidence from the interview on which that score rests. It isn't a formality after the fact, but the instrument that makes your judgement testable: a colleague should be able to see from your form why candidate A got a 4 and candidate B a 2.

Each competency has four fields. The first is the score: a score on a fixed scale. The second is the evidence: the quote or action from the STARR on which that score rests, in the candidate's own words. The third is the weighting factor and the minimum threshold that you set when drawing up the profile, plus the question of whether the candidate comes out above the minimum threshold on this must-have. The fourth is the outstanding questions: what you haven't been able to assess. That isn't a low score, it's an agenda point for a second interview.

Fill in the form within fifteen minutes of the interview ending, and certainly before the next candidate comes in. Your memory for exact wording is shorter than you think.

Giving the scale meaning

A score without a description means something different to every interviewer. So set out in advance what you mean by each score, in behavioural terms. A five-point scale works well. A 1 you give if you heard only behaviour that runs counter to the competency, without any fitting example. A 2 if there is one weak example, or behaviour that stays below the requirement of the role. A 3 if there is a complete example showing that the candidate has mastered the competency at the level the role requires. A 4 if there are several examples, with initiative that goes further than was asked. And a 5 if the candidate has brought others along on this point or has improved the working method.

Comparing candidates

After the final interviews, three or four completed forms lie on the table. The temptation to say straight away who was the nicest is strong. Instead, first line up all the scores side by side, per competency, with the weighting factor alongside.

A weighted total gives you a ranking, but it is not a result. First you test the minimum thresholds: a candidate who falls below the threshold on a must-have drops out, even if their total is the highest. Only then do you compare the remaining candidates, competency by competency, with the quotations from your form alongside.

Five pitfalls in assessment

Assessment pitfalls are systematic distortions in your judgement: you score something other than what the candidate actually demonstrated. They operate outside your awareness, so the only way to catch them is to test your judgement out loud.

• Halo and horn: one strong impression colours all the other scores. A candidate who talks fluently and warmly also scores highly on planning, even though you haven't noted down a single example of it. With a poor first impression it works just as strongly in the other direction.
• Contrast erroryou assess the candidate against the previous one rather than against the requirements of the role. After a weak interview, a mediocre candidate suddenly looks excellent.
• Recency: the last interview is the sharpest in your mind. As a result, the candidate from Tuesday morning consistently gets less room in the discussion.
• Similarity error: you recognise your own education, hobby or career and unconsciously rate it higher. That is how you build a team that resembles you.
• Resourceswhen in doubt, you give a 3 across the board. That wipes out every difference between candidates, and in the end your gut feeling decides.

A workable agreement with your fellow interviewers: every score above or below 3 is backed up with a quote. If the quote can't be found, the score changes, not the memory.

After the interview

A second interview is not a repeat with different people. You use it to close the open points from round one: the competencies for which you do not yet have a complete STAR, and the doubts you noted down while comparing.

Only call references once the candidate has given permission. There too, do not ask "what was he like", but ask for the same kind of facts as in the interview: which tasks, which share, which result. And in the discussion about terms of employment you are no longer assessing: there you finalise the agreements within the range you set beforehand. Put that range on paper before you go into that conversation, including your maximum, and confirm the outcome in writing the same week.

Summary

• The evaluation form sets out, for each competency, a score, the evidence from the STARR, the weighting factor with the minimum threshold, and what is still outstanding.
• Fill it in within fifteen minutes of finishing, and in any case before the next candidate comes in.
• Give the scale meaning in advance: 1 is only contrary behaviour, 2 is below the requirement of the role, 3 is a complete example at the level of the role, 4 is several examples with extra initiative, 5 is having brought others along or improved the working method.
• First test the minimum thresholds of your must-haves. Anyone falling below them drops out, even with the highest total.
• The five pitfalls: halo and horn, contrast, recency, similarity and central tendency.
• Every score above or below 3 is substantiated with a quotation. If the quotation is not there, the score changes, not the memory.
• The second interview closes your open points; you question referees on facts, not on impressions.