You're Doing Interviews Wrong
What traits would you want in a teacher or school leader?
Hard working
Has integrity and honesty
Believes students can succeed
Pays attention to the progress of their students
Curious as to why things work or don’t work, and how to improve them
Likes to build relationships
Wants to develop subject knowledge
Now, imagine you stop rating your candidates for their answer to each question.
Instead, you listen to their answers and rate them just against your chosen traits (you don’t have to have mine).
Only once you have done this, can you then make a gut reaction in deciding who is best. This means that you might not employ the person with the highest score (though they would still have a very high score).
Does that sound like a mad idea?
Well, it is one backed by a Nobel Prize winner in psychology, Daniel Kahneman.
I listened to him on the Adam Grant podcast.
Here’s an extract of the relevant part:
Daniel Kahneman:
My attitude to intuition is not that I've spent my life, you know, saying that it's no good. Uh, in, in the book that we're right writing—just finished writing—our advice is not to do without intuition. It is to delay it. That is, it is not to decide prematurely and not to have intuitions very early. If you can delay your intuitions, I think that they are your best guide, probably about what you should be doing.
Adam Grant:
Okay. So two questions there. One is, how? The other is, why?
Daniel Kahneman:
Well, you delay your intuitions. Now I'm talking about formal decisions, decision, that might be taken within an organization or a decision that an interviewer might take in deciding whether or not to hire a candidate and here the advice of delaying intuition is simply because when you have formed an intuition, you are no longer taking in information. You are just, rationalizing your own decision, or you're confirming your own decision. And there's a lot of research indicating that this is actually what happens in interviews. That interviewers spent a lot of time, they make their mind up very quickly and they spend the rest of the interview confirming what they believe, which is really a waste of time.
Adam Grant:
Yes. Yes. So the idea of delaying your intuition is to make sure that you've gathered comprehensive, accurate, unbiased information. So that then when your intuition forms it's based on better sources, better data, is that, is that what you're after?
Daniel Kahneman:
Yes, because I don't think you can make decisions without their being endorsed by your intuitions. You have to feel conviction. You have to feel that there is some good reason to be doing what you're doing. So ultimately intuition must be involved, but if it's involved, if you jump to conclusions too early or jump to decisions too early, uh, then you're going to make avoidable mistakes.
Adam Grant:
This is an interesting twist on, I guess, how I've thought about intuition, especially in a hiring context, but I think it applies to a lot of places. My, my advice for a long time has been don't trust your intuition, test your intuition, because I think about intuition as subconscious pattern recognition, and I want to make those patterns conscious so I can figure out whether whatever relationship I've detected in the past is relevant to the present.
And it, it seems like that's what, what you've argued as well. When you've said, look, you know, you can trust your intuition. If you're in a predictable environment, you have regular practice and you get immediate feedback on your judgment. I think the tension for me here is I don't know how capable people are of delaying their intuition. And I wonder if what might be more practical is to say, okay, let's make your intuition explicit instead of implicit early on. So that then you can rigorously challenge it and figure out if it's valid in this situation.
Daniel Kahneman:
I've been deeply influenced by something that I did very early in my career. I mean, I was 22 years old. I set up an interviewing system for the Israeli army. Um, for it was to determine suitability for combat units. The interview system that I designed broke up the problem so that you had six traits that you were interviewing about, you're asking factual questions about each trait at the time. And you were scoring each trait once you had completed the questions about that trait.
Adam Grant:
Jumping in here, because this is such a cool example, but it needs a little explaining. Danny created a system for interviewers to rate job candidates on specific traits-- like work ethic, analytical ability, or integrity. But interviewers did not take it well.
Daniel Kahneman:
They really hated that system when I introduced it. And they told me, I mean, I vividly remember one of them saying, 'you're turning us into robots,'
Adam Grant:
Danny decided to test which approach worked best. Was it their intuition or their ratings from the data? The answer… was both. Their ratings plus their intuition. But not their intuition at the beginning… their intuition at the end, after they did the ratings.
Daniel Kahneman:
That is you rate those six traits, and then close your eyes and just have an intuition. it turned out that that intuition, that, that intuition at the end was the best single predictor. It was just as good as the average of the six traits and it added information. You know, I was surprised. You know, I just was doing that as a favor to them, letting them have intuitions, but the discovery was very clear and we ended up with a system in which the average of the sixth traits and the final intuition had equal weight.
Adam Grant:
It sounds like what you recommend then concretely is for a manager to make a list of the skills and values that they're trying to select on. To do ratings that are anchored on those dimensions. So, you know, I might judge somebody who's coding skills, if they're a programmer or their ability to sell, if they're a sales person. And then I might also be interested in whether they, you know, they're aligned on our organizational values. And then once I've done that, I want to form an overall impression of the candidate because I may have picked up on other pieces of information that didn't fit the model that I had.
Daniel Kahneman:
I think that's about right.