This post does its best to outline a process that attempts to remove bias and create a comfortable interview for both interviewer and interviewee. This process is not static. One of the goals of this post is to show that any such system needs to take into account feedback and be able to change over time. The important part is that we can use the processes to gather feedback and change what isn't working. Feedback is welcome and encouraged.

Key take aways:

• what is the role?
• what skills does the role require?
• skill groups consist of interview slots.
• interviewers value everyones time.
• be transparent with the interviewee about requirements, questions, and format.
• stop using old techniques like white boarding and algorithm/puzzle questions.
• start asking questions that are real problems you have to solve at work.
• collect feedback to find what is and isn't working.
• tweak the process.

My Background

Over the years I've done many interviews and I've been on both sides of the table. I was once part of the Hiring Machine at Yahoo!. We were a group of engineers tasked with a goal of growing our engineering organization. I stopped writing code for a bit and my focus was primarily on hiring. There was a process to follow, but it lacked a cohesive strategy. Through that entire time I came to realize that the process had an extreme focus on tweaking the metrics to make it look like a lot of good work was being done. I learned a lot about what did and did not work. The Hiring Machine did teach me to track work being done by who and what stage we were at with a potential candidate, but had little direction once that potential candidate became a candidate. It also lacked a process for feedback.

At other companies in my career I've had the opportunity to change small things about the hiring process, but interviewing has always felt odd. I've done my best to make candidates feel comfortable. I know for sure I've made tons of mistakes. Writing this down was a reflection of all the time I have spent on this over the years. These are not original ideas though. An Elegant Puzzle: Systems of Engineering by Will Larson has a lot of great insights on hiring, and tons of other great information on running organizations. I also think a lot about The Hiring Post by Thomas Ptacek. That post was in the back of my mind when writing this.

Before the Interview

Agreement on Roles

What I've learned is that moving between companies means learning about new levels at the company that you are interviewing with. Because roles vary between companies we need to define roles that you are interviewing for as clear as possible. Even more important is that the interviewers need to agree on what the role is and the skills required to perform the role.

Interview Slots

Each skill (or groups of skills) listed in the requirements for the role should be an interview slot. For example, if we are hiring for a front end role the required skills could include html, css, javascript, and general knowledge about the web platform. That would then consist of one interview slot. Two interviewers should cover an interview slot. Having a big group of people in the room can be intimidating for folks. People are already coming into an interview nervous. The two interviewers should be knowledgeable about the topic, prepared, and trained on how to conduct an interview.

The format of each interview slot depends on the role and skills required. For example, one slot could focus on coding, and another could role play a conflict resolution situation between peers. The important thing is that the format of each slot is clear and concise.

Interviewers are prepared and on time

They should do their best to not run over time. Stick to questions and discussions within the slot. There is a lot of ground to cover and this person is going to have a long day interviewing, let's make the best use of their time and the other interviewers time and stick as close as possible to the schedule. If you do go over time, then let's make sure to shift the schedule so that we don't limit the evaluation of the other slots.

Let the interviewee prepare for the interview

We want to hire smart people that are able to grow in a reasonable time frame into all the responsibilities required of their job. For example, if we are hiring for a role and we would like to asses a candidates systems design skills then we should come up with a specific systems design question and let the candidate know it ahead of time. For some interviewers there will be a reaction that this is giving them the ability to look up the answer before hand. I hope they do. Because that is research and is what you would do in real life. If they can do the research and come to the table to talk about it and discuss the design and all the trade offs, then that makes for a much better bucket of data to evaluate how that person would act in that situation in the real world. This also goes for coding questions. If we are going to test their knowledge of a specific framework or technology then we should be transparent about it and give them adequate time to prepare.

The Interview

Whiteboard Optional

Whiteboards can be great, but not everyone is comfortable writing on them in an interview environment. They can be ok for drawing diagrams, but are terrible for writing code. A simple fix is to make them optional. If the candidate wants to use it to draw out a diagram then great. We can get some laptops from IT with some of the popular editors installed that the candidate can use if they want during coding questions.

Stop Asking Unrealistic Questions

A whole genre of books exists around how to study specifically for unrealistic coding questions. This is a reaction to interviewers picking up poor interview habits. Instead of asking questions that you yourself never have to write, try to steer the questions to situations that you work on in real life. For example, one of the best interviews I ever had the person asked me how to code a specific thing on their site. I explained how I would do it and then they asked me to write the code. They gave me a laptop with the required code without that specific piece in place. I did it, was able to run it and see the results. This was an actual problem that they were trying to solve and while a small problem, it had real value to the company and was a real problem someone was going to have to solve.

Manager Interview

A manager should interview the candidate and this is the last interview slot. This way the manager can asks questions and gather feedback on how the interview itself went. This is important because it allows us to tune our process.

After the Interview

The aftermath of the interview should involve each pair of interviewers to write up feedback. They should do so independently and without discussing. I know this is difficult. You want to discuss what went well and went bad. Save it for after you have written it down. Then feel free to discuss. The feedback should have a standard way of collection that everyone uses. I've had good experiences with a custom spreadsheet with each interview slot as a column with a score of how knowledgeable the person was in that area. It also has columns for notes and of course the interviewees name and role they are interviewing for. This makes it so you can see how the interviewee stacks against other candidates. This is one example of a way to do it, but the important thing here is making sure that we collect feedback and that we have a more object way at considering candidates. The pair of interviewers should avoid talking to the interviewers scheduled for other slots. Again, I realize this is difficult. To get honest and open feedback about the candidate that is as unbiased as possible, it is important that we try as hard as possible to stick to these steps. There is no chance we can be completely unbiased, but we can at least build a process that is as similar as possible for both the interviewer and interviewee.

Metrics

Collect metrics on everything about the process. Your friends in recruiting probably have very valuable metrics.

Some metrics you might be interested in:

• sourcing vs direct applicants vs referral (referrals can be great, but can also prevent diversity of candidates)
• how many interviews is each interviewer doing per week? (are they doing too many? Interviews can be taxing on both sides of the table)
• how many phone screen are becoming on-sites? how many on-sites are becoming offers? (do we need to tweak our sourcing methods or interview questions?)

This allows us to make educated decisions and tweak our interview process when it appears that we need to. If we have a feeling that something isn't working, let's try to prove that it isn't working and tweak the controls until it works. Be skeptical about the current situation and use science and the controls that we have to try and make it better.