Hiring Engineers: Why We Do Open Book Code Challenges

One of the few topics my friends and colleagues in the industry are willing to talk about are their strategies for recruiting and hiring talented engineers. It’s kind of funny because as an industry of practice, software engineers are so open about everything. Recruiting and hiring is such a competitive exercise, it feels like only the big giants like Google, Facebook and Apple go out of their way to talk about their approaches.

For this blog, I am going to talk about the topic of interviewing candidates. Like most companies, the process takes time and effort. Most candidates often make upwards of two visits to our office. Our average time to go through the process of introduction to offer takes about 4 weeks. In the tech world, that’s like the equivalent of two decades.

It’s Not Just a Job

When I got out of college in the late 90’s, people would jump jobs every 4 to 8 months. If your resume wasn’t littered with startup after startup, or enterprise after enterprise, you weren’t really playing the game right. For a young pup like myself, who watched his father have the same job for 50 years running his own dental practice, it was mind blowing to think the industry approved of this rampant mobility.


I approach things from the perspective that a job isn’t just a job. Rather, a job is an experience in your life. It becomes part of your DNA, just like going to high school or college. Your job consumes 1/3 or more of your time each week, so you better make sure it’s the right place.

We want our candidates to want to be at Contrast. I’m sure everybody says that and wants to believe that phrase to be the gospel. We truly believe it. We want Contrast to be your startup…your top choice. We’ve passed on some really talented engineers that didn’t see much value in our process.

Meeting the Team

I read an article a few weeks back about Etsy which referenced the word “culture fit”. It made me chuckle to say the least. We try our hardest to build a team of skilled engineers who want to work in an environment that values respect for one another and offers the chance to speak open and honestly. We don’t look for clones or brogrammers. We look for engineers that want to make the Internet safer and want to be a part of a high-performing team.


I think it’s important to sell my team from the start. I want every candidate, whether their an intern or a full-time engineer to trek to our office for a meet and greet. The meeting can take anywhere from 30 to 60 minutes. While it consumes a good amount of my time and pulls my engineers away from their assignments, it’s the first step in establishing a relationship of trust and openness with a candidate.

We are pretty transparent from the start. This initial meeting is not an interview for the candidate, but rather the chance for the candidate to interview us. We want our candidates to come into the office with a boat load of questions to make sure this is the right place to interview.

The Project

Every engineer that’s been hired at Contrast since I started has had to do a project. The project has changed over the years. The link above shows you the current state of the project. The project is open book and candidates do not have a time limit to complete. We often advise candidates to work on it over the course of a week or two at most, otherwise they won’t complete it.

The project is wholly owned by the author. We don’t want our names mentioned in the project. We do ask that the author codes the project and submits it to Github. Some folks are uneasy about a coding project in the public persona. Most like the project as it gives them a reason to build up their public coding profile.

We ask candidates to do projects for a couple of reasons. First, the project is designed to showcase a candidate’s ability to take on new work and see how serious they are about working with us. The really serious candidates crush through the assignment in less than two weeks. Second, the project is used as the focal point for the majority of the interview (see below). Third, the project gives us a chance to see how well you deal with unfamiliar technologies, as well as demonstrate your creativity.

The Interview

Most technical interviews are awkward, frustrating, unrealistic to name a few attributes. Our technical interview isn’t about how great of a developer you are. Rather, it’s about going through the experience of working a project and receiving/giving peer feedback.

We generally perform a small code review of 2 to 4 engineers with a candidate. We once had 12 people in a room with a single candidate, which we really blew. I won’t make that mistake again. Apologies to that candidate as the interview was way too overwhelming.

The code review starts with the candidate walking us through the project. He/she tells us how they interpreted the problem and how they decomposed their work. Next, they walk us through their design and implementation. They are then given the chance to call-out things they would do differently before we ask questions.

The interview team then asks upwards of 4 to 6 questions about the project, design choices, implementation decisions and further elaboration of what’s next. It’s not all questions though…my engineers will always add commentary of what they enjoyed or were concerned about with any project. Most of the time the candidate takes the feedback real well and comes back with a good follow-up question or response. Occasionally, we have a candidate that’s upset with the feedback, but can’t offer a good path forward. When this happens, both parties realize it’s not an ideal match.

The Experience


The overall experience of meeting the team twice and working on a project can be overwhelming to some who are looking for quick work. For the folks who’ve joined our team, it was a good formula for making the decision about whether this was the right place to work.

It may take time and in our space, time is not unlimited. It’s safe to say that it’s the right time for everyone to make an educated decision about their next job or next hire.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s