1. 16 Jan 2014

    8 Ways To Identify Great Candidates in Job Interviews

    By: Jason Bishop

    Lessons Learned in Hiring for a Small Team

    I wanted to share some of the things I’ve learned over the last 6-7 years during which I’ve been involved with various phases of the hiring process. Most recently, this last year, Mitchell and I built out a team of 10 developers (including ourselves) of varying experience levels, programming language backgrounds, and academic pedigrees. Our team consistently beat deadlines and pushed features. In short, we wrote some solid code - all while having a great time.

    Before I tell you how I found the talent we ended up hiring, I will first spend some time talking about what I mean when I say “great developer”. This term I believe has a different meaning to everyone, but I will attempt to distill it into some basic qualities which I think universally apply. Stay with me and I promise by the end of the post, I will have provided you with tangible ways to smartly expand your team.

    If you want to get to the part about sussing out candidates, skip to the good part.

    The 2 essential qualities of a developer are:

    1. They are a pleasure to work with. This means they are good communicators, are accountable for the code they write, and know how to have fun. In short, they are a great “culture fit”.
    2. They are talented and prolific when it comes to actually writing code. This means 1) They can offer insightful feedback on how to make some code or system better; 2) features they implement are complete, robust, and on-time; 3) their actual code is reflective of best practices, well-tested, well-documented, and pragmatic.

    Exceptional developers ...

    • Talk about code and write code even when it does not relate to work.
    • Have something to teach you about code that you were previously unaware of.
    • Work well on teams.

    Poor developers:

    • Treat writing code as nothing more than work.
    • Don’t take pride in their profession and workplace.
    • Don’t respect the time of their peers: they show up to meetings late, fail to deliver complete features on time, and are unavailable when collaboration is necessary.
    • Can’t or don’t communicate.

    “Red Herrings”

    Some “red herrings” - qualities which may seem good or bad but in practice work out differently.

    1. Red Herring #1: Impressive ability to code but unwillingness to work on a team. I call these types of developers “black boxes”. They can generally write features well enough, but they don’t participate in daily scrums, do not make themselves available when other developers are working with their code, and are unwilling to compromise on various code conventions. Essentially, these “black boxes” do something, but you don’t really know how it gets done and you can’t take it apart to get more information. On a dev team (and in engineering in general), these are fundamentally undesirable qualities.
    2. Red Herring #2: Lack of any of the following: professional experience, academic pedigree, or specific language competency (*). In my experience, both poor and excellent programmers come from a wide range of backgrounds. I have found that these 3 factors make poor heuristics in sussing out raw engineering talent. Instead they might be useful for:
      1. Professional experience - I’ve found this to be a better indicator when you are looking for someone who has a deep history with a specific technology. This doesn’t make them a great developer, per se, but in some cases you are looking for someone who is just a repository of how to use a certain technology.
      2. Academic pedigree - I find this to be the most useless indicator of the 3. I think it goes without saying that many factors outside of code competency go into a persons ability to go to a good college or even attend college. The only situation I can think of this mattering is if you were filling a position in academia.
      3. Specific language competency. I find this to be a better indicator when you are looking for someone who can hit the ground running on day 1. Even then, it is a bit questionable, as even veteran, highly-seasoned developers need time to acclimate to new codebases. Due to the extremely difficult nature of hiring talent, I find it is often wiser to hire someone with strong ability to write code in general, and concede that it might take 3-6+ months for them to be strong with the language your codebase uses. Waiting those 3-6+ months for the perfect <insert language here>-ist can be extremely costly both in terms of money and time.
    3. Red Herring #3: They want a lot of money. This brings up a larger issue about how salary is awarded in your company. It is a complex issue and different companies approach it in different ways, for example: 1) salary according to experience/seniority/title; 2) salary according to “value you provide to the team”; 3) salary according to “what the employee negotiated for”. From my standpoint - and I realize this might be a little idealistic - money should be largely irrelevant when making the decision to hire someone. If you want someone on your team, you want them on your team - bottom line. This person is going to provide value to someone, one way or another - can you afford to lose them to another business? Passing on an ideal engineer at salary X in order to hire 2 “average” engineers at salary 2X is not an equivalent trade. Average hires will bring the rest of your team down and create technical debt. As I said before though, the world is not perfect and the financials may not work out, but I highly suggest “going to bat” when you find the perfect, if expensive, hire.
    4. Red Herring #4: They are unlike you, a bit “different”, or seem idiosyncratic. Since I’ve worked professionally on engineering teams, I’ve met a huge spectra of personality types. It is unfair, and in many cases illegal, to discriminate (I believe that is the only word that works here.) against people who are, for example:
      1. Older than the rest of the team.
      2. Diagnosed autistic.
      3. Female.
      4. Hopefully all the readers of this blog are highly enlightened individuals and the aforementioned point goes without saying. And hopefully everyone understands that none of these types of people should be thought of in a pejorative sense. In any case, the point is that there are a great number of fabulously talented devs and many of them might not fit the mould that has emerged in tech over the past decade.

    This leads to the main question ...

    How do you suss it all out during an interview?

    1. Ask them if they have any personal hack projects. Some answers that impressed me:
      1. “I am working on a voxel engine which, at the time, was the most performant voxel engine as far as I could research on the internet.”
      2. “I am new to Rails but I’ve implemented the first 20 Project Euler questions in Ruby on my GitHub if you would like to see them.”
      3. “<Insert well-known open source project here> has merged one of my pull requests.”
    2. Ask them if over the past week they have read any interesting news pertaining to software. Great developers are interested in programming outside of its utility in completing their task at work. Because the technologies change so quickly today, the best engineers are often those who stay current on tech news.
    3. Go ahead and give them a “technical question that has no practical application in typical software engineering”. Make it language-agnostic; ask them something that requires no outside APIs or function calls. Great developers have a fundamental understanding of how code works. Certainly, don’t let this preclude asking them about specific tools and frameworks; rather, think of this as a way to determine how they might solve a novel problem you come across in your codebase. Great developers think on their feet and are capable of handling these types of scenarios.
    4. Ask them questions that can “unfold” into deeper and more difficult questions. A popular one that Mitchell and I like to use is:
      1. In code: Write a function that accepts a string and returns a hash that stores a count for each character that occurs in the string.
      2. Verbally: Adapt the function you just wrote to instead accept a URL and return a hash that stores the count for each character that occurs on the webpage.
      3. Verbally: Describe an app that would accept a base domain and return a hash that stores the count for each character that occurs any page hosted at that domain. For example, www.wikipedia.org.
      4. Verbally: Describe an app that does this same thing, but can accomplish it extremely fast - under an hour.
    5. Have someone else (a recruiter or hiring manager, for instance) pre-screen the candidate with the “gimme” interview questions. You’d be surprised what a time-saver this is. Example questions would be FizzBuzz or Fibonacci. Some developers object to this for the following reasons:
      1. “Great developers will be insulted and therefore turned off by being asked such an easy question”. Of the > 120 candidates I interviewed between August and December of 2012, zero declined a second interview after being asked their screener question. Also, ask yourself if a person like that is really someone you want to work with.
      2. “These questions are too easy”. Of the > 120 candidates I interviewed between August and December of 2012, over half failed either FizzBuzz or Fibonacci. Considering each in-person interview takes about an hour, this saved me massive amounts of time.
      3. Variations of 1) “These questions are too hard”; 2) “These questions aren’t indicative of ‘practical engineering’ competencies”; 3) “I don’t think I could do FizzBuzz under pressure, but I’m a good developer”.
        1. I think we can all agree that these are essentially college entry-level computer science questions. Let’s hope that the great developers we desire can produce better code than college freshmen new to coding.
        2. Perhaps, but other questions can suss that out later. These questions are simply there to establish a baseline competency level.
        3. I heard this one time and I simply said: “I don’t believe you.” and made them code FizzBuzz on the spot, which they wrote 100% correctly. So much for that objection.

    Warning Signs:

    1. The candidate responds to a question with a negative attitude. As I indicated earlier, I ask some pure-programming questions in job interviews I conduct. A small minority of candidates come in expecting to be asked solely about various frameworks, such as Ruby on Rails or jQuery. A small minority of these candidates fail to answer these questions correctly and react with aggression about the nature of the question itself. In my view, a wrong answer can be compensated for with a correct answer, but a negative attitude with a complete stranger is a very strong indicator that the candidate is not a good match.
    2. The candidate is late to an interview or does not show up for an interview. This is definitely a strike against a candidate, but I’m willing to set this aside under extenuating circumstances. Shit happens and it’s not worth losing a great developer over. That said, if you have any inkling that this is a habitual thing, be on your guard. Scrums, for example, are an essential component of our Agile-flavored work environment and if someone can’t make it to a job interview, what are the chances they will take a 5-minute stand-up seriously?
    3. The candidate conflates similar terms or concepts. One common example of this is when a candidate doesn’t recognize the difference between a framework and the programming language in which the framework is implemented (e.g. jQuery vs. JavaScript, Ruby vs. Rails, etc.). This is a very basic thing you can identify quite quickly and is a strong indicator of a poor understanding of programming.

    Hopefully this has been a good high-level rundown of how to identify great developers in job interviews. If you have any other tips or corrections, please don’t hesitate to email us!