May 17, 2017 · 41m · y-combinator
Hiring Engineers with Ammon Bartram · Y Combinator
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this Y Combinator podcast episode, Triplebyte co-founder Ammon Bartram analyzes the systemic flaws in traditional technical hiring, explains Triplebyte's data-driven interview methodology, and offers practical career advice for software engineering candidates and startup founders.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The partners as informed peer 1.6 Guest teaching 6.3 Guest disagreement 1.3 The partners pushing back 0.2
speaking balance: gold is the partners, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 36:04 Ammon dismisses most hiring books and conventional wisdom
Ammon bluntly rejects the usefulness of industry management literature, asserting that roughly 80 percent of what is written about interviewing is baseless.
Hardest push from the partners ▶ 21:03 Craig questions whether prep classes select for dependent candidatesCraig challenges the trend of companies hosting prep courses, asking whether it inadvertently filters for people who need constant hand-holding.
Biggest teaching moment ▶ 13:00 Ammon demonstrates interview variance matches random movie reviewsAmmon shares empirical data from cross-company candidate placements showing that interviewer score agreement is no better than online film critic agreement.
The partners hold their own ▶ 15:13 Craig points out large tech companies have the luxury to trainCraig adds an astute analytical observation that big tech firms can afford abstract generalist testing because they possess the capacity and time to train junior hires.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The partners as informed peer | Guest teaching | Guest disagreement | The partners pushing back | Why |
|---|---|---|---|---|---|---|
| Overview of Triplebyte's Recruiting Startup Model | 1 | 6 | 1 | 0 | Craig asks a general question about what startups should seek in engineering hires. Ammon educates the host on how different companies have fundamentally divergent, conflicting definitions of what constitutes a 'good engineer'. | |
| Addressing Interviewer Bias and Engineering Cultural Skew | 1 | 6 | 1 | 0 | Craig asks how founders can diagnose what kind of engineer they need. Ammon explains how interviewers bias hiring toward their own individual strengths, skewing company engineering culture over time. | |
| Evaluating Credentials: CS Degrees, Bootcamps, and Seniority | 2 | 7 | 2 | 0 | Craig inquires about distinguishing CS degree holders from bootcamp graduates. Ammon delivers a counterintuitive take, explaining that both lack practical experience and that junior candidates often outperform senior engineers on standard interview puzzles. | |
| Triplebyte's Structured Four-Part Interview Breakdown | 1 | 5 | 0 | 0 | Craig asks about Triplebyte's specific evaluation process. Ammon systematically outlines their four modular interview components based on data from 2,000 interviews. | |
| Flaws in Single Brain Teasers and Interview Inter-Rater Reliability | 1 | 8 | 2 | 0 | Craig asks what common tech interview practices are flawed. Ammon drops empirical data showing that inter-rater agreement between tech interviewers is as noisy as random online movie reviewers. | |
| Big Company vs. Small Startup Hiring Priorities | 3 | 6 | 1 | 1 | Craig asks where brainteaser-proficient candidates should apply and correctly notes that large tech firms have the capacity to train people. Ammon explains why simple tasks have higher predictive power than difficult dynamic programming puzzles. | |
| Interview Preparation Strategies for Computer Science Graduates | 2 | 5 | 1 | 1 | Craig asks how new graduates should prepare and questions whether corporate prep classes merely filter for candidates who need hand-holding. Ammon breaks down core CS topics and time-pressured practice. | |
| Demystifying Big Tech Questions and Reframe Rejection | 2 | 7 | 2 | 0 | Craig recounts failing consulting brainteasers and asks about candidate side projects. Ammon reframes rejection by noting top candidates only pass 80% of interviews, and explains why side projects are rarely scored during actual technical interviews. | |
| Navigating Candidate Location Preferences in Hiring | 2 | 6 | 1 | 0 | Craig asks about location advantages and common interview formats. Ammon compares standard interviews to democracy as the least-bad option and explains why whiteboard coding unfairly penalizes practical programmers. | |
| Inside Triplebyte's Internal Engineering Hiring Process | 2 | 5 | 0 | 0 | Craig asks how Triplebyte interviews internally and whether candidates must chase cutting-edge tech. Ammon breaks down Triplebyte's specialized testing and the 10-year risk of neglecting modern tech stacks. | |
| Debunking Hiring Myths and Standardizing Interviewer Hints | 1 | 8 | 3 | 0 | Craig asks for engineering management book recommendations. Ammon rejects the premise, stating 80% of hiring literature is bogus and recounting how testing past-project storytelling proved non-predictive. |
Statements from this episode (31)
Bartram: Conflicting definitions of "good engineer" create hiring noise
“Most companies think that they are trying to hire good engineers. That's, that, that's what they say to themselves. And that, and they, what they don't realize is that, you know, Company A's definition of a good engineer is significantly different from Company…”
Bartram: Later-stage companies must evaluate multiple engineering archetypes
“For companies at a bit of a larger stage, I think the obvious answer is you want to hire both those people. And so it's about building a process that can identify more broadly different types of skill.”
Bartram: Interviewers skew company culture by testing their own strengths
“When people are interviewing an engineer, they tend to ask about the things that they
Are the best at.
There's this overlap between the things that you're the best at and the things that you think are the most important.
Every engineer thinks the things that t…”
Bartram: Google's computer-science-skewed engineering culture worked well
“Google, you know, has, intentionally or unintentionally, grown very much in a computer science direction, and that's obviously worked out, you know, very well for them.”
Bartram: Bootcamp grads perform comparably to CS degree holders in interviews
“Bootcamps versus CS degree, I don't think are all that different.”
Bartram: Recent grads often outperform senior engineers in technical interviews
“People who are fresh out of university and boot camps actually in many cases, because they've been practicing, are better at the kind of problem that gets asked in an interview than your very senior, you know, eight, 10 years of experience at that large compan…”
Bartram: Experienced engineers get job offers despite worse interview performances
“If you have five years of experience, it's just flat out easier to pass an interview. You will get a job offer You know, after a worse performance”
Bartram: Traditional interviews fail to test real-world debugging skills
“And I think this does a great job of solving some of those problems, basically, because the, you know, this is a skill that, that comes from experience that is often missed by more, more, more traditional interviews.”
Bartram: Coding interview question results correlate far less than companies assume
“If you just take a bunch of candidates and in a controlled setting, have them all answer, you know, three or four of these questions, you'll see there's just, you know, there's some correlation, but there's way less correlation than you would think.”
Bartram: Tech interviewer agreement rates match online movie reviewer agreement
“A pretty interesting stat we dug up is I compared the rate of agreement between interviewers at companies with a data set of users reviewing movies online, right? And the numbers were actually, were basically, and the integrator agreement was equivalent.”
Bartram: Big tech prioritizes innate ability; startups prioritize specific stack experience
“Bigger companies care more about measuring your innate ability and less about measuring whether you can jump into their particular code base and be productive on day one. So it's way more likely that a smaller company is going to say, we're using, you know, Ru…”
Bartram: Easy interview questions predict engineering success better than hard questions
“Asking pretty easy interview questions is often more generally predictive than asking harder interview questions.”
Bartram: Hash tables and BFS represent 40% of tech interview questions
“A surprising percentage of a new question ends up being Slightly obscured applications of sort of those, especially sort of, you know, hash tables and breadth first search. Those two things by themselves represent probably 40% of the questions that are asked b…”
Bartram: Cracking the Coding Interview's non-coding advice does not fit startups
“Cracking the Coding Interview has a pretty good list of questions. It's, the other advice in that book I don't think really applies to startups very much but the questions are good.”
Bartram: Facebook provides interview prep classes to all applicants
“Facebook, for example, has started providing a sort of an interview prep class to everyone who applies so that they're sort of going over the material.”
Bartram: CS Applications Form the Majority of Big Tech Interview Questions
“Hard application, you know, sort of practical application of a computer science topic represents The significant majority of questions at big companies.”
Bartram: Top Engineers Pass at Most 80% of Their Tech Interviews
“One thing, one number we have that's interesting is that the engineers who do the best At companies go on to pass, ah, about 80% of their interviews, but not a hundred. No, almost no one passes more than 80% of their interviews at companies.”
Bartram: Companies only look at side projects during initial resume screening
“So companies, Don't actually pay very much attention to side projects. Except for at the screening stage. So resume screen, you know, candidate applies to company, the company decides if they're going to interview the person at all. And there's some adverse se…”
Bartram: 80% of software engineers lack side projects to show
“The reason it's the right decision is that most engineers don't have side projects. Most engineers have been, have, you know, been working at a company, and it's all proprietary code, and there's very little they can show. That's, you know, eight out of 10 eng…”
Bartram: Evaluating engineer skill from large codebases is startlingly difficult
“It's actually, it's startlingly hard to look at a big bit of code and decide if you think the programmer who wrote it is skilled. Just again, like, there's so much context, you can't tell what bugs they spend hours over, like, finding, like, finding bugs, like…”
Bartram: Big tech companies do not care where candidates live
“Big companies don't care at all where you're based. They have, they fly people in, you know, by the hundreds every week.”
Bartram: Bay Area candidates have a 10-20% advantage at small startups
“Smaller companies Do show a slight preference to local candidates. And so if your goal is to work at a small, let's say sub, you know, 20 person startup you're probably at a I don't know, 10 to 20% advantage if you're based in the Bay Area.”
Bartram: Trial employment periods drive away top software engineering candidates
“If you've worked with someone for a week, you have a far better read of their skill than I think anyone can get during a three to four hour interview. The problem is that there's a pretty strong bias in who's willing to do trial employment. And that's, it's ad…”
Bartram: Interview consistency matters more than which questions companies ask
“So like to come, just like to make sure that you're asking everyone the same question and make sure that you're evaluating them in the same way. And I think that's more important than what you're actually asking, right?”
Bartram: Whiteboard coding interviews make productive programmers look bad
“Whiteboard coding tends to skew toward the academic. It tends to Give preference to people who are really good at breaking their thoughts down into this sort of structured academic way and writing with a small amount of code. So you often have people who are a…”
Bartram: Very few companies evaluate candidates on flashy new tech
“Very few companies, especially only, generally only smaller ones, are like directly evaluating flashy new tech.”
Bartram: 80% of Published Interview Advice Fails Under Empirical Testing
“I truly believe that, like, 80% of what's written out about interviewing just doesn't actually hold up.”
Bartram: Past-Project Discussions Are Far Less Predictive Than Coding Tests
“We tried scoring engineers, and I had to have them talk about past projects. And scoring them and trying, like, even go, even like a full hour, like going into depth in the project, talking technical details, scoring it just talking skill. Ability to spin a ta…”
Bartram: Cutting bad interviews short damages company reputation
“I'm generally Against cutting interviews short, actually, I think it's, I think, except in the case where the interview where the candidate is in pain, I think it's not worth doing. I think you save some time, but you damage your reputation, you know, they rea…”
Bartram: Switch from evaluation to teaching mode when candidates fail interviews
“And a trick that, that we use that I think helps in that case is to sort of, in the case where a candidate is totally failing the interview, like, flipping a switch in your brain and going from, like, evaluation mode into teaching mode. You know, you're like, …”
Bartram: Interview panels should never exceed two interviewers per candidate
“Like an interview panel definitely increases the stress. So we max out at two to one. So it's, training is important, right? So if you're trying to keep it consistent, you need to have Continue across, like, across, and people need to watch each other's interv…”