Forsgren: Software delivery performance does not statistically vary by company size
Nicole Forsgren · How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, Github) · Jul 30, 2023 · at 22:55
Dr. Nicole Forsgren discusses whether DevOps and DORA delivery benchmarks only apply to large enterprises versus early-stage startups.
0:00 / 0:09exact quote · 9.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We saw no statistical significance between Small companies and large companies, the only statistical significant, statistically significant difference was with retail.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Nicole Forsgren
Forsgren: DORA metrics alone are insufficient in the AI era
“If that's all you're looking at, it's not going to be sufficient anymore because AI has now changed the way we think about feedback loops.”
Forsgren: Software delivery speed and stability move in tandem, not trade-offs
“When we started measuring these, we found that they move in tandem. Now, like, with very strong significance, right, from a statistical standpoint. Now, what this means is, now we say speed and stability move together.”
Forsgren: AI Coding Tools Should Free Up Cognitive Space, Not Prompt Layoffs
“Anyone who's looking at that and like, dear CEOs or whoever who are like, now I can lay off half my workforce. That's not what this is about, right? It's not about taking a task and cutting your time in half because now what we've enabled is your ability to do…”
Forsgren: Developer surveys almost always prove more accurate than telemetry during conflicts
“If there is ever a disagreement between the surveys and the instrumentation, which is incredibly advanced, almost every time that I've ever heard of the surveys are correct and not the instrumentation.”
Forsgren: Most developer productivity metrics are a lie
“I will say most productivity metrics are a lie.”
Forsgren: Non-deterministic LLM code demands active evaluation over direct acceptance
“LLMs are non-deterministic, right? Now we can't just Put in a command and get something back and accept it. We really need to evaluate it. So, you know, are we seeing hallucinations? What's the reliability? Does it meet like the style that we would typically w…”