Metrics
topic on 9 shows · 23 statements across 21 episodes
the Y Combinator Startup Podcast 
In Depth 
More or Less 
the Knowledge Project 
Lenny's Podcast 
My First Million 
No Priors 
the Official SaaStr Podcast 
20VC
23 statements about Metrics, every show
Puri: Customer anecdotes usually prove that your operational metrics are incomplete
“What I'm telling you is that the anecdotes typically show you that your metric is incomplete, that it's not capturing some part of the customer's experience that actually matters, and both the metric is right and the customer is right. The metric is just incom…”
Karpathy: Autonomous training loops will overfit metrics via Goodhart's Law
“I do think that if you had an autonomous loop over those metrics, there's gonna be a lot of like good harding going on where the system will like overfit to those metrics.”
Alex Schultz: A company North Star is a goal, not a metric
“Well, the most important thing is the North Star is not a metric. So the North Star is your goal. Your goal is like, what are you trying to achieve?”
Baxley: Successful products treat metrics as consequences, not drivers
“I've never seen a product be successful that used metrics as a driver for what they were doing. I've seen a lot of companies be really successfully seeing metrics as a consequence and a way to evaluate the quality of their decisions”
Chennapragada: Using mature metrics for zero-to-one products creates false precision
“When you're looking at something zero to one,
If you decide on a metric too prematurely, that's false precision, first of all, right?
Like, you kind of I mean, CTR, when you have, like, thousand people, doesn't mean anything.
you know, retention also may not …”
Yehoshua: PMs should not over-rely on metrics over intuition
“One of the things that I caution product managers about is that you don't want to be too overly reliant on metrics. And you want to also have an intuition. You want product managers who understand intuitively their customers and their product, and sometimes yo…”
Dave Morin: Default-private teen accounts will hurt Instagram's engagement metrics
“Changing the default is a huge deal. And I can't think of another technology company that is willingly, or perhaps under some pressure here, changed the default. It's something that actually really will, you know, influence the metrics. Perhaps damage the busi…”
Founders make a deadly mistake by tracking too many dashboard metrics
“I think the biggest mistake that I see founders making day to day is not having a handle on their metrics, right? I think everybody measures metrics in like a different way. And I think we've kind of entered an era of death by a thousand dashboards. And I woul…”
Sutherland: Rigid corporate metrics distort behavior and demotivate employees
“The metrics often actually get gained or they lead to a complete distortion of behavior. The second thing with metrics is that either people gain them to their own advantage or they obey them, but find them stupid. But the loss of autonomy that results when yo…”
Franceschi: Companies must evaluate financial metrics in the most punitive way
“I have this belief that That we need to look at metrics the most punitive way possible.”
Weinstein: Product metrics should measure customer value, not internal events
“Metrics at their best are A numerical representation of the value we're providing for the customer.”
Weinstein: Metrics are almost always wrongly defined for weeks
“Metrics are almost always wrong for many weeks. We didn't quite get the definition correctly. You have to live in the metric for quite a while before you really believe in it.”
Blomfield: Debating Metric Definitions Is Worse Than Having No Metrics
“Constant arguments about what your key metrics are even worse than having no metrics at all.”
Larson: Half of metrics' value is educating leadership mental models
“I think people want metrics to show reality. They want to show the truth, but half of metrics is showing the truth. The other half is educating people to inform their mental model about how the truth works.”
Rachitsky: There is always one primary lever that impacts goals most
“And then it turns out there's always one thing that actually impacts the bigger goal most. So you just end up working on that thing anyway.”
Balfour: Metrics measure if a growth strategy is working, not define it
“Your metrics are there to help answer the question, is your strategy working? Right. And so they are not there to determine your strategy.”
Merritt: Companies Start with No Metrics, Add Too Many, Then Consolidate
“We did all those standard metrics, and I think you start with kind of none, and then you have too many, and then you shrink back, you're like, actually, these are the ones that matter. That's like the arc for us on the ones that we did measure.”
Mehta: Goal setting must target understanding before trying to move metrics
“And if you don't understand how to move a particular metric, then the right goal is to set a goal to increase your understanding, not to move that metric.”
Stanley: Short-term metric gains in complex problems are deceptive dead ends
“Even if your score on a metric is going up in the short run, it doesn't mean it will all get all the way to the point that you want it to get. That's a fundamental problem. It's called deception. And it's a fundamental problem of all complex problems.”
Stanley: Metrics are useless for hard problems because required stepping stones are counterintuitive
“If the actual stepping stones that lead to where we want to go are counterintuitive, in other words, they're not what you would expect, then the metrics are useless, right? Because they won't detect those stepping stones because they don't look like what the m…”
Rushkoff: Focusing Exclusively on Displayed Metrics Distorts Real Value
“The metric you put on the wall is the metric that you're going to get, you know, and it may not be an ideal single focus for more than just some provisional fun experiment.”
Puttagunta: The venture ecosystem over-rotates on managing early-stage startups by metrics
“Yeah, I think there's been, frankly, an over rotation into just running businesses simply on metrics.”