Domain Expert

“The people who are, who have developed an expertise who can see nuance, whether it's around political topics or tech or whatever their field, medical healthcare the people who can do that are still way ahead of where AI is today.”

“Our view is you don't need 100,000 of people. You need the smartest people in a given domain.”

“Benevolent dictator is just a catchy term for the fact that when you're doing this open coding, a lot of teens get bogged down in having a committee do this. And for a lot of situations, that's wholly unnecessary. Like, You know, people get really uncomfortabl…”

“This is this really counterintuitive thing where, and you've talked about specialization on the last one is like it, the biggest gains of AI go to people who have some degree of expertise in an area to know what is actually true. What is not going to work? Wha…”

“I think the domain experts, like in our case, clinicians, they're really good at like debugging model outputs, meeting with users, distilling that feedback into something actionable, maybe annotating or doing evals, but they don't necessarily have like, you kn…”

“Domain experts are driving more of the product decisions.”

“I think that they'll play a role in creating their own evals, where they, like, yeah, where they might come up with certain criteria for what a good response looks like, and humans validate that criteria. However, I think you often need to ground this in, like…”

“One really huge thing about LLM as a judge, people love LLM as a judge, but you really have to make sure that you can trust the LLM as a judge. And so how do you trust and how do you trust anything is that you have to check it and you have to measure how good …”

“And the reality is you have to hire very good lawyers who can actually evaluate these systems, and the same within tax and these other areas, and they can't be too junior, because if they were too junior and they were able to eval it, they would be senior, rig…”

“My usual question is, explain to me why this can't work. That's the only question I ever ask experts with domain expertise, and if they can't identify anything, then I'm perfectly fine, like investing in a company that's going to try to reinvent, you know, som…”

“So these, like, existing mental models for a generalist help you kind of, like, spot Opportunities. I think, like, sometimes, like, a domain expert get really tunnel vision on.”

“And so I sympathize with, like, everyone that actually needs and have some domain expertise, but won't get into code anytime soon, even with Copilot or whatsoever, and that still need to leverage their data in order to get things done.”

“However, I believe that, and this is based on my experience training these models, it's actually the domain experts will be driving the best models out there. It won't be people like me who can actually do all the model training, et cetera.”

“They always make the mistake of choosing the domain expert. I would say you should almost always choose the sales expert. A good salesperson, a good AE will learn your product, will learn your market. That's what they've done their whole careers. Trying to tea…”

“Anyone who's in the venture business saying, I'm a domain expert, I'm only investing in X as an industry sector, I think that's a very fragile and brittle strategy over the time span of decades.”

“There's very few people that have domain expertise that can write, that can explain things.”