The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 9 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Tech is likely near the peak of the current generative AI hype cycle
“I think what we're going through now is we're probably close to the peak of the current hype cycle that was introduced earlier this year with, you know, ChatGPT and everything.”
Word clouds and topic modeling fail to capture nuanced customer feedback
“You know, people have done word clouds, word counts topic modeling, but, you know, none of those really capture the nuance of what people are saying. And you know, it doesn't account for the fact that people could be saying multiple things per response.”
Machine learning engineering skills transfer broadly across vision, NLP, and audio domains
“One of the things about machine learning and AI in this industry is that there's a lot of crossover between these different data domains. You know, if you're able to solve a computer vision problem pretty well, a lot of those skills are transferable to natural…”
Mandich: User research synthesis lacks an objective universal ground truth
“You can take two expert user researchers, give them the same list of, you know, 500 responses, tell them to distill them down into 10 actionable takeaways, and those 10 will be completely different, or even if they are the same 10 takeaways, the responses that…”
Analyzing machine learning data in batch produces better results than real-time streaming
“From an ML point of view. I think it's advantageous to analyze as much data in batch as possible. You tend to get more information to work with.”
Complex non-deterministic LLM tasks still require manual human evaluation
“And that's just, that's something that's really hard to evaluate in an automated manner, at least right now, because it's a more complex task, because the output is, you know, non-deterministic and freeform. For now, it really just does require some manual eva…”
Hosted LLMs shift hiring demand toward full-stack machine learning engineers
“I think given our current switch to hosted LLMs and away from, you know, taking Google's Burt and fine tuning it ourselves, more towards, I guess we call like a full stack ML engineer, somebody who's able to, you know, help integrate this and actually implemen…”
The next 12 to 18 months of AI will target hallucinations and math
“To your original question, I think issues like those, hallucinations, lack of quant answering, those are probably going to be the major things that get released over the next couple years. I don't know what that looks like. You know, it might be an LLM that's …”
Large language models currently fail at basic math and quantitative reasoning
“Right now a lot of these models fall flat and for an application like ours and for a lot of applications out there, that's kind of a pretty glaring omission. You know, these models are fantastic at summarization. They're really good at generating text based on…”