The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 38 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Trojanowski: Applying the Bitter Lesson to Complex Tax Workflows Is a Mistake
“I think it is a mistake to throw out those learnings and say, you know, bitter lesson, throw out those learnings. We're just going to have the agents at runtime develop an entirely new way to do a tax return that is, you know, because bitter lesson, yada, yada…”
Trojanowski: Technical AI moats are not real moats
“Technical moats are not real moats. Like, I, there's no portion of basis's long-term terminal value that stems from some, you know, secret RL trick we found that nobody else found.”
Trojanowski: AI coding agents remain worse than junior engineers over two-week projects
“Even now with coding, like, the agents are not yet they're not human level at being coherent over long periods of time. That's obvious because they can't code like a junior engineer on a project for two weeks. So they can't, they're, that's worse than a human …”
Trojanowski: Enterprise agents need process reliability, not 'Move 37' breakthroughs
“And so the thing that you know, someone is buying from us is not, this will be the best ever tax return. They're buying that, you know, the confidence... They're buying that it's going to be consistent and reliable and something that they can trust that actual…”
Trojanowski: 10-Hour Agent Runs Are Not Black Boxes
“Not thinking that an agent operating over 10 hours is a black box. It's not. It has a lot of data, and you're probably doing a disservice to your customers if you don't understand, like, how it's going about the work.”
Trojanowski: Closing the agent self-improvement loop will be close by year-end
“I think that closing the loop is gonna happen pretty fast.
I think you'll have, like, I don't know about the entire loop being closed, but I think you'll be
Relatively close by end of year.”
Trojanowski: English prompt context matters more than code quality in agents
“A lot of engineers, they treat the code as more precious than the English, when actually the English is more precious because the English affects the performance.
The code does not affect the performance, right?
If the logic, assuming the logic's the same, it …”
Trojanowski: Scaling Verifiable Rewards Won't Reliably Automate Tax Returns
“I don't believe that if you were to train a model, you know, and you scale up the amount of pre-training computing, you scale up the amount of Like post training from like perfectly verifiable rewards that suddenly will output a model that will do a tax return…”
Trojanowski: AI models will subsume agent harness engineering in 2-5 years
“I think that how long it will take to get swallowed up. I don't know exactly. You know, I think it's probably sub five years. I don't think it's sub two years. I think it's probably sub five years.”
Trojanowski: AI agents prefer raw spoken thoughts over summarized text
“Speaking is just so much faster than writing things down, and in fact, when you try to write things down, you are Actually, essentially trying to summarize all the crazy thoughts in your head, and so that's why it takes a lot of time. And it's useful for your …”
Trojanowski: Claude 3 Opus Understood Long Contexts That GPT-4 Turbo Failed
“I would say they were Opus III, which I think goes underappreciated, but I think was the first model to truly be able to like actually understand that long context. Before, if you put anything in the, like, 80,000 tokens in the GPT-IV Turbo, it could not under…”
Trojanowski: OpenAI o3 proved post-training improves per-token reasoning quality
“And then I think after a one, it was oh three, because I think oh three helped prove that not only could you scale the amount of reasoning at inference time, but with better training, with more compute, better data, et cetera, in the post-training phase, you c…”
Trojanowski: DeepSeek-R1 succeeded by scaling outcome supervision over process supervision
“If you look, you know, if you fast forward a bit and you look at the, like the DeepSeq R-one paper where they effectively laid out, you know, what I think all the labs were doing at that time, or at least OpenAI was doing in terms of you know, RLVR reasoning f…”
Trojanowski: METR AI Agent Benchmark Is Inaccurate With a Low Bar
“I think the meter chart is somewhat inaccurate these days because it's so hard to measure. And also the bar is pretty low, right? The bar is pretty low. Yeah, yeah. The bar is pretty low, and I think, I'm not super familiar with their dataset, but my understan…”
Trojanowski: Verifiable rewards cannot solve software engineering because coding is subjective
“At the end of the day, coding is subjective. It's an art and you're not going to solve an art through verifiable rewards.”
Trojanowski: Good AI agent design should replicate how human organizations verify work
“Humans are already used to working with non-deterministic systems. It's just the systems are normally their coworkers, not their computers. And in many ways, like companies and processes, it's all about how do you design a system for non-deterministic entities…”
Trojanowski: Context engineering is runtime training with far less data
“I think the framework I like is thinking about it as training data. Except you're just training the model at runtime. It is training data. And because you, the model's learning at inference time, the total amount of training data is far lower, right? Like the …”
Trojanowski: Nothing Paradigm-Shifting Has Changed Since o3
“Things have really not changed since oh three, I would say. Almost everything since oh three has been relatively on, I don't want to say on trend and then like I knew this exact trend, but I would say it's all within the same paradigm. Like nothing paradigm sh…”
Trojanowski: Cheaper inference is replacing graphs and embeddings for agent ontologies
“As models are getting cheaper and cheaper, more and more of that actually can just be done using inference instead of using determined, like things like graphs or things like embeddings.”
Trojanowski: Most engineering historically was execution-oriented, not systems thinking
“Now, I think the majority of engineering historically has not been really systems thinking based. You know, it's been a little bit more execution oriented, but if you think about like the hardest engineering, like, Hey, I'm trying to design like what this is g…”
Trojanowski: Legal drafting requires the same skills as AI context engineering
“I think law actually is kind of like that. You know, in many ways I think about, I think maybe the founding fathers would have been really good context engineers or agent managers, because you had to write, you know, a piece of English that was going to be, yo…”
Trojanowski: AI models are far worse at agent engineering than software
“Right now they are far, far, far worse at engineering agent systems than they are at engineering most software.
Far worse.
because by definition, like that kind of work, which is so novel, has not seen a large amount in their training data.
and so they have …”
Trojanowski: Accounting Automation Requires Long-Horizon Coherence Across Many Actions
“Accounting is difficult. It's not something that is just purely text in text out. And so it requires the ability for like AIs to be able to, you know, perform lots of actions over long periods of time and actually you know, be coherent over that period of time…”
Trojanowski: Accounting Acts as Information Compression Over the Economy
“And accounting is actually the art of compressing all of that into something that is structured that now people can look at and understand and make decisions. So something about accounting you could argue in a meta way is kind of like an intelligence over the …”
Trojanowski: Autonomous agents must optimize for reviewer legibility, not just completion
“And you can like optimize, not just for getting the work done, but for making it easy for your reviewer to understand the decisions that you made. And that's obviously very true in software engineering, and it's actually true in, I think, most professions and …”
Trojanowski: Long-Horizon Agents Must Use Reasoning for State Management
“Once you start getting to longer horizons where you're beyond the context window or you're getting to context rot, you need to kind of use in sort of brute force your reasoning to build out your environment. Whether that be with sub agents or compaction, we ta…”
Trojanowski: Better to Give AI Agents Principles and Context Than Step-by-Step Instructions
“As people who are good at building agents know, like it's much better at the margins to be able to give principles and the whys and more context and let them figure it out.”
Trojanowski: LLM agent review requires uncorrelated trajectories to avoid bias
“So there's some activation state that by definition is going to be biased to that current trajectory. And so maybe for review, you want an uncorrelated trajectory, right? Where it's like a new box and it's just a smart.”
Trojanowski: Frontier agent architectures combine LLM intuition with org design
“If you build these kind of LLM intuitions and you combine them with, you know, maybe basic principles of, like, organizational design and management I think you start to get to maybe what is, like, the frontier of agent building.”
Trojanowski: Best Agent Intuition Comes From Automating Your Own Work
“A lot of the people with the best agent intuition, actually, yes, a lot of people come from ML backgrounds, but people who don't, a lot of them are ones who are just really good at automating their own work.”
Trojanowski: Agent Behavior Specs Align Human Teams, Not Just Models
“It is actually both a spec and a rubric. We call this specs and there were some people who asked, isn't this a rubric? And it is, it's both. The reason it's both is because it is not just used to grade or potentially reward the agent. It's also used to align t…”
Trojanowski: Codebase ontology matters as much for AI agents as for engineers
“The ontology of your code base, it always mattered for engineers. It matters just as much, if not more, for really good agents over time.”
Trojanowski: Maintaining company canon will be a key human role in agent-native firms
“And that's why I think, you know, for true agent native companies especially in the future today, I think it's still quite early, but especially in the future, having a clear understanding of what your company canon is and organizing that in an ontology that m…”
Trojanowski: Key skill for agent roles is systems thinking across abstractions
“I think one thing, one skill that really matters is good systems thinking. And where does good systems thinking come from? It comes from people who have had to think about some abstraction, some system, something, and design it in such a way that it performs i…”
Trojanowski: Basis Prioritizes Agent Orchestration Over Model Weight Updates
“Today we don't go directly into the weights. And primarily the reason we don't do that is because A lot of the advancements the models are having when it comes to orchestrating themselves yield far more performance gains than benefits you would have of like up…”
Trojanowski: Over Three Million Accountants Work in the US
“It is, one of, if not the largest knowledge work profession in the country. There are over three million, you know, combined kind of accountants in the country.”
Trojanowski: Preparing a Form 1065 tax return takes 20+ hours of human labor
“Performing a 1065 can take a human, you know like 20 plus hours easily of actual work. I don't mean like it took them a day. I meant like literal sitting down work. And it could actually take much longer for very complicated returns.”
Trojanowski: Basis uses full AI agents as behavior evaluation judges
“So right now, at least the behavior judging is relatively expensive because it's a pretty advanced judge in that it is also an agent. It's not a judge in the traditional sense. It's literally an agent because it has to look at the trajectory.”