Opinion
Trojanowski: Applying the Bitter Lesson to Complex Tax Workflows Is a Mistake
“I think it is a mistake to throw out those learnings and say, you know, bitter lesson, throw out those learnings. We're just going to have the agents at runtime develop an entirely new way to do a tax return that is, you know, because bitter lesson, yada, yada…”
Insight
Trojanowski: Technical AI moats are not real moats
“Technical moats are not real moats. Like, I, there's no portion of basis's long-term terminal value that stems from some, you know, secret RL trick we found that nobody else found.”
Assertion Not checkable as stated
Trojanowski: AI coding agents remain worse than junior engineers over two-week projects
“Even now with coding, like, the agents are not yet they're not human level at being coherent over long periods of time. That's obvious because they can't code like a junior engineer on a project for two weeks. So they can't, they're, that's worse than a human …”
Opinion
Trojanowski: Enterprise agents need process reliability, not 'Move 37' breakthroughs
“And so the thing that you know, someone is buying from us is not, this will be the best ever tax return. They're buying that, you know, the confidence... They're buying that it's going to be consistent and reliable and something that they can trust that actual…”
Insight
Trojanowski: 10-Hour Agent Runs Are Not Black Boxes
“Not thinking that an agent operating over 10 hours is a black box. It's not. It has a lot of data, and you're probably doing a disservice to your customers if you don't understand, like, how it's going about the work.”
Prediction Not checkable as stated
Trojanowski: Closing the agent self-improvement loop will be close by year-end
“I think that closing the loop is gonna happen pretty fast.
I think you'll have, like, I don't know about the entire loop being closed, but I think you'll be
Relatively close by end of year.”
Insight
Trojanowski: English prompt context matters more than code quality in agents
“A lot of engineers, they treat the code as more precious than the English, when actually the English is more precious because the English affects the performance.
The code does not affect the performance, right?
If the logic, assuming the logic's the same, it …”
Prediction Not checkable as stated
Trojanowski: Scaling Verifiable Rewards Won't Reliably Automate Tax Returns
“I don't believe that if you were to train a model, you know, and you scale up the amount of pre-training computing, you scale up the amount of Like post training from like perfectly verifiable rewards that suddenly will output a model that will do a tax return…”
Prediction Not checkable as stated
Trojanowski: AI models will subsume agent harness engineering in 2-5 years
“I think that how long it will take to get swallowed up. I don't know exactly. You know, I think it's probably sub five years. I don't think it's sub two years. I think it's probably sub five years.”
Insight
Trojanowski: AI agents prefer raw spoken thoughts over summarized text
“Speaking is just so much faster than writing things down, and in fact, when you try to write things down, you are Actually, essentially trying to summarize all the crazy thoughts in your head, and so that's why it takes a lot of time. And it's useful for your …”
Opinion
Trojanowski: Claude 3 Opus Understood Long Contexts That GPT-4 Turbo Failed
“I would say they were Opus III, which I think goes underappreciated, but I think was the first model to truly be able to like actually understand that long context. Before, if you put anything in the, like, 80,000 tokens in the GPT-IV Turbo, it could not under…”
Insight
Trojanowski: OpenAI o3 proved post-training improves per-token reasoning quality
“And then I think after a one, it was oh three, because I think oh three helped prove that not only could you scale the amount of reasoning at inference time, but with better training, with more compute, better data, et cetera, in the post-training phase, you c…”
Assertion Supported
Trojanowski: DeepSeek-R1 succeeded by scaling outcome supervision over process supervision
“If you look, you know, if you fast forward a bit and you look at the, like the DeepSeq R-one paper where they effectively laid out, you know, what I think all the labs were doing at that time, or at least OpenAI was doing in terms of you know, RLVR reasoning f…”
Opinion
Trojanowski: METR AI Agent Benchmark Is Inaccurate With a Low Bar
“I think the meter chart is somewhat inaccurate these days because it's so hard to measure. And also the bar is pretty low, right? The bar is pretty low. Yeah, yeah. The bar is pretty low, and I think, I'm not super familiar with their dataset, but my understan…”
Insight
Trojanowski: Verifiable rewards cannot solve software engineering because coding is subjective
“At the end of the day, coding is subjective. It's an art and you're not going to solve an art through verifiable rewards.”
Insight
Trojanowski: Good AI agent design should replicate how human organizations verify work
“Humans are already used to working with non-deterministic systems. It's just the systems are normally their coworkers, not their computers. And in many ways, like companies and processes, it's all about how do you design a system for non-deterministic entities…”
Insight
Trojanowski: Context engineering is runtime training with far less data
“I think the framework I like is thinking about it as training data. Except you're just training the model at runtime. It is training data. And because you, the model's learning at inference time, the total amount of training data is far lower, right? Like the …”
Opinion
Trojanowski: Nothing Paradigm-Shifting Has Changed Since o3
“Things have really not changed since oh three, I would say. Almost everything since oh three has been relatively on, I don't want to say on trend and then like I knew this exact trend, but I would say it's all within the same paradigm. Like nothing paradigm sh…”
Insight
Trojanowski: Cheaper inference is replacing graphs and embeddings for agent ontologies
“As models are getting cheaper and cheaper, more and more of that actually can just be done using inference instead of using determined, like things like graphs or things like embeddings.”
Opinion
Trojanowski: Most engineering historically was execution-oriented, not systems thinking
“Now, I think the majority of engineering historically has not been really systems thinking based. You know, it's been a little bit more execution oriented, but if you think about like the hardest engineering, like, Hey, I'm trying to design like what this is g…”
Insight
Trojanowski: Legal drafting requires the same skills as AI context engineering
“I think law actually is kind of like that. You know, in many ways I think about, I think maybe the founding fathers would have been really good context engineers or agent managers, because you had to write, you know, a piece of English that was going to be, yo…”
Assertion Not checkable as stated
Trojanowski: AI models are far worse at agent engineering than software
“Right now they are far, far, far worse at engineering agent systems than they are at engineering most software.
Far worse.
because by definition, like that kind of work, which is so novel, has not seen a large amount in their training data.
and so they have …”
Insight
Trojanowski: Accounting Automation Requires Long-Horizon Coherence Across Many Actions
“Accounting is difficult. It's not something that is just purely text in text out. And so it requires the ability for like AIs to be able to, you know, perform lots of actions over long periods of time and actually you know, be coherent over that period of time…”
Insight
Trojanowski: Accounting Acts as Information Compression Over the Economy
“And accounting is actually the art of compressing all of that into something that is structured that now people can look at and understand and make decisions. So something about accounting you could argue in a meta way is kind of like an intelligence over the …”
