The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 50 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Sachs: AI Model Quality Varies Between First-Party APIs and Cloud Providers
“Companies that say they're selling the same model through different vendors, whether it be through first party or Bedrock, Azure, et cetera, we do see different qualities sometimes, and that's not necessarily what's advertised.”
Notion Will Beat Frontier Labs in Collaboration Like Datadog Beat AWS
“Datadog could not exist without cloud storage, right? That it's kind of fundamental that that works. And AWS has like a cloud watch product, but Datadog is an expert on understanding how people want observability on the products they launch. And we're experts …”
Simon Last: 99% of Agent Failures Are Tool Bugs, Not Model Flaws
“And actually, 99% of the time, it's a bug in one of the tools. Right. And so, just fix the bug.”
Notion Engineers Faced an Identity Crisis Shifting From Coding to AI Delegation
“Every software engineer in Notion this summer went through like this sheer one of our engineering leads at the company called it like, every software engineer is going through the identity crisis that every manager goes through, where all of a sudden they real…”
Supervising AI Agents Is a Deeply Technical Systems Problem, Unlike Managing Humans
“There's a critical difference
To being a manager, which is that like, it is actually very deeply technical.
The problem of, you know, humans are very like fuzzy and you can't like treat a team of humans like a rigorous system where like, you know, PRs like fle…”
Over-Simplifying AI Agent Interfaces Abstracts Interpretability and Nerfs Agent Capabilities
“I'd actually say we don't try and make it as easy as possible to use because the more we do that, the more we abstract away that interpretability that Simon's talking about that basically nerfs the model or nerfs the agent from being super capable.”
Claude Sonnet Was the Breakthrough That Unlocked Notion's Custom Agents
“The big unlock was Probably like Sonic 3.6 or seven early last year. And that's when we started working on our agent, which we shipped last year.”
Notion Custom Agents Use Standard Pages and Databases as Their Memory
“Another example of this is we have no built-in memory concept. Memory is, is just pages and databases. And so if you want to give it memory, just give it a page and give it access to that page.”
Frontier AI Labs Altered Final Model Snapshots Based on Notion's Enterprise Feedback
“Like they have shipped models that were not the snapshots that we wanted, and they have changed the snapshots that they shipped based on the feedback that we give, because our feedback tends to be more enterprise work focused and not coding agent focused.”
Frontier AI Product Development Requires Knowing When to Stop Fighting Model Limits
“Is really what will set notion apart for every new capability is we have like two skills that are crucial when it comes to frontier capabilities. One is not letting yourself swim upstream. So like quickly realizing if you're just pressing against model capabil…”
Notion Partnered With Anthropic and OpenAI to Build 30% Pass Rate Evals
“And then what we have, what we call Frontier Headroom evals, where we actively want to be at 30% pass rate. And that's actually been a effort that we took in partnership with Anthropic and OpenAI in the past maybe two or three months, because we actually hit a…”
Optimizing Vector Embedding Models Is No Longer the Right RAG Retrieval Focus
“We don't spend a lot of time trying to optimize what vector embedding we use anymore. That was a period of time, but that's just not the right level of optimization, right?”
Model Behavior Engineers Do Not Need Traditional Engineering Backgrounds
“And so, we have, like, very firm conviction And we have had for a number of years now that that is its own career path, and we have always welcomed the misfits, so to speak. So we really firmly believe that you don't need an engineering background to be the be…”
The Majority of Notion Enterprise Search Traffic Now Comes From AI Agents
“We're at a point right now in our business and enterprise or AI enabled plans where the search load and the search traffic, a majority of it's coming from agents, not humans.”
Notion Fine-Tunes Its AI Capabilities Entirely Without Relying on User Data
“There are certain aspects of Notion where we do fine tune and do reinforce and fine tuning on our own capabilities. But that's not necessarily trained on user data. You don't need that, that much data in the first place.”
Fine-Tuning Models on Internal Tools Unnecessarily Slows Down Rapid Product Development
“It would actually really slow us down to have a model that was fine tuned on our tools because we'd have to retrain it and cut a new model every time we did that.”
AI Labs Cluster at Extremes and Ignore Intermediate Intelligence-Latency Tiers
“Everyone's clustered in capability, where everyone's clustered, I mean, Haiku's not that much cheaper. No one's really in the middle. Like, people really tend to cluster around two. Like, this is really capable, and it's really fast, but it's really expensive,…”
Agent Environments Must Favor Model Preferences Over Internal System Architecture
“I mean, that was, I would say that was a big learning is just, you know, really be savvy and really careful thinking about what the model wants in terms of, you know, its environment and cater around that and really try so hard not to expose it to any complexi…”
Turbopuffer Bought Oregon Dark Fiber to Serve Notion Across Clouds
“It started getting really painful in like mid-twenty-twenty-four, because we were closing deals with Notion actually, that was running in AWS, and we're like, trust us, you really want us to run this in GCP?
And they were like, no, I don't know about that, lik…”
Polu: Airbyte's Notion connector output is not useful for AI models
“And the reality is that if you look at Notion, Airby does the job of taking Notion and putting it in a structured way, but that's a way that is not really usable to actually make it available to models in a useful way. Because you get all the blocks, details, …”
AI Teams Lose Velocity When Building Cool Tools Without User Journeys
“I actually think the failure of our team is when we focus too much on what are tools that are cool tools. I actually think that's when we make have the least velocity because you still need some sort of focus on a user journey.”
Notion Observes Increased AI API Latency During Standard Business Hours
“We've definitely noticed, particularly for some providers that things are slower during working hours.”
Notion Bypasses Third-Party MCP Search to Maintain Internal Quality Control
“There are a lot of things where we choose not to use MCP because we want to add more high touch to quality. I think search and agentic find is like the largest instance of that, where we have slack and linear and JIRA search and notion that is not using necess…”
Notion's Design Team Builds Live Code Prototypes Instead of Static Mocks
“Our design team made a whole separate GitHub repo called the design playground. And it's basically just to create a bunch of like helper components and for quickly throwing together UIs and it's become like actually quite sophisticated. Like it has like an age…”
Frictionless AI Prototyping Requires Strong Product Conviction to Avoid Feature Bloat
“I think it's put a big pressure point on us to have really strong product conviction, because if anything can be demoed, You really need a strong filter of making sure that if, you know, you're doing X amount of work, you're making the, you're focusing on one …”
Claude Sonnet Crashed on Duplicate Tool Names While OpenAI Handled the Error
“Sonic couldn't handle two tools with the same name in OpenAI, GPT, 5.2. It was like, ah, I can figure this out. So that was an interesting one that we learned by accident through a SEV.”
Executing Deterministic Code via CLI Avoids Recurring LLM Token Fees From MCP
“And so if we can have our agents properly execute code that calls on CLI deterministically, it's a one time cost, right? Versus constantly having a language model integrate with an MCP over and over and over and paying those like repeated token fees.”
The Majority of Notion AI Tasks Do Not Require Opus-Level Intelligence
“It's not Opus, that's for sure, because a majority of the tasks people are doing aren't Opus level intelligence.”
Agent Search Prioritizes Top-K Retrieval Depth Over Precise Positional Ranking
“Positional ranking matters less, but top care retrieval mode matters more.”
Goal-Driven Tool Definitions Allowed Notion to Distribute Agent Tool Ownership
“When we went away from few shots to describing the goal of the tool and like goal driven, basically moving from a DAG to like a true system with feedback, that's when we could distribute tool ownership to the teams much better.”
MCP Lacks a Trigger Protocol, Forcing Notion to Build Custom Trigger Infrastructure
“MCP is a really great way to gain access to tools. Works really well. But you just looked at the trigger UI, for example, there's no trigger protocol. And so those are things we had to build ourselves.”
Notion Feature Teams Own Their AI Evaluations Running Nightly in CI
“We maintain the eval framework. Every team owns their own evals and a lot of them we've integrated to opt into CI or we run them nightly and we have a team, a custom agent that triggers to a team to look at the major failures.”
Eskildsen: Anthropic, Notion, and Cursor use Turbopuffer across three deployment models
“You can run Turbo Puffer either in SAS, right? That's what cursor does. You can run it in a single tenant cluster. So it's just you. That's what Notion does. And then you can run it in, in, in BYOC where everything is inside the customer's VPC. That's what, fo…”
Grinberg: AI Coding Agents Need Full Workplace Context, Not Just IDEs
“Everyone expects these you know, agentic systems to perform at the level of a human, right? Because that's what they're always going to compare them to. But in a lot of cases, they'll have these agents just in the IDE. And that's like the equivalent of onboard…”
Crivello: AI Agent Community Will Mirror Webflow, Zapier, and Airtable Users
“I think the no-code tinkerers is the community. Yeah. It is going to be the same sort of community as what, yeah, Webflow, Zapier, Airtable, Notion, to some extent.”
Notion Rebuilt Its Agent Framework Four or Five Times Before Launch
“It's probably the fourth or fifth time that we rebuilt that.”
Notion Fine-Tuned Custom Function-Calling Models Before Standard Tool Calling Existed
“Before function calling came out, we were trying to fine tune with the frontier labs and with fireworks, like a function calling model on notion functions.”
Notion's AI Team Reviews p99 Token-Exhaustive Agent Transcripts Every Friday
“We'll all sit down every Friday and look at the P. 99 of like the most token exhaustive custom agent transcript and just look at why it didn't do well and cut a bunch of tasks.”
Notion's Custom Agent Operates With Over 100 Distinct Internal Feature Tools
“So there's now like over a hundred tools just for all, all the crazy notion stuff.”
Notion Will Soon Release Direct Agent-to-Agent Invocation Settings
“So I think it's actually not released yet. Releasing it like next week is in the settings for an agent, you can give it access to invoke any other agent. So you can have them just talk directly.”
Notion's Database Autofill Will Soon Become Agentic With Usage-Based Pricing
“The database autofill feature will soon be agentic. That will be associated with usage based pricing”
Notion Dedicates 50 Engineers to Core AI and 40 to Packaging
“I manage, ah, the team that's what we'll call like core AI capabilities and infrastructure. That's about 50 people. But then we have, I partner teams that do packaging, so how it shows up in the corner chat versus custom agents versus meeting notes, that's ano…”
Notion's Meeting Notes Automatically Mention Individuals Referenced in Generated Summaries
“One thing that the meeting notes team added recently that was But I've been blowing my mind is they, we have, they made it so it actually, when it makes the summary will actually mention the people that were referenced in it. So I now get notifications wheneve…”
Self-Healing Agent Capabilities Are Next on Notion's Product Roadmap
“Obviously we should build product of self-healing. That's next on our roadmap.”
Notion's Internal Bug-Triaging Custom Agent Completely Changed Company Operations
“One of our biggest use cases is bug triaging. So if someone posts something in Slack and you just have a custom agent that lives there that has its own routing constitution of what team this belongs to. Creates a task in your task database and then posts in th…”
Notion Commits to Supporting MCP Integrations as Long as Users Demand Them
“So we will always support our MCP insofar as other people are using MCPs, right?”