why aren't all 126 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 6 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Prediction Not checkable as stated
Shafqat: NotebookLM will theoretically cover most languages soon
“So I guess high level, like we're definitely working on adding more languages. That's like top priority. We're going to start small, but like theoretically we should be able to cover like most languages pretty soon.”
Disclosure
Martin: NotebookLM is prioritizing real-time interactive audio chat
“We're actively, that's one of the things we're actively prioritizing.
Actually, one of the interesting things is now we're like, why would anyone want to do that?
Right?
Like, what are the actual, like kind of going back to sort of having a strong POV about th…”
Insight
Yi Tay: AI researchers should draft papers and titles before running experiments
“I usually start writing the thing while working on that thing itself. Like, so, even, like, let's say, like, if you want to launch something, like, then the end goal is, like, a blog post, or shipping something, everything, right? I like, or not really a launc…”
Assertion Supported
Patel: Google TPU v5e is half the size of TPU v5p
“TPUv-V-V-E is, like the new one, but it's mostly, mostly an inference chip. It's a small chip. It's a little bit, it's about half the size of a TPUv-V-V. That chip, you know, you can get very good performance on, like, Of Lama, 70 B inference, right?”
Prediction Held up
Rizk predicts Google will soon automatically flag AI-generated media
“The second thing is, frankly, I don't think it's up to you, whether you tell them or not. Very, very soon, like, Google is just gonna mark things as AI generated.”
Assertion Partly supported
Hotz: Google's TPU compiler is a closed-source 32MB binary blob
“Not only is the chip closed source, But all of XLA is open source, but the XLA to TPU compiler is a 32 megabyte binary blob called lib TPU on Google's cloud instances. It's all closed source.”
Disclosure
Younis: Majority of Applied Intuition's first 400 hires came from Google
“I mean, our first of 400 people, majority were Googlers. Like a majority of the company came from, you know, this giant company we worked at.”
Assertion Not checkable as stated
Younis: Google discussed creating an internal university around 2014-2016
“Google as most, as recent as probably 10 years ago was thinking of starting a university internally, there was discussions on it.”
Disclosure
Dean: Google evaluates models against held-out internal benchmarks absent from training data
“So we have a bunch of held out internal benchmarks that we really look at where we know That wasn't represented in the training data at all. There are capabilities that we want the model to have that it doesn't have now, and then we can work on, you know, asse…”
Assertion Supported
Dean: Google Search index updates improved from monthly to sub-minute
“So the update rate actually is the parameter that changed the most. Surprisingly. So it used to be once a month. And then we went to a system that could update any particular page in, like, sub one minute.”
Assertion Partly supported
Jeff Dean: Distillation originated to compress 50-model ensembles into serviceable form
“Distillation was originally motivated because we were seeing that we had a very large image data set at the time, you know, three hundred million images that we could train on with, you know, I forget, like 20,000 categories or something, so much bigger than I…”
Assertion Supported
Dean: Google moved its entire search index into memory in 2001
“So in 2001, we introduced we put our entire index in memory. And what that enabled from a quality perspective was amazing, because before, you had to be really careful about You know, how many different terms you looked at for a query, because every one of the…”
Assertion Supported
Borovik: Google built internal AI coding tools in Cider IDE for years
“Being Google, we built a lot of our own tools and, you know, cider, which, you know, folks maybe have heard of is our, you know, our internal IDE and we've had all kinds of, you know, capabilities and tools there for a while. So yes, you know, we certainly hav…”
Assertion Partly supported
Morcos: Gemini tech report names data quality as single most important factor
“If you look at like the data section of like the Gemini tech report, it basically says like data quality was the single most important thing for making great model.”
Assertion Supported
DeepMind and OpenAI used different reduction methods to solve IMO Problem 1
“Google has one method and then OpenAI AI also have another method but both kind of works. Basically you just reduced any n to three, and then you just do case by case analysis.”
Assertion Contradicted
Google Gemini Flash-Lite ranks in top five on Galileo Agent Leaderboard
“In the, on the original leaderboard, we also have flashlight, which is, nobody talks about, I think, even now, but I think it's got a very decent score, which is, like, in the top five models. It's extremely cheap model, like, it's so dirt cheap that it makes …”
Assertion Supported
Kilpatrick: Google's implicit caching automatically passes cost savings to developers
“Explicit caching is nice. Like there's definitely use cases where it makes sense, but people want implicit caching. So I'm happy passing the cost saving on to developers. You don't have to do anything. It just works right now and you're saving money.”
Assertion Supported
Mallick: Gemini officially supports 24 languages but responds in Klingon
“We officially support 24 languages, but you can try talking to the model and cling on and it'll respond to you.”
Disclosure
Mallick: Google released experimental proactive audio for Gemini native audio
“One of the features that we've pushed out A little more experimental, but would love for people to test it is what we're calling proactive audio, and it's available only in the native audio, in the audio to audio architecture right now. And what this feature d…”
Assertion Supported
Google simplifies Gemini Flash pricing to flat 10 cents per million tokens
“So it went from seven and a half cents per million tokens to 10 cents. The sort of way that this was offset was we used to distinguish. I don't know if folks are familiar with this, but we used to distinguish based on input token volume. So it was like over a …”
Assertion Supported
Sehgal: Gemini Deep Research retains all browsed websites in context
“We actually keep everything in context, like all the sites that it's read remain in context. So if there's a piece of missing information, it can just fetch that.”
Disclosure
Sehgal: Early Gemini Deep Research testers never edited initial research plans
“Actually like in early rounds of testing, we saw no one was editing, and so we were just like, if we just put a button here, Maybe people will, like, engage more.”
Insight
Taylor: Application Speed Drives Significant Percentage Points of Growth at Scale
“It turns out like it's faster, just being faster is worth a lot of percentage points of growth. At a scale of Google.”
Disclosure
Bryk: Exa entered YC with pitch to beat Google
“We entered YC with We are better than Google.”
Assertion Contradicted
Swyx: Gemini Flash accounts for 50% of OpenRouter requests
“Gemini Flash, according to Open Router, is now 50% of their Open Router requests.”
Disclosure
Martin: Google is exploring developer API access for NotebookLM technology
“I think at the same time, right, there are a lot of developers that are interested in using the same technology to build their own thing. We're going to look into that. How soon that's going to be ready, I can't really comment, but these are the things that li…”
Assertion Supported
Yi Tay: UL2 is a 20B encoder-decoder model using T5 pre-training data
“So I think UL-II is an encoder, decoder, 20 B model. I think when we got it approved, it was, like kind of, you know, it was released as, like, kind of, like, the big brother of T-Five, you know, kind of like, okay, we updated T-Five with, like, a new objectiv…”
Assertion Supported
Frankle: OpenAI, Google, Meta, and Apple have data deals with Shutterstock
“And you know, I, at least I've heard in the news, like opening, I Google, Meta Apple have all called Shutterstock and made those deals.”
Assertion Supported
Frankle: Google TPUs provide much higher bandwidth-to-compute ratios
“TPUs have a very different network bandwidth to compute ratio. They have a lot more bandwidth just objectively and TPUs per chip tend to be a little bit less compute intensive and have a little bit less memory.”
Disclosure
Wang: a16z invested in Character.AI in January 2023 before Google deal
“So we invested in character in January, 23, which feels like eons ago. I mean, three years ago, feels like lifetimes ago, but and then they did the IP licensing deal with Google in August, 20 four.”