OpenAI's Assistants API increased its file limit to 10,000 files
Michelle Pokrass · Building AGI with OpenAI's Structured Outputs API · Sep 17, 2024 · at 51:45
Michelle Pokrass, OpenAI Tech Lead, explains improvements made to OpenAI's Assistants API file search and RAG architecture.
0:00 / 0:27exact quote · 27.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Before, we only supported, I think, like, 20 files per assistant, and the way we used those files was, like, less effective. Basically, the model would decide, based on the file name, whether to search a file, and there's, like, not a ton of information in there. So our new offering, which we shipped a few months ago, I think, now allows 10 K files per assistant, which is, like, dramatically more. And also, it's a kind of different operation, so you can search semantically over all files at once, rather than just kind of the model choosing one up front.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Michelle Pokrass
Pokrass predicts developers will abandon RAG vector stores for direct long-context
“So we do expect a lot of developers to start, you know, uploading their full context more directly to the model. So for smaller tasks, you maybe don't need The whole vector store.”
Pokrass: AI model gains now driven by post-training, not larger pre-trains
“We find that actually a significant amount of the gains come from new post-training techniques. So I think in the past the narrative is that you need to pre-train these larger and larger models to get better performance, and we're finding that we're able to sq…”
Pokrass: Prototype with GPT-4.1, then downscale for latency or upscale for reasoning
“I think the answer is always going to be the fastest model that accomplishes your task, right? So maybe you start prompting 4.1 as a starting point if it does your task super well, Then maybe you could drop down a 4.1 mini and save latency, or even nano. Where…”
Pokras: Vision fine-tuning is the most underrated release for bespoke OCR
“Vision fine-tuning is so underrated. For the past, like, two months, whenever I talk to founders, they tell me this is the thing they need most. A lot of people are doing, like, OCR on, on very bespoke formats, like government documents, and vision fine-tuning…”
Pokrass: Every successful company eventually outgrows Postgres for NoSQL
“At some point, every company gets the scale, every successful company gets the scale where Postgres is not cutting it. And then you migrate to some sort of NoSQL database.”
Function calling benchmarks like BFCL are largely saturated
“I find that a lot of these evals are mostly saturated, like for BFCL. All the models are near, near the top. Already. And kind of the errors are more, I would say like just differences in default behaviors. I think most of the models on the leaderboard can kin…”