Prefill Tokens
topic on 1 show · 1 statements across 1 episodes
1 statements about Prefill Tokens, every show
Patel: AI coding startup costs are driven by pre-fill context, not decode tokens
“If you look at coding as an application and you like look at these coding companies and how much they're paying for prefill versus decode, actually majority of their cost is prefill tokens, not decode tokens, because the context is just so large and it's switc…”