inference cost
also referred to as: inference costs
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Jan 23, 2025 by Tomasz Tunguz · across every show →
Everything said about inference cost, oldest first
Jan 23, 2025 neutral
Tunguz: 100x cost difference between small and largest AI models
“I was just looking at the analysis between the smallest models, which are about four or two to four billion parameters, and the very largest models, which about four or fifty billion parameters, you have a hundred X difference in inference costs.”