throughput

“There's a fundamental trade off on the GPU between being throughput oriented or latency optimized. And everyone has chosen latency optimization because the shape of usage was chatbot oriented.”