May: AT&T cut AI token costs 90% using small language models
Rob May · Why the 7th Biggest Player in a Huge Market Beats the Winner in a Small One · Apr 10, 2026 · at 14:25
Rob May discusses LLM inference cost optimization strategies with host Sean Weisbrot.
0:00 / 0:18exact quote · 18.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We just saw an article in late February from AT&T or AT&T was doing eight billion tokens per month through their core AI systems. And it was costing them too much money, and they rewrote their entire architecture to take advantage of small task-specific language models, and were able to cut those costs by 90%.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Rob May
May: OpenAI and Anthropic will be pushed to raise inference prices
“I think it's going to get worse before it gets better, because my guess is as open AI and anthropic IPO, wall street's going to scrutinize their unit economics. And it's probably going to ask them to raise prices, push them to raise prices on inference.”
May: Many senior executives are imitators rather than adaptable problem solvers
“You know, a lot of people that have climbed really high in their career are just great imitators. They just take a playbook that they saw somewhere. They don't always fully understand it. They don't always know how to change it for changing circumstances, but …”
May: Best GTM strategy for IT buyers is open sourcing core value
“One of the things I've learned having built companies that sold to like sales and marketing people and companies that sold to IT people is like your comment on, well, you can't give things away for free for people who don't value it. Very true in the marketing…”
May: Massive markets forgive mistakes, unlike winner-take-all smaller niches
“The reason you wanna do that is because when you're in a big market, you know, you can make a lot of mistakes and still Come out on head. You could be the seventh biggest player in the industry and still do really well, you know whereas in, in, in, you know, i…”
Rob May: Good executives should immediately know who they would hire next
“I think as an entrepreneur, and also I expect this of the executives that work for me, that they should constantly be cultivating talent. You know, a couple of times a month, like, you know, going to events or taking people out to dinner or having coffees with…”