Insight
Tamir: Direct OpenAI or Anthropic APIs outperform cloud intermediary platforms
“Using a first party solution with say OpenAI or with Anthropic is going to have different performance, different latency than if you do it via an intermediary, like a cloud. So you maybe get some simplicity by not having to onboard different vendors and differ…”
Insight
Tamir: On-premise infrastructure is cheaper for AI training if already established
“Maybe you buy your own infrastructure, which is also a cheaper answer. If you have, if you already have an infrastructure, an on-prem infrastructure system, that's really going to depend on what your AI training expectations are anticipated.”
Assertion Not checkable as stated
Tamir: Serving 70B open-source LLMs forces teams to buy reserved GPUs
“What if you want to do something like a seventy billion Lama or minstrel or mixed or any of these, those are going to be very hard to get on an ad hoc basis. And you're going to end up having to pay the price for a reserve instance, if you want to be able to s…”
Insight
Tamir: E-commerce platforms should use dynamic discounts, never dynamic pricing
“You never want to do dynamic pricing. You want to do dynamic discounts as a product framing bit of advice”
Insight
Tamir: Search must balance direct and complementary relevance for optimal revenue
“And maybe what you would want to do is have one metric that shows for direct relevance and another that shows complementary relevance, right? And then it is somewhat a product decision of how much you want to balance these, but it's also something that you can…”
Insight
Tamir: Successful ML teams require serving engineers and embedded product managers
“You need to have engineering for serving. You need to have your ML scientists and engineers, and you need to have product. You need to have product every year in the trenches with your ML builders.”
Insight
Tamir: Per-token commercial LLM costs are unscalable for every enterprise query
“And if you pay a commodity LLM, which right now does usually beat out open source LLMs, you're going to end up paying per token and you can't scalably, if you have the right size of a business, do this for every query or every kind of use case.”
Insight
Tamir: Search retrieval models learn most from near-miss negative training examples
“If I search for boots and I show a Christmas tree, that's not gonna, the model's not gonna learn much from saying, oh, that was wrong, right? It's gonna learn a lot more by showing its sneakers or by showing its sandals and saying that's because it's closer to…”
Insight
Tamir: Gap between ML loss metrics and product goals determines AI success
“There, that gap between what I want to mathematically measure, which is how the machine is going to learn based on that error and what my actual product result is. Really matters.”
Insight
Tamir: Rigorous evaluation culture is critical as non-experts deploy pre-trained AI models
“Having that strong culture to back up the performance of your model before you go into release is probably one of the biggest issues in terms of having a strong ML culture when you're, as it gets easier and easier for non-experts to start leveraging these tool…”
