Hotz: Tinygrad could replicate PyTorch's API in two engineer-months
George Hotz · Ep 18: Petaflops to the People — with George Hotz of tinycorp · Jun 20, 2023 · at 31:32
George Hotz discusses Tinygrad's API compatibility compared to PyTorch and Mojo.
0:00 / 0:04exact quote · 4.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Replicating the PyTorch API. Is something I can do with a couple, you know, like an engineer month or two.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from George Hotz
Hotz: GPT-4 is an 8-way mixture model with 220B parameters per head
“GPT-IV is two hundred twenty billion in each head, and then it's an eight-way mixture model.”
Hotz: Meta attracts researchers who want to publish while OpenAI keeps ideologues
“OpenAI can keep ideologues who, you know, believe ideological stuff, and Facebook can keep every researcher who's like, dude, I just want to build AI and publish it.”
Hotz: Analog computing for AI won't work and clockless chips aren't practical
“Analog computing just won't work. And clockless computing sure, it might work in theory, but your ETA tools are, maybe AIs will be able to design clockless chips, but not humans.”
Hotz: Systolic arrays are the wrong architectural choice for AI chips
“I think systolic arrays are the wrong choice. Systolic array, I think they have systolic arrays because that was the guy's PhD. And of course Amazon makes... They are very power efficient, but it becomes hard to schedule a lot of stuff. On them, if you're not …”