Together AI

“Yeah, yeah, so I just joined this week actually, and it's been really exciting.”

“We are 38 people on the team and we are hiding across all the areas you know.”

“I think there were some errors in that analysis. In particular we were trying to decode it, and one of the things we noticed is that it assumed that input tokens weren't being priced. So I think that may have been an error in the model.”

“I would say right now the top five models on our inference stack are probably all fine-tuned versions of open models.”

“Everyone sort of had a reaction to it because it just didn't match their benchmarks that we've all run internally against different services.”

“I think on the inference stack, there are open source inference stacks which are pretty good, and it gives us, you know, definitely today it gives us a competitive advantage to have the best one, and So we're not sort of rushing out to release everything about…”

“A good benchmark should think about how it's going to incentivize the field to actually move forward, right? So the benchmark will become kind of standard. How are people going to over optimize to the benchmark because people are going to do that? And when peo…”

“Combining all those techniques all together, right? So we'll give you essentially another boost, right? So that kind of one thing that we learn on the technical side.”

“We have close to seven to 8000 GPUs today. It's growing monthly.”

“There are applications that are, you know, consuming the tokens that are produced from unmodel, so they're not necessarily being read or heard by humans. So that's a place where we see that level of requirement today that really nobody can quite satisfy.”

“So, so I think one fundamental thing that changed in the last year, essentially, in the beginning when people think about data, is, is always like a byproduct of the model, right? You release the model, you also release the data, right? The data side is there …”

“The way we structure our cloud is by combining data centers around the world instead of you know we are today not located in hyperscalers. We have built a footprint of you know, AI supercomputers in this sort of a disaggregated, decentralized manner.”

“We are running into you know, the limits of how fast you can make transformers and you know, we want inference at 5000 tokens per second. And I don't think we will get there with transformers”

“Once you can get sort of an immediate answer from a model it starts working in a different way and you know, new types of applications will be created.”

“I think you need to have some kind of marketplace for figuring out how to get this you know, data into models and have, I think you'll increasingly see more of that”

“It's like 40, 45% I was counting this morning.”

“There aren't really big data moats around foundation models. They are built from a subset of the web. What is difficult is the cost of capital to build these, and what, one of the ways in which you can reduce this cost is by making more efficient systems.”

“If you only push on one direction, you are going to reach diminution return really, really quickly. Yeah, there's only that much you can do on the system side, only that much you can do on the algorithm side. And since the only big thing that's going to happen…”

“So that's why in REST-PYRON V-II, we kind of overlay the data set, it's like, 40 different pre-computed quality signal, right? If you want to reproduce your best effort, like, C-Four filter, it's kind of like, 20 lines of code.”

“I don't think we are running out of data on earth. Right, so think about it globally... But I do think there are many organizations in the world have enough data to actually train, like, very, very good models, right? So, I mean, they are not public available,…”

“I think, like, Together and Fireworks and all these people are trying to build some faster CUDA kernels and faster, like, you know, hardware kernels in general. But those modes only last for a month or two. Like, these ideas quickly propagate.”

“They are nowhere near as reliable as, I mean, every single time I use any of those products and run a benchmark, I find a bug, text the CEO, and they fix something. It's nowhere near where OpenAI is.”

“I think most people that I talk to are not seriously pursuing alternative architectures. There are some notable exceptions, primarily together with the Mombard architecture recursal with RWKV. There's like the XLSTM that was created by a separate Hogwriter. An…”

“Mosaic was the first one to really recognize that there was a huge opportunity in making this easy. And now this has largely been commoditized by things like SageMaker and Together and lots of different folks that help you on the training side. But on the data…”