Q And what was the origin of it? How did you come up with the idea? Uh, how did you get people to buy in? And then maybe what were one or two of the pivotal moments early on that kind of made it the standard for, for these things?
A Yeah. Yeah. Chatbot Arena project was started last year in April, May, around that. Before that, we were basically experimenting in the lab how to fine-tune a chatbot open source based on the Lama-one model that had released. At that time, Lama-one was like a base model, and people didn't really know how to fine-tune it, so we were doing some explorations. We were inspired by Stanford's Alpaca project. So, we basically, yeah, grow a data set from the internet, which is called shared gpt data set, which is, like, a dialog data set between user and chat gpt conversation. And it turns out to be, like, pretty high-quality data, dialog data. So, we fine-tune on it, and then we try to, and release the model called wikunia. And people were very excited about it because it kind of like demonstrate open way model can reach this conversation capability similar to ChatGPT. And then we basically released the model ways and also do the demo website. The model. That, people were very excited about it, but during the development, the biggest challenge to us at the time was, like, how do we even evaluate it? How do we even argue this model we trend is better than others? And, like, what's the gap between this open source model and other proprietary offering? At that time, it was, like, GPT-FOR was just announced. It's, like, cloud one, right? What's the difference between them? And then after …
AI assessment note: “we quickly realized that people need a tool to compare between different models.”