Kambadur: Synthetic data generation is a highly popular use case for LLMs
Melanie Kambadur · A Conversation on The State of AI | Melanie Kambadur, Meta & Gideon Mann, Bloomberg · Feb 2, 2023 · at 19:55
Melanie Kambadur, Research Engineering Manager at Meta's FAIR Labs, explains how large language models generate synthetic long-tail data to train specialized models like hate speech classifiers.
0:00 / 0:08exact quote · 8.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“A very popular use of these LLMs is to generate synthetic data to train other models, because they're really good at generating sort of long tail data.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Melanie Kambadur
Kambadur: AI voice models replicate human speech from three seconds of audio
“So now we have voice models that can listen to Three seconds, I think, that we had recently. This new volley of your voice, and then replicate your voice. We have models that can get, like, a very short sentence and generate, like, minutes-long video that desc…”
Kambadur: Generative 3D AI tools will vastly accelerate building the metaverse
“That's kind of like how the metaverse is gonna happen. We're gonna be able to like walk around and generated three D worlds in VR, and they're all gonna be built Much more quickly than maybe we expected a few years ago.”
Kambadur: Triggering unsafe OpenAI outputs requires clever adversarial prompt engineering
“I've seen really huge advances from open AI's models in their responsible AI and their safety. Like now it takes a lot of clever prompt engineering is what we call, you know, using different ways of talking to the model to get it to reveal interesting stuff. I…”
Kambadur: AI inference costs tens of cents for text, dollars for video
“Every time the user wants to call the model, it might cost, you know, tens of cents for a text model or dollars for a video model.”
Kambadur: Stack Overflow banned ChatGPT for generating plausible but incorrect answers
“ChatGPT got banned from stack overflow because it sounded convincing, but wasn't quite right all the time.”
Kambadur: OpenAI sought top 10th percentile doctors and lawyers as data annotators
“Their model itself has gotten so good that they now need, like, extreme experts to be able to train the model, so we saw an ad for their annotators, which are the people who, like, help give feedback to the model, and they asked for, like, top 10 percentile ch…”