Shreya Shankar

4 statements across 3 episodes · 3 bullish · 0 bearish · 2 people on the record · first statement Sep 28, 2024 by Shreya Shankar · across every show →

On the record as a speaker too: Shreya Shankar's record, appearances and statements → this page counts the times other people say the name.

Everything said about Shreya Shankar, oldest first

Sep 28, 2024 bullish
Disclosure
Shankar: New LLM evaluation framework to be open-sourced in ChainForge
“Not yet. It's not out yet, but we will. So the conference is in three weeks. We have to have it out by then, but it'll be implemented in chainforge.ai, which is an open source LLM pipeline building tool.”
Shreya Shankar Sep 28, 2024 ▶ 56:49 [Paper Club] Who Validates the Validators? Aligning LLM-Judges with Humans (w/ Eugene Yan)
Sep 28, 2024
Insight
Shankar: Decompose LLM pipelines into unit tasks with standalone intermediate assertions
“The idea is to have each node in your graph kind of be a standalone, do a standalone thing that you can have standalone assertions for. And if you think about, you know, infinitely many inputs flowing through your pipeline, there's going to be some fraction of…”
Shreya Shankar Sep 28, 2024 ▶ 57:35 [Paper Club] Who Validates the Validators? Aligning LLM-Judges with Humans (w/ Eugene Yan)
Nov 29, 2024 positive
Insight
Shreya Shankar: Chunking efficacy is task-specific, requiring automated pipeline optimization
“Sometimes it's beneficial to chunk and sometimes you should not chunk. And we have observed this in a number of workloads and the insight that we've gained is that we will never know what it's all task specific and data specific. And we are so further convince…”
Shreya Shankar Nov 29, 2024 ▶ 19:31 [Paper Club] DocETL: Agentic Query Rewriting + Eval for Complex Document Processing w Shreya Shankar
Mar 13, 2025 bullish
Prediction Not checkable as stated
Husain: HCI and workflow evaluations will enter AI tools by 2027
“If I were to fast forward one or two years, I would expect to see those and all the tools. It just hasn't arrived yet.”
Hamel Husain Mar 13, 2025 ▶ 26:15 [Lightning Pod] Evals: How to Improve AI Consistently — with Hamel Husain and Shreya Shankar
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.