Assertion Partly supported AI assessment confidence: 85% certainty 3/5 debate potential 1/5

Madheswaran: BERT cut entity extraction training data requirements to ten documents

Jay Madheswaran · How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show · Feb 10, 2026 · at 6:02

Eve Legal founder Jay Madheswaran discusses the technological breakthroughs in natural language processing (NLP) around 2018 that led him to leave venture capital and build an AI startup.

0:00 / 0:11exact quote · 11.9s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“So, right, so you went from needing to train thousands about thousands of documents to 10 documents, maybe, to extract things like dollar amounts from invoice totals simple things like that.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

The introduction of BERT and transfer learning reduced the training data requirement for entity extraction in documents from thousands of samples down to approximately 10 documents.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

The claim is partially true. Prior to the adoption of transfer learning and pretrained transformer encoders like BERT (Devlin et al., 2018), supervised sequence-labeling models (e.g., BiLSTM-CRF) typically required thousands of fully annotated documents or sentences (such as CoNLL-2003's 14,041 sentences across 946 documents) to achieve viable performance. However, attributing a reduction in training data requirements down to '~10 documents' specifically to the introduction of BERT is an overstatement. Empirical benchmarks (e.g., Few-NERD, Ding et al., 2021; Huang et al., 2021) demonstrate that standard fine-tuning of BERT on 5-to-10 shot support sets suffered severe degradation, typically achieving F1-scores below 35%. Viable extraction from approximately 5–10 examples was only realized later through specialized few-shot metric learning frameworks (e.g., StructShot, CONTaiNER), multimodal document transformers (LayoutLM), and primarily through large language model (LLM) in-context learning (e.g., GPT-3/GPT-4).

Pre-BERT supervised entity extraction models required thousands of annotated sentences/documents (e.g., CoNLL-2003 benchmark).
Standard fine-tuning of vanilla BERT under 5-shot to 10-shot regimes yielded low F1 scores (20% to 35% on Few-NERD and domain benchmarks).
Viable few-shot extraction on ~10 document examples required later architectures (e.g., LayoutLM, StructShot) and autoregressive LLM prompting rather than standard BERT fine-tuning.

Checked against

[1] pragnakalp.com [2] researchgate.net [3] arxiv.org [4] github.com [5] researchgate.net [6] nih.gov [7] edpsciences.org [8] arxiv.org

BERT transfer learning data efficiency named entity recognition few shot baseline "BERT" "few-shot" "NER" "10 examples" OR "10 shots" OR "10 documents" F1 score BERT entity extraction document few shot 10 documents samples BERT vs BiLSTM-CRF training data requirements few-shot NER "LayoutLM" few-shot document entity extraction "10 documents" OR "10 pages" OR "10 samples" pre trained language models reduce training data named entity recognition 10 samples BERT few shot named entity recognition training data reduction 10 examples OR 20 examples

assessed as of 2026-09-05

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

BERT entity extraction natural language processing

⚑ Spot an error? Report it

More from Jay Madheswaran

Assertion Supported

Madheswaran: EvenUp relied on offshore labor before layering in AI

“Versus even for the time was kind of done by offshore human labor on two very specific tasks. And they're now layering in AI.”

Jay Madheswaran Feb 10, 2026 ▶ 32:27 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Insight

Madheswaran: Plaintiff Lawyers Share Tool Recommendations Outside Their Local Markets

“They are competitors, but they're competitors locally, right? So if you look at it, you know, in, I don't know, Los Angeles, in one neighborhood, there might be two or three personal injury labor and employment attorneys. Representing that area. They'd be comp…”

Jay Madheswaran Feb 10, 2026 ▶ 43:56 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Insight

Madheswaran: Fundraising requires triggering VC greed then reducing perceived risk

“You know, they want some level of greed to be clicked on. It's like, oh, If they do this, I can see why they would make a lot of money and make me a lot of money. And you have to then, once you do that, reduce risk, right? Basically show a through line between…”

Jay Madheswaran Feb 10, 2026 ▶ 10:02 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Insight

Madheswaran: Generative AI cannot reach 99.9% document extraction accuracy alone

“That's where it gets harder. You know, when you're chasing from 99% accuracy to 99.9. Gen AI still doesn't fully help with that. So you still have to kind of do more typical machine learning”

Jay Madheswaran Feb 10, 2026 ▶ 12:38 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Disclosure

Madheswaran: Eve AI directly talks to, qualifies, and signs legal clients

“And now Eve actually literally talks to the client, which is kind of cool, you know, helps get them signed and qualifies them without violating any laws. Like you can't accidentally do unauthorized practice of law.”

Jay Madheswaran Feb 10, 2026 ▶ 15:45 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Insight

Madheswaran: LLMs made automating hundreds of bespoke workflows economically feasible

“And you have to take that and convert that into those hundred bespoke workflows. Which was economically unfeasible before LLMs, but with LLMs, we saw that we were actually able to go deep into the workflows, and as a result, provide more value for the firms, a…”

Jay Madheswaran Feb 10, 2026 ▶ 19:54 How I grew to $3M ARR in 6 months—& to a $1B valuation in 2 years. | Jay, Founder of Eve Legal · PMF Show

Madheswaran: BERT cut entity extraction training data requirements to ten documents

More from Jay Madheswaran

Turn any episode into a week of clips.