Jun 11, 2024 · 1h 5m · latent-space
How AI is Eating Finance - with Mike Conover of Brightwave
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the Latent Space podcast, Brightwave founder Mike Conover discusses the technical and strategic realities of building vertical AI for financial services. He explains how modular system architectures, expert-annotated data moats, and structured reasoning pipelines transform generative models from generic summarizers into actionable intelligence engines for asset managers.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 4.9 Guest teaching 4.7 Guest disagreement 1.1 The hosts pushing back 1.1
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 1:03:30 Mike corrects Alessio on Dolly's benchmark status
When Alessio reminisces about Dolly being the best open-source model, Mike immediately shuts down the claim, stating Dolly was never state of the art and was merely a demonstration of instruction following.
Hardest push from the hosts ▶ 18:34 Alessio pushes back against needle-in-a-haystack metricsAlessio actively challenges standard evaluation trends, arguing needle-in-a-haystack testing forces models to over-attend to irrelevant legal and regulatory boilerplate.
Biggest teaching moment ▶ 13:34 Mike deconstructs large context window synthesis mythsMike methodically educates listeners on token generation probability and output length constraints, explaining why a million-token context window fails at nuanced synthesis compared to specialized subsystems.
The host holds their own ▶ 52:07 Alessio maps out statistical mechanics of VC vs HFTAlessio demonstrates deep venture and quantitative expertise by contrasting normal distributions in momentum trading with non-linear power-law distributions in early-stage investment.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Mike Conover's Background in Large-Scale Systems and Finance | 4 | 5 | 1 | 1 | Alessio sets up the interview by citing Mike's Nature paper and background at Workday, prompting Mike to explain how digital trace data and complex systems relate to financial markets. Mike provides deep historical context from his DARPA and LinkedIn days. | |
| Founding Brightwave and Co-Founder Background | 3 | 4 | 0 | 0 | Alessio asks about Brightwave's founding story and co-founder Brandon Katara. Mike explains Brandon's background spanning regulated exchange architecture and early deep learning search systems. | |
| Team Composition for Vertical AI in Finance | 4 | 4 | 1 | 1 | Alessio queries how to balance AI talent with domain expertise in vertical AI startups. Mike articulates that AI companies urgently need systems engineering alongside deep non-consensus financial domain experts. | |
| Brightwave Product Overview and Customer Personas | 4 | 5 | 0 | 0 | Mike details Brightwave's product capabilities, showing how it unpacks complex second-order supply chain impacts like gallium and germanium export controls for crossover hedge funds and wealth managers. | |
| Evolution of Context Sizes and Systems-of-Systems Approach | 5 | 6 | 2 | 1 | Alessio asks about the transition from small context sizes like Dolly to modern million-token windows. Mike disabuses the notion that huge context windows solve deep document synthesis, arguing instead for a decomposed systems-of-systems approach. | |
| Generating Actionable Insights Beyond Summarization | 6 | 4 | 1 | 2 | Alessio brings up his own critique that needle-in-a-haystack benchmarks misdirect focus toward boilerplate retention. Mike agrees and discusses chunking and selective document semantic parsing in financial filings. | |
| Ensuring Factuality and Product Affordances for LLMs | 4 | 5 | 1 | 1 | Alessio queries how Brightwave handles factuality in LLMs. Mike discusses multi-pass verification, internal entailment evaluation models, and design-led product affordances that let users trigger discretionary compute to double check findings. | |
| User Feedback, Personalization, and Revealed Preferences | 5 | 4 | 1 | 1 | Alessio asks about capturing feedback when subjective investment insights take months or years to validate. Mike draws parallels to recommendation systems, explaining how Brightwave tracks revealed preferences through user exploration paths. | |
| LLM Evaluation Strategies and Human Annotation Flywheels | 6 | 5 | 1 | 2 | Alessio references previous guest David Luan on evals and asks about building internal data flywheels. Mike shares practical insights on LLM-as-a-judge supervision, calibrated against small high-quality human annotation rubrics. | |
| Managing Temporality and Quantitative Data in Financial RAG | 5 | 5 | 1 | 1 | Alessio asks how Brightwave manages temporal shifts and numerical precision in financial RAG. Mike explains how semantic query intent routing separates breaking news retrieval from long-term thematic analysis. | |
| Confidential Data Security and Context-Aware Prompting | 5 | 4 | 1 | 1 | Alessio asks about customer confidentiality and handling proprietary alpha data alongside public sources. Mike details context-aware prompting and metadata propagation across multi-step inference chains. | |
| Knowledge Graph Extraction for Granular Financial Reasoning | 4 | 5 | 0 | 0 | Alessio asks about knowledge graph extraction versus standard vector search. Mike explains single-pass structured extraction from massive unstructured corpora to build rich economic entity-relation graphs. | |
| Fine-Tuning as Behavioral Differentiation and Classical ML | 5 | 5 | 2 | 1 | Alessio notes Mike's early stance against building proprietary foundation models. Mike details his philosophy that fine-tuning acts like stem-cell behavioral differentiation in a finite state machine rather than a way to inject knowledge, pushing back on anthropomorphic agent frameworks. | |
| The Role of Financial Modeling and the Legacy of Spreadsheets | 5 | 5 | 1 | 2 | Alessio asks why Brightwave avoids direct Excel spreadsheet generation. Mike explains that spreadsheets lack fault tolerance and that financial modeling is a deeply personal thinking process, drawing a historical parallel to VisiCalc. | |
| The Future of AI in Thematic Investing and Venture Capital | 8 | 4 | 2 | 3 | Alessio gives a detailed monologue contrasting the normal distributions of quantitative HFT with the extreme power-law dynamics of venture capital. Mike validates the point and demonstrates how Brightwave surfaces second- and third-order derivative theses. | |
| The Evolution and Future of Open-Source Foundation Models | 5 | 5 | 3 | 1 | Alessio asks about the future of open-source models and benchmark gaming. Mike predicts diminishing returns on pre-training foundation models, followed by a direct, crisp correction of Alessio's nostalgic comment regarding Dolly's performance. |
Statements from this episode (31)
Conover: Labor flow networks predict next-quarter S&P 500 market cap changes
“We demonstrated that five hundred million jobs transitions can be hierarchically clustered as a network of labor flows and in our predictive next quarter S and P 500 market gap changes.”
Brightwave raises $6M Seed round led by Decibel Partners
“We, we've raised a six million dollar seed round including participate lead by decibel we love working with and including Participation from .7 to two, one of the largest hedge funds in the world and moon fire ventures.”
Conover: Identifying mispriced assets is ill-suited to human intellect
“If you think of the job of an active asset manager, the work to be done is to understand something about the market that nobody else has seen in order to identify a mispriced asset. And it's our view that that is not a task that is well suited to human intelle…”
Conover: AI models outclass human working memory, not core reasoning
“It's not clear that you get superhuman reasoning capabilities from human level demonstrations of skill. And by that, I mean the pre-training corpus, but then additionally, the fine tuning corpuses, I think you largely mimic the demonstrations that are present …”
Conover: AI companies hire systems engineers, traditional software hires AI talent
“All of the traditional software companies are trying to hire AI talent and all the AI companies are trying to hire systems engineers, and that is 100% the case.”
Conover: Financial AI requires deep domain expertise for non-consensus insights
“Grammarly is a good example of a company that has Generative work product that is valuable by most humans. Whereas in finance, the character of the insight, the depth of insight and the non-consensusness of the insight really requires fairly deep domain expert…”
Conover: A $20B crossover hedge fund uses Brightwave for equity research
“We have a twenty billion dollar crossover hedge fund and their equities team uses this tool to go deep on a thesis.”
Conover: Commercial LLMs struggle to generate 5,000 output tokens in one generation
“There is a characteristic output length for these models. Let's say it's about 1200 tokens. Like it is very difficult to get any of the commercial LMs or LLAMA to write 5000 tokens.”
Conover: Million-token context windows fail to extract deep insights from SEC filings
“It, I think empirically is not the case that you can just throw all of the SEC filings in, you know, a million token context window and get deep insight that is useful out the other end.”
Conover: One-year patient adherence rate for Ozempic is only 35%
“Or for example, that adherence rates to a Zenpec after a year, just 35%.”
Brightwave trains custom models to verify outputs against primary source material
“We train our own models to assess you know, you can think of this like entailment, like is, is this supported by the underlying primary sources?”
Conover: AI personalization must rely on revealed rather than stated preferences
“Getting a person to articulate everything that they believe is not a realistic task. Netflix doesn't ask you to describe what kinds of movies you like and they give you the option to vote, but nobody does this. And so what I think you do is you observe people'…”
Brightwave uses LLM supervision alongside human annotation benchmarks for model evaluation
“We pay human annotators to evaluate the quality of the generative outputs, and I think that that is always the reference standard, but we frequently first turn to LLM supervision as a way to
Have whether it's at fine tuning time or even for subsystems that ar…”
Conover: Repeatable AI annotation pipelines require unsexy people management
“It's one thing to do like a single monolithic push to create a,
Training data set like that, or an evaluation corpus, but I think it's another to have a repeatable process, and a lot of that, I think, realistically is pretty unsexy, like, people management wor…”
Conover: Nearest-neighbor search without metadata yields convincingly wrong AI answers
“If I just look for something that is a nearest neighbor without any of that temporal or other qualitative metadata overlay, you're just going to get a kind of a bag of facts. And that, that is like explicitly not helpful. Because the worst
Failure state for th…”
Conover: The most alpha-generating information in finance is often private
“Frequently the most interesting and alpha generating material is not publicly available”
Brightwave uses composable, context-aware prompting conditioned on retrieved context semantics
“You can have prompts that are composable and that have different sort of command units that like may or may not be present based on the semantics of the content that is being populated into the rag context window. And so that, that's something we make great us…”
Conover: Programming LLMs as zero-marginal-cost machine learning systems is underappreciated
“I think that it is underappreciated how powerful, there's the generative capabilities of language models, but there's also the ability to program them to function as arbitrary machine learning systems, basically for marginally zero cost.”
Conover: LLMs enable knowledge graphs more granular than Bloomberg or LinkedIn
“We believe that there's an opportunity to create a knowledge graph that has resolution that greatly exceeds what any You know, whether it's Bloomberg or LinkedIn currently has access to where we're getting as granular as person X submitted congressional testim…”
Conover: Unbounded AI agents are useless compared to finite state machines
“Specifically, like, I don't think that unbounded agentic behaviors are useful and that instead a useful LLM system is more like a finite state machine where the behavior of the system is occupying one of many different behavioral regimes and making decisions a…”
Conover: LLMs do not need anthropomorphic personas for high-quality reasoning
“Our experience has been that You can get really, really high quality reasoning from roughly an agentic system without needing to be too cute about it. You can describe the task and you know, within well-defined bounds you don't need to treat the LLM like a per…”
Conover: Classical ML provides statistical output guarantees that LLMs cannot offer
“Traditional machine learning has a real material role to play in producing a system that hangs together, and there are, you know, guaranteeable Like statistical promises that classical machine learning systems to include traditional deep learning can make abou…”
Conover: Building automated Excel spreadsheets is an explicit non-goal for Brightwave
“I think what is an explicit non-goal for the company is to create Excel spreadsheets.”
Conover: Push-button automated financial modeling misses the core purpose of modeling
“The other piece of this is that the financial modeling is often very, when we talk to our users, it's very personal. So they have a specific view of how a company is structured. They have the, you know, one key driver of asset performance that they think is re…”
Conover: Systematic hedge fund trading desks operate like large ML teams
“The more that I have learned about How teams at hedge funds actually behave, and you look at, like, systematics desks, or semi-systematic trading groups, man, it's a lot like a big machine learning team.”
Conover: Investment thesis idea generation is absolutely automatable with AI
“I think that process of idea generation is absolutely automatable.”
Conover: AI models can parse documents to identify second-order derivative bets
“It is very straightforward to take a model and say, parse through all of these documents and find second order derivative bets and say, oh, it turns out that energy is like very, very adjacent to investments in AI and may not be priced in the same way that GPU…”
Conover: AI model developers are absolutely overfitting to public evaluation benchmarks
“And I think the work around over, you know, overfitting on the test, I think is like that. 100% is happening.”
Conover: Economic incentives to pre-train commodity foundation models from scratch are diminishing
“The incentives, the economic incentives for companies to train their own foundation models, I think, are diminishing. So the, like, window in which you are the dominant pre-train, and let's say that you spend five to forty million dollars, you know, for like a…”