StarZero All shows Try the podcast clipper →

Apr 9, 2018 · 21m · mad

Data Pipelines at Braze // Jon Hyman, Braze (FirstMark's Data Driven)

Jon Hyman · 17m spoken Matt Turck · 26s spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

Jon Hyman, Co-founder and CTO of Braze, presents a deep dive at Data Driven NYC into the high-throughput data pipelines and real-time streaming architecture powering personal customer messaging at scale. He discusses external data streams like Braze Currents, internal decision-logging telemetry with Kafka and Elasticsearch, and how real-time interaction feedback loops create competitive advantages for enterprise brands.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 2.4% of the talking time here. How this is scored →

Matt as informed peer 0.8 Guest teaching 1.2 Guest disagreement 0.3 Matt pushing back 0.5
05100:0010:0020:000:51–4:22 · Matt as informed peer 0/10 Mobile Paradigm Shift and Messaging Personalization at Scale Jon presents a solo monologue on Braze's business model and the industry shift from usage-based to interaction-based personalization. The host does not participate in this segment.4:22–8:26 · Matt as informed peer 0/10 Braze Currents: Streaming Architecture with Kafka Jon delivers a technical overview of Braze Currents built on Kafka Streams and Kafka Connect. The host is inactive during this presentation segment.8:26–10:28 · Matt as informed peer 0/10 Case Study: Postmates Supply and Demand Balancing Jon explains a Postmates case study on balancing courier supply and demand using real-time engagement data. The segment contains no host dialogue.10:28–13:52 · Matt as informed peer 0/10 Evaluating Decision Rules and Adopting Elasticsearch Jon details logging product decision rules into Elasticsearch to evaluate campaign delivery and uninstall rates. No host interaction occurs during this segment.13:52–17:16 · Matt as informed peer 4/10 Future Technical Roadmap and Recruitment Pitch Host Matt Turck joins to question why complex options like Kafka and Elastic were chosen over simpler tools, and asks for clarification when Jon mentions moving off managed Amazon Elasticsearch. Jon clarifies that they are moving to self-hosted instances on AWS rather than leaving AWS entirely.17:16–21:23 · Matt as informed peer 1/10 Audience Q&A: Real-Time Data Shifts and Competitive Moats Audience members ask about market drivers for streaming data and the backstory behind the Appboy rebranding. Jon educates the audience on competitive recommendation moats and metallurgy terminology.0:51–4:22 · Guest teaching 0/10 Mobile Paradigm Shift and Messaging Personalization at Scale Jon presents a solo monologue on Braze's business model and the industry shift from usage-based to interaction-based personalization. The host does not participate in this segment.4:22–8:26 · Guest teaching 0/10 Braze Currents: Streaming Architecture with Kafka Jon delivers a technical overview of Braze Currents built on Kafka Streams and Kafka Connect. The host is inactive during this presentation segment.8:26–10:28 · Guest teaching 0/10 Case Study: Postmates Supply and Demand Balancing Jon explains a Postmates case study on balancing courier supply and demand using real-time engagement data. The segment contains no host dialogue.10:28–13:52 · Guest teaching 0/10 Evaluating Decision Rules and Adopting Elasticsearch Jon details logging product decision rules into Elasticsearch to evaluate campaign delivery and uninstall rates. No host interaction occurs during this segment.13:52–17:16 · Guest teaching 4/10 Future Technical Roadmap and Recruitment Pitch Host Matt Turck joins to question why complex options like Kafka and Elastic were chosen over simpler tools, and asks for clarification when Jon mentions moving off managed Amazon Elasticsearch. Jon clarifies that they are moving to self-hosted instances on AWS rather than leaving AWS entirely.17:16–21:23 · Guest teaching 3/10 Audience Q&A: Real-Time Data Shifts and Competitive Moats Audience members ask about market drivers for streaming data and the backstory behind the Appboy rebranding. Jon educates the audience on competitive recommendation moats and metallurgy terminology.0:51–4:22 · Guest disagreement 0/10 Mobile Paradigm Shift and Messaging Personalization at Scale Jon presents a solo monologue on Braze's business model and the industry shift from usage-based to interaction-based personalization. The host does not participate in this segment.4:22–8:26 · Guest disagreement 0/10 Braze Currents: Streaming Architecture with Kafka Jon delivers a technical overview of Braze Currents built on Kafka Streams and Kafka Connect. The host is inactive during this presentation segment.8:26–10:28 · Guest disagreement 0/10 Case Study: Postmates Supply and Demand Balancing Jon explains a Postmates case study on balancing courier supply and demand using real-time engagement data. The segment contains no host dialogue.10:28–13:52 · Guest disagreement 0/10 Evaluating Decision Rules and Adopting Elasticsearch Jon details logging product decision rules into Elasticsearch to evaluate campaign delivery and uninstall rates. No host interaction occurs during this segment.13:52–17:16 · Guest disagreement 1/10 Future Technical Roadmap and Recruitment Pitch Host Matt Turck joins to question why complex options like Kafka and Elastic were chosen over simpler tools, and asks for clarification when Jon mentions moving off managed Amazon Elasticsearch. Jon clarifies that they are moving to self-hosted instances on AWS rather than leaving AWS entirely.17:16–21:23 · Guest disagreement 1/10 Audience Q&A: Real-Time Data Shifts and Competitive Moats Audience members ask about market drivers for streaming data and the backstory behind the Appboy rebranding. Jon educates the audience on competitive recommendation moats and metallurgy terminology.0:51–4:22 · Matt pushing back 0/10 Mobile Paradigm Shift and Messaging Personalization at Scale Jon presents a solo monologue on Braze's business model and the industry shift from usage-based to interaction-based personalization. The host does not participate in this segment.4:22–8:26 · Matt pushing back 0/10 Braze Currents: Streaming Architecture with Kafka Jon delivers a technical overview of Braze Currents built on Kafka Streams and Kafka Connect. The host is inactive during this presentation segment.8:26–10:28 · Matt pushing back 0/10 Case Study: Postmates Supply and Demand Balancing Jon explains a Postmates case study on balancing courier supply and demand using real-time engagement data. The segment contains no host dialogue.10:28–13:52 · Matt pushing back 0/10 Evaluating Decision Rules and Adopting Elasticsearch Jon details logging product decision rules into Elasticsearch to evaluate campaign delivery and uninstall rates. No host interaction occurs during this segment.13:52–17:16 · Matt pushing back 3/10 Future Technical Roadmap and Recruitment Pitch Host Matt Turck joins to question why complex options like Kafka and Elastic were chosen over simpler tools, and asks for clarification when Jon mentions moving off managed Amazon Elasticsearch. Jon clarifies that they are moving to self-hosted instances on AWS rather than leaving AWS entirely.17:16–21:23 · Matt pushing back 0/10 Audience Q&A: Real-Time Data Shifts and Competitive Moats Audience members ask about market drivers for streaming data and the backstory behind the Appboy rebranding. Jon educates the audience on competitive recommendation moats and metallurgy terminology.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 0% · guest 100%0:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 17.9% · guest 82.1%15:00 · Matt 17.9% · guest 82.1%18:00 · Matt 0% · guest 100%18:00 · Matt 0% · guest 100%21:00 · Matt 6.3% · guest 93.7%21:00 · Matt 6.3% · guest 93.7%
Sharpest disagreement ▶ 16:54 Jon reframes infrastructure hosting misunderstanding

Jon gently reframes the host's impression that Braze is leaving AWS, clarifying the technical distinction between managed Amazon Elasticsearch and self-managed EC2 instances.

 Hardest push from Matt ▶ 16:48 Matt presses on Elasticsearch hosting plans

Matt picks up on a perceived contradiction regarding Amazon hosting and directly asks where else they would run Elasticsearch if not on Amazon.

 Biggest teaching moment ▶ 16:54 Clarifying managed vs self-hosted cloud infrastructure

Jon educates the host on cloud deployment nuances, explaining how switching from AWS managed Elasticsearch to self-hosted clusters on AWS provides needed configuration flexibility.

 Matt holds his own ▶ 15:07 Matt challenges tech stack complexity

Matt demonstrates architectural insight by questioning why Braze opted for heavy technologies like Kafka and Elastic instead of simpler alternative solutions.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Mobile Paradigm Shift and Messaging Personalization at Scale 0000 Jon presents a solo monologue on Braze's business model and the industry shift from usage-based to interaction-based personalization. The host does not participate in this segment.
Braze Currents: Streaming Architecture with Kafka 0000 Jon delivers a technical overview of Braze Currents built on Kafka Streams and Kafka Connect. The host is inactive during this presentation segment.
Case Study: Postmates Supply and Demand Balancing 0000 Jon explains a Postmates case study on balancing courier supply and demand using real-time engagement data. The segment contains no host dialogue.
Evaluating Decision Rules and Adopting Elasticsearch 0000 Jon details logging product decision rules into Elasticsearch to evaluate campaign delivery and uninstall rates. No host interaction occurs during this segment.
Future Technical Roadmap and Recruitment Pitch 4413 Host Matt Turck joins to question why complex options like Kafka and Elastic were chosen over simpler tools, and asks for clarification when Jon mentions moving off managed Amazon Elasticsearch. Jon clarifies that they are moving to self-hosted instances on AWS rather than leaving AWS entirely.
Audience Q&A: Real-Time Data Shifts and Competitive Moats 1310 Audience members ask about market drivers for streaming data and the backstory behind the Appboy rebranding. Jon educates the audience on competitive recommendation moats and metallurgy terminology.

Statements from this episode (8)

Assertion Not checkable as stated
Hyman: Braze helps client brands reach over 1.25 billion monthly active users
“We're helping them engage more than one and a quarter billion monthly active users.”
Jon Hyman Apr 9, 2018 ▶ 2:07
Assertion Not checkable as stated
Braze processes 500 billion monthly data points to deliver personalized brand messages
“And the way that we do this is by processing nearly half a trillion pieces of data every single month, and then our customers, these marketing and growth and engagement teams of these brands, can then use that to personalize and send out tens of billions of me…”
Jon Hyman Apr 9, 2018 ▶ 2:12
Assertion Not checkable as stated
Collecting 99th percentile mobile app engagement data takes up to four weeks
“We've seen for some apps and this varies by app and by country and everything like that, it can take up to four weeks for us to get that 99th percentile of data collected.”
Jon Hyman Apr 9, 2018 ▶ 5:22
Assertion Not checkable as stated
Jon Hyman says Braze processes about 100 megabytes per second through Kafka
“So the scale that we're operating at, we're pushing about a hundred megabytes per second through Kafka.”
Jon Hyman Apr 9, 2018 ▶ 6:51
Insight
On-demand marketplaces must control promotion visibility to balance supply and demand
“These on-demand marketplaces, similarly like Lyft, it's challenging where you need to control the amount of people who can even see your promotional materials let alone who are acting on it.”
Jon Hyman Apr 9, 2018 ▶ 9:17
Assertion Not checkable as stated
Postmates uses Braze engagement data to adjust courier supply
“So Postmates can use this engagement data to then adjust the supply on their side.”
Jon Hyman Apr 9, 2018 ▶ 9:26
Assertion Not checkable as stated
Braze CTO: Failed push notifications are mostly caused by app uninstalls
“It actually is mostly attributed to uninstalls. When people uninstall your app, you can't send push to them anymore and so they end up bouncing.”
Jon Hyman Apr 9, 2018 ▶ 13:30
Disclosure
Braze plans to self-host Elasticsearch on AWS rather than using managed service
“What we're doing is we're going to be running our own Elasticsearch instances ourselves that we can then fully configure and it'll just give us a lot more flexibility.”
Jon Hyman Apr 9, 2018 ▶ 16:57
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more