Jan 16, 2020 · 16m · a16z
Explaining AI
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In his keynote address at the a16z Summit, Harry Shum outlines Microsoft's breakthroughs in artificial intelligence while emphasizing the urgent need to address algorithmic bias and develop explainable AI systems.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 0.0 Guest teaching 2.8 Guest disagreement 0.2 The host pushing back 0.0
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 9:10 Challenging AI Bias Assumptions
Harry Shum playfully challenges the audience with word association prompts that expose implicit gender biases embedded within standard AI training datasets.
Hardest push from the host ▶ 4:44 Absence of Host PushbackBecause this transcript represents a solo keynote presentation, the host does not speak or offer any pushback.
Biggest teaching moment ▶ 6:30 Educating on Systemic AI Gender BiasHarry Shum clearly demonstrates how changing a single pronoun in a bio drastically alters a machine learning model's output, educating the audience on systemic dataset bias.
The host holds their own ▶ 0:00 Absence of Host ParticipationThe host is non-participatory during this monologue presentation, resulting in no host hit-back moments.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Approaching Human Parity in Perception and Language Tasks | 0 | 2 | 0 | 0 | This monologue segment consists entirely of Harry Shum presenting AI benchmark achievements in perception and speech recognition. The host is non-existent in the audio, requiring host-side scores to be zero. | |
| Case Study: XiaoIce Chatbot and the Power of EQ in AI | 0 | 2 | 0 | 0 | Harry Shum explains the XiaoIce chatbot case study, focusing on EQ integration and high engagement metrics. Because this is a keynote presentation without host interaction, host scores are strictly zero. | |
| Business Applications of XiaoIce in Retail and Finance | 0 | 3 | 0 | 0 | The guest presents real-world applications of XiaoIce in retail and finance before introducing machine learning bias. The host remains silent throughout the monologue. | |
| Analyzing and Debiasing Word Embeddings in NLP | 0 | 4 | 1 | 0 | Harry Shum interacts with the audience using word embedding analogies to illustrate gender bias in training data. Host activity remains non-existent. | |
| Three Reasons Why Explainable AI is Critical | 0 | 3 | 0 | 0 | The speaker outlines three core reasons why explainable AI is necessary for human trust and safety. No host presence is detected in this segment. | |
| Prediction Accuracy vs. Model Explainability Spectrum | 0 | 3 | 0 | 0 | Harry Shum concludes his keynote by discussing the trade-off between model accuracy and explainability. Host scores remain zero due to the solo presentation format. |
Statements from this episode (14)
Shum: Microsoft AI approaches human parity in perception and language
“We actually gradually approaching human parody in a number of those human tasks, especially perception. From computer vision to speech to more and more natural language.”
Shum: Microsoft Research Beijing invented ResNet, the most popular vision network
“ResNet now is the most popular deep neural net, ah, in computer vision. Ah, we actually invented in our research lab in Beijing by my students, ah, using 152 layers of neural networks.”
Shum: Microsoft AI surpassed human transcribers with 5.1% error rate
“In speech, ah, there's this very standard dataset called the switchboard dataset that is really recording of two, ah, from two sides and, ah, phone conversations. Ah, two years ago, we accomplished the 5.1% of the error rate. Ah, that is really, really, ah, am…”
Shum: Microsoft's XiaoIce social chatbot has 120 million monthly active users
“Ah, XiaoEyes now is very popular, ah, with a hundred twenty million, ah, monthly active users.”
Shum: Microsoft's XiaoIce chatbot averages 23 conversation turns per session
“But Xiaomi's CPS, or conversation turns possession, No, on average, actually achieves 23 times.”
Shum: Longest XiaoIce conversation lasted over 29 hours and 7,000 turns
“Incredibly, the longest conversation any human user ever talked to XiaoAis is actually over 29 hours of 7000 turns.”
Shum: XiaoIce distributed 1M coupons with 40% in-store conversion rate
“So we did experiment with the coupons, and the amazing that, you know, we distributed a million coupons in 13 hours, and the in-stock conversion rate hit a 40% in the next four days.”
Shum: XiaoIce powers 90% of Chinese corporate earnings summaries
“We actually now power 90% of, ah the quarterly earning report summary, ah, for 90% of, you know, Chinese companies, and they're also powered by Xiaoice.”
Shum: Goldman Sachs's Apple Card system likely contains AI bias
“You probably would, you know, agree with me that very likely there's some kind of AI buyers in the machine learning system they have built.”
Shum: Blindly training AI on internet data yields biased systems
“The data we collected over the internet has some inherent bias. So if we just blindly use the data to train AI, we know we have problems.”
Shum: AI models with trillions of parameters are black boxes
“Because we now train all this kind of complex models with millions, even trillions of parameters. So effectively we are building AIs as black boxes.”
Shum: Opaque AI in medical and military decisions can be deadly
“You might think it's probably harmless to get some kind of Netflix, Netflix recommendation about the movies you to watch. But the political ads on social networks can be problematic. Medical diagnosis, or even military decisions based on AI, you know, can be d…”