Assertion Supported AI assessment confidence: 98% certainty 5/5 debate potential 3/5

Webster: DeepSeek hard-censors 85% of politically sensitive topics in Promptfoo benchmark

Ian Webster · How to use DeepSeek safely · Feb 28, 2025 · at 7:47

Ian Webster, CEO of Promptfoo, presents benchmark testing data regarding Chinese political censorship embedded in DeepSeek's AI models.

0:00 / 0:10exact quote · 10.9s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“We did a benchmark on Chinese politically sensitive topics that, that found that about 85% of those topics in our test set were hard censored.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

A benchmark on Chinese politically sensitive topics conducted by the speaker's team found that roughly 85% of tested topics were hard-censored by DeepSeek.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

The resolution criteria require public release or documentation of the benchmark report or dataset by the researcher's organization confirming the roughly 85% hard-censorship rate across the test set. On January 28, 2025, Ian Webster's organization, Promptfoo, published a benchmark report titled '1,156 Questions Censored by DeepSeek' along with the 'CCP-sensitive-prompts' dataset of 1,360 prompts on HuggingFace and Google Sheets. The results confirmed an approximate 85% hard refusal rate (1,156 out of 1,360 prompts) on DeepSeek-R1, fully satisfying the criteria.

Promptfoo published the report '1,156 Questions Censored by DeepSeek' on January 28, 2025.
Promptfoo publicly released the benchmark dataset 'CCP-sensitive-prompts' on HuggingFace and Google Sheets.
The benchmark evaluation confirmed an 85% censorship/refusal rate across the test set.

Checked against

[1] a16z.com [2] simplecast.com [3] promptfoo.dev [4] promptfoo.dev [5] qwe.edu.pl [6] mischadohler.com [7] enkryptai.com [8] taipeitimes.com

Ian Webster DeepSeek censorship benchmark 85 percent Ian Webster a16z podcast DeepSeek censored topics DeepSeek politically sensitive topics benchmark hard censored 85 percent

assessed as of 2026-07-28

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

DeepSeek Promptfoo

⚑ Spot an error? Report it

Webster: DeepSeek hard-censors 85% of politically sensitive topics in Promptfoo benchmark

Turn any episode into a week of clips.