The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 9 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Uber deployed trip similarity algorithms to fight incentive fraud in China
“Uber used to run this incentive program in China, and, ah, people are trying to game it, so they used to always have, like, similar trips simulated From their, ah, various different devices. So what we did was actually, like, try to find an algorithm where we …”
Murugesan: Uber practically had no data infrastructure when he joined in 2014
“We practically did not have an infrastructure is what the honest reality is.”
Schemaless JSON fails as companies scale beyond 50 employees
“It works well, like, if you're, like, a ten-person company, or, like, even to a fifty-person company, when you're, like, scaling to, like, thousands of people, like, you actually need a proper negotiation in between.”
Murugesan: City operations personnel constitute most of Uber's data consumers
“Most of Uber's data consumers are actually these operations people.”
Uber consolidated all log and business data into an HDFS data lake
“What we really created was, like, using HDFS, like, a data lake, where we basically copied the whole data sets from, like whatever we get from, like, analytical logs or, like, all our business data sources, too, into HDFS.”
Streamific powers all data ingestion and aggregation at Uber
“There's a system called Streamific, which powers all of the data ingestion aggregation at this point.”
Uber transitioned from ETL into Vertica to EL into Hadoop
“We went from an ETL model, where we scraped from, like, the original source, transformed the data and loaded to Vertica, to, like, just an EL model, where we just, like, just copy the data as soon as possible into, like, Hadoop, and all the transformation can …”
Uber engineers frequently crashed Kafka clusters with unthrottled Spark executor writes
“Kafka was, in general, like, a nice way where people used to pipe the results of, like, their Spark jobs. But often cases, what they do is, like, they hit Kafka hard and bring Kafka down because they're trying to, like, actually send data from, like, hundred e…”
Uber uses cost accounting chargebacks to track data storage costs by unit
“We have something called cost accounting chargebacks is what we call it. So we actually try to, we have a lineage model where we try to figure out who's storing the data. And, ah, we actually kind of can go get a dollar amount of what we are storing.”