Google Spanner

“This allows us to offer developers a consistency model called external consistency, where even though Spanner might be processing hundreds of transactions concurrently they all appear to have committed In serial order according to their timestamp.”

“When the reload goes back to normal, We can then eventually remove these shard points that we added earlier, and this is a natural way of the system cleaning up itself so that we then shrink back the number of shards to reduce both unnecessary shards and also …”

“In fact, this is a case where one of the earlier methods of sharding just by row would potentially work better, because if your incoming new workload is just randomly Distributed across rows perfectly. It will distribute across all the shards very well.”

“We use this technology called TrueTime, which allows us to get an estimate of what is the exact accurate time anywhere in the world to assign a unique timestamp to every transaction that Spanner processes.”

“Well, Spanner is Google's relational database. It has asset semantics and SQL, sorry, asset transactions and SQL semantics.”

“The mission critical database for Google, it supports hundreds of Google applications, including Google AdWords and Google Play, and we manage, you know, petabytes of data and have tens of millions of QPS every day.”

“With every shard being on a side lead on a different row, sorry, every row being on a different shard, you end up needing to check all shards to answer this query, which kind of defies this prior, this criteria of locality that I talked about earlier.”

“One of the biggest problems you can face with a dynamic sharding system is that it takes a while for us to adapt to your workload.”

“What's special about Spanner is that it's able to synchronously replicate data between different data centers using algorithms such as Paxos and two-phase commit.”

“The really nice thing about having a location service is that you can adaptively change your shards rather than having to hard code it upfront.”

“Between shards, You still have full transaction guarantees and full SQL capabilities. In a lot of SQL sharding schemes, you, within charts, you have full SQL capabilities, but across charts, there's no such thing.”

“The problem here is that, you know, one day you're going to get some singer that's so popular and so prolific and has so many songs that That one singer shard is gonna be huge, and then you are kind of in trouble, right? You either need to do a lot of special …”

“With some breakthroughs in computer science, there was the Google Spanner paper that came out, and it said, hey, you can get transactions. You can have these types of guarantees in a distributed environment, and that was really the foundation for the work that…”