Chia: All Spanner RPCs address specific data rather than machine names
Daniel Chia · Dynamic Range Sharding with Spanner // Daniel Chia, Google Spanner (FirstMark's Data Driven NYC) · Feb 25, 2019 · at 13:07
Daniel Chia, Senior Software Engineer at Google Spanner, explains the internal dynamic sharding and coprocessor framework architecture of Spanner at Data Driven NYC.
0:00 / 0:05exact quote · 5.3s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“All RPCs in Spanner are addressed to specific pieces of data rather than the machine name.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Daniel Chia
Chia: Shard granularity is exceptionally difficult to alter post-implementation
“Once you shard by a certain level of granularity, it's very hard to redesign your system to try and think about it a different way.”
Chia: Google Spanner eliminates operational differences between and within shards
“And in Spanner, it's a little bit nicer in that every shard behaves the same, and because we have technologies like, ah, two-phase commit, We are, you don't really have any difference between operating within a shard or across shards, but it's not always true …”
Daniel Chia: Sharding database tables purely by entity keys risks giant hot shards
“The problem here is that, you know, one day you're going to get some singer that's so popular and so prolific and has so many songs that That one singer shard is gonna be huge, and then you are kind of in trouble, right? You either need to do a lot of special …”
Daniel Chia: Row-based sharding can outperform dynamic range sharding for uniformly random workloads
“In fact, this is a case where one of the earlier methods of sharding just by row would potentially work better, because if your incoming new workload is just randomly Distributed across rows perfectly. It will distribute across all the shards very well.”
Google Spanner provides full SQL capabilities and transaction guarantees across shards
“Between shards, You still have full transaction guarantees and full SQL capabilities. In a lot of SQL sharding schemes, you, within charts, you have full SQL capabilities, but across charts, there's no such thing.”