The Exchanges

Q And maybe to drive it home, you know, again, very basic way. What's a couple of examples of, uh, I'm an analyst in a company. And I'm in charge of BI. What kind of queries do I get and need to answer through my BI tools?

A It's been around for so long. I think the general thing that a BI person gets asked is one of two things. Can you please update this dashboard? By which they mean, I want another data source included, or I want a different type of visualization or something like that. Uh, or the second request they get is, can you extract this data set to me and just email it to me? That's what BI people, I think, typically do. I think what they want to do is figure out for the business what is actually going on in the data. You know, what's happening to our marketing performance? Is it getting more or less efficient? What's happening to our inventory? But they end up servicing a bunch of very rudimentary, relentless requests from users who are one step removed from the data that they want. So I think both sides are very unhappy with this process. But unfortunately, I think that's probably a reasonably good description of what we see people doing in classic BI jobs today.

AI assessment note: “Can you please update this dashboard?... or... can you extract this data set”

Q Fantastic. So to get into the meat of the product and the more slightly technical discussion, a big part of the idea is that you live on top of Snowflake, all modern data warehouses. Do you want to talk from a product and technical perspective about how that works, where the data lives, where the computer lives? How do you do it?

A Yep. So one of the things that changed, obviously, with the Databricks and the Snowflakes and the Big Queries and so forth is that the data volumes got huge. One of the things that I knew before, I was working in infrastructure, and one of the things I knew before joining Sigma was that the price per terabyte of storing data in AWS for five years in a row was -65%. So you don't have to, you know, sort of graduate with an economics tree to understand that if you have declining prices year over year, you're going to have increasing volumes. That's exactly what was happening. Snowflake and the like came around and just figured out how to make, or better said, I like to think of it as organize that data so you could use it. So the first thing that Sigma had to do architecturally, uh, is abandon everything BI products had done in the past around managing for performance. All products a priori were built with caching layers. We do, we do not do any caching. So everything you do in Sigma is live query on the warehouse. Our customers have up to trillions of records. Uh, and our overhead on that transaction is less than one second. So you log in through Sigma, you build your assets in a Sigma controller layer, but we push down every bit of the query. So the data never leaves the warehouse. This has a number of very salient, uh, beneficial impacts. Number one is super high performance, o…

AI assessment note: “we push down every bit of the query. So the data never leaves the warehouse.”

Q And clearly you're all in on cloud. There is a little bit of a theme around cloud repatriation, maybe with AI, you know, people wanting the models to be very close to the data. Is that something A, you see, B, you worry about?

A I think that the models are going to live next to the data in these warehouses, and we're already seeing that. Um, Anthropic, uh, doing, you know, if you know Dario, doing that deal with Snowflake is a clear attempt to make sure that they're as close as they can be to the enterprise data, that they're getting great performance, that they're adhering to security models. Uh, I think that's a trend that we are going to see. I also think we're going to see people, uh, building, uh, smaller models in these warehouses as well. Either tailored to whatever business they're in or trying to manage cost. So I think that's inevitable. Uh, there's some trade-offs, of course, to not working on the, on the, on the premises side, but I think the, I always think about things as a train track in this particular case, like the trains left the station, it's gaining momentum. There's no point in building backwards.

AI assessment note: “I think that the models are going to live next to the data in these warehouses”

Q hosted in partnership with our friends at Foursquare. If you care about BI, AI, or just want to understand the data goal rush, stick around for this very entertaining and insightful conversation with Mike. Mike, welcome. You have, uh, on the Sigma computing website, a line that I found super interesting, which is, uh, forget the past 20 years of BI. It's been boring. What do you mean by that?

A Uh, we talk about the fact that, you know, the world did not need another BI product. This is, uh, we didn't come up with that line. It was actually someone, uh, from JP Morgan that told us that, and I think, He was right. Uh, BI as an industry is boring. Uh, the idea that, I know I'm in an audience of data people, some of you have done this job, so I, no offense intended, but, you know, when you, when we were living in this world of like client, uh, uh, you had storage and compute on premises, and you had to figure ways to connect to those things, and that was difficult from a security point of view and a networking point of view, and then you had these special skills like writing SQL, And then you could create dashboards for people who really wanted the data. That just didn't seem like a great model. Combined with the fact that increasingly strangely over time, the chart became the proxy for the data. It's funny because if you look at older BI tools like Business Objects and even MicroStrategy and Qlik, they were very tabular or in terms of their interface, and they gave you row-level data. And then we sort of, in my opinion, devolved into Tableau. And we sort of treated people like they were idiots, and all they could really understand was a colorful pie chart.

AI assessment note: “we sort of devolved into Tableau. And we sort of treated people like they were idiots”

Q of people are wondering whether this idea of inputting a query, a BI query in natural language, and have it translate into very precise SQL, which is the language that one uses to query databases. It's something that can actually work, and that's where you go into the semantic layer stuff as well. All the things. What are you guys doing and how you think about the opportunities and challenges?

A It's a terrible idea, and so it doesn't matter whether it works. It could be the best executed terrible idea ever. No one wants to speak SQL to a system. Very few people think in terms of SQL. I don't think around like, what bagel do I want today? I select the bagel, the poppy sea bagel from the second shelf. When? I mean, no one does this. So the idea that I'm supposed to adopt a SQL semantic concept into my natural language is the ultimate hubris of engineering. You know, like, oh, these idiots don't write SQL. So what we're going to do is we're going to get them to be able to speak SQL easier to us. This is not what AI was supposed to do for end users. So I, I think these products are a joke. Um, I don't think that they're going to succeed and we don't care about them. Um, so what we do care about though is helping people And helping systems understand each other. So I think we think a lot about context to data. Who are you? What data do you tend to use? Who do you collaborate with on an ongoing basis? What did you do yesterday? What are people like you in the system doing? Combined with systems level understanding, what tables are out there? What are, what are being actively used? Uh, and you put the two together and you can come up with some really good answers. So ask is the product that we are putting into public data on, uh, April first, and then for GA on May first, an…

AI assessment note: “It's a terrible idea, and so it doesn't matter whether it works.”

Q the BI solution that lives on top of modern data warehouses. Another big principle is this, uh, concept of spreadsheets like interface. Uh, so how does that work? And, um, I saw as I was prepping for this that you guys just released an update, uh, making Sigma click. How does that work? What was challenging to do? What are you learning? Uh, and why did you need to refresh?

A So, a spreadsheet's difficult to do on top of a warehouse because you're dealing with basically a flat table against a relational database, and these two concepts don't really, uh, work well together. Uh, and that's kind of the magic that we mostly made work, and when I say mostly, I always tell folks, what's the difference between a spreadsheet in Sigma and a spreadsheet like Excel, and the difference is in Sigma, there is no cell reference, right? Everything is columnar, right? A table doesn't know about a cell. So there's no, there's no way to make that proxy really work without massive performance implications. Um, but the fun thing about all of this is if you look at Sigma as a, as a BI product and you want to build a pie chart, you can do that. And you can do that with AI, by the way, you can do all your favorite BI things. But the fun part is like a reference JP Morgan, uh, Who uses our product, and when they connected the first time, the guys, I think, is sort of like, I can swear in here.

AI assessment note: “a spreadsheet's difficult to do on top of a warehouse because you're dealing with”

Q people can query and having a common language around query. Uh, what, what, um, has been sort of specific and unique about your approach and, um, you know, since we mentioned Tableau up front, uh, you know, Looker, which was acquired by Google for. 2.7 billion a few years ago. The secret sauce was Luke ML, which was that intermediary layer. So how do you guys go about that part?

A So maybe we'll start with what, what is, what is a semantic layer? Uh, we, we ran a little study, by the way, I'll give you a stat to, to indicate why these things become important. So we looked at all of our enterprise customers Table utilization over a 60 day period, and .1% of the tables in their warehouses were ever touched. So think about the data that you're storing, uh, and only using .1, not, not .1% of the data, .1% of the tables. And we're pretty convinced that one of the problems that exists in most enterprises is just awareness of the data that's available to people, period. So with that as background, semantic layer helps provide meaning to people on data models, right? It gives you an understanding of what this data means. Uh, and obviously, DBT went out there as kind of one of the more successful ones a few years ago, uh, with an open source model, and they've been since trying to monetize that, and there are a few other, uh, vendors that are doing the same, and I suspect you might see warehouse providers do the same. Sigma does a semantic job for you, but is not trying to replicate the semantic layer. We, you can data model in Sigma. You can build all kinds of referencing capabilities so that you can search and identify that data. You can build metrics in Sigma so you know, for example, what revenue is and that there's a standard calculation for revenue on the s…

AI assessment note: “Sigma does a semantic job for you, but is not trying to replicate the semantic layer.”

Q Once the venture has gotten off the ground a little bit. What was that story? When did you come in, come in, and what was the context?

A You know, to the extent that we can say Sigma's in any way successful today, and I hate to say that cause I'm a super paranoid person, but we're doing okay. Uh, and then the, the, the expression is every overnight success is 10 years in the making. This being just about 10 years since Sigma was founded, we're right on cue. Uh, but those are really hard. If you've built your own companies, it's never a linear path. Uh, you spend, matter of fact, it's a lot like evolution. It happens in big bangs and then, you know, nothing happens for a long time. And I think Rob and Jason, and I'll say this on their behalf, because I've told this story with them and for them for a long time now. They, the idea that drives the company today is the first idea, which was, why is the average person never given a technology in Silicon Valley? Like if you live in Silicon, I live in California, I live in San Francisco, all the companies that you know and love that weren't consumer, uh, were built, uh, for technical people. You know, whether it was VMware or anything that Cisco ever invented, everything was built for a technical person. Nothing was built for a normal person. And the normal people are just, by the way, I'll just call them normal people. I consider myself a normal person or like the long tail, but the vast majority of the people at every single company that you go to. They work in market…

AI assessment note: “the idea that drives the company today is the first idea”