data engineering

“Data infrastructure. Or data engineering, which is the engineering of systems where the design of the system itself is dependent on the nature of the data that flows through it”

“If you're hiring a bunch of data scientists and then making them dependent on a different organization for engineering is basically Hiring some really expensive people, and then tying their hands behind their back, and you know, that's not a smart thing to do,…”

“We see ourselves very much as a data company, and we thought that data science in the early days would be a really key part of our strategy, and it is, but the data engineering challenge is so immensely harder that kind of data science kind of, in terms of tea…”

“That is like the death of the data analyst as like a productive member of your team. They will get frustrated. They will leave. They, they're, they don't have great career paths. All of these like negative outcomes.”

“An amazing engineering achievement happened in the early, you know, throughout the 2010, which was solving these massive pure technical scale problems. But now we're talking about organizational scale, dealing with complexity and dealing with developer product…”

“I think one of the big changes would be, we'll move from this specialized and specialization to generalization. So some of the things that we consider specialization today, like data engineering, a large portion of what we call data science becomes basic engin…”

“You know, and in fact, I think, candidly, data engineering will suffer the same thing. It will happen. We write about this in the last chapter of our book, in the fact that data engineering as a title could morph into something else, but that's what titles do.”

“A lot of the core issues in data engineering actually haven't changed that much in 20 years, and so people got so excited about technology in the transition in the 2000, the internet bubble, rise of Google, the rise of Facebook, that they lost sight of that”

“If you're really good at the data part, then you got that covered, right? But what you probably lack is, like, rigorous software engineering skills. That's what I would focus on, hands down.”

“One of the core beliefs about the, that we have about the profession is that data and software are not that different. And when you're building a production data system, you're building a production software system, which means that we should be taking lessons…”

“I do think like if you're early starting out, There's no problem in probably bringing on some sort of consultant to do maybe more of the data engine work and then bring on a full-time maybe analyst to kind of work on top of that is what I'd imagine would be go…”

“SQL you can learn quickly. What's going to take time is just getting a sense for how data Operates and flows and data sets in general.”

“I see a lot of people move laterally into data engineering, often from either data analyst or software engineer. Data analyst, because I think there's just the sheer quantity of data analytics jobs are More.”