Blog

The Most Impactful Differentiator for Modern Data Warehouses

Aug 05, 2021
Jeff Haws

Whether your company has invested in one year or not, you’re probably at least passingly familiar with modern data warehouses like Snowflake and Google BigQuery. They’re one of the hottest data-related tools, particularly for enterprises with massive amounts of data. If you have tons of data that you want to consolidate and make accessible to marketing in an efficient and affordable way, these are likely the best options for you.

But what makes them so great? There’s a number of advantages to using them rather than some of the other data-storage options, but let’s focus on one huge differentiator that you may have heard but may not fully understand its significance: separating storage from compute.

Let’s start by defining these two terms simply. Storage is just what it sounds like: a system that allows you to store data online in the cloud. Compute is how you work on projects with that data. It’s essentially the way you put that data to use. SaaS is an example of cloud computing that many are familiar with.

Now, let’s look at why this matters.

What does separating them even mean?

Traditionally, the two have been tied together in on-premise and cloud storage options, meaning they had to 1) scale together, and 2) always be “on” (which means you’re being charged for them)

So, let’s say you’re putting around 35% of your stored data to use. Every time your customer base grows, you’re storing more and more data that’s just sitting there. And with storage and compute linked together, you have to pay for compute to scale up at the same pace of storage even though you’re only using a relatively small percentage of what’s being stored.

On the flip side, you might decide to do more with the data you have, expanding projects and sending more campaigns. This requires more compute ability, which means you have to likewise increase storage even if you don’t need it.

While with compute, even when you’re not actively using it — such as overnight — it still has to be running at the same rate it does during the day because it’s tied to storage, driving up costs during times when you’re doing nothing with the data.

How does this impact your team? 

If you understand that if you want to be an organization that cares about and recognizes the immense importance of your data — your marketing team can’t keep up with the competition without unfettered access to it — then your data team’s problems are the marketing team’s problems.

There are multiple ways separating storage and compute will be a huge boon for your data team, therefore making your lives easier as well when you want to send better, more personalized campaigns at enterprise scale.

Agility

Since everything scaled together in the old world, there’s a good bit of guesswork involved in trying to manage data storage. Having an idea of the demands on storage or compute well ahead helped the data team plan how they expand or contract the other, along with the spend involved. And there’s no real way to know for sure. So, yeah. They end up guessing.

Removing that constraint means no need to guess. If either storage or compute can scale up — or down … or off completely, as a matter of fact — suddenly there’s no guesswork. Each goes up and down alongside your needs, and data teams can just do the work that’s needed at the time rather than planning out complex deployments months in advance.

Scalability

When storage and compute are tied together, rapid growth in one or the other can be unnecessarily costly and challenging to manage. Scaling infrastructure to handle it all and ensuring nothing goes awry is a constant pain for your data team, and it will create significant headaches.

Once they’re separated, though, scaling happens without having to do much of anything on your end. You get complete flexibility in managing resources to keep pace with your storage and compute needs at any time, reducing the need for data replication and syncs that can hold up your marketing work.

Cost

With traditional setups, you often pay for whatever is provisioned upfront to account for peak usage. So, no matter your current needs, you’re going to be paying pretty much peak costs — with extra charges and challenges for overages.

When storage and compute are independent of each other, though, it’s simple to set up a model where you pay for them at the rate they’re needed. Your data team doesn’t need to anticipate unexpected surges and other changes, and you don’t need to pay for what you’re not using. This can greatly reduce costs, and free your data team to focus on how they can help you utilize your data better.

Takeaways

While there are lots of advantages to using a modern data warehouse, this may be the biggest — and sometimes least understood for marketers — differentiator for them. Hopefully, this will help you better understand why your data team might be particularly excited about the idea of getting your data set up in a solution like Snowflake.

We’d love to talk to you more about this. If we can answer any questions or help you understand how MessageGears can help you take full advantage of the power of the modern data warehouse by connecting directly to it so you have access to your data live and in real time, reach out to set up a call.

About the Author

Jeff Haws

As MessageGears’ Senior Marketing Manager, Jeff is focused on producing engaging and thoughtful content that resonates with enterprise marketers, helping them to better understand how MessageGears makes their jobs easier. He’s passionate about understanding the way data impacts messaging, and he’s also hopelessly obsessed with baseball.