Data Modeling in the Modern Stack

Data modeling is the most impactful decision a data team makes, because it sets the architecture and the path everyone follows from then on. The modern data stack hasn't changed that, even though cheap storage and powerful cloud warehouses have made the old arguments about performance and disk space less important. Four approaches cover most of what teams do: Inmon style normalized modeling with departmental data marts, Kimball style dimensional modeling with a star schema, Data Vault with hubs, links and satellites, and one big table where staging goes straight to wide marts. Each trades off redundancy, joins, compute cost and complexity differently. What I see work most often is a hybrid: a star schema underneath for structure and mental clarity, with wide one big table style marts on top for reporting and reverse ETL.

Why data modeling still matters

Over the last five to ten years the modern data stack brought a wave of new tools, and with it a lot of disagreement about modeling. The camps look something like this:

• Traditional modeling is dead.
• Cloud databases are so strong you don't need a model at all.
• The modern stack causes more harm than good.

Whichever camp you're in, there are four reasons modeling is still worth taking seriously.

1. The number of sources

A business uses more data sources than ever, and as an engineer the only good way to get a hold of them is a modeling strategy.

The unfortunate reality is that sources get thrown over the fence to the engineer to figure out. Without a plan for where each one goes you get bombarded and overwhelmed.

2. The range of consumers

There are more types of data consumers than ever, each with different expectations of what they want to see and how they'll use it.

Without an organizational model behind the data it's harder to answer all of those questions, or at least to feel confident that you can.

3. Optimization: speed and cost

The guidelines differ between traditional row based databases and modern cloud column stores. In the modern stack we're mostly talking about cloud computing, where the expensive component is computation rather than storage.

These databases handle far more than the old ones and they're incredibly efficient, but there's still a limit. A good data model helps you balance that and avoid running too many outlandish queries.

4. Mental clarity

This is the most important one to me. A defined model gives everyone a clear strategy to follow:

• How tables get built
• How queries get written
• What goes where

It's also much easier to onboard people, especially if you're following a well known approach they can recognize on day one.

Common approaches

There are technically a million ways to model data, but four cover most of what teams actually do.

1. Normalized modeling (Inmon)

Made famous by Bill Inmon and around for a long time. Sources feed a staging layer, and the warehouse is designed to mimic the source systems themselves.

Everything is normalized, meaning no data redundancy, which also means a lot of joins to get a final result.

The other distinguishing feature is that the warehouse feeds individual data marts per department: finance, HR, operations, each with their own reporting needs.

The upside is that the warehouse is a true single source of truth, because it represents each source exactly. The downsides:

• Join heavy. Not ideal for column based cloud warehouses.
• Conflicting marts. Each department's mart can define things differently depending on how it's built.
• No single place for everything. You don't get full access to all of the data from one spot.

2. Dimensional modeling (Kimball)

Also called denormalized modeling, made famous by Ralph Kimball, and the home of the star schema. Rather than individual normalized tables that each mirror a source, you build flattened, denormalized facts and dimensions.

There's some redundancy in the warehouse, but at the analytics layer there are fewer joins and everything is available from one single source of truth.

The other difference is that it's designed around business function rather than around source data.

This is probably the most common strategy, and the one whose validity has been questioned most in the modern stack: is it still necessary to do all of this?

3. Data Vault

A more complex approach. You still have sources and staging, but the data is split into three kinds of table:

• Hubs. Key metadata.
• Links. More key metadata, connecting the hubs.
• Satellites. The descriptive values and context around them.

One of the main goals is that the raw vault loads source data without compromising any future logic. No transformations, just all the data allocated into its place. The business vault then applies more subtle transformations on top, following the same organizing principle.

This design isn't conducive to analytical reporting. It's for storage and organization, which is why you'll usually see a presentation layer on top that mimics a dimensional model for analytics.

The upside is that it's very organized with a clear strategy for what goes where. The downside is that it's complicated, and it's really intended for environments with a lot of different sources.

4. One big table (OBT)

One of the newer approaches. You go straight to very wide, denormalized models, from staging directly to the final reporting mart.

Two theories sit behind it: storage is cheap, so wide tables are fine, and modern cloud databases compute so well that you don't need to store intermediate structures.

You might see an intermediate layer in the middle, but those are often ephemeral: not materialized, there to support one mart rather than being reused.

The upside is obvious: it's easier, and you're up and running quickly. The downsides are real:

• Compute creep. Each pipeline becomes an extensive query with a lot of business logic inside it, and the more complicated those get, the more expensive the compute. Fine at the start, unwieldy at enterprise scale.
• Redundancy. Each mart repeats the same underlying data.
• At the mercy of the sources. With no modeling in the middle, you have to be deliberate about each pipeline or you end up taking everything and building a pipeline for each, which gets out of hand fast.

Things to consider

Slow down, then embrace what's new

The new tools make the debate about performance and storage less decisive than it used to be. The values of organization and mental clarity are as important as ever, and a model is the foundation for handling every new source that comes at you.

It's tempting to go fast and build models quickly. In the long run that introduces problems you could have avoided by slowing down and building the warehouse strategically.

At the same time, embrace what's new: column stores, cheap storage, the idea of one big table.

The hybrid approach

Take the wide marts and put a star schema underneath them. That's the hybrid I see most often.

To the end user it looks the same, a wide mart they can query. To the engineers there's a structure for modeling the data, and the end to end queries are broken into manageable pieces instead of one enormous pipeline.

The cost is that you're arguably doubling up storage with the extra mart layer, but storage is the cheap part.

Marts serve more than reports

The mart layer matters for another reason. The modern stack brought reverse ETL, which syncs warehouse data back into business applications. So the model is now serving:

• People running queries
• Reporting tools
• Reverse ETL syncs into business applications

Design the marts to handle all of those scenarios.

Team dynamics

Finally, think about responsibilities. Job titles have blurred, but a hybrid like this creates a natural split: data engineers focus on the core data model, and analytics engineers or analysts focus on the data marts and the analytics, with overlap between them.

It doesn't have to be exactly that, but it's the kind of thing to decide up front rather than figuring it out on the fly after you've already built everything.

Related reading

• Comparing 3 Types of Data Modeling
• The Struggle of Data Modeling (Facts vs Dimensions)
• How to Create a 3 Layer Data Model Pipeline
• Data Architecture with dbt (Putting It All Together)

Final takeaway

The hybrid described here is the shape most of the warehouses I've helped build or review have ended up in. Usually that's after a team started with one big table for speed and then needed structure once the number of sources and consumers grew.

The approaches above are the options I walk through with a team before that decision gets made for them.