MVC is an interesting concept from the late 70s that separates the View (presentation) from the Controller via the Model. It has been used in designing web applications and is still heavily used, for example, in Ruby on Rails or Laravel, a popular PHP framework.

This design pattern got me thinking: Wouldn't it be convenient to separate the presentation from the storage through a data modeling layer, similar to the model layer? A place where we can declaratively define our metrics, measures, and business KPIs so that we can update them confidently and version them.

Today, we have the beginning of such a separation of concerns with a Semantic Layer. Let's explore the parallels between MVC and data.

What Is an Active Record and How Does It Fit In?

To understand how this parallel works, let's start with Active Record, the intermediary, and concrete implementation of the Model layer in Ruby on Rails. An Active Record is the "M" in MVC (Model View Controller) and maps database tables to Ruby classes. It facilitates the creation and use of business objects whose data requires persistent storage in a database, also called ORM (Object-Relational Mapping).

A data modeling language such as LookML could be the ORM to a semantic layer, mapping raw data sources to business metrics or existing defined tables within the semantic layer, also called Views.

The Active Record provides methods to perform CRUD operations without writing SQL. Semantic layers typically don't update or change data but provide extensive API components that abstract complex consumer queries and data operations. The Active Record also represents relationships between models where a semantic layer's data modeling component defines relationships between various data entities and metrics.

An Active Record validates data before persisting in the database; the same goes for the semantic layer, with its access control and declarative data modeling, which allows parsing definitions and ensures data integrity and access rights.

Web Development's MVC Applied to Data

We can see similar analogies if we take a step back and look at the MVC model.

In MVC, the View represents the visual presentation of information, such as charts, diagrams, or tables, and the Model represents the application's dynamic data structure. And we use the Controller to combine the two. Image of the MVC and how they interact. Image of the MVC and how they interact.

The Controller usually accepts user input and converts it into commands for the Model or View. In Rails, client requests are sent to a router, which directs them to specific controller methods. The controller then interacts with models and prepares responses using views.

This solved the problem of maintainability and, most importantly, separation of concerns. If we contrast that with the data world, where data transformations are made within data pipelines, dbt transformations, in your business intelligence tools, or anywhere along the data warehouse chain, we have the same problems that key metrics are mixed with technical implementation logic, usually defined multiple times in several BI tools, data apps or dashboards.

Traditional data access is done through SQL queries via ODBC connection or a data pipeline. In the context of newer data architecture, such as a data lake or data lakehouse, we duplicate metrics definitions on top of this extracted data. Often, creating another dbt model is much easier than using an existing one.

This duplication of definitions obscures the maintainability of key KPI declarations, creating inconsistency, making them harder to apply, and affecting data governance.

Think of the Model1 from the model-view-controller pattern and how it could solve these issues with a central, consistent, and declarative way of defining our metrics. This is what a semantic layer is created for.

📝 Still Want to Know More About MVC or Active Record?

For a deeper understanding of these fundamental patterns with detailed explanations of concepts we're borrowing for our data architecture patterns:

The MVC for Data as a Semantic Layer

If we apply the semantic layer to the MVC model: Can we map it to one of the components, the model or the controller? Let's check.

Like the Controller, which accepts user input and converts it into actions on the Model or View, the semantic layer acts as an intermediary between data consumers (like BI tools or data apps) and data sources. It receives queries or requests from users and translates them through the APIs (SQL, MDX, REST, GraphQL, AI, etc.) into executable commands against the data sources. It routes user intentions to the correct data retrieval or manipulation, abstracting the underlying, potentially complex, architecture.

At the same time, a semantic layer also overlaps and has traits from the Model. The Model from MVC holds the application's data and business logic, similar to a semantic layer that holds the data modeling and key business metrics, this aligns closely with the Model's responsibility for data representation and business rules. Secondly, the Model ensures data integrity and validation logic. Similarly, the semantic layer enforces best practices data governance rules with access controls and, with it, data model language that checks the validity of rules almost like a compiler.

So, can the semantic layer be seen as a Controller? Partially, yes: It acts as a mediator between the user's requests and the data, but not exclusively: It also embodies characteristics of the Model by defining data structures, business metrics, and relationships.

That could look something like this: The image shows the semantic layer MVC, comparing the traditional MVC above.

Breaking it down - looking only at the query side (not updating the data as seen with dotted lines) - we can put the semantic layer in the middle, replacing the Controller, and at the same time, the View component goes through the semantic layer as an abstraction:

  • Semantic Layer ↔ Model: The semantic layer applies business logic and abstracts complex data access, governing the Model from direct querying by users and views.
  • Semantic Layer ↔ View: The View uses the semantic layer as an abstraction to access data via APIs, ensuring metrics are only defined once and are consistent.
  • Semantic Layer ↔ User: The user interacts with the semantic layer to update business metrics and retrieve data, maybe checking the lineage of a given KPI. The layer ensures metrics are updated consistently and aligned with the defined business rules.

The connections are left with dotted lines, as they might still occur, but not in the lens of the semantic layer:

  • Semantic Layer ↔ Controller: The controller is only involved in direct CRUD operations on the Model, which would bypass the semantic layer.
  • Model ↔ View: For example, clicking a button that causes a database entry to be updated. It is not a use case a semantic layer supports.

A More Semantic Layer-Centric View

As we have seen, the semantic layer acts mostly as an intermediary between the raw data (Model) and the analytics tools or applications (View/Controller). If we recall the key components of a semantic layer:

  • Data modeling: Unified, consistent metric definition in a declarative manner.
  • APIs (GraphQL, REST, SQL, MDX, AI): Integrate with any endpoint and deliver trusted data.
  • Access control: Centralize and fine-grain governance and security policies.
  • Caching: Query optimization, pre-aggregations; deliver faster more cost-efficient results.

We can update and draw another image by interacting with all the sources (Model) and have all the visualizations with data apps, BI tools, spreadsheets, and AI models (View). In the below adapted MVC model for a semantic layer, we represent the raw data sources as the Model.

The image represents the MVC pattern for a Semantic Layer, interacting with various ways data is consumed and visualized.

The semantic layer essentially absorbs some Model-like responsibilities (data modeling) while primarily functioning as an advanced Controller that manages how data is accessed, processed, and delivered to the View components.

What Are the Benefits of the MVC in Data?

How does the MVC pattern improve our lives within the data engineering lifecycle?

In summary, a semantic layer achieves a separation of concerns, which benefits complex organizations. With its declarative nature, we get flexibility and more straightforward modification and extension of individual metrics, as well as reusability through models that can be reused across different views (dashboards, data apps, etc.), which is instrumental in following the DRY principles for your most precious Key Performance Indicator.

Ultimately, it serves much better data governance, maintainability, a single source of definition, and simplifies metrics in large organizations with a declarative way of defining metrics.

💡 Related Architectural Patterns

There are also other related Patterns we didn't get into. Repository Pattern and Facade Pattern. They use different terms but serve similar purposes to achieve separation of concerns. A simplified view and comparison:

  • Repository Pattern: Client → Repository → Domain Objects Person
  • MVC: View → Controller → Model
  • Facade Pattern: Client → Facade → Complex Subsystems

What's Next

Organizations like Jobber have demonstrated the practical benefits of evolving from traditional patterns like Active Record to modern semantic layers. Their transition from Active Record to Cube shows how semantic layers can benefit organizations directly.

As we have seen, a semantic layer can benefit MVC and Active Records. It ensures a central, consistent, and declarative way of defining metrics, improving maintainability and governance in complex data systems.

In Part 2, we'll examine the benefits of having a semantic layer and focus on its strengths and advantages above and beyond those discussed in Part 1, as well as how it fits into today's data architecture.

1 Note: We will explore a bit of a nuanced definition of a semantic layer as a mix of controller and model.