From Ledgers to Intelligence Part 8: The Modern Data Stack dbt, Fivetran, and the Composable Analytics Era

Digital Transformation | June 2026

By 2016, the cloud data warehouse was established. Snowflake, Redshift, and BigQuery had demonstrated that organisations could run production analytical workloads at scale without owning hardware. What had not yet changed was the rest of the stack: the ingestion pipelines, the transformation processes, the metadata management, the consumption layer. These remained expensive, complex, and particularly for transformation the domain of specialist ETL engineers.

The Modern Data Stack was not a single product or a single invention. It was a philosophy a set of principles about how analytical infrastructure should be built that emerged from a cluster of startups founded between 2012 and 2018, each addressing a specific layer of the analytics architecture. Together, they assembled into a composable, cloud-native alternative to the monolithic enterprise BI suite that had dominated the previous decade.

Fivetran: Managed Connectors for ELT Ingestion

Fivetran was founded in 2012 with a deceptively simple proposition: managed connectors that replicate data from source systems into cloud data warehouses with zero maintenance. Before Fivetran, building and maintaining ingestion pipelines was significant engineering work. Each source system had a different API, authentication mechanism, and data model; APIs changed and broke pipelines; engineering teams spent weeks building connectors for standard SaaS applications (Salesforce, HubSpot, Stripe) that every analytics team needed.

Fivetran’s model was: pay a subscription, select your sources, provide credentials, and data flows. The engineering team maintaining Fivetran’s connector library handled API changes, authentication updates, and schema evolution. For the customer, a new data source went from “weeks of engineering work” to “fifteen minutes of configuration.” The maintenance burden disappeared entirely.

Fivetran’s architectural philosophy aligned perfectly with ELT: load data as-is into the warehouse and transform it there. The connector’s job was faithful replication, not transformation. This narrowed the scope of what Fivetran needed to do and made it very good at that narrow scope.

dbt: SQL-First Transformation and the Analytics Engineer

dbt (data build tool), created by Fishtown Analytics (later dbt Labs) and open-sourced in 2016, made a bet that proved transformative: all meaningful data transformation in a cloud data warehouse environment could be done in SQL, and that SQL-based transformation could be managed with the same software engineering practices applied to application code version control, automated testing, documentation, and modularity.

A dbt project consists of SQL files (models) that define SELECT statements. dbt compiles these models into the appropriate DDL/DML for the target warehouse (Snowflake, BigQuery, Redshift, etc.), manages dependencies between models, runs them in the correct order, and materialises the results as tables or views. A data engineer writes a SQL model that references upstream models; dbt resolves the dependency graph and executes everything correctly.

The impact on data team practices was profound. Before dbt, data transformation was a closed system: ETL jobs ran in proprietary tools, transformation logic was not version-controlled, testing was manual or nonexistent, and documentation was absent or immediately out of date. After dbt, transformation code lived in git, was reviewed in pull requests, tested via automated assertions, and self-documented through dbt’s docs generator. The practices that software engineers took for granted became standard in data teams.

This shift created a new role: the analytics engineer. Coined by Tristan Handy (CEO of dbt Labs) in a 2019 blog post, the analytics engineer sits between the data engineer (who builds ingestion pipelines and infrastructure) and the data analyst (who builds reports and insights). The analytics engineer owns the transformation layer the dbt models that produce the clean, trusted datasets that analysts consume. The role acknowledged a reality that had existed for years but lacked a name: someone needed to own the modelling layer between raw data and business reports.

🏗️ Architecture: The Modern Data Stack (2016–2022)

☁️ SaaS Sources (Salesforce, Stripe, HubSpot, Postgres) → 🔌 Ingestion (Fivetran / Airbyte Managed ELT) → 🏭 Cloud Warehouse (Snowflake / BigQuery / Redshift) → 🔄 Transform (dbt Models SQL + Git) → 🗺️ Semantic Layer (LookML / dbt Semantic) → 📊 Consumption (Looker / Metabase / Mode)

Looker and the Semantic Layer Redefined

Looker, founded in 2012 and acquired by Google in 2019 for $2.6 billion, approached the BI consumption layer differently. Rather than building a standalone reporting tool with its own semantic model, Looker made the semantic layer defined in its LookML modelling language the central product. LookML descriptions of database objects, relationships, and metrics were version-controlled, peer-reviewed, and the single source of truth for all metric definitions. Every chart, every dashboard, every ad-hoc query ran through LookML.

This architecture addressed the metric sprawl problem that had plagued the enterprise BI era: when revenue was defined in LookML, it was defined once. A change to the revenue definition propagated automatically to every downstream report. There was no way to build a “unofficial” revenue figure that bypassed the LookML model the model was the only path to the data.

The ELT Paradigm Shift

The Modern Data Stack’s central architectural departure from its predecessor was the inversion of the transformation step. Traditional ETL transformed data before loading. ELT loads data first and transforms it inside the warehouse using the warehouse’s own compute SQL executed at scale. This shift was made economically viable by the elastic, cheap compute of cloud warehouses and made practically accessible by dbt’s SQL-first transformation framework.

ELT had cascading implications. It reduced the complexity of the ingestion layer (no transformation logic in the ingestion tool). It moved transformation logic into a version-controlled, testable, documented codebase. It made transformation faster cloud warehouse SQL engines are highly optimised. And it made transformation accessible to a broader population: analysts who knew SQL could now contribute to the transformation layer without learning Java or proprietary ETL scripting languages.

---

References

1. Handy, T. (2019). The Analyst in the Modern Data Stack. dbt Labs Blog.
2. dbt Labs (2020). dbt Documentation: How dbt works.
3. Fivetran (2020). The Modern Data Stack: An Introduction. Fivetran Blog.
4. Google Cloud (2019). Google acquires Looker, expanding its multi-cloud analytics platform.
5. Stancil, B. (2021). The Modern Data Stack: Past, Present, and Future. Mode Analytics Blog.
6. Airbyte (2021). Airbyte Open-Source ELT Documentation.
7. Moses, B. & Gavish, L. (2021). Fundamentals of Data Observability. O’Reilly Media.