dbt Core turns warehouse SQL into a reviewed dependency graph

01The central move is to treat SQL as a build graph

dbt Core lets analysts and engineers define models primarily as select statements. It compiles project code, resolves dependencies, and asks the configured data platform to materialize the results as views, tables, incremental relations, or other adapter-supported forms. The data generally stays in the warehouse. dbt supplies the project system around the transformation rather than becoming a second database engine.

That boundary explains the repository's value. A transformation gains a file path, name, dependencies, configuration, tests, documentation, and version history. Changes can pass through pull requests instead of being edited directly in a dashboard. It also explains the limit: a syntactically valid model can be expensive, logically wrong, or built from unreliable source data. dbt makes analytical work inspectable and repeatable. It does not decide the grain, business definition, privacy rule, or acceptable warehouse bill.

02ref and source make lineage executable

The ref() function names another project resource and creates a dependency edge. dbt resolves the relation appropriate to the current target and builds upstream models before downstream consumers. source() names declared raw inputs and lets teams attach freshness expectations, descriptions, and tests to their boundary. These functions are preferable to scattered hard-coded database names because they give the compiler enough information to construct a directed acyclic graph.

A graph is useful only when its edges reflect reality. A macro that queries an undeclared table, a dashboard that bypasses curated models, or a copied relation name can create hidden dependencies. Cycles also reveal a modeling problem that no build order can solve. Keep source declarations near ownership information, use ref for model relationships, and inspect the manifest when behavior is surprising. Lineage should help answer who is affected by a change, not merely produce an attractive diagram after deployment.

03Materialization chooses the storage and refresh contract

A view stores query logic and evaluates it when read. A table stores results and must be rebuilt to reflect new inputs. An incremental model processes selected new or changed rows according to project logic. Ephemeral models can be inlined into dependents rather than created as database relations. Adapters can support additional materializations. The choice affects freshness, query latency, storage, rebuild time, and the complexity of correcting history.

Incremental models deserve particular skepticism because they contain two programs: the initial build and later updates. A timestamp predicate can miss late records, mutable source rows can leave stale output, and a changed transformation may apply only to future batches. Unique keys and merge strategies differ by adapter. Every incremental model needs a full-refresh plan, a late-data policy, and a comparison that proves incremental results match a clean rebuild. Use incremental work to solve a measured scale problem, not as a reflex for every large-looking table.

04Tests are queries with an operating consequence

A test suite needs severity and ownership. Testing uniqueness on a model without defining its grain can create a false sense of rigor, while scanning a huge relation after every small commit can create unsustainable cost. Some conditions should warn, some should stop publication, and some belong at source ingestion rather than downstream. Preserve failing rows or enough diagnostic context to investigate without exposing sensitive data. A green run means the declared checks passed on that data at that time, not that every analytical conclusion is correct.

05Jinja and macros are power with a readability tax

dbt uses Jinja templating to parameterize SQL and macros to reuse logic. This can remove repetitive unions, generate consistent schemas, wrap adapter differences, or centralize a careful convention. Packages distribute reusable macros and models across projects. The compiler then renders ordinary SQL for the target platform, and the compiled output is often the best debugging artifact when a template behaves unexpectedly.

Excessive abstraction makes an analytical model harder to review than the repeated SQL it replaced. A macro can hide joins, filters, dynamic relation access, or warehouse-specific behavior behind a friendly name. Package upgrades can change generated SQL across a large graph. Keep macros small, document inputs and output shape, pin package versions, and review dependencies just like application libraries. Prefer direct SQL for one-off logic. Reuse is valuable when it preserves meaning, not when it demonstrates that a template language can generate anything.

06Artifacts make a run available to other systems

dbt Core writes JSON artifacts such as the manifest, run results, sources, and catalog-related outputs according to the command and version. The manifest describes project resources, configurations, and dependencies. Run results record execution outcomes and timing. These files let documentation, lineage, CI analysis, and external tooling reason about what was parsed or executed without scraping terminal text.

Artifact schemas are versioned interfaces. A consumer should validate the schema version it understands and avoid depending on incidental fields. Artifacts can expose database names, compiled SQL, descriptions, metadata, and invocation details, so publication requires a privacy review. Keep the artifacts from a production run with its code revision and target context when they support audit or debugging. Do not assume that a newly generated documentation site represents the same revision and environment as the tables users are currently querying.

07State selection can make CI faster without making it simple

State-aware selection compares a current project with a prior manifest, enabling selectors for modified resources and related graph nodes. Deferral can resolve references to relations from another environment for resources not built locally. Together, these features support slim CI patterns where a pull request builds changed work in an isolated schema while relying on stable upstream production objects.

The comparison artifact becomes part of correctness. Using a manifest from the wrong branch, target, or deployment can select the wrong graph. A changed macro may affect many models even when their SQL files are untouched, and environment-specific data can hide a failure. Define where the baseline manifest comes from, include downstream consumers when the contract changes, and periodically run the full graph. Faster feedback is useful only when the selection policy is explicit enough for a reviewer to know what did not run.

08Production requires more than dbt run on a schedule

dbt Core is a command-line engine, not by itself a complete managed scheduler, secrets service, multi-user IDE, or incident system. An orchestrator or CI platform must supply timing, retries, concurrency control, credentials, environment isolation, logs, alerts, and artifact retention. Selection syntax can divide a graph into jobs, but overlapping schedules may contend in the warehouse or publish downstream tables before a coherent upstream run finishes.

Use separate development and production targets, least-privilege warehouse roles, deterministic dependency installation, and explicit timeouts. Record invocation parameters because one project can behave differently under another target, variable, selector, or full-refresh flag. Estimate compute from scans and rebuilds, then alert on both failure and surprising success duration. A completed command can still have warnings, stale sources, or an unexpectedly small selection. The deployable unit is the project revision plus its adapter, packages, profile policy, and invocation, not SQL files alone.

09Core, adapters, packages, and the dbt platform are distinct products

The dbt-labs/dbt repository, renamed from dbt-labs/dbt-core, is licensed under Apache-2.0. Its main branch now holds dbt v2.0, a ground-up rewrite in Rust, while the v1 Python implementation continues on the 1.latest branch. The default dbt v2 distribution adds dbt-specific customizations under a separate dbt product license. With v1, a usable installation also needs an adapter that implements behavior for the target data platform, and those adapters are separately versioned packages that can move compatibility boundaries independently. Core v2.0 bundles its adapter layer. Community packages and their transitive dependencies have their own maintainers and licenses. Pin a tested set, read release notes, and upgrade through a representative project rather than updating the word dbt as if it named one binary.

dbt Labs also offers the hosted dbt platform and the dbt platform CLI with collaboration and managed-service capabilities outside the promise of this repository. Documentation can cover both experiences, so feature availability should be checked against Core before it enters a design. The distinction is not an argument for one edition. It is procurement and architecture hygiene. Evaluate the Apache-licensed engine on what it runs locally, then evaluate hosted IDE, scheduling, metadata, semantic, and enterprise features under their current plans and terms.