Using dbt with Dagster, part one: Set up the dbt project#

The first part of this tutorial doesn't involve Dagster at all. You'll get a dbt project running on your computer.

In this part, you'll:

• Download a dbt project
• Configure your dbt project to run with DuckDB
• Build your dbt project

Step 1: Download the sample dbt project#

Let's get started by downloading a sample dbt project. We'll use the standard dbt Jaffle Shop example.

1. First, create a folder that will ultimately contain both your dbt project and Dagster code.

   mkdir tutorial-dbt-dagster
   

2. Then, navigate into that folder:

   cd tutorial-dbt-dagster
   

3. Finally, download the sample dbt project into that folder.

   git clone https://github.com/dbt-labs/jaffle_shop.git
   

Step 2: Configure your dbt project to run with DuckDB#

Running dbt requires a data warehouse to store the tables that are created from the dbt models. For our data warehouse, we'll use DuckDB, because setting it up doesn't require any long-running services or external infrastructure.

You'll set up dbt to work with DuckDB by configuring a dbt profile:

1. Navigate into the jaffle_shop folder, which was created when you downloaded the project, inside your tutorial-dbt-dagster folder:

   cd jaffle_shop
   

2. In this folder, with your text editor of choice, create a file named profiles.yml and add the following code to it:

   jaffle_shop:
     target: dev
     outputs:
       dev:
         type: duckdb
         path: tutorial.duckdb
         threads: 24
   

Step 3: Build your dbt project#

With the profile configured above, your dbt project should now be usable. To test it out, run:

dbt build

This will run all the models, seeds, and snapshots in the project and store a set of tables in your DuckDB database.

What's next?#

At this point, you should have a fully-configured dbt project that's ready to work with Dagster. The next step is to load the dbt models into Dagster as assets.