Connecting dbt projects to Census
Setting it up
- Connect to your GitHub repository. We currently only support dbt projects stored in GitHub. If you’d prefer to use a different service, please let us know!
- Select the branch (if any) you’d like Census to use. Census will refresh the project on a regular basis and detect any changes to your models. You can force a refresh at any point from the models' page.
- Customize the Census model selector. Any model exposed to Census becomes available as a source for syncing your data to external tools. By default, Census looks for models with the
censustag but you can customize the filter.
- All models with a tag:
- All models in a directory:
- All models:
- Finally, specify where intermediate models are materialized. If the models you expose in Census have dependencies, we will attempt to use these materialized tables. You may need to ensure that our database connection has read access to these tables.
Once you’ve configured your project repository, Census will analyze your project and display the models you’ve made available. You’re now ready to start using these models as part of Census syncs!
Managing data warehouse permissions
Census doesn't necessarily require the same permissions your dbt project needs because Census only runs the models you've exposed to Census during set up. Census only requires read access to your selected models and any of their materialized dependencies. That means you can use dbt's materialize configuration flag to create permissions boundaries. Once materialized dependencies are generated by dbt runner, Census will reference the materialized results when accessing your models.
Features not currently supported
If you’re using dbt, you likely already have a dbt runner so Census doesn’t run your dbt project in dbt terminology, it simply does the compile step. As a result, several dbt features are not available with Census(please let us know what you’d like to see as we’re always improving the product).
- Materialization directives. Census doesn’t currently materialize your tables back to your data warehouse. Census will however use materialized tables by your dbt runner to speed up the execution
- Pre and post hooks
- Non-public packages