Fivetran's starter project which acts as the foundation to building dbt packages Here's a template of the README for both transformation + source packages
This package models [connector name] data from Fivetran's connector. It uses data in the format described by [this ERD](link to the connector ERD).
[High level objective of package]. It achieves this by:
- [major thing the package does #1]
- [ #2 ]
- [ #3] ...
Please be aware the dbt_connector and dbt_connector_source packages will only work with the Fivetran connector schema released after [some date]. If your Jira connector was set up prior to [some date], you will need to fully resync or set up a new [connector name] connector in order for the Fivetran dbt [connector] packages to work.
This package contains transformation models, designed to work simultaneously with our [Connector source package](link to source package repo). A dependency on the source package is declared in this package's packages.yml
file, so it will automatically download when you run dbt deps
. The primary outputs of this package are described below. Intermediate models are used to create these output models.
model | description |
---|---|
[conector__model_1](link to model sql file on github) | toot toot |
[conector__model_2](link to model sql file on github) | beep beep |
This package contains staging models, designed to work simultaneously with our [Connector modeling package](link to transformation package repo). The staging models:
- Remove any rows that are soft-deleted
- Name columns consistently across all packages:
- Boolean fields are prefixed with
is_
orhas_
- Timestamps are appended with
_at
- ID primary keys are prefixed with the name of the table. For example, a user table's ID column is renamed user_id.
- Foreign keys include the table that they refer to. For example, a project table's owner ID column is renamed owner_user_id.
- Boolean fields are prefixed with
- [anything else?]
Check dbt Hub for the latest installation instructions, or read the dbt docs for more information on installing packages.
By default, this package looks for your [Connector] data in the [connector_name]
schema of your target database. If this is not where your [Connector] data is, add the following configuration to your dbt_project.yml
file:
# dbt_project.yml
...
config-version: 2
vars:
connector_database: your_database_name
connector_schema: your_schema_name
[brief explanation]
If you want to [do something], add the following variable to your dbt_project.yml
file:
# dbt_project.yml
...
config-version: 2
vars:
connector:
CONNECTOR__example_list_variable: ['the', 'list', 'of', 'values']
By default this package will build the CONNECTOR staging models within a schema titled (<target_schema> + _stg_CONNECTOR
) and the CONNECTOR final models with a schema titled (<target_schema> + _CONNECTOR
) in your target database. If this is not where you would like your modeled CONNECTOR data to be written to, add the following configuration to your dbt_project.yml
file:
# dbt_project.yml
...
models:
CONNECTOR:
+schema: my_new_schema_name # leave blank for just the target_schema
CONNECTOR_source:
+schema: my_new_schema_name # leave blank for just the target_schema
Don't see a model or specific metric you would have liked to be included? Notice any bugs when installing
and running the package? If so, we highly encourage and welcome contributions to this package!
Please create issues or open PRs against master
. Check out this post on the best workflow for contributing to a package.
This package has been tested on BigQuery, Snowflake and Redshift. Coming soon -- compatibility with Spark
- Provide feedback on our existing dbt packages or what you'd like to see next
- Have questions, feedback, or need help? Book a time during our office hours here or email us at solutions@fivetran.com
- Find all of Fivetran's pre-built dbt packages in our dbt hub
- Learn how to orchestrate dbt transformations with Fivetran here
- Learn more about Fivetran overall in our docs
- Check out Fivetran's blog
- Learn more about dbt in the dbt docs
- Check out Discourse for commonly asked questions and answers
- Join the chat on Slack for live discussions and support
- Find dbt events near you
- Check out the dbt blog for the latest news on dbt's development and best practices