Project Structure

Created: May 20, 2021, Updated: January 26, 2024

This chapter contains all the information about what your Bizzflow project should look like in your git repository.

A Bizzflow project is basically a directory structure containing configuration files (JSON or YAML and SQL files). Typical project structure should look somewhat like this:

.
├── dags/
|   └── custom_dag.py
├── extractors/
    ├── hubspot.json
    └── root_mysql.json
├── transformations/
    └── main/
      ├── 00_init_invoices.sql
      ├── 05_init_deals.sql
      ├── 10_process_invoices.sql
      └── 20_process_deals.sql
├── datamarts.json
├── orchestrations.json
├── project.json
├── step.json
├── transformations.json
├── notifications.json
├── datasets.json
├── writers/
    └── hubspot.json
└── .bizzflow_project

A Bizzflow project

Below you will find each of the directory’s and file’s description in order to understand what the structure means for your project.

dags/

dags is a directory that is empty by default. It is a place you can put your custom Airflow DAGs to and they will appear in your Airflow UI.

extractors/

This directory will contain all your extractors' configuration files. They are always JSON / YAML files with keys specified in a separate Extractors chapter.

transformations/

With combination of transformations.json (or transformations.yaml) file this directory makes it possible for you to add transformations to your project. While the configuration file contains transformation’s metadata, this is the directory where you will put the transformation SQL files into. See more in Transformations.

datamarts.json

See Datamarts for more information.

orchestrations.json

See Orchestrations for more information.

project.json

See General Project Configuration for more information.

step.json

See Step for more information.

transformations.json

With combination of transformations/ directory this file makes it possible for you to add transformations to your project. While the configuration file contains transformation’s metadata, the directory will contain the transformation SQL files. See more in Transformations.

datasets.json

See Datasets for more information.

notifications.json

See Notifications for more information.

writers/

This directory will contain all your writers' configuration files. They are always JSON / YAML files with keys specified in a separate Writers chapter.

.bizzflow_project

This is a file that servers for Bizzflow to recognize the repository as a Bizzflow project. You do not need to worry about this file at all.