Skip to content

ranggaaldosas/etl-fraud-metrics

Repository files navigation

Project ETL Fraud

End-to-end data pipeline with write-audit-publish (WAP) pattern

  • Data Stack
    • Orchestration = Apache Airlow
    • Connector = Airbyte
    • Data Transformation = dbt
    • Data Quality Platform = Soda
    • Database = Postgre & MySQL & AWS S3
    • OLAP/Data Warehouse = Snowflake

A. System Design

system_design

B. Airflow Graph

flow

C. Data Quality Check

soda

D. Snowflake Data Warehouse

dwh

About

End-to-end data pipeline with write-audit-publish (WAP) pattern

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published