Data Pipeline in a Box (Airflow + dbt)

Overview: From raw events to analytics-ready marts with Airflow, dbt, and Great Expectations.
Key Capabilities: Ingestion DAGs (batch + optional CDC), retries, SLAs, lineage; dbt staging→core→marts with tests & docs;
Great Expectations suites with Slack/webhook alerts; Terraform samples for schedulers/secrets; backfill/late data handling.
Architecture: Modular Airflow DAGs; pluggable warehouse profiles (Postgres/Redshift/BigQuery/ClickHouse);
dbt models with snapshots and documented contracts; DQ/observability via GX and metadata export.
Security & Compliance: Least-privilege IAM; encrypted connections; dataset ACLs; secrets in Connections/Vault; audit logs.
Performance & Ops: Partitioning & clustering strategies; parallelization; cost-aware materializations; example SLAs.
Quick Start: terraform apply (optional) → configure profiles.yml and Airflow connections → compose up → dbt seed/run/test → enable DAGs.
Deliverables: DAGs, dbt project, GX suites, Terraform, runbooks (incidents, backfills, cost).