Advanced · Multi-Source Data Engineering Platform

Nimbus — Medallion Data Platform

A full Bronze → Silver → Gold pipeline over four independent sources (orders, CRM, product catalog, clickstream). Runs a 14-task orchestrated DAG with data-quality gating, SCD Type 2 dimension history, point-in-time joins, sessionization, RFM segmentation, cohort retention, and anomaly detection — entirely client-side.

Medallion Architecture SCD Type 2 Point-in-Time Join Sessionization RFM Segmentation Cohort Retention Z-Score Anomaly Detection DAG Orchestration + Retry
Orchestration — 14 Tasks · 3 Layers
BRONZEraw ingest
ingest_ordersPENDING
source: orders_stream (Web/App/Marketplace)
ingest_customersPENDING
source: crm_customers (3 point-in-time snapshots)
ingest_productsPENDING
source: product_catalog
ingest_clickstreamPENDING
source: web_events (raw, unsessionized)
SILVERcleaned + modeled
dq_validatePENDING
needs: all 4 bronze tasks
build_dim_customer_scd2PENDING
needs: ingest_customers, dq_validate
build_dim_productPENDING
needs: ingest_products, dq_validate
build_fact_ordersPENDING
needs: ingest_orders + both dims (as-of join)
sessionize_clickstreamPENDING
needs: ingest_clickstream, dq_validate
GOLDanalytics-ready
compute_rfm_segmentsPENDING
needs: build_fact_orders
compute_cohort_retentionPENDING
needs: build_fact_orders
detect_revenue_anomaliesPENDING
needs: build_fact_orders
build_conversion_funnelPENDING
needs: sessionize_clickstream, build_fact_orders
publish_gold_warehousePENDING
needs: all 4 gold compute tasks

DATA QUALITY RULE ENGINE

RuleTableResult
Run the pipeline to evaluate rules.

PIPELINE LOGS

Idle. Click RUN FULL PIPELINE to start the job…

DIM_CUSTOMER_SCD2 — SAMPLE VERSION HISTORY

Slowly Changing Dimension (Type 2): every attribute change on a customer (tier upgrade, city move) opens a new version instead of overwriting history, so build_fact_orders can join each order to the customer attributes that were true on the order date — not today's attributes.
customer_idtiercityvalid_fromvalid_tocurrent
Run the pipeline to build dimension history.
Gold Layer — Analytics Dashboard
Total Revenue
$0
—
Fact Orders Loaded
0
—
Scored Customers (RFM)
0
—
Revenue Anomalies
0
—
Session → Purchase
0%
—

DAILY REVENUE — Z-SCORE ANOMALY DETECTION

RFM SEGMENTATION — RECENCY × FREQUENCY × MONETARY

COHORT RETENTION — % ACTIVE BY MONTHS SINCE FIRST ORDER

CLICKSTREAM → PURCHASE FUNNEL

WAREHOUSE EXPLORER

Run the pipeline to populate the warehouse.

DATA LINEAGE

Select a Gold table above to trace its lineage back to raw sources.