feat: dev_run.sh — auto-run pipeline on first startup

On the first `./scripts/dev_run.sh` invocation (serving.duckdb absent),
automatically run extract → transform → export_serving from the repo root
so the dashboard is populated without any manual steps.

Subsequent runs skip the pipeline for a fast startup. Delete serving.duckdb
from the repo root to force a full pipeline re-run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-02-22 11:15:34 +01:00
parent cb799ff019
commit ac8ab47448

View File

@@ -54,6 +54,35 @@ info "Building CSS"
make css-build
ok "CSS built"
# -- Pipeline (first-time only) ----------------------------------------------
# Runs extract → transform → export_serving from the repo root if serving.duckdb
# does not exist yet. Subsequent dev_run.sh invocations skip this — delete
# serving.duckdb from the repo root to force a full re-run.
REPO_ROOT="$(cd .. && pwd)"
PIPELINE_LANDING="$REPO_ROOT/data/landing"
PIPELINE_DUCKDB="$REPO_ROOT/local.duckdb"
PIPELINE_SERVING="$REPO_ROOT/serving.duckdb"
if [ ! -f "$PIPELINE_SERVING" ]; then
info "First run — fetching and transforming data (this may take a few minutes)"
if (
cd "$REPO_ROOT"
LANDING_DIR="$PIPELINE_LANDING" DUCKDB_PATH="$PIPELINE_DUCKDB" \
uv run materia pipeline run extract &&
LANDING_DIR="$PIPELINE_LANDING" DUCKDB_PATH="$PIPELINE_DUCKDB" \
uv run materia pipeline run transform &&
DUCKDB_PATH="$PIPELINE_DUCKDB" SERVING_DUCKDB_PATH="$PIPELINE_SERVING" \
uv run materia pipeline run export_serving
); then
ok "Pipeline complete — dashboard ready"
else
warn "Pipeline did not complete — dashboard will show empty data"
fi
else
ok "Data ready (delete $PIPELINE_SERVING to force a pipeline re-run)"
fi
# -- Process management ------------------------------------------------------
PIDS=()