fix(supervisor): use sqlmesh plan --auto-apply instead of run

'run' requires the prod environment to already exist. 'plan --auto-apply' initializes the environment on first run and applies pending changes on subsequent runs — fully self-healing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(supervisor): run sqlmesh against prod environment
2026-02-27 15:40:37 +01:00 · 2026-02-27 15:39:55 +01:00 · 2026-02-27 15:39:39 +01:00 · 2026-02-27 15:23:12 +01:00 · 2026-02-27 14:03:40 +01:00 · 2026-02-27 13:59:50 +01:00
18 changed files with 1223 additions and 31 deletions
--- a/.env.prod.sops
+++ b/.env.prod.sops
@@ -53,8 +53,12 @@ BING_SITE_URL=ENC[AES256_GCM,data:M33VI97DyxH8gRR3ZUXoXg4QrEv5og==,iv:GxZtwfbBVi
 #ENC[AES256_GCM,data:OTUMKNkRW0zrupNppXthwE1oieILhNjM+cjx5hFn69g=,iv:48ID2qtSe9ggD2X+G/iUqp3v2uwEc7fZw8lxHIvVXmk=,tag:okBn0Npk1K9dDOFWA/AB1A==,type:comment]
 GEONAMES_USERNAME=ENC[AES256_GCM,data:UXd/S2TzXPiGmLY=,iv:OMURM5E6SFEsaqroUlH76DEnr7C/ujNk9UQnbWT0hK4=,tag:VsjjS12QDbudiEhdAQ/OCQ==,type:str]
 CENSUS_API_KEY=ENC[AES256_GCM,data:9RbKlxSD17LqIuuNXaOKSgZ8LnFh9Wbze3XHgpctfV/1TqBMZTIedQ==,iv:WwsmR3HLUEcgUpLliGRaUPhGM9vFNPMGXSAQQ6+9UVc=,tag:R4EMNy5MxxvK0UTaCL0umA==,type:str]
-sops_age__list_0__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBqck9GdHVkUmIzNnlvMW5k\nVkNtazZ0ZytzZ25vMU5SckdFLzcrTFNYOVZZCmNjbU9yV0lTRlB5cEpMVC81QTdu\nS2ZDc0ZkNnRBNFhFMEN1bjY3YVhwZEEKLS0tIGE5TEdYenVOV1IwcE0wYnlKNElF\ncXV1K0xuczZzZ3JnL1lrSC9QWHIwNGsKfW4ARke6Cj83BpQc8weayL3v8SVgQ+Fp\n99aVWp103O1fumksR1w4u0X7fSNRrgAmpY/yyZuEvsoIY8ELFVcqgQ==\n-----END AGE ENCRYPTED FILE-----\n
+sops_age__list_0__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBaUVk0UEVqdmtsM3VzQnpZ\nZjJDZ1lsM0VqWFpVVXUvNzdQcCtHbVJLNjFnCmhna01vTkVBaFQ5ZVlXeGhYNXdH\ncWJ5Qi9PdkxLaHBhQnR3cmtoblkxdEUKLS0tIDhHamY4NXhxOG9YN1NpbTN1aVRh\nOHVKcEN1d0QwQldVTDlBWUU4SDVDWlUKRJU+CTfTzIx6LLKin9sTXAHPVAfiUerZ\nCqYVFncsCJE3TbMI424urQj7kragPoGl1z4++yqAXNTRxfZIY4KTkg==\n-----END AGE ENCRYPTED FILE-----\n
 sops_age__list_0__map_recipient=age1f5002gj4s78jju45jd28kuejtcfhn5cdujz885fl7z2p9ym68pnsgky87a
+sops_age__list_1__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBmVEticFRVemlzZnlzek4x\nbWJ0d0h5ejJVUk5remo1VkdxNjVpdllqbFhFClc1UXlNd09xVVA5MnltMlN5MWRy\nYUlNRmNybHh1RGdPVC9yWlYrVmRTdkkKLS0tIHBUbU9qSDMrVGVHZDZGSFdpWlBh\nT3NXTGl0SmszaU9hRmU5bXI0cDRoRW8KLvbNYsBEwz+ITKvn7Yn+iNHiRzyyjtQt\no9/HupykJ3WjSdleGz7ZN6UiPGelHp0D/rzSASTYaI1+0i0xZ4PUoQ==\n-----END AGE ENCRYPTED FILE-----\n
+sops_age__list_1__map_recipient=age1wjepykv3glvsrtegu25tevg7vyn3ngpl607u3yjc9ucay04s045s796msw
+sops_age__list_2__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBFeHhaOURNZnRVMEwxNThu\nUjF4Q0kwUXhTUE1QSzZJbmpubnh3RnpQTmdvCjRmWWxpNkxFUmVGb3NRbnlydW5O\nWEg3ZXJQTU4vcndzS2pUQXY3Q0ttYjAKLS0tIE9IRFJ1c2ZxbGVHa2xTL0swbGN1\nTzgwMThPUDRFTWhuZHJjZUYxOTZrU00KY62qrNBCUQYxwcLMXFEnLkwncxq3BPJB\nKm4NzeHBU87XmPWVrgrKuf+PH1mxJlBsl7Hev8xBTy7l6feiZjLIvQ==\n-----END AGE ENCRYPTED FILE-----\n
+sops_age__list_2__map_recipient=age1c783ym2q5x9tv7py5d28uc4k44aguudjn03g97l9nzs00dd9tsrqum8h4d
 sops_lastmodified=2026-02-26T14:32:28Z
 sops_mac=ENC[AES256_GCM,data:pyHQHwTtjh7OLiMqbqhUjfrmetEtYS7yB342C/TWfDCwEotWLVwnGWlC4+HIl53pw9+3AgoBVRnW0t86e4kG9O8KyHnk68S9qBcpUsybW3lyGPNXmBydv1W9gQHuK8f/4WGIbkhNxyIToKg9ZAmYWFxNhRKSoYKm5P9Uh7B7CF4=,iv:syrX8VdL3JsDsawvFWbX04Ygcr18hjSSHfEwHkyKETk=,tag:qrhWkh/e+21OKGU2+rCeyg==,type:str]
 sops_unencrypted_suffix=_unencrypted
--- a/.gitignore
+++ b/.gitignore
@@ -61,3 +61,4 @@ web/src/padelnomics/static/css/output.css

 # Generated report PDFs (built locally via make report-pdf, not committed)
 data/content/reports/_build/
+_serving_meta.json
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -17,7 +17,9 @@ test:

 tag:
  stage: tag
-  image: alpine/git
+  image:
+    name: alpine/git
+    entrypoint: [""]
  script:
    - git tag "v${CI_PIPELINE_IID}"
    - git push "https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git" "v${CI_PIPELINE_IID}"
--- a/.sops.yaml
+++ b/.sops.yaml
@@ -1,3 +1,3 @@
 creation_rules:
  - path_regex: \.env\..+\.sops$
-    age: age1f5002gj4s78jju45jd28kuejtcfhn5cdujz885fl7z2p9ym68pnsgky87a,age1wjepykv3glvsrtegu25tevg7vyn3ngpl607u3yjc9ucay04s045s796msw
+    age: age1f5002gj4s78jju45jd28kuejtcfhn5cdujz885fl7z2p9ym68pnsgky87a,age1wjepykv3glvsrtegu25tevg7vyn3ngpl607u3yjc9ucay04s045s796msw,age1c783ym2q5x9tv7py5d28uc4k44aguudjn03g97l9nzs00dd9tsrqum8h4d
--- a/infra/setup_server.sh
+++ b/infra/setup_server.sh
@@ -40,7 +40,7 @@ fi

 log "Creating directories..."
 mkdir -p "${APP_DIR}" "${DATA_DIR}/landing"
-chown "${SERVICE_USER}:${SERVICE_USER}" "${APP_DIR}"
+chown -R "${SERVICE_USER}:${SERVICE_USER}" "${APP_DIR}"
 chown -R "${SERVICE_USER}:${SERVICE_USER}" "${DATA_DIR}"

 # ── Docker ────────────────────────────────────────────────────────────────────
--- a/scratch/lineage-ux-prototype.html
+++ b/scratch/lineage-ux-prototype.html
@@ -0,0 +1,860 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>Lineage UX Prototype</title>
+<link rel="preconnect" href="https://fonts.googleapis.com">
+<link href="https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;600&family=DM+Mono:wght@400;500&display=swap" rel="stylesheet">
+<style>
+  *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+
+  :root {
+    --green-bg: #F0FDF4; --green-border: #BBF7D0; --green-accent: #16A34A;
+    --green-fill: #DCFCE7; --green-text: #14532D;
+    --blue-bg: #EFF6FF; --blue-border: #BFDBFE; --blue-accent: #1D4ED8;
+    --blue-fill: #DBEAFE; --blue-text: #1E3A8A;
+    --amber-bg: #FFFBEB; --amber-border: #FDE68A; --amber-accent: #D97706;
+    --amber-fill: #FEF3C7; --amber-text: #78350F;
+    --slate-50: #F8FAFC; --slate-100: #F1F5F9; --slate-200: #E2E8F0;
+    --slate-400: #94A3B8; --slate-500: #64748B; --slate-700: #334155;
+    --slate-800: #1E293B; --slate-900: #0F172A;
+    --panel-w: 340px;
+    --font-sans: 'DM Sans', ui-sans-serif, system-ui, sans-serif;
+    --font-mono: 'DM Mono', ui-monospace, monospace;
+  }
+
+  body {
+    font-family: var(--font-sans);
+    background: var(--slate-50);
+    color: var(--slate-800);
+    min-height: 100vh;
+    display: flex;
+    flex-direction: column;
+  }
+
+  /* ── Page shell ── */
+  .page-header {
+    background: white;
+    border-bottom: 1px solid var(--slate-200);
+    padding: 0.75rem 1.5rem;
+    display: flex;
+    align-items: center;
+    gap: 1rem;
+  }
+  .page-header h1 { font-size: 0.875rem; font-weight: 600; color: var(--slate-800); }
+  .badge {
+    font-size: 0.6875rem; font-weight: 500; padding: 2px 7px;
+    border-radius: 99px; background: var(--slate-100); color: var(--slate-500);
+    letter-spacing: 0.02em;
+  }
+  .hint {
+    font-size: 0.75rem; color: var(--slate-400); margin-left: auto;
+  }
+
+  .workspace {
+    display: flex;
+    flex: 1;
+    overflow: hidden;
+    position: relative;
+  }
+
+  /* ── DAG canvas ── */
+  .canvas-wrap {
+    flex: 1;
+    overflow: auto;
+    padding: 1.5rem;
+    transition: margin-right 0.22s cubic-bezier(0.4,0,0.2,1);
+  }
+  .canvas-wrap.panel-open { margin-right: var(--panel-w); }
+
+  .card {
+    background: white;
+    border: 1px solid var(--slate-200);
+    border-radius: 10px;
+    overflow: hidden;
+  }
+  .card-header {
+    padding: 0.75rem 1rem;
+    font-size: 0.8125rem;
+    font-weight: 600;
+    border-bottom: 1px solid var(--slate-100);
+    color: var(--slate-700);
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+  }
+  .svg-wrap { padding: 1rem 0.75rem 0.75rem; overflow-x: auto; }
+
+  /* ── Lineage SVG node/edge states ── */
+  .lineage-node { cursor: pointer; }
+  .lineage-node rect:first-child { transition: filter 0.1s; }
+  .lineage-node:hover rect:first-child { filter: brightness(0.93); }
+  .lineage-node.selected rect:first-child { filter: brightness(0.88); }
+  .lineage-edge { transition: stroke 0.12s, stroke-width 0.12s, opacity 0.12s; }
+  .lineage-edge.hi { stroke: #1D4ED8 !important; stroke-width: 2 !important; opacity: 1 !important; }
+  .lineage-edge.dim { opacity: 0.1; }
+
+  /* ── Hover tooltip ── */
+  #tooltip {
+    position: fixed;
+    z-index: 100;
+    pointer-events: none;
+    opacity: 0;
+    transition: opacity 0.1s;
+    filter: drop-shadow(0 4px 12px rgba(0,0,0,0.12));
+  }
+  #tooltip.visible { opacity: 1; }
+  .tt-box {
+    background: var(--slate-900);
+    color: white;
+    border-radius: 8px;
+    padding: 10px 12px;
+    min-width: 200px;
+    max-width: 260px;
+  }
+  .tt-header {
+    display: flex;
+    align-items: center;
+    gap: 6px;
+    margin-bottom: 8px;
+    padding-bottom: 7px;
+    border-bottom: 1px solid rgba(255,255,255,0.08);
+  }
+  .tt-name {
+    font-family: var(--font-mono);
+    font-size: 0.6875rem;
+    font-weight: 500;
+    color: white;
+    flex: 1;
+    min-width: 0;
+    overflow: hidden;
+    text-overflow: ellipsis;
+    white-space: nowrap;
+  }
+  .tt-layer {
+    font-size: 0.5625rem;
+    font-weight: 600;
+    letter-spacing: 0.08em;
+    padding: 2px 6px;
+    border-radius: 4px;
+    flex-shrink: 0;
+    text-transform: uppercase;
+  }
+  .tt-layer.staging  { background: rgba(22,163,74,0.25); color: #86EFAC; }
+  .tt-layer.foundation { background: rgba(29,78,216,0.3); color: #93C5FD; }
+  .tt-layer.serving  { background: rgba(217,119,6,0.25); color: #FCD34D; }
+
+  .tt-row {
+    display: flex;
+    align-items: baseline;
+    gap: 6px;
+    padding: 2px 0;
+    font-size: 0.6875rem;
+  }
+  .tt-col { font-family: var(--font-mono); color: #CBD5E1; flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; }
+  .tt-type { font-family: var(--font-mono); color: #64748B; font-size: 0.625rem; flex-shrink: 0; }
+  .tt-more { font-size: 0.625rem; color: #64748B; margin-top: 5px; padding-top: 5px; border-top: 1px solid rgba(255,255,255,0.06); }
+
+  /* ── Detail panel ── */
+  #detail-panel {
+    position: fixed;
+    top: 0;
+    right: 0;
+    bottom: 0;
+    width: var(--panel-w);
+    background: white;
+    border-left: 1px solid var(--slate-200);
+    display: flex;
+    flex-direction: column;
+    transform: translateX(100%);
+    transition: transform 0.22s cubic-bezier(0.4,0,0.2,1);
+    z-index: 50;
+    overflow: hidden;
+  }
+  #detail-panel.open { transform: translateX(0); }
+
+  .panel-top {
+    padding: 0.875rem 1rem;
+    border-bottom: 1px solid var(--slate-100);
+    display: flex;
+    flex-direction: column;
+    gap: 0.375rem;
+    flex-shrink: 0;
+  }
+  .panel-title-row {
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+  }
+  .panel-model-name {
+    font-family: var(--font-mono);
+    font-size: 0.8125rem;
+    font-weight: 500;
+    color: var(--slate-800);
+    flex: 1;
+    min-width: 0;
+    overflow: hidden;
+    text-overflow: ellipsis;
+  }
+  .panel-close {
+    background: none;
+    border: none;
+    cursor: pointer;
+    color: var(--slate-400);
+    padding: 2px;
+    border-radius: 4px;
+    line-height: 1;
+    font-size: 1rem;
+    transition: color 0.1s, background 0.1s;
+  }
+  .panel-close:hover { color: var(--slate-700); background: var(--slate-100); }
+
+  .panel-meta {
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+    font-size: 0.6875rem;
+    color: var(--slate-500);
+  }
+  .meta-chip {
+    padding: 2px 7px;
+    border-radius: 99px;
+    font-size: 0.625rem;
+    font-weight: 600;
+    letter-spacing: 0.05em;
+    text-transform: uppercase;
+  }
+  .meta-chip.staging  { background: var(--green-fill); color: var(--green-text); border: 1px solid var(--green-border); }
+  .meta-chip.foundation { background: var(--blue-fill); color: var(--blue-text); border: 1px solid var(--blue-border); }
+  .meta-chip.serving  { background: var(--amber-fill); color: var(--amber-text); border: 1px solid var(--amber-border); }
+
+  .panel-body {
+    flex: 1;
+    overflow-y: auto;
+    padding: 0;
+  }
+
+  .panel-section {
+    border-bottom: 1px solid var(--slate-100);
+    padding: 0.75rem 1rem;
+  }
+  .panel-section:last-child { border-bottom: none; }
+  .section-label {
+    font-size: 0.625rem;
+    font-weight: 700;
+    letter-spacing: 0.08em;
+    text-transform: uppercase;
+    color: var(--slate-400);
+    margin-bottom: 0.5rem;
+  }
+
+  /* Schema table */
+  .schema-table {
+    width: 100%;
+    border-collapse: collapse;
+    font-size: 0.6875rem;
+  }
+  .schema-table th {
+    text-align: left;
+    font-weight: 600;
+    color: var(--slate-500);
+    padding: 0 0 0.375rem;
+    font-size: 0.625rem;
+    letter-spacing: 0.04em;
+    text-transform: uppercase;
+  }
+  .schema-table td {
+    padding: 3px 0;
+    vertical-align: middle;
+  }
+  .schema-table tr + tr td { border-top: 1px solid var(--slate-100); }
+  .col-name { font-family: var(--font-mono); color: var(--slate-800); font-weight: 500; }
+  .col-type { font-family: var(--font-mono); color: var(--slate-400); font-size: 0.625rem; }
+  .col-null { font-size: 0.5625rem; color: var(--slate-300); text-align: right; }
+  .col-null.yes { color: var(--amber-accent); }
+
+  /* Row count */
+  .stat-row {
+    display: flex;
+    align-items: baseline;
+    gap: 0.375rem;
+  }
+  .stat-val {
+    font-family: var(--font-mono);
+    font-size: 1rem;
+    font-weight: 500;
+    color: var(--slate-800);
+  }
+  .stat-unit { font-size: 0.6875rem; color: var(--slate-500); }
+
+  /* Dep lists */
+  .dep-list { display: flex; flex-direction: column; gap: 3px; }
+  .dep-item {
+    display: flex;
+    align-items: center;
+    gap: 6px;
+    padding: 4px 7px;
+    border-radius: 5px;
+    cursor: pointer;
+    transition: background 0.1s;
+    font-family: var(--font-mono);
+    font-size: 0.6875rem;
+    color: var(--slate-700);
+  }
+  .dep-item:hover { background: var(--slate-50); }
+  .dep-dot {
+    width: 6px; height: 6px; border-radius: 50%; flex-shrink: 0;
+  }
+  .dep-dot.staging  { background: var(--green-accent); }
+  .dep-dot.foundation { background: var(--blue-accent); }
+  .dep-dot.serving  { background: var(--amber-accent); }
+
+  .empty-state { color: var(--slate-400); font-size: 0.6875rem; font-style: italic; }
+
+  /* ── "Click to explore" annotation ── */
+  .interaction-hint {
+    font-size: 0.6875rem; color: var(--slate-400);
+    display: flex; align-items: center; gap: 4px;
+  }
+  .kbd {
+    display: inline-flex; align-items: center; justify-content: center;
+    background: var(--slate-100); border: 1px solid var(--slate-200);
+    border-radius: 3px; padding: 1px 5px;
+    font-family: var(--font-mono); font-size: 0.5625rem;
+    color: var(--slate-600); line-height: 1.4;
+  }
+</style>
+</head>
+<body>
+
+<div class="page-header">
+  <h1>Data Lineage</h1>
+  <span class="badge">26 models</span>
+  <span class="badge">staging → foundation → serving</span>
+  <span class="hint interaction-hint">
+    hover to preview schema &nbsp;·&nbsp; click to inspect
+  </span>
+</div>
+
+<div class="workspace">
+  <div class="canvas-wrap" id="canvas-wrap">
+    <div class="card">
+      <div class="svg-wrap" id="svg-wrap">
+        <!-- SVG injected by JS below -->
+      </div>
+    </div>
+  </div>
+
+  <!-- Detail panel -->
+  <div id="detail-panel">
+    <div class="panel-top">
+      <div class="panel-title-row">
+        <span class="panel-model-name" id="panel-model-name">—</span>
+        <button class="panel-close" id="panel-close" title="Close">✕</button>
+      </div>
+      <div class="panel-meta">
+        <span class="meta-chip" id="panel-layer-chip">—</span>
+        <span id="panel-materialization">view</span>
+      </div>
+    </div>
+    <div class="panel-body" id="panel-body">
+      <!-- injected by JS -->
+    </div>
+  </div>
+</div>
+
+<!-- Floating tooltip -->
+<div id="tooltip">
+  <div class="tt-box" id="tt-box"></div>
+</div>
+
+<script>
+// ── Mock data (would come from DuckDB DESCRIBE in production) ──────────────
+const SCHEMA = {
+  stg_padel_courts: {
+    layer: 'staging', materialization: 'view',
+    rows: null,
+    columns: [
+      { name: 'court_id',      type: 'VARCHAR',   nullable: false },
+      { name: 'name',          type: 'VARCHAR',   nullable: true  },
+      { name: 'lat',           type: 'DOUBLE',    nullable: true  },
+      { name: 'lon',           type: 'DOUBLE',    nullable: true  },
+      { name: 'country_code',  type: 'VARCHAR',   nullable: true  },
+      { name: 'source_file',   type: 'VARCHAR',   nullable: false },
+      { name: 'extracted_at',  type: 'TIMESTAMP', nullable: false },
+    ],
+  },
+  stg_playtomic_venues: {
+    layer: 'staging', materialization: 'view', rows: null,
+    columns: [
+      { name: 'tenant_id',     type: 'VARCHAR',   nullable: false },
+      { name: 'name',          type: 'VARCHAR',   nullable: true  },
+      { name: 'city',          type: 'VARCHAR',   nullable: true  },
+      { name: 'country_code',  type: 'VARCHAR',   nullable: true  },
+      { name: 'lat',           type: 'DOUBLE',    nullable: true  },
+      { name: 'lon',           type: 'DOUBLE',    nullable: true  },
+      { name: 'sport_ids',     type: 'VARCHAR[]', nullable: true  },
+    ],
+  },
+  stg_playtomic_resources: {
+    layer: 'staging', materialization: 'view', rows: null,
+    columns: [
+      { name: 'resource_id',   type: 'VARCHAR', nullable: false },
+      { name: 'tenant_id',     type: 'VARCHAR', nullable: false },
+      { name: 'name',          type: 'VARCHAR', nullable: true  },
+      { name: 'sport_id',      type: 'VARCHAR', nullable: true  },
+    ],
+  },
+  stg_playtomic_availability: {
+    layer: 'staging', materialization: 'view', rows: null,
+    columns: [
+      { name: 'slot_id',       type: 'VARCHAR',   nullable: false },
+      { name: 'tenant_id',     type: 'VARCHAR',   nullable: false },
+      { name: 'resource_id',   type: 'VARCHAR',   nullable: false },
+      { name: 'start_at',      type: 'TIMESTAMP', nullable: false },
+      { name: 'duration_min',  type: 'INTEGER',   nullable: false },
+      { name: 'price_cents',   type: 'INTEGER',   nullable: true  },
+      { name: 'currency',      type: 'VARCHAR',   nullable: true  },
+    ],
+  },
+  stg_population: {
+    layer: 'staging', materialization: 'view', rows: null,
+    columns: [
+      { name: 'city_code',  type: 'VARCHAR', nullable: false },
+      { name: 'city_name',  type: 'VARCHAR', nullable: true  },
+      { name: 'population', type: 'INTEGER', nullable: true  },
+      { name: 'year',       type: 'INTEGER', nullable: false },
+    ],
+  },
+  dim_venues: {
+    layer: 'foundation', materialization: 'table', rows: 4821,
+    columns: [
+      { name: 'venue_hk',       type: 'VARCHAR',   nullable: false },
+      { name: 'tenant_id',      type: 'VARCHAR',   nullable: true  },
+      { name: 'overpass_id',    type: 'VARCHAR',   nullable: true  },
+      { name: 'name',           type: 'VARCHAR',   nullable: true  },
+      { name: 'country_code',   type: 'VARCHAR',   nullable: true  },
+      { name: 'city',           type: 'VARCHAR',   nullable: true  },
+      { name: 'lat',            type: 'DOUBLE',    nullable: true  },
+      { name: 'lon',            type: 'DOUBLE',    nullable: true  },
+      { name: 'court_count',    type: 'INTEGER',   nullable: true  },
+      { name: 'has_playtomic',  type: 'BOOLEAN',   nullable: false },
+      { name: 'loaded_at',      type: 'TIMESTAMP', nullable: false },
+    ],
+  },
+  dim_cities: {
+    layer: 'foundation', materialization: 'table', rows: 1203,
+    columns: [
+      { name: 'city_hk',        type: 'VARCHAR',   nullable: false },
+      { name: 'city_name',      type: 'VARCHAR',   nullable: false },
+      { name: 'country_code',   type: 'VARCHAR',   nullable: false },
+      { name: 'population',     type: 'INTEGER',   nullable: true  },
+      { name: 'nuts2_code',     type: 'VARCHAR',   nullable: true  },
+      { name: 'income_eur',     type: 'DOUBLE',    nullable: true  },
+      { name: 'venue_count',    type: 'INTEGER',   nullable: false },
+      { name: 'lat',            type: 'DOUBLE',    nullable: true  },
+      { name: 'lon',            type: 'DOUBLE',    nullable: true  },
+    ],
+  },
+  dim_venue_capacity: {
+    layer: 'foundation', materialization: 'table', rows: 4812,
+    columns: [
+      { name: 'tenant_id',      type: 'VARCHAR', nullable: false },
+      { name: 'court_count',    type: 'INTEGER', nullable: false },
+      { name: 'open_hours_wday',type: 'DOUBLE',  nullable: true  },
+      { name: 'open_hours_wend',type: 'DOUBLE',  nullable: true  },
+    ],
+  },
+  fct_daily_availability: {
+    layer: 'foundation', materialization: 'table', rows: 382104,
+    columns: [
+      { name: 'date',           type: 'DATE',    nullable: false },
+      { name: 'tenant_id',      type: 'VARCHAR', nullable: false },
+      { name: 'resource_id',    type: 'VARCHAR', nullable: false },
+      { name: 'slots_total',    type: 'INTEGER', nullable: false },
+      { name: 'slots_booked',   type: 'INTEGER', nullable: false },
+      { name: 'occupancy_rate', type: 'DOUBLE',  nullable: true  },
+      { name: 'avg_price_eur',  type: 'DOUBLE',  nullable: true  },
+    ],
+  },
+  venue_pricing_benchmarks: {
+    layer: 'serving', materialization: 'table', rows: 4201,
+    columns: [
+      { name: 'tenant_id',       type: 'VARCHAR', nullable: false },
+      { name: 'p25_price_eur',   type: 'DOUBLE',  nullable: true  },
+      { name: 'p50_price_eur',   type: 'DOUBLE',  nullable: true  },
+      { name: 'p75_price_eur',   type: 'DOUBLE',  nullable: true  },
+      { name: 'peak_price_eur',  type: 'DOUBLE',  nullable: true  },
+      { name: 'avg_occupancy',   type: 'DOUBLE',  nullable: true  },
+      { name: 'sample_days',     type: 'INTEGER', nullable: false },
+    ],
+  },
+  city_market_profile: {
+    layer: 'serving', materialization: 'table', rows: 987,
+    columns: [
+      { name: 'city_hk',         type: 'VARCHAR', nullable: false },
+      { name: 'city_name',       type: 'VARCHAR', nullable: false },
+      { name: 'country_code',    type: 'VARCHAR', nullable: false },
+      { name: 'venue_count',     type: 'INTEGER', nullable: false },
+      { name: 'avg_price_eur',   type: 'DOUBLE',  nullable: true  },
+      { name: 'median_occ_rate', type: 'DOUBLE',  nullable: true  },
+      { name: 'population',      type: 'INTEGER', nullable: true  },
+      { name: 'income_eur',      type: 'DOUBLE',  nullable: true  },
+      { name: 'opportunity_score',type:'DOUBLE',  nullable: true  },
+    ],
+  },
+  pseo_city_costs_de: {
+    layer: 'serving', materialization: 'table', rows: 847,
+    columns: [
+      { name: 'city_slug',       type: 'VARCHAR', nullable: false },
+      { name: 'city_name_de',    type: 'VARCHAR', nullable: true  },
+      { name: 'avg_build_cost',  type: 'DOUBLE',  nullable: true  },
+      { name: 'land_cost_m2',    type: 'DOUBLE',  nullable: true  },
+      { name: 'venue_count',     type: 'INTEGER', nullable: false },
+      { name: 'opportunity_score',type:'DOUBLE',  nullable: true  },
+    ],
+  },
+};
+
+// fallback for models not in mock
+function getSchema(model) {
+  if (SCHEMA[model]) return SCHEMA[model];
+  const layer = model.startsWith('stg_') ? 'staging'
+              : (model.startsWith('dim_') || model.startsWith('fct_')) ? 'foundation'
+              : 'serving';
+  return { layer, materialization: 'view', rows: null,
+    columns: [
+      { name: 'id',         type: 'VARCHAR',   nullable: false },
+      { name: 'created_at', type: 'TIMESTAMP', nullable: false },
+    ]
+  };
+}
+
+// ── DAG definition ────────────────────────────────────────────────────────
+const DAG = {
+  stg_padel_courts: [],
+  stg_playtomic_venues: [],
+  stg_playtomic_resources: [],
+  stg_playtomic_opening_hours: [],
+  stg_playtomic_availability: [],
+  stg_population: [],
+  stg_population_usa: [],
+  stg_population_uk: [],
+  stg_population_geonames: [],
+  stg_income: [],
+  stg_income_usa: [],
+  stg_city_labels: [],
+  stg_nuts2_boundaries: [],
+  stg_regional_income: [],
+  stg_tennis_courts: [],
+  dim_venues: ['stg_playtomic_venues','stg_playtomic_resources','stg_padel_courts'],
+  dim_cities: ['dim_venues','stg_income','stg_city_labels','stg_population','stg_population_usa','stg_population_uk','stg_population_geonames'],
+  dim_locations: ['stg_population_geonames','stg_income','stg_nuts2_boundaries','stg_regional_income','stg_income_usa','stg_padel_courts','stg_tennis_courts'],
+  dim_venue_capacity: ['stg_playtomic_venues','stg_playtomic_resources','stg_playtomic_opening_hours'],
+  fct_availability_slot: ['stg_playtomic_availability'],
+  fct_daily_availability: ['fct_availability_slot','dim_venue_capacity'],
+  venue_pricing_benchmarks: ['fct_daily_availability'],
+  city_market_profile: ['dim_cities','venue_pricing_benchmarks'],
+  planner_defaults: ['venue_pricing_benchmarks','city_market_profile'],
+  location_opportunity_profile: ['dim_locations'],
+  pseo_city_costs_de: ['city_market_profile','planner_defaults','location_opportunity_profile'],
+  pseo_city_pricing: ['venue_pricing_benchmarks','city_market_profile'],
+  pseo_country_overview: ['pseo_city_costs_de'],
+};
+
+// Compute downstream map
+const DOWNSTREAM = {};
+Object.keys(DAG).forEach(n => DOWNSTREAM[n] = []);
+Object.entries(DAG).forEach(([name, deps]) => {
+  deps.forEach(dep => {
+    if (!DOWNSTREAM[dep]) DOWNSTREAM[dep] = [];
+    DOWNSTREAM[dep].push(name);
+  });
+});
+
+function classifyLayer(name) {
+  if (name.startsWith('stg_')) return 'staging';
+  if (name.startsWith('dim_') || name.startsWith('fct_')) return 'foundation';
+  return 'serving';
+}
+
+// ── SVG rendering (mirrors Python logic) ─────────────────────────────────
+const COLORS = {
+  staging:    { bg:'#F0FDF4', border:'#BBF7D0', accent:'#16A34A', fill:'#DCFCE7', text:'#14532D' },
+  foundation: { bg:'#EFF6FF', border:'#BFDBFE', accent:'#1D4ED8', fill:'#DBEAFE', text:'#1E3A8A' },
+  serving:    { bg:'#FFFBEB', border:'#FDE68A', accent:'#D97706', fill:'#FEF3C7', text:'#78350F' },
+};
+const LANE_ORDER = ['staging','foundation','serving'];
+const LANE_LABELS = { staging:'STAGING', foundation:'FOUNDATION', serving:'SERVING' };
+
+function buildSVG() {
+  const CW = 7.4, PAD_H = 10, NH = 26, VGAP = 10, PAD_TOP = 52, PAD_BOT = 24;
+  const INNER_W = 210, LANE_GAP = 40, LANE_PAD_L = 16;
+
+  // downstream counts
+  const dnCount = {};
+  Object.keys(DAG).forEach(n => dnCount[n] = 0);
+  Object.values(DAG).forEach(deps => deps.forEach(d => dnCount[d] = (dnCount[d]||0)+1));
+
+  const layers = { staging:[], foundation:[], serving:[] };
+  Object.keys(DAG).forEach(n => layers[classifyLayer(n)].push(n));
+  LANE_ORDER.forEach(l => layers[l].sort((a,b) => (dnCount[b]||0)-(dnCount[a]||0)||a.localeCompare(b)));
+
+  const nodeW = n => Math.max(n.length * CW + PAD_H*2, 80);
+
+  const laneX = {};
+  let xc = 0;
+  LANE_ORDER.forEach(l => { laneX[l] = xc; xc += INNER_W + LANE_PAD_L*2 + LANE_GAP; });
+
+  const pos = {};
+  const laneH = {};
+  LANE_ORDER.forEach(l => {
+    let y = PAD_TOP;
+    layers[l].forEach(n => { pos[n] = [laneX[l]+LANE_PAD_L, y]; y += NH+VGAP; });
+    laneH[l] = y + PAD_BOT - VGAP;
+  });
+
+  const W = xc - LANE_GAP;
+  const H = Math.max(...Object.values(laneH));
+
+  let parts = [];
+
+  parts.push(`<defs>
+    <marker id="arr" markerWidth="6" markerHeight="6" refX="5" refY="3" orient="auto">
+      <path d="M0,0 L0,6 L6,3 z" fill="#CBD5E1"/>
+    </marker>
+    <marker id="arr-hi" markerWidth="6" markerHeight="6" refX="5" refY="3" orient="auto">
+      <path d="M0,0 L0,6 L6,3 z" fill="#1D4ED8"/>
+    </marker>
+  </defs>`);
+
+  LANE_ORDER.forEach(l => {
+    const c = COLORS[l], lx = laneX[l], lw = INNER_W+LANE_PAD_L*2, lh = laneH[l];
+    parts.push(`<rect x="${lx}" y="0" width="${lw}" height="${lh}" rx="10" fill="${c.bg}" stroke="${c.border}" stroke-width="1"/>`);
+    parts.push(`<text x="${lx+lw/2}" y="28" text-anchor="middle" font-family="'DM Sans',sans-serif" font-size="10" font-weight="700" letter-spacing="1.5" fill="${c.accent}">${LANE_LABELS[l]}</text>`);
+    parts.push(`<line x1="${lx+12}" y1="36" x2="${lx+lw-12}" y2="36" stroke="${c.border}" stroke-width="1"/>`);
+  });
+
+  // Edges
+  Object.entries(DAG).forEach(([name, deps]) => {
+    const [tx, ty] = pos[name];
+    const tgt_cx = tx, tgt_cy = ty + NH/2;
+    deps.forEach(dep => {
+      if (!pos[dep]) return;
+      const [sx, sy] = pos[dep];
+      const sw = nodeW(dep);
+      const src_cx = sx+sw, src_cy = sy+NH/2;
+      const cpx1 = src_cx+(tgt_cx-src_cx)*0.45;
+      const cpx2 = tgt_cx-(tgt_cx-src_cx)*0.45;
+      parts.push(`<path class="lineage-edge" data-from="${dep}" data-to="${name}" d="M${src_cx},${src_cy} C${cpx1},${src_cy} ${cpx2},${tgt_cy} ${tgt_cx},${tgt_cy}" fill="none" stroke="#CBD5E1" stroke-width="1" marker-end="url(#arr)"/>`);
+    });
+  });
+
+  // Nodes
+  Object.keys(DAG).forEach(name => {
+    const l = classifyLayer(name);
+    const c = COLORS[l];
+    const [rx, ry] = pos[name];
+    const rw = nodeW(name);
+    const tx = rx+PAD_H, ty = ry+NH/2+4;
+    parts.push(`<g class="lineage-node" data-model="${name}" tabindex="0" role="button" aria-label="${name}">
+      <rect x="${rx}" y="${ry}" width="${rw}" height="${NH}" rx="5" fill="${c.fill}" stroke="${c.border}" stroke-width="1"/>
+      <rect x="${rx}" y="${ry}" width="3" height="${NH}" rx="5" fill="${c.accent}"/>
+      <text x="${tx}" y="${ty}" font-family="'DM Mono',monospace" font-size="11" fill="${c.text}">${name}</text>
+    </g>`);
+  });
+
+  return `<svg class="lineage-svg" viewBox="0 0 ${W} ${H}" xmlns="http://www.w3.org/2000/svg" style="width:100%;height:auto;min-width:${Math.ceil(W)}px">${parts.join('\n')}</svg>`;
+}
+
+// ── Inject SVG ────────────────────────────────────────────────────────────
+document.getElementById('svg-wrap').innerHTML = buildSVG();
+
+// ── Tooltip logic ─────────────────────────────────────────────────────────
+const tooltip = document.getElementById('tooltip');
+const ttBox = document.getElementById('tt-box');
+const PREVIEW_COUNT = 4;
+
+function showTooltip(model, x, y) {
+  const s = getSchema(model);
+  const preview = s.columns.slice(0, PREVIEW_COUNT);
+  const extra = s.columns.length - PREVIEW_COUNT;
+  const layerClass = s.layer;
+
+  ttBox.innerHTML = `
+    <div class="tt-header">
+      <span class="tt-name">${model}</span>
+      <span class="tt-layer ${layerClass}">${s.layer}</span>
+    </div>
+    ${preview.map(c => `
+      <div class="tt-row">
+        <span class="tt-col">${c.name}</span>
+        <span class="tt-type">${c.type}</span>
+      </div>
+    `).join('')}
+    ${extra > 0 ? `<div class="tt-more">+${extra} more column${extra===1?'':'s'} — click to view all</div>` : ''}
+    ${extra <= 0 && s.columns.length > 0 ? `<div class="tt-more" style="color:#94A3B8;font-style:italic">click to inspect</div>` : ''}
+  `;
+
+  // Position: prefer right of cursor, flip if near right edge
+  const W = tooltip.offsetWidth || 260, H = tooltip.offsetHeight || 120;
+  const vw = window.innerWidth, vh = window.innerHeight;
+  let left = x + 14, top = y - 10;
+  if (left + W > vw - 12) left = x - W - 14;
+  if (top + H > vh - 12) top = vh - H - 12;
+  tooltip.style.left = left + 'px';
+  tooltip.style.top  = top + 'px';
+  tooltip.classList.add('visible');
+}
+
+function hideTooltip() {
+  tooltip.classList.remove('visible');
+}
+
+// ── Panel logic ───────────────────────────────────────────────────────────
+const panel = document.getElementById('detail-panel');
+const canvasWrap = document.getElementById('canvas-wrap');
+const panelModelName = document.getElementById('panel-model-name');
+const panelLayerChip = document.getElementById('panel-layer-chip');
+const panelMat = document.getElementById('panel-materialization');
+const panelBody = document.getElementById('panel-body');
+
+let activeModel = null;
+
+function fmt(n) {
+  return n == null ? '—' : n.toLocaleString();
+}
+
+function openPanel(model) {
+  activeModel = model;
+  const s = getSchema(model);
+  panelModelName.textContent = model;
+  panelLayerChip.textContent = s.layer;
+  panelLayerChip.className = 'meta-chip ' + s.layer;
+  panelMat.textContent = s.materialization;
+
+  const ups = DAG[model] || [];
+  const downs = DOWNSTREAM[model] || [];
+
+  panelBody.innerHTML = `
+    <div class="panel-section">
+      <div class="section-label">Row count</div>
+      <div class="stat-row">
+        <span class="stat-val">${fmt(s.rows)}</span>
+        ${s.rows != null ? '<span class="stat-unit">rows</span>' : ''}
+      </div>
+      ${s.rows == null ? '<div class="empty-state" style="margin-top:2px">staging views have no row count</div>' : ''}
+    </div>
+
+    <div class="panel-section">
+      <div class="section-label">Schema · ${s.columns.length} columns</div>
+      <table class="schema-table">
+        <thead>
+          <tr>
+            <th>column</th>
+            <th>type</th>
+            <th style="text-align:right">nullable</th>
+          </tr>
+        </thead>
+        <tbody>
+          ${s.columns.map(c => `
+            <tr>
+              <td class="col-name">${c.name}</td>
+              <td class="col-type">${c.type}</td>
+              <td class="col-null ${c.nullable?'yes':''}">${c.nullable?'null':'—'}</td>
+            </tr>
+          `).join('')}
+        </tbody>
+      </table>
+    </div>
+
+    <div class="panel-section">
+      <div class="section-label">Upstream · ${ups.length}</div>
+      ${ups.length ? `<div class="dep-list">${ups.map(d => `
+        <div class="dep-item" data-model="${d}" onclick="openPanel('${d}')">
+          <span class="dep-dot ${classifyLayer(d)}"></span>${d}
+        </div>
+      `).join('')}</div>` : '<div class="empty-state">no upstream dependencies</div>'}
+    </div>
+
+    <div class="panel-section">
+      <div class="section-label">Downstream · ${downs.length}</div>
+      ${downs.length ? `<div class="dep-list">${downs.map(d => `
+        <div class="dep-item" data-model="${d}" onclick="openPanel('${d}')">
+          <span class="dep-dot ${classifyLayer(d)}"></span>${d}
+        </div>
+      `).join('')}</div>` : '<div class="empty-state">nothing depends on this model</div>'}
+    </div>
+  `;
+
+  panel.classList.add('open');
+  canvasWrap.classList.add('panel-open');
+
+  // Highlight selected node
+  document.querySelectorAll('.lineage-node').forEach(n => {
+    n.classList.toggle('selected', n.dataset.model === model);
+  });
+}
+
+function closePanel() {
+  panel.classList.remove('open');
+  canvasWrap.classList.remove('panel-open');
+  document.querySelectorAll('.lineage-node').forEach(n => n.classList.remove('selected'));
+  activeModel = null;
+}
+
+document.getElementById('panel-close').addEventListener('click', closePanel);
+
+// ── Wire up SVG nodes ─────────────────────────────────────────────────────
+const svg = document.querySelector('.lineage-svg');
+const nodes = svg.querySelectorAll('.lineage-node');
+const edges = svg.querySelectorAll('.lineage-edge');
+
+nodes.forEach(g => {
+  const model = g.dataset.model;
+
+  g.addEventListener('mouseenter', e => {
+    // Highlight edges
+    edges.forEach(edge => {
+      if (edge.dataset.from === model || edge.dataset.to === model) {
+        edge.classList.add('hi');
+        edge.classList.remove('dim');
+        edge.setAttribute('marker-end', 'url(#arr-hi)');
+      } else {
+        edge.classList.add('dim');
+        edge.classList.remove('hi');
+      }
+    });
+    showTooltip(model, e.clientX, e.clientY);
+  });
+
+  g.addEventListener('mousemove', e => {
+    showTooltip(model, e.clientX, e.clientY);
+  });
+
+  g.addEventListener('mouseleave', () => {
+    edges.forEach(e => {
+      e.classList.remove('hi', 'dim');
+      e.setAttribute('marker-end', 'url(#arr)');
+    });
+    hideTooltip();
+  });
+
+  g.addEventListener('click', () => {
+    hideTooltip();
+    if (activeModel === model) {
+      closePanel();
+    } else {
+      openPanel(model);
+    }
+  });
+});
+
+// Close panel on Escape
+document.addEventListener('keydown', e => {
+  if (e.key === 'Escape') closePanel();
+});
+</script>
+</body>
+</html>
--- a/src/padelnomics/supervisor.py
+++ b/src/padelnomics/supervisor.py
@@ -250,7 +250,7 @@ def run_transform() -> None:
    """Run SQLMesh — it evaluates model staleness internally."""
    logger.info("Running SQLMesh transform")
    ok = run_shell(
-        f"uv run sqlmesh -p transform/sqlmesh_padelnomics run",
+        "uv run sqlmesh -p transform/sqlmesh_padelnomics plan prod --auto-apply",
    )
    if not ok:
        send_alert("SQLMesh transform failed")
--- a/transform/sqlmesh_padelnomics/models/foundation/dim_venue_capacity.sql
+++ b/transform/sqlmesh_padelnomics/models/foundation/dim_venue_capacity.sql
@@ -34,6 +34,7 @@ SELECT
  v.tenant_id,
  v.country_code,
  v.city,
+  v.city_slug,
  cc.active_court_count,
  ROUND(wh.hours_open_per_week, 1)                AS hours_open_per_week,
  ROUND(wh.avg_hours_open_per_day, 1)             AS avg_hours_open_per_day,
@@ -42,6 +43,6 @@ SELECT
  ROUND(cc.active_court_count * wh.avg_hours_open_per_day, 1)  AS capacity_court_hours_per_day,
  -- Total bookable court-hours per week
  ROUND(cc.active_court_count * wh.hours_open_per_week, 1)     AS capacity_court_hours_per_week
-FROM staging.stg_playtomic_venues v
+FROM foundation.dim_venues v
 JOIN court_counts cc ON v.tenant_id = cc.tenant_id
 JOIN weekly_hours wh ON v.tenant_id = wh.tenant_id
--- a/transform/sqlmesh_padelnomics/models/foundation/dim_venues.sql
+++ b/transform/sqlmesh_padelnomics/models/foundation/dim_venues.sql
@@ -98,6 +98,8 @@ SELECT
  court_count,
  indoor_court_count,
  outdoor_court_count,
+  -- Conformed city key: enables deterministic joins to dim_cities / venue_pricing_benchmarks
+  LOWER(REGEXP_REPLACE(LOWER(COALESCE(city, '')), '[^a-z0-9]+', '-')) AS city_slug,
  extracted_date
 FROM ranked
 QUALIFY ROW_NUMBER() OVER (
--- a/transform/sqlmesh_padelnomics/models/foundation/fct_daily_availability.sql
+++ b/transform/sqlmesh_padelnomics/models/foundation/fct_daily_availability.sql
@@ -44,6 +44,7 @@ SELECT
  sa.tenant_id,
  cap.country_code,
  cap.city,
+  cap.city_slug,
  cap.active_court_count,
  cap.capacity_court_hours_per_day,
  sa.available_slot_count,
--- a/transform/sqlmesh_padelnomics/models/serving/city_market_profile.sql
+++ b/transform/sqlmesh_padelnomics/models/serving/city_market_profile.sql
@@ -57,7 +57,7 @@ WITH base AS (
  FROM foundation.dim_cities c
  LEFT JOIN serving.venue_pricing_benchmarks vpb
    ON c.country_code = vpb.country_code
-    AND LOWER(TRIM(c.city_name)) = LOWER(TRIM(vpb.city))
+    AND c.city_slug = vpb.city_slug
  WHERE c.padel_venue_count > 0
 ),
 scored AS (
--- a/transform/sqlmesh_padelnomics/models/serving/planner_defaults.sql
+++ b/transform/sqlmesh_padelnomics/models/serving/planner_defaults.sql
@@ -21,6 +21,7 @@ city_benchmarks AS (
  SELECT
    country_code,
    city,
+    city_slug,
    median_peak_rate,
    median_offpeak_rate,
    median_occupancy_rate,
@@ -128,7 +129,7 @@ SELECT
 FROM city_profiles cp
 LEFT JOIN city_benchmarks cb
  ON cp.country_code = cb.country_code
-  AND LOWER(TRIM(cp.city_name)) = LOWER(TRIM(cb.city))
+  AND cp.city_slug = cb.city_slug
 LEFT JOIN country_benchmarks ctb
  ON cp.country_code = ctb.country_code
 LEFT JOIN hardcoded_fallbacks hf
--- a/transform/sqlmesh_padelnomics/models/serving/pseo_city_pricing.sql
+++ b/transform/sqlmesh_padelnomics/models/serving/pseo_city_pricing.sql
@@ -41,6 +41,6 @@ FROM serving.venue_pricing_benchmarks vpb
 -- Join city_market_profile to get the canonical city_slug and country metadata
 INNER JOIN serving.city_market_profile c
  ON  vpb.country_code = c.country_code
-  AND LOWER(TRIM(vpb.city)) = LOWER(TRIM(c.city_name))
+  AND vpb.city_slug = c.city_slug
 -- Only cities with enough venues for meaningful pricing statistics
 WHERE vpb.venue_count >= 2
--- a/transform/sqlmesh_padelnomics/models/serving/venue_pricing_benchmarks.sql
+++ b/transform/sqlmesh_padelnomics/models/serving/venue_pricing_benchmarks.sql
@@ -17,6 +17,7 @@ WITH venue_stats AS (
    da.tenant_id,
    da.country_code,
    da.city,
+    da.city_slug,
    da.price_currency,
    AVG(da.occupancy_rate)                              AS avg_occupancy_rate,
    MEDIAN(da.median_price)                             AS median_hourly_rate,
@@ -29,12 +30,13 @@ WITH venue_stats AS (
  WHERE TRY_CAST(da.snapshot_date AS DATE) >= CURRENT_DATE - INTERVAL '30 days'
    AND da.occupancy_rate IS NOT NULL
    AND da.occupancy_rate BETWEEN 0 AND 1.5
-  GROUP BY da.tenant_id, da.country_code, da.city, da.price_currency
+  GROUP BY da.tenant_id, da.country_code, da.city, da.city_slug, da.price_currency
  HAVING COUNT(DISTINCT da.snapshot_date) >= 3
 )
 SELECT
  country_code,
  city,
+  city_slug,
  price_currency,
  COUNT(*)                                              AS venue_count,
  -- Pricing benchmarks
@@ -54,4 +56,4 @@ SELECT
  SUM(days_observed)                                    AS total_venue_days_observed,
  CURRENT_DATE                                          AS refreshed_date
 FROM venue_stats
-GROUP BY country_code, city, price_currency
+GROUP BY country_code, city, city_slug, price_currency
--- a/web/src/padelnomics/admin/pipeline_routes.py
+++ b/web/src/padelnomics/admin/pipeline_routes.py
@@ -100,7 +100,7 @@ _DAG: dict[str, list[str]] = {
        "stg_regional_income", "stg_income_usa", "stg_padel_courts", "stg_tennis_courts",
    ],
    "dim_venue_capacity": [
-        "stg_playtomic_venues", "stg_playtomic_resources", "stg_playtomic_opening_hours",
+        "dim_venues", "stg_playtomic_resources", "stg_playtomic_opening_hours",
    ],
    "fct_availability_slot": ["stg_playtomic_availability"],
    "fct_daily_availability": ["fct_availability_slot", "dim_venue_capacity"],
@@ -767,6 +767,12 @@ async def pipeline_trigger_extract():

 # ── Lineage tab ───────────────────────────────────────────────────────────────

+# Compute downstream map once at import time (DAG is static).
+_DOWNSTREAM: dict[str, list[str]] = {n: [] for n in _DAG}
+for _name, _deps in _DAG.items():
+    for _dep in _deps:
+        _DOWNSTREAM.setdefault(_dep, []).append(_name)
+

@bp.route("/lineage")
@role_required("admin")
@@ -780,6 +786,67 @@ async def pipeline_lineage():
    )


+@bp.route("/lineage/schema/<model>")
+@role_required("admin")
+async def pipeline_lineage_schema(model: str):
+    """JSON: schema details for a lineage node.
+
+    Returns columns + types from information_schema (serving models only —
+    staging/foundation live in lakehouse.duckdb which the web app cannot open).
+    Row count is included for serving models when the table exists.
+    """
+    from quart import jsonify
+
+    from ..analytics import fetch_analytics
+
+    if model not in _DAG:
+        return jsonify({"error": "unknown model"}), 404
+
+    layer = _classify_layer(model)
+    upstream = _DAG[model]
+    downstream = _DOWNSTREAM.get(model, [])
+
+    row_count = None
+    columns: list[dict] = []
+
+    if layer == "serving":
+        col_rows = await fetch_analytics(
+            """
+            SELECT column_name, data_type, is_nullable
+            FROM information_schema.columns
+            WHERE table_schema = 'serving' AND table_name = ?
+            ORDER BY ordinal_position
+            """,
+            [model],
+        )
+        columns = [
+            {
+                "name": r["column_name"],
+                "type": r["data_type"],
+                "nullable": r["is_nullable"] == "YES",
+            }
+            for r in col_rows
+        ]
+        if columns:
+            # model is validated against _DAG keys — safe to interpolate
+            count_rows = await fetch_analytics(
+                f"SELECT count(*) AS n FROM serving.{model}"
+            )
+            if count_rows:
+                row_count = count_rows[0]["n"]
+
+    return jsonify(
+        {
+            "model": model,
+            "layer": layer,
+            "upstream": upstream,
+            "downstream": downstream,
+            "row_count": row_count,
+            "columns": columns,
+        }
+    )
+
+
 # ── Catalog tab ───────────────────────────────────────────────────────────────


--- a/web/src/padelnomics/admin/routes.py
+++ b/web/src/padelnomics/admin/routes.py
@@ -881,7 +881,7 @@ async def marketplace_activity():
           FROM lead_forwards lf
           JOIN suppliers s ON s.id = lf.supplier_id
           UNION ALL
-           SELECT 'credit' as event_type, id as ref_id, supplier_id as ref2_id,
+           SELECT 'credit' as event_type, cl.id as ref_id, cl.supplier_id as ref2_id,
               s.name as actor, cl.event_type as detail,
               CAST(cl.delta AS TEXT) as extra, cl.created_at
           FROM credit_ledger cl
--- a/web/src/padelnomics/admin/templates/admin/base_admin.html
+++ b/web/src/padelnomics/admin/templates/admin/base_admin.html
@@ -156,7 +156,7 @@

      <a href="{{ url_for('pipeline.pipeline_dashboard') }}" class="{% if active_section == 'pipeline' %}active{% endif %}">
        <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" d="M20.25 6.375c0 2.278-3.694 4.125-8.25 4.125S3.75 8.653 3.75 6.375m16.5 0c0-2.278-3.694-4.125-8.25-4.125S3.75 4.097 3.75 6.375m16.5 0v11.25c0 2.278-3.694 4.125-8.25 4.125s-8.25-1.847-8.25-4.125V6.375m16.5 0v3.75m-16.5-3.75v3.75m16.5 0v3.75C20.25 16.153 16.556 18 12 18s-8.25-1.847-8.25-4.125v-3.75m16.5 0c0 2.278-3.694 4.125-8.25 4.125s-8.25-1.847-8.25-4.125"/></svg>
-        Pipeline
+        Data Platform
      </a>

      <div class="admin-sidebar__divider"></div>
--- a/web/src/padelnomics/admin/templates/admin/partials/pipeline_lineage.html
+++ b/web/src/padelnomics/admin/templates/admin/partials/pipeline_lineage.html
@@ -5,45 +5,296 @@
    <span class="text-xs font-normal text-slate ml-2">
      {{ node_count }} models &mdash; staging &rarr; foundation &rarr; serving
    </span>
+    <span class="text-xs font-normal text-slate" style="margin-left:auto">
+      hover to preview &middot; click to inspect
+    </span>
  </p>
  <div style="overflow-x:auto;padding:1rem 0.5rem 0.5rem">
    {{ lineage_svg | safe }}
  </div>
 </div>

+<!-- Detail panel: fixed right, slides in on node click -->
+<div id="ln-panel" style="
+  position:fixed;top:0;right:0;bottom:0;width:320px;
+  background:#fff;border-left:1px solid #E2E8F0;
+  display:flex;flex-direction:column;
+  transform:translateX(100%);
+  transition:transform 0.2s cubic-bezier(0.4,0,0.2,1);
+  z-index:200;overflow:hidden;
+  box-shadow:-4px 0 24px rgba(0,0,0,0.06);
+">
+  <div style="padding:0.75rem 1rem;border-bottom:1px solid #F1F5F9;flex-shrink:0">
+    <div style="display:flex;align-items:center;gap:0.5rem;margin-bottom:0.375rem">
+      <span id="ln-model-name" style="
+        font-family:'Commit Mono',ui-monospace,monospace;
+        font-size:0.8125rem;font-weight:500;color:#1E293B;
+        flex:1;min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;
+      ">—</span>
+      <button id="ln-close" style="
+        background:none;border:none;cursor:pointer;color:#94A3B8;
+        font-size:1rem;padding:2px 4px;border-radius:4px;line-height:1;
+        transition:color 0.1s,background 0.1s;
+      " title="Close (Esc)">✕</button>
+    </div>
+    <div style="display:flex;align-items:center;gap:0.5rem">
+      <span id="ln-layer-chip" style="
+        font-size:0.625rem;font-weight:700;letter-spacing:0.06em;
+        text-transform:uppercase;padding:2px 7px;border-radius:99px;
+      ">—</span>
+      <span id="ln-mat" style="font-size:0.6875rem;color:#64748B"></span>
+    </div>
+  </div>
+  <div id="ln-body" style="flex:1;overflow-y:auto"></div>
+</div>
+
+<!-- Hover tooltip -->
+<div id="ln-tooltip" style="
+  position:fixed;z-index:300;pointer-events:none;
+  opacity:0;transition:opacity 0.08s;
+  filter:drop-shadow(0 4px 14px rgba(0,0,0,0.14));
+">
+  <div id="ln-tt-inner" style="
+    background:#0F172A;color:#fff;border-radius:8px;
+    padding:10px 12px;min-width:190px;max-width:250px;
+  "></div>
+</div>
+
 <style>
-  .lineage-node { cursor: default; }
-  .lineage-node rect:first-of-type { transition: filter 0.12s; }
-  .lineage-node:hover rect:first-of-type { filter: brightness(0.94); }
+  .lineage-node { cursor: pointer; }
+  .lineage-node rect:first-of-type { transition: filter 0.1s; }
+  .lineage-node:hover rect:first-of-type { filter: brightness(0.92); }
+  .lineage-node.ln-selected rect:first-of-type { filter: brightness(0.86) !important; }
  .lineage-edge { transition: stroke 0.12s, stroke-width 0.12s, opacity 0.12s; }
  .lineage-edge.hi { stroke: #1D4ED8 !important; stroke-width: 2 !important; marker-end: url(#arr-hi) !important; }
-  .lineage-edge.dim { opacity: 0.12; }
+  .lineage-edge.dim { opacity: 0.1; }
+
+  .ln-section { border-bottom: 1px solid #F1F5F9; padding: 0.75rem 1rem; }
+  .ln-section:last-child { border-bottom: none; }
+  .ln-label {
+    font-size: 0.5875rem; font-weight: 700; letter-spacing: 0.08em;
+    text-transform: uppercase; color: #94A3B8; margin-bottom: 0.5rem;
+  }
+  .ln-schema-table { width: 100%; border-collapse: collapse; font-size: 0.6875rem; }
+  .ln-schema-table th {
+    text-align: left; font-weight: 600; color: #64748B;
+    padding: 0 0 0.375rem; font-size: 0.5875rem; letter-spacing: 0.04em; text-transform: uppercase;
+  }
+  .ln-schema-table td { padding: 3px 4px 3px 0; vertical-align: middle; }
+  .ln-schema-table tr + tr td { border-top: 1px solid #F1F5F9; }
+  .ln-col-name { font-family: 'Commit Mono', ui-monospace, monospace; color: #1E293B; font-weight: 500; }
+  .ln-col-type { font-family: 'Commit Mono', ui-monospace, monospace; color: #94A3B8; font-size: 0.625rem; }
+  .ln-col-null { font-size: 0.5625rem; color: #CBD5E1; text-align: right; }
+  .ln-col-null.yes { color: #D97706; }
+
+  .ln-dep-item {
+    display: flex; align-items: center; gap: 6px; padding: 4px 6px;
+    border-radius: 5px; cursor: pointer;
+    font-family: 'Commit Mono', ui-monospace, monospace;
+    font-size: 0.6875rem; color: #334155; transition: background 0.1s;
+  }
+  .ln-dep-item:hover { background: #F8FAFC; }
+  .ln-dep-dot { width: 6px; height: 6px; border-radius: 50%; flex-shrink: 0; }
+  .ln-dep-dot.staging    { background: #16A34A; }
+  .ln-dep-dot.foundation { background: #1D4ED8; }
+  .ln-dep-dot.serving    { background: #D97706; }
+
+  .ln-chip-staging    { background: #DCFCE7; color: #14532D; border: 1px solid #BBF7D0; }
+  .ln-chip-foundation { background: #DBEAFE; color: #1E3A8A; border: 1px solid #BFDBFE; }
+  .ln-chip-serving    { background: #FEF3C7; color: #78350F; border: 1px solid #FDE68A; }
 </style>

 <script>
 (function () {
+  var SCHEMA_BASE = "{{ url_for('pipeline.pipeline_lineage_schema', model='MODEL') }}".replace('/MODEL', '/');
  var svg = document.querySelector('.lineage-svg');
  if (!svg) return;
  var nodes = svg.querySelectorAll('.lineage-node');
  var edges = svg.querySelectorAll('.lineage-edge');
+  var panel = document.getElementById('ln-panel');
+  var panelBody = document.getElementById('ln-body');
+  var tooltip = document.getElementById('ln-tooltip');
+  var ttInner = document.getElementById('ln-tt-inner');
+  var activeModel = null;
+  var cache = {};

-  nodes.forEach(function (g) {
-    var model = g.dataset.model;
-    g.addEventListener('mouseenter', function () {
+  // ── Helpers ────────────────────────────────────────────────────────────
+  function layer(model) {
+    if (model.startsWith('stg_')) return 'staging';
+    if (model.startsWith('dim_') || model.startsWith('fct_')) return 'foundation';
+    return 'serving';
+  }
+
+  function fmt(n) {
+    return n == null ? '—' : Number(n).toLocaleString();
+  }
+
+  // ── Edge highlight ─────────────────────────────────────────────────────
+  function highlightEdges(model) {
    edges.forEach(function (e) {
      if (e.dataset.from === model || e.dataset.to === model) {
-          e.classList.add('hi');
-          e.classList.remove('dim');
+        e.classList.add('hi'); e.classList.remove('dim');
      } else {
-          e.classList.add('dim');
-          e.classList.remove('hi');
+        e.classList.add('dim'); e.classList.remove('hi');
      }
    });
+  }
+
+  function clearEdges() {
+    edges.forEach(function (e) { e.classList.remove('hi', 'dim'); });
+  }
+
+  // ── Schema fetch (cached) ──────────────────────────────────────────────
+  function fetchSchema(model, cb) {
+    if (cache[model]) { cb(cache[model]); return; }
+    fetch(SCHEMA_BASE + encodeURIComponent(model))
+      .then(function (r) { return r.json(); })
+      .then(function (d) { cache[model] = d; cb(d); })
+      .catch(function () {
+        var fallback = { model: model, layer: layer(model), columns: [], upstream: [], downstream: [], row_count: null };
+        cache[model] = fallback;
+        cb(fallback);
      });
+  }
+
+  // ── Tooltip ────────────────────────────────────────────────────────────
+  function showTooltip(data, x, y) {
+    var cols = data.columns || [];
+    var preview = cols.slice(0, 4);
+    var extra = cols.length - 4;
+    var lc = layer(data.model);
+    var badge = lc === 'staging'
+      ? 'background:rgba(22,163,74,0.25);color:#86EFAC'
+      : lc === 'foundation'
+        ? 'background:rgba(29,78,216,0.3);color:#93C5FD'
+        : 'background:rgba(217,119,6,0.25);color:#FCD34D';
+
+    ttInner.innerHTML =
+      '<div style="display:flex;align-items:center;gap:6px;margin-bottom:8px;padding-bottom:7px;border-bottom:1px solid rgba(255,255,255,0.08)">' +
+        '<span style="font-family:\'Commit Mono\',monospace;font-size:0.6875rem;font-weight:500;flex:1;overflow:hidden;text-overflow:ellipsis;white-space:nowrap">' + data.model + '</span>' +
+        '<span style="font-size:0.5625rem;font-weight:700;letter-spacing:0.07em;text-transform:uppercase;padding:2px 5px;border-radius:3px;flex-shrink:0;' + badge + '">' + lc + '</span>' +
+      '</div>' +
+      (cols.length === 0
+        ? '<div style="font-size:0.6875rem;color:#475569;font-style:italic">schema in lakehouse only</div>'
+        : preview.map(function (c) {
+            return '<div style="display:flex;align-items:baseline;gap:6px;padding:2px 0">' +
+              '<span style="font-family:\'Commit Mono\',monospace;font-size:0.6875rem;color:#CBD5E1;flex:1;overflow:hidden;text-overflow:ellipsis">' + c.name + '</span>' +
+              '<span style="font-family:\'Commit Mono\',monospace;font-size:0.625rem;color:#475569;flex-shrink:0">' + c.type + '</span>' +
+            '</div>';
+          }).join('') +
+          (extra > 0
+            ? '<div style="font-size:0.625rem;color:#475569;margin-top:5px;padding-top:5px;border-top:1px solid rgba(255,255,255,0.06)">+' + extra + ' more &mdash; click to view all</div>'
+            : '<div style="font-size:0.625rem;color:#334155;margin-top:5px;padding-top:5px;border-top:1px solid rgba(255,255,255,0.06);font-style:italic">click to inspect</div>')
+      );
+
+    var vw = window.innerWidth, vh = window.innerHeight;
+    var left = x + 14, top = y - 10;
+    if (left + 252 > vw - 12) left = x - 252 - 14;
+    if (top + 160 > vh - 12) top = vh - 160 - 12;
+    tooltip.style.left = left + 'px';
+    tooltip.style.top = top + 'px';
+    tooltip.style.opacity = '1';
+  }
+
+  function hideTooltip() { tooltip.style.opacity = '0'; }
+
+  // ── Panel ──────────────────────────────────────────────────────────────
+  function depItems(list) {
+    if (!list.length) return '<div style="font-size:0.6875rem;color:#94A3B8;font-style:italic">none</div>';
+    return list.map(function (d) {
+      return '<div class="ln-dep-item" data-model="' + d + '"><span class="ln-dep-dot ' + layer(d) + '"></span>' + d + '</div>';
+    }).join('');
+  }
+
+  function renderPanel(data) {
+    var cols = data.columns || [];
+    var lc = data.layer || layer(data.model);
+
+    document.getElementById('ln-mat').textContent =
+      cols.length > 0 ? 'table' : (lc === 'serving' ? '' : '');
+
+    panelBody.innerHTML =
+      '<div class="ln-section">' +
+        '<div class="ln-label">Row count</div>' +
+        (data.row_count != null
+          ? '<div style="font-family:\'Commit Mono\',monospace;font-size:1rem;font-weight:500;color:#1E293B">' + fmt(data.row_count) +
+            ' <span style="font-size:0.6875rem;font-weight:400;color:#64748B">rows</span></div>'
+          : '<div style="font-size:0.6875rem;color:#94A3B8;font-style:italic">' +
+            (lc !== 'serving' ? 'staging/foundation — in lakehouse.duckdb' : 'not yet built') + '</div>') +
+      '</div>' +
+
+      '<div class="ln-section">' +
+        '<div class="ln-label">Schema &middot; ' + cols.length + ' columns</div>' +
+        (cols.length > 0
+          ? '<table class="ln-schema-table"><thead><tr><th>column</th><th>type</th><th style="text-align:right">null?</th></tr></thead><tbody>' +
+            cols.map(function (c) {
+              return '<tr><td class="ln-col-name">' + c.name + '</td><td class="ln-col-type">' + c.type + '</td>' +
+                '<td class="ln-col-null' + (c.nullable ? ' yes' : '') + '">' + (c.nullable ? 'null' : '—') + '</td></tr>';
+            }).join('') + '</tbody></table>'
+          : '<div style="font-size:0.6875rem;color:#94A3B8;font-style:italic">schema available in lakehouse.duckdb only</div>') +
+      '</div>' +
+
+      '<div class="ln-section"><div class="ln-label">Upstream &middot; ' + (data.upstream || []).length + '</div>' + depItems(data.upstream || []) + '</div>' +
+      '<div class="ln-section"><div class="ln-label">Downstream &middot; ' + (data.downstream || []).length + '</div>' + depItems(data.downstream || []) + '</div>';
+
+    panelBody.querySelectorAll('.ln-dep-item').forEach(function (el) {
+      el.addEventListener('click', function () { openPanel(el.dataset.model); });
+    });
+  }
+
+  function openPanel(model) {
+    activeModel = model;
+    document.getElementById('ln-model-name').textContent = model;
+    var chip = document.getElementById('ln-layer-chip');
+    var lc = layer(model);
+    chip.textContent = lc;
+    chip.className = 'ln-chip-' + lc;
+    chip.setAttribute('style',
+      'font-size:0.625rem;font-weight:700;letter-spacing:0.06em;text-transform:uppercase;padding:2px 7px;border-radius:99px;');
+    document.getElementById('ln-mat').textContent = '';
+    panelBody.innerHTML = '<div style="padding:2rem 1rem;font-size:0.75rem;color:#94A3B8">Loading\u2026</div>';
+    panel.style.transform = 'translateX(0)';
+
+    nodes.forEach(function (n) { n.classList.toggle('ln-selected', n.dataset.model === model); });
+    highlightEdges(model);
+
+    fetchSchema(model, renderPanel);
+  }
+
+  function closePanel() {
+    panel.style.transform = 'translateX(100%)';
+    clearEdges();
+    nodes.forEach(function (n) { n.classList.remove('ln-selected'); });
+    activeModel = null;
+  }
+
+  document.getElementById('ln-close').addEventListener('click', closePanel);
+  document.addEventListener('keydown', function (e) { if (e.key === 'Escape') closePanel(); });
+
+  // ── Node wiring ────────────────────────────────────────────────────────
+  nodes.forEach(function (g) {
+    var model = g.dataset.model;
+
+    g.addEventListener('mouseenter', function (e) {
+      if (activeModel === null) highlightEdges(model);
+      // prefetch so tooltip has data on arrive
+      fetchSchema(model, function (data) { showTooltip(data, e.clientX, e.clientY); });
+    });
+
+    g.addEventListener('mousemove', function (e) {
+      if (cache[model]) showTooltip(cache[model], e.clientX, e.clientY);
+    });
+
    g.addEventListener('mouseleave', function () {
-      edges.forEach(function (e) {
-        e.classList.remove('hi', 'dim');
+      hideTooltip();
+      if (activeModel === null) clearEdges();
+      else highlightEdges(activeModel);
    });
+
+    g.addEventListener('click', function () {
+      hideTooltip();
+      if (activeModel === model) closePanel();
+      else openPanel(model);
    });
  });
 })();
Author	SHA1	Message	Date
Deeman	143ad28854	fix(supervisor): use sqlmesh plan --auto-apply instead of run Some checks failed CI / test (push) Has been cancelled Details CI / tag (push) Has been cancelled Details 'run' requires the prod environment to already exist. 'plan --auto-apply' initializes the environment on first run and applies pending changes on subsequent runs — fully self-healing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 15:40:37 +01:00
Deeman	415d28afa9	fix(supervisor): run sqlmesh against prod environment Without the 'prod' argument sqlmesh defaults to dev_<username>, which doesn't exist on the server (padelnomics_service user). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 15:39:55 +01:00
Deeman	66d7cdea21	update	2026-02-27 15:39:39 +01:00
Deeman	9c2bf51c73	fix(infra): chown -R APP_DIR so service user owns full tree Without -R, a manual uv sync or git operation run as root would create files under /opt/padelnomics owned by root, breaking uv for the service user (Permission denied on .venv/bin/python3). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 15:23:12 +01:00
Deeman	7e0b06a2ad	prototype	2026-02-27 14:03:40 +01:00
Deeman	dca198c17d	fix(ci): clear alpine/git entrypoint in tag job alpine/git sets ENTRYPOINT ["git"], so GitLab's shell executor was invoking `git sh <script>` instead of `sh <script>`. Override with entrypoint: [""]. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 13:59:50 +01:00
Deeman	49820391ab	fix(admin): qualify ambiguous column name in marketplace_activity query `credit_ledger cl` joined with `suppliers s` — both have `id`, so SQLite raised OperationalError. Qualify as `cl.id` and `cl.supplier_id`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 13:59:30 +01:00
Deeman	f048e8276f	style(admin): rename nav label "Pipeline" → "Data Platform" Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 13:59:13 +01:00
Deeman	bcacc7aae6	merge(pipeline-lineage): conform geographic dimension hierarchy via city_slug	2026-02-27 13:31:44 +01:00
Deeman	00393933ca	merge: lineage hover tooltip + click schema panel	2026-02-27 13:24:20 +01:00
Deeman	89ff931212	feat(lineage): hover tooltip + click-to-inspect schema panel - New route GET /admin/pipeline/lineage/schema/<model> — returns JSON with columns+types (from information_schema for serving models), row count, upstream and downstream model lists. Validates model against _DAG to prevent arbitrary table access. - Precomputes _DOWNSTREAM map at import time from _DAG. - Lineage template: replaces minimal edge-highlight JS with full UX — hover triggers schema prefetch + floating tooltip (layer badge, top 4 columns, "+N more" note); click opens 320px slide-in panel showing row count, full schema table, upstream/downstream dep lists. Dep items in panel are clickable to navigate between models. Schema responses are cached client-side to avoid repeat fetches. Staging/foundation models show "schema in lakehouse.duckdb only". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 13:23:54 +01:00
Deeman	4e82907a70	refactor(transform): conform geographic dimension hierarchy via city_slug Propagates the conformed city key (city_slug) from dim_venues through the full pricing pipeline, eliminating 3 fragile LOWER(TRIM(...)) fuzzy string joins with deterministic key joins. Changes (cascading, task-by-task): - dim_venues: add city_slug computed column (REGEXP_REPLACE slug derivation) - dim_venue_capacity: join foundation.dim_venues instead of stg_playtomic_venues; carry city_slug alongside country_code/city - fct_daily_availability: carry city_slug from dim_venue_capacity - venue_pricing_benchmarks: carry city_slug from fct_daily_availability; add to venue_stats GROUP BY and final SELECT/GROUP BY - city_market_profile: join vpb on city_slug = city_slug (was LOWER(TRIM)) - planner_defaults: add city_slug to city_benchmarks CTE; join on city_slug - pseo_city_pricing: join city_market_profile on city_slug (was LOWER(TRIM)) - pipeline_routes._DAG: dim_venue_capacity now depends on dim_venues, not stg_playtomic_venues Result: dim_venues.city_slug → dim_cities.(country_code, city_slug) forms a fully conformed geographic hierarchy with no fuzzy string comparisons. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-02-27 13:23:03 +01:00