merge: Market Score v4 + Opportunity Score v5

- CHANGELOG.md: document Market Score v4 and Opportunity Score v5 changes
- pipeline_routes.py: add dim_countries to location_profiles dependency list

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CHANGELOG.md

@@ -7,6 +7,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ## [Unreleased]
 
 ### Changed
+- **Market Score v3 → v4** — fixes Spain averaging 54 (should be 65-80). Four calibration changes: count gate threshold lowered from 5 → 3 venues (3 establishes a market pattern), density ceiling lowered from LN(21) → LN(11) (10/100k is reachable for mature markets), demand evidence fallback raised from 0.4 → 0.65 multiplier with 0.3 floor (existence of venues IS evidence of demand), economic context ceiling changed from income/200 → income/25000 (actual discrimination instead of free 10 pts for everyone).
+- **Opportunity Score v4 → v5** — fixes structural flaws: supply gap (30pts) + catchment gap (15pts) merged into single supply deficit (35pts, GREATEST of density gap and distance gap) eliminating ~80% correlated double-count. New sports culture signal (10pts) using tennis court density as racquet-sport adoption proxy. New construction affordability signal (5pts) using income relative to PLI construction costs from `dim_countries`. Economic power reduced from 20 → 15pts. New dependency on `foundation.dim_countries` for `pli_construction`.
+
 - **Unified `location_profiles` serving model** — merged `city_market_profile` and `location_opportunity_profile` into a single `serving.location_profiles` table at `(country_code, geoname_id)` grain. Both Marktreife-Score (Market Score) and Marktpotenzial-Score (Opportunity Score) are now computed per location. City data enriched via LEFT JOIN `dim_cities` on `geoname_id`. Downstream models (`planner_defaults`, `pseo_city_costs_de`, `pseo_city_pricing`) updated to query `location_profiles` directly. `city_padel_venue_count` (exact from dim_cities) distinguished from `padel_venue_count` (spatial 5km from dim_locations).
 - **Both scores on all map tooltips** — country map shows avg Market Score + avg Opportunity Score; city map shows Market Score + Opportunity Score per city; opportunity map shows Opportunity Score + Market Score per location. All score labels use the trademarked "Padelnomics Market Score" / "Padelnomics Opportunity Score" names.
 - **API endpoints** — `/api/markets/countries.json` adds `avg_opportunity_score`; `/api/markets/<country>/cities.json` adds `opportunity_score`; `/api/opportunity/<country>.json` adds `market_score`.

Dockerfile

@@ -26,6 +26,7 @@ RUN mkdir -p /app/data && chown -R appuser:appuser /app
 COPY --from=build --chown=appuser:appuser /app .
 COPY --from=css-build /app/web/src/padelnomics/static/css/output.css ./web/src/padelnomics/static/css/output.css
 COPY --chown=appuser:appuser infra/supervisor/workflows.toml ./infra/supervisor/workflows.toml
+COPY --chown=appuser:appuser content/ ./content/
 USER appuser
 ENV PYTHONUNBUFFERED=1
 ENV DATABASE_PATH=/app/data/app.db

infra/supervisor/supervisor.sh

@@ -42,7 +42,7 @@ do
         # The web app detects the inode change on next query — no restart needed.
         DUCKDB_PATH="${DUCKDB_PATH:-/data/padelnomics/lakehouse.duckdb}" \
         SERVING_DUCKDB_PATH="${SERVING_DUCKDB_PATH:-/data/padelnomics/analytics.duckdb}" \
-            uv run python -m padelnomics.export_serving
+            uv run python src/padelnomics/export_serving.py
 
     ) || {
         if [ -n "${ALERT_WEBHOOK_URL:-}" ]; then

transform/sqlmesh_padelnomics/models/serving/location_profiles.sql

@@ -5,28 +5,36 @@
 --
 -- Two scores per location:
 --
--- Padelnomics Market Score (Marktreife-Score v3, 0–100):
+-- Padelnomics Market Score (Marktreife-Score v4, 0–100):
 --   "How mature/established is this padel market?"
 --   Only meaningful for locations matched to a dim_cities row (city_slug IS NOT NULL)
 --   with padel venues. 0 for all other locations.
 --
---   40 pts  supply development — log-scaled density (LN ceiling 20/100k) × count gate
---   25 pts  demand evidence   — occupancy when available; 40% density proxy otherwise
+--   v4 changes: lower count gate (5→3), lower density ceiling (LN(21)→LN(11)),
+--   better demand fallback (0.4→0.65 with 0.3 floor), economic context discrimination (200→25K).
+--
+--   40 pts  supply development — log-scaled density (LN ceiling 10/100k) × count gate (3)
+--   25 pts  demand evidence   — occupancy when available; 65% density proxy + 0.3 floor otherwise
 --   15 pts  addressable market — log-scaled population, ceiling 1M
---   10 pts  economic context  — income PPS normalised to 200 ceiling
+--   10 pts  economic context  — income PPS normalised to 25,000 ceiling
 --   10 pts  data quality      — completeness discount
 --
--- Padelnomics Opportunity Score (Marktpotenzial-Score v3, 0–100):
+-- Padelnomics Opportunity Score (Marktpotenzial-Score v5, 0–100):
 --   "Where should I build a padel court?"
---   Computed for ALL locations — zero-court locations score highest on supply gap.
---   H3 catchment methodology: addressable market and supply gap use a regional
+--   Computed for ALL locations — zero-court locations score highest on supply deficit.
+--   H3 catchment methodology: addressable market and supply deficit use a regional
 --   H3 catchment (res-5 cell + 6 neighbours, ~24km radius).
 --
+--   v5 changes: merge supply gap + catchment gap → single supply deficit (35 pts),
+--   add sports culture proxy (10 pts, tennis density), add construction affordability (5 pts),
+--   reduce economic power from 20 → 15 pts.
+--
 --   25 pts  addressable market        — log-scaled catchment population, ceiling 500K
---   20 pts  economic power     — income PPS, normalised to 35,000
---   30 pts  supply gap         — inverted catchment venue density; 0 courts = full marks
---   15 pts  catchment gap      — distance to nearest padel court
---   10 pts  sports culture     — tennis courts within 25km
+--   15 pts  economic power            — income PPS, normalised to 35,000
+--   35 pts  supply deficit            — max(density gap, distance gap); eliminates double-count
+--   10 pts  sports culture            — tennis court density as racquet-sport adoption proxy
+--    5 pts  construction affordability — income relative to construction costs (PLI)
+--   10 pts  market validation         — country-level avg market maturity (from market_scored CTE)
 --
 -- Consumers query directly with WHERE filters:
 --   cities API:       WHERE country_slug = ? AND city_slug IS NOT NULL
@@ -105,7 +113,7 @@ city_match AS (
     ORDER BY c.padel_venue_count DESC
   ) = 1
 ),
--- Pricing / occupancy from Playtomic (via city_slug) + H3 catchment
+-- Pricing / occupancy from Playtomic (via city_slug) + H3 catchment + country PLI
 with_pricing AS (
   SELECT
     b.*,
@@ -118,6 +126,7 @@ with_pricing AS (
     vpb.median_occupancy_rate,
     vpb.median_daily_revenue_per_venue,
     vpb.price_currency,
+    dc.pli_construction,
     COALESCE(ct.catchment_population, b.population)::BIGINT           AS catchment_population,
     COALESCE(ct.catchment_padel_courts, b.padel_venue_count)::INTEGER AS catchment_padel_courts
   FROM base b
@@ -129,9 +138,11 @@ with_pricing AS (
     AND cm.city_slug = vpb.city_slug
   LEFT JOIN catchment ct
     ON b.geoname_id = ct.geoname_id
+  LEFT JOIN foundation.dim_countries dc
+    ON b.country_code = dc.country_code
 ),
--- Both scores computed from the enriched base
-scored AS (
+-- Step 1: market score only — needed first so we can aggregate country averages.
+market_scored AS (
   SELECT *,
     -- City-level venue density (from dim_cities exact count, not dim_locations spatial 5km)
     CASE WHEN population > 0
@@ -144,34 +155,38 @@ scored AS (
       WHEN population > 0 OR  COALESCE(city_padel_venue_count, 0) > 0 THEN 0.5
       ELSE 0.0
     END AS data_confidence,
-    -- ── Market Score (Marktreife-Score v3) ──────────────────────────────────
+    -- ── Market Score (Marktreife-Score v4) ──────────────────────────────────
     -- 0 when no city match or no venues (city_padel_venue_count NULL or 0)
     CASE WHEN COALESCE(city_padel_venue_count, 0) > 0 THEN
       ROUND(
         -- Supply development (40 pts)
+        -- density ceiling 10/100k (LN(11)), count gate 3 venues
         40.0 * LEAST(1.0, LN(
             COALESCE(
               CASE WHEN population > 0
                 THEN COALESCE(city_padel_venue_count, 0)::DOUBLE / population * 100000
                 ELSE 0 END
-            , 0) + 1) / LN(21))
-             * LEAST(1.0, COALESCE(city_padel_venue_count, 0) / 5.0)
+            , 0) + 1) / LN(11))
+             * LEAST(1.0, COALESCE(city_padel_venue_count, 0) / 3.0)
         -- Demand evidence (25 pts)
+        -- with occupancy: scale to 65% target. Without: 65% of supply proxy + 0.3 floor
+        -- (existence of venues IS evidence of demand)
         + 25.0 * CASE
             WHEN median_occupancy_rate IS NOT NULL
               THEN LEAST(1.0, median_occupancy_rate / 0.65)
-            ELSE 0.4 * LEAST(1.0, LN(
+            ELSE GREATEST(0.3, 0.65 * LEAST(1.0, LN(
                 COALESCE(
                   CASE WHEN population > 0
                     THEN COALESCE(city_padel_venue_count, 0)::DOUBLE / population * 100000
                     ELSE 0 END
-                , 0) + 1) / LN(21))
-                     * LEAST(1.0, COALESCE(city_padel_venue_count, 0) / 5.0)
+                , 0) + 1) / LN(11))
+                     * LEAST(1.0, COALESCE(city_padel_venue_count, 0) / 3.0))
           END
         -- Addressable market (15 pts)
         + 15.0 * LEAST(1.0, LN(GREATEST(population, 1)) / LN(1000000))
         -- Economic context (10 pts)
-        + 10.0 * LEAST(1.0, COALESCE(median_income_pps, 100) / 200.0)
+        -- ceiling 25,000 PPS discriminates between wealthy and poorer markets
+        + 10.0 * LEAST(1.0, COALESCE(median_income_pps, 15000) / 25000.0)
         -- Data quality (10 pts)
         + 10.0 * CASE
             WHEN population > 0 AND COALESCE(city_padel_venue_count, 0) > 0 THEN 1.0
@@ -180,25 +195,57 @@ scored AS (
           END
       , 1)
     ELSE 0
-    END AS market_score,
-    -- ── Opportunity Score (Marktpotenzial-Score v3, H3 catchment) ──────────
+    END AS market_score
+  FROM with_pricing
+),
+-- Step 2: country-level avg market maturity — used as market validation signal (10 pts).
+-- Filter to market_score > 0 (cities with padel courts only) so zero-court locations
+-- don't dilute the country signal. ES proven demand → ~60, SE struggling → ~35.
+country_market AS (
+  SELECT
+    country_code,
+    ROUND(AVG(market_score), 1) AS country_avg_market_score
+  FROM market_scored
+  WHERE market_score > 0
+  GROUP BY country_code
+),
+-- Step 3: add opportunity_score using country market validation signal.
+scored AS (
+  SELECT ms.*,
+    -- ── Opportunity Score (Marktpotenzial-Score v5, H3 catchment) ──────────
     ROUND(
       -- Addressable market (25 pts): log-scaled catchment population, ceiling 500K
       25.0 * LEAST(1.0, LN(GREATEST(catchment_population, 1)) / LN(500000))
-      -- Economic power (20 pts): income PPS normalised to 35,000
-      + 20.0 * LEAST(1.0, COALESCE(median_income_pps, 15000) / 35000.0)
-      -- Supply gap (30 pts): inverted catchment venue density
-      + 30.0 * GREATEST(0.0, 1.0 - COALESCE(
+      -- Economic power (15 pts): income PPS normalised to 35,000
+      + 15.0 * LEAST(1.0, COALESCE(median_income_pps, 15000) / 35000.0)
+      -- Supply deficit (35 pts): max of density gap and distance gap.
+      -- Merges old supply gap (30) + catchment gap (15) which were ~80% correlated.
+      + 35.0 * GREATEST(
+          -- density-based gap (H3 catchment): 0 courts = 1.0, 8/100k = 0.0
+          GREATEST(0.0, 1.0 - COALESCE(
             CASE WHEN catchment_population > 0
-            THEN catchment_padel_courts::DOUBLE / catchment_population * 100000
+              THEN GREATEST(catchment_padel_courts, COALESCE(city_padel_venue_count, 0))::DOUBLE / catchment_population * 100000
               ELSE 0.0
-          END, 0.0) / 8.0)
-      -- Catchment gap (15 pts): distance to nearest court
-      + 15.0 * COALESCE(LEAST(1.0, nearest_padel_court_km / 30.0), 0.5)
-      -- Sports culture (10 pts): tennis courts within 25km
-      + 10.0 * LEAST(1.0, tennis_courts_within_25km / 10.0)
+            END, 0.0) / 8.0),
+          -- distance-based gap: 30km+ = 1.0, 0km = 0.0; NULL = 0.5
+          COALESCE(LEAST(1.0, nearest_padel_court_km / 30.0), 0.5)
+        )
+      -- Sports culture (10 pts): tennis density as racquet-sport adoption proxy.
+      -- Ceiling 50 courts within 25km. Harmless when tennis data is zero (contributes 0).
+      + 10.0 * LEAST(1.0, COALESCE(tennis_courts_within_25km, 0) / 50.0)
+      -- Construction affordability (5 pts): income purchasing power relative to build costs.
+      -- PLI construction is EU27=100 index. High income + low construction cost = high score.
+      + 5.0 * LEAST(1.0,
+          COALESCE(median_income_pps, 15000) / 35000.0
+          / GREATEST(0.5, COALESCE(pli_construction, 100.0) / 100.0)
+        )
+      -- Market validation (10 pts): country-level avg market maturity.
+      -- ES (~70/100): proven demand → ~7 pts. SE (~35/100): emerging → ~3.5 pts.
+      -- NULL (no courts in country yet): 0.5 neutral → 5 pts (untested, not penalised).
+      + 10.0 * COALESCE(cm.country_avg_market_score / 100.0, 0.5)
     , 1) AS opportunity_score
-  FROM with_pricing
+  FROM market_scored ms
+  LEFT JOIN country_market cm ON ms.country_code = cm.country_code
 )
 SELECT
   s.geoname_id,

transform/sqlmesh_padelnomics/models/serving/pseo_country_overview.sql

@@ -18,13 +18,14 @@ SELECT
   country_slug,
   COUNT(*)                                                        AS city_count,
   SUM(padel_venue_count)                                         AS total_venues,
-  ROUND(AVG(market_score), 1)                                    AS avg_market_score,
+  -- Population-weighted: large cities (Madrid, Barcelona) dominate, not hundreds of small towns
+  ROUND(SUM(market_score * population) / NULLIF(SUM(population), 0), 1) AS avg_market_score,
   MAX(market_score)                                              AS top_city_market_score,
   -- Top 5 cities by venue count (prominence), then score for internal linking
   LIST(city_slug ORDER BY padel_venue_count DESC, market_score DESC NULLS LAST)[1:5]     AS top_city_slugs,
   LIST(city_name ORDER BY padel_venue_count DESC, market_score DESC NULLS LAST)[1:5]     AS top_city_names,
-  -- Opportunity score aggregates (NULL-safe: cities without geoname_id match excluded from AVG)
-  ROUND(AVG(opportunity_score), 1)                               AS avg_opportunity_score,
+  -- Opportunity score aggregates (population-weighted: saturated megacities dominate, not hundreds of small towns)
+  ROUND(SUM(opportunity_score * population) / NULLIF(SUM(population), 0), 1) AS avg_opportunity_score,
   MAX(opportunity_score)                                         AS top_opportunity_score,
   -- Top 5 opportunity cities by population (prominence), then opportunity score
   LIST(city_slug ORDER BY population DESC, opportunity_score DESC NULLS LAST)[1:5] AS top_opportunity_slugs,
@@ -36,6 +37,8 @@ SELECT
   -- Use the most common currency in the country (MIN is deterministic for single-currency countries)
   MIN(price_currency)                                            AS price_currency,
   SUM(population)                                                AS total_population,
+  ROUND(SUM(lat * population) / NULLIF(SUM(population), 0), 4)  AS lat,
+  ROUND(SUM(lon * population) / NULLIF(SUM(population), 0), 4)  AS lon,
   CURRENT_DATE                                                   AS refreshed_date
 FROM serving.pseo_city_costs_de
 GROUP BY country_code, country_name_en, country_slug

web/src/padelnomics/admin/pipeline_routes.py

@@ -111,7 +111,7 @@ _DAG: dict[str, list[str]] = {
     "fct_daily_availability": ["fct_availability_slot", "dim_venue_capacity"],
     # Serving
     "venue_pricing_benchmarks": ["fct_daily_availability"],
-    "location_profiles": ["dim_locations", "dim_cities", "venue_pricing_benchmarks"],
+    "location_profiles": ["dim_locations", "dim_cities", "dim_countries", "venue_pricing_benchmarks"],
     "planner_defaults": ["venue_pricing_benchmarks", "location_profiles"],
     "pseo_city_costs_de": [
         "location_profiles", "planner_defaults",

web/src/padelnomics/admin/routes.py

@@ -27,6 +27,7 @@ from quart import (
 from ..auth.routes import role_required
 from ..core import (
     EMAIL_ADDRESSES,
+    REPO_ROOT,
     config,
     count_where,
     csrf_protect,
@@ -2182,7 +2183,7 @@ async def scenario_pdf(scenario_id: int):
 # Article Management
 # =============================================================================
 
-_ARTICLES_DIR = Path(__file__).parent.parent.parent.parent.parent / "data" / "content" / "articles"
+_ARTICLES_DIR = REPO_ROOT / "content" / "articles"
 _FRONTMATTER_RE = re.compile(r"^---\s*\n(.*?)\n---\s*\n", re.DOTALL)
 
 
@@ -2792,7 +2793,7 @@ async def article_new():
         (build_dir / f"{article_slug}.html").write_text(body_html)
 
         # Save markdown source
-        md_dir = Path("data/content/articles")
+        md_dir = REPO_ROOT / "content" / "articles"
         md_dir.mkdir(parents=True, exist_ok=True)
         (md_dir / f"{article_slug}.md").write_text(body)
 
@@ -2863,7 +2864,7 @@ async def article_edit(article_id: int):
             build_dir.mkdir(parents=True, exist_ok=True)
             (build_dir / f"{article['slug']}.html").write_text(body_html)
 
-            md_dir = Path("data/content/articles")
+            md_dir = REPO_ROOT / "content" / "articles"
             md_dir.mkdir(parents=True, exist_ok=True)
             (md_dir / f"{article['slug']}.md").write_text(body)
 
@@ -3054,7 +3055,7 @@ async def _rebuild_article(article_id: int):
         )
     else:
         # Manual article: re-render from markdown file
-        md_path = Path("data/content/articles") / f"{article['slug']}.md"
+        md_path = REPO_ROOT / "content" / "articles" / f"{article['slug']}.md"
         if not md_path.exists():
             return
         raw = md_path.read_text()

web/src/padelnomics/api.py

@@ -8,6 +8,7 @@ daily when the pipeline runs).
 from quart import Blueprint, abort, jsonify
 
 from .analytics import fetch_analytics
+from .auth.routes import login_required
 from .core import fetch_all, is_flag_enabled
 
 bp = Blueprint("api", __name__)
@@ -26,6 +27,7 @@ async def _require_maps_flag() -> None:
 
 
 @bp.route("/markets/countries.json")
+@login_required
 async def countries():
     """Country-level aggregates for the markets hub map."""
     await _require_maps_flag()
@@ -96,23 +98,3 @@ async def city_venues(country_slug: str, city_slug: str):
     )
     return jsonify(rows), 200, _CACHE_HEADERS
 
-
-@bp.route("/opportunity/<country_slug>.json")
-async def opportunity(country_slug: str):
-    """Location-level opportunity scores for the opportunity map."""
-    await _require_maps_flag()
-    assert country_slug, "country_slug required"
-    rows = await fetch_analytics(
-        """
-        SELECT location_name, location_slug, lat, lon,
-               opportunity_score, market_score,
-               nearest_padel_court_km,
-               padel_venue_count, population
-        FROM serving.location_profiles
-        WHERE country_slug = ? AND opportunity_score > 0
-        ORDER BY opportunity_score DESC
-        LIMIT 500
-        """,
-        [country_slug],
-    )
-    return jsonify(rows), 200, _CACHE_HEADERS

web/src/padelnomics/content/__init__.py

@@ -17,14 +17,14 @@ import yaml
 from jinja2 import ChainableUndefined, Environment
 
 from ..analytics import fetch_analytics
-from ..core import slugify, transaction, utcnow_iso
+from ..core import REPO_ROOT, slugify, transaction, utcnow_iso
 
 logger = logging.getLogger(__name__)
 
 # ── Constants ────────────────────────────────────────────────────────────────
 
 TEMPLATES_DIR = Path(__file__).parent / "templates"
-BUILD_DIR = Path("data/content/_build")
+BUILD_DIR = REPO_ROOT / "data" / "content" / "_build"
 
 # Threshold functions per template slug.
 # Return True → article should be noindex (insufficient data for quality content).

web/src/padelnomics/content/routes.py

@@ -9,7 +9,15 @@ from jinja2 import Environment, FileSystemLoader
 from markupsafe import Markup
 from quart import Blueprint, abort, g, redirect, render_template, request
 
-from ..core import capture_waitlist_email, csrf_protect, feature_gate, fetch_all, fetch_one
+from ..analytics import fetch_analytics
+from ..core import (
+    REPO_ROOT,
+    capture_waitlist_email,
+    csrf_protect,
+    feature_gate,
+    fetch_all,
+    fetch_one,
+)
 from ..i18n import get_translations
 
 bp = Blueprint(
@@ -18,7 +26,7 @@ bp = Blueprint(
     template_folder=str(Path(__file__).parent / "templates"),
 )
 
-BUILD_DIR = Path("data/content/_build")
+BUILD_DIR = REPO_ROOT / "data" / "content" / "_build"
 
 RESERVED_PREFIXES = (
     "/admin", "/auth", "/planner", "/billing", "/dashboard",
@@ -196,6 +204,14 @@ async def markets():
     )
 
     articles = await _filter_articles(q, country, region)
+    map_countries = await fetch_analytics("""
+        SELECT country_code, country_name_en, country_slug,
+               city_count, total_venues,
+               avg_market_score, avg_opportunity_score,
+               lat, lon
+        FROM serving.pseo_country_overview
+        ORDER BY total_venues DESC
+    """)
 
     return await render_template(
         "markets.html",
@@ -205,6 +221,7 @@ async def markets():
         current_q=q,
         current_country=country,
         current_region=region,
+        map_countries=map_countries,
     )
 
 

web/src/padelnomics/content/templates/markets.html

@@ -92,10 +92,8 @@
     });
   }
 
-  fetch('/api/markets/countries.json')
-    .then(function(r) { return r.json(); })
-    .then(function(data) {
-      if (!data.length) return;
+  var data = {{ map_countries | tojson }};
+  if (data.length) {
     var maxV = Math.max.apply(null, data.map(function(d) { return d.total_venues; }));
     var lang = document.documentElement.lang || 'en';
     data.forEach(function(c) {
@@ -112,7 +110,7 @@
         .on('click', function() { window.location = '/' + lang + '/markets/' + c.country_slug; })
         .addTo(map);
     });
-    });
+  }
 })();
 </script>
 {% endblock %}

web/src/padelnomics/core.py

@@ -27,6 +27,9 @@ from quart import g, make_response, render_template, request, session  # noqa: E
 
 load_dotenv()
 
+# Repo root: web/src/padelnomics/core.py → 4 levels up
+REPO_ROOT = Path(__file__).parents[3]
+
 
 def _env(key: str, default: str) -> str:
     """Get env var, treating empty string same as unset."""

web/src/padelnomics/public/routes.py

@@ -87,6 +87,46 @@ async def opportunity_map():
     return await render_template("opportunity_map.html", countries=countries)
 
 
+@bp.route("/opportunity-map/data")
+async def opportunity_map_data():
+    """HTMX partial: opportunity + reference data islands for Leaflet map."""
+    from ..core import is_flag_enabled
+    if not await is_flag_enabled("maps", default=True):
+        abort(404)
+    country_slug = request.args.get("country", "")
+    if not country_slug:
+        return ""
+    opp_points = await fetch_analytics(
+        """
+        SELECT location_name, location_slug, lat, lon,
+               opportunity_score, market_score,
+               nearest_padel_court_km, padel_venue_count, population
+        FROM serving.location_profiles
+        WHERE country_slug = ? AND opportunity_score > 0
+        ORDER BY opportunity_score DESC
+        LIMIT 500
+        """,
+        [country_slug],
+    )
+    ref_points = await fetch_analytics(
+        """
+        SELECT city_name, city_slug, lat, lon,
+               city_padel_venue_count AS padel_venue_count,
+               market_score, population
+        FROM serving.location_profiles
+        WHERE country_slug = ? AND city_slug IS NOT NULL
+        ORDER BY city_padel_venue_count DESC
+        LIMIT 200
+        """,
+        [country_slug],
+    )
+    return await render_template(
+        "partials/opportunity_map_data.html",
+        opp_points=opp_points,
+        ref_points=ref_points,
+    )
+
+
 @bp.route("/imprint")
 async def imprint():
     lang = g.get("lang", "en")

web/src/padelnomics/public/templates/opportunity_map.html

@@ -24,7 +24,10 @@
 
   <div class="card mb-4" style="padding: 1rem 1.25rem;">
     <label class="form-label" for="opp-country-select" style="margin-bottom: 0.5rem; display:block;">Select a country</label>
-    <select id="opp-country-select" class="form-input" style="max-width: 280px;">
+    <select id="opp-country-select" name="country" class="form-input" style="max-width:280px;"
+            hx-get="{{ url_for('public.opportunity_map_data') }}"
+            hx-target="#map-data"
+            hx-trigger="change">
       <option value="">— choose country —</option>
       {% for c in countries %}
       <option value="{{ c.country_slug }}">{{ c.country_name_en }}</option>
@@ -33,6 +36,7 @@
   </div>
 
   <div id="opportunity-map"></div>
+  <div id="map-data" style="display:none;"></div>
 
   <div class="mt-4 text-sm text-slate">
     <strong>Circle size:</strong> population &nbsp;|&nbsp;
@@ -86,18 +90,27 @@
          : (p || '');
   }
 
-  function loadCountry(slug) {
+  function renderMap() {
     oppLayer.clearLayers();
     refLayer.clearLayers();
-    if (!slug) return;
+    var oppEl = document.getElementById('opp-data');
+    var refEl = document.getElementById('ref-data');
+    if (!oppEl) return;
+    var oppData = JSON.parse(oppEl.textContent);
+    var refData = JSON.parse(refEl.textContent);
 
-    fetch('/api/opportunity/' + slug + '.json')
-      .then(function(r) { return r.json(); })
-      .then(function(data) {
-        if (!data.length) return;
-        var maxPop = Math.max.apply(null, data.map(function(d) { return d.population || 1; }));
+    refData.forEach(function(c) {
+      if (!c.lat || !c.lon || !c.padel_venue_count) return;
+      L.marker([c.lat, c.lon], { icon: REF_ICON })
+        .bindTooltip(c.city_name + ' — ' + c.padel_venue_count + ' existing venues',
+          { className: 'map-tooltip', direction: 'top', offset: [0, -7] })
+        .addTo(refLayer);
+    });
+
+    if (!oppData.length) return;
+    var maxPop = Math.max.apply(null, oppData.map(function(d) { return d.population || 1; }));
     var bounds = [];
-        data.forEach(function(loc) {
+    oppData.forEach(function(loc) {
       if (!loc.lat || !loc.lon) return;
       var size = 8 + 40 * Math.sqrt((loc.population || 1) / maxPop);
       var color = oppColor(loc.opportunity_score);
@@ -115,24 +128,10 @@
       bounds.push([loc.lat, loc.lon]);
     });
     if (bounds.length) map.fitBounds(bounds, { padding: [30, 30] });
-      });
-
-    // Existing venues as small gray reference dots (drawn first = behind opp dots)
-    fetch('/api/markets/' + slug + '/cities.json')
-      .then(function(r) { return r.json(); })
-      .then(function(data) {
-        data.forEach(function(c) {
-          if (!c.lat || !c.lon || !c.padel_venue_count) return;
-          L.marker([c.lat, c.lon], { icon: REF_ICON })
-            .bindTooltip(c.city_name + ' — ' + c.padel_venue_count + ' existing venues',
-              { className: 'map-tooltip', direction: 'top', offset: [0, -7] })
-            .addTo(refLayer);
-        });
-      });
   }
 
-  document.getElementById('opp-country-select').addEventListener('change', function() {
-    loadCountry(this.value);
+  document.body.addEventListener('htmx:afterSwap', function(e) {
+    if (e.detail.target.id === 'map-data') renderMap();
   });
 })();
 </script>

web/src/padelnomics/public/templates/partials/opportunity_map_data.html

@@ -0,0 +1,2 @@
+<script id="opp-data" type="application/json">{{ opp_points | tojson }}</script>
+<script id="ref-data" type="application/json">{{ ref_points | tojson }}</script>

web/src/padelnomics/worker.py

@@ -11,6 +11,7 @@ from datetime import datetime, timedelta
 
 from .core import (
     EMAIL_ADDRESSES,
+    REPO_ROOT,
     config,
     execute,
     fetch_all,
@@ -710,9 +711,8 @@ async def handle_run_extraction(payload: dict) -> None:
       If absent, runs all extractors via the umbrella `extract` entry point.
     """
     import subprocess
-    from pathlib import Path
 
-    repo_root = Path(__file__).resolve().parents[4]
+    repo_root = REPO_ROOT
     extractor = payload.get("extractor", "").strip()
     if extractor:
         cmd_name = f"extract-{extractor.replace('_', '-')}"
@@ -743,9 +743,8 @@ async def handle_run_transform(payload: dict) -> None:
     2-hour absolute timeout — same as extraction.
     """
     import subprocess
-    from pathlib import Path
 
-    repo_root = Path(__file__).resolve().parents[4]
+    repo_root = REPO_ROOT
     result = await asyncio.to_thread(
         subprocess.run,
         ["uv", "run", "sqlmesh", "-p", "transform/sqlmesh_padelnomics", "plan", "prod", "--auto-apply"],
@@ -769,9 +768,8 @@ async def handle_run_export(payload: dict) -> None:
     10-minute absolute timeout.
     """
     import subprocess
-    from pathlib import Path
 
-    repo_root = Path(__file__).resolve().parents[4]
+    repo_root = REPO_ROOT
     result = await asyncio.to_thread(
         subprocess.run,
         ["uv", "run", "python", "src/padelnomics/export_serving.py"],
@@ -791,9 +789,8 @@ async def handle_run_export(payload: dict) -> None:
 async def handle_run_pipeline(payload: dict) -> None:
     """Run full ELT pipeline: extract → transform → export, stopping on first failure."""
     import subprocess
-    from pathlib import Path
 
-    repo_root = Path(__file__).resolve().parents[4]
+    repo_root = REPO_ROOT
 
     steps = [
         (