# Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] ### Fixed - **pSEO error details collapse** — clicking "Error" on a job row expanded the details, but they collapsed after ~2s because HTMX polling replaced the ``. Jobs with errors now stop polling, keeping the `
` element stable. - **UNIQUE constraint on article slug** — `ON CONFLICT(url_path, language)` upsert failed because a separate single-column `UNIQUE` on `slug` fired first. Migration 0030 drops the redundant `UNIQUE` from `slug` (keeps the index for lookups and the composite `UNIQUE(url_path, language)`). - **Map country names** — 22 countries (PL, RO, CO, HU, ZA, KE, BR, CZ, QA, NZ, HR, LV, MT, CR, CY, PA, SV, DO, PE, VE, EE, ID) that appeared as bare ISO codes on the markets map and dropdown now show proper English/German names. Added country names to `dim_countries.sql`, `COUNTRY_LABELS` (i18n.py), and both locale files. Map tooltips and dropdown are now fully localised via `get_country_name()`. - **Map score tooltip clarity** — tooltip now shows both "Avg. Score" (country average) and "Top City" (highest location score) with separate color dots, making clear the map bubble color represents the country average — not a cap. ### Added - **Microsoft Clarity integration** — consent-gated heatmaps and session recordings (project ID via `CLARITY_PROJECT_ID` env var). Script only loads when the user has accepted functional cookies; bootstraps immediately on consent without requiring a reload. Privacy policy (EN + DE) updated with Clarity disclosure: data collection, sub-processor, cookies (`_clck`, `_clsk`), and international transfers. - **IndexNow integration** — push-notify Bing, Yandex, Seznam, and Naver when articles are published/unpublished/edited or suppliers are created. Bulk operations batch all URLs into a single request. Skips silently in dev (no key configured). Serves key verification file at `/{key}.txt`. ### Fixed - **SEO audit fixes** — sitemap: replaced `/market-score` with `/padelnomics-score`, added `/opportunity-map`, removed `/billing/pricing` (blocked by robots.txt), deduplicated articles query (was producing 4 entries per article instead of 2). Fixed `/market-score` redirect chain (1 hop instead of 2). Moved default OG tags inside `{% block head %}` so child templates replace rather than duplicate them. Added JSON-LD WebPage + BreadcrumbList schema to features, planner, and directory pages. Added meta descriptions to export pages. ### Changed - **Opportunity Score v7 → v8** — better spread and discrimination across the full 0-100 range. Addressable market weight reduced (20→15 pts) with steeper sqrt curve (ceiling 1M, was LN/500K). Economic power reduced (15→10 pts). Supply deficit increased (40→50 pts) with market existence dampener: countries with zero padel venues get max 5 pts supply deficit (factor 0.1), scaling linearly to full credit at 50+ venues. NULL nearest-court distance now treated as 0 (assume nearby) instead of 0.5. Added `country_percentile` output column (PERCENT_RANK within country). Target: P5-P95 spread ≥40 pts (was 22), zero-venue countries avg <30. - **Opportunity Score v6 → v7 (calibration fix)** — two fixes for inflated scores in saturated markets. (1) `dim_locations` now sources venue coordinates from `dim_venues` (deduplicated OSM + Playtomic) instead of `stg_padel_courts` (OSM only), making Playtomic-only venues visible to spatial lookups. (2) Country-level supply saturation dampener on the 40-pt supply deficit component: saturated countries (Spain ~4.5/100k) get dampened supply deficit (×0.55 → 22 pts max), emerging markets (Germany ~0.7/100k) are nearly unaffected (×0.93 → ~37 pts). - **Single-score simplification** — consolidated two public-facing scores (Market Score + Opportunity Score) into one **Padelnomics Score** (internally: `opportunity_score`). All maps, tooltips, article templates, and the methodology page now show a single score. Dual-ring markers reverted to single-color markers. `/market-score` route renamed to `/padelnomics-score` (old URL 301-redirects). All `mscore_*` i18n keys replaced with `pnscore_*`. Business plan queries `opportunity_score` from `location_profiles` (replaces legacy `city_market_overview` view). Map tooltip strings now i18n'd via `window.__MAP_T` (12 keys, EN + DE). ### Fixed - **Non-Latin city names on map** — GeoNames entries with CJK/Cyrillic/Arabic characters (e.g. "Seelow" showing Japanese) now filtered in `stg_population_geonames` via Latin-only regex. - **GeoNames regex DuckDB compatibility** — replaced Python-style `\u00C0` Unicode escapes in `stg_population_geonames` regex with literal Unicode characters (`À-ɏḀ-ỿ`) for DuckDB compatibility. - **Score range safety** — `location_profiles` clamps both scores to 0-100 via `LEAST/GREATEST`. - **Pipeline cast fix** — `venue_pricing_benchmarks.sql` defensively casts `snapshot_date` VARCHAR to DATE. ### Changed - **Dual-ring map markers** — map markers now encode two scores visually: inner core = primary score, outer ring = secondary score. Markets hub and country overview show Market Score (core) + Opportunity Score (ring). Opportunity map shows Opportunity Score (core) + Market Score (ring). City venue maps unchanged (navy dots). Color scale upgraded from 3-tier (green/amber/red) to 5-tier (deep green ≥80, teal ≥60, amber ≥40, orange-red ≥20, red <20) with distinct luminance at each tier for colorblind safety. Markers < 18px fall back to single-layer (no ring). Muted markers (cities without articles) show dashed ring outline. Highlighted markers (user's geo city) get blue outer glow. Opportunity map markers with score ≥75 pulse gently to highlight top investment targets. Tooltip lines now have inline color dots matching marker layers. All map scripts share a single `map-markers.js` module (`PNMarkers.scoreColor` + `PNMarkers.makeIcon`), replacing 3 duplicated implementations. ### Added - **Geo headers on city/region hubs** — Cloudflare geo headers (`CF-IPCountry`, `CF-IPCity`) now used across location-based pages. Opportunity map pre-selects and auto-loads the user's country. Country overview maps highlight the user's city with a blue ring (best-effort CF-IPCity name match). `window.__GEO` JS global injected via `base.html` for client-side map code. ### Fixed - **Opportunity map color scale** — low-score bubbles used blue (`#3B82F6`) instead of red (`#DC2626`), inconsistent with the unified `scoreColor()` scale used everywhere else. Fixed in `oppColor()`, legend, and `article-maps.js` tooltip colors. Thresholds aligned to ≥60/30/\<30. ### Changed - **Opportunity Score v5 → v6** — calibrates for saturated markets (Spain avg dropped from ~78 to ~50-60 range). Density ceiling lowered from 8 → 5/100k (Spain at 6-16/100k now hits zero-gap). Supply deficit weight increased from 35 → 40 pts. Addressable market reduced from 25 → 20 pts. Market validation inverted → "market headroom": high country avg maturity now reduces opportunity (saturated market = less room for new entrants). - **Markets page map legend** — bubble map now has a visual legend explaining size = venue count, color = Market Score. Opportunity score tooltip color unified to same green/amber/red scale (was using blue for low scores, inconsistent). - **Geo-localized article sorting** — `/markets` page sorts articles by user proximity using Cloudflare CF-IPCountry header. User's country first, nearby countries second (DACH, Iberia, Nordics, etc.), rest by published_at. Map bubbles re-ordered so user's country renders on top. Falls back to chronological order when header is absent (local dev). - **Score v6: Global economic data** — `dim_countries.median_income_pps` and `pli_construction` now cover all target markets, not just EU. World Bank WDI indicators (GNI per capita PPP + price level ratio) fill gaps for non-EU countries (AR, MX, AE, AU, etc.) with values calibrated to the Eurostat scale using Germany as anchor. EU countries keep exact Eurostat values. New extractor (`worldbank.py`), staging model (`stg_worldbank_income`), and `dim_countries` fallback CTEs. No changes to scoring formulas — the fix is upstream in the data layer. - **Market Score v3 → v4** — fixes Spain averaging 54 (should be 65-80). Four calibration changes: count gate threshold lowered from 5 → 3 venues (3 establishes a market pattern), density ceiling lowered from LN(21) → LN(11) (10/100k is reachable for mature markets), demand evidence fallback raised from 0.4 → 0.65 multiplier with 0.3 floor (existence of venues IS evidence of demand), economic context ceiling changed from income/200 → income/25000 (actual discrimination instead of free 10 pts for everyone). - **Opportunity Score v4 → v5** — fixes structural flaws: supply gap (30pts) + catchment gap (15pts) merged into single supply deficit (35pts, GREATEST of density gap and distance gap) eliminating ~80% correlated double-count. New sports culture signal (10pts) using tennis court density as racquet-sport adoption proxy. New construction affordability signal (5pts) using income relative to PLI construction costs from `dim_countries`. Economic power reduced from 20 → 15pts. New dependency on `foundation.dim_countries` for `pli_construction`. - **Unified `location_profiles` serving model** — merged `city_market_profile` and `location_opportunity_profile` into a single `serving.location_profiles` table at `(country_code, geoname_id)` grain. Both Marktreife-Score (Market Score) and Marktpotenzial-Score (Opportunity Score) are now computed per location. City data enriched via LEFT JOIN `dim_cities` on `geoname_id`. Downstream models (`planner_defaults`, `pseo_city_costs_de`, `pseo_city_pricing`) updated to query `location_profiles` directly. `city_padel_venue_count` (exact from dim_cities) distinguished from `padel_venue_count` (spatial 5km from dim_locations). - **Both scores on all map tooltips** — country map shows avg Market Score + avg Opportunity Score; city map shows Market Score + Opportunity Score per city; opportunity map shows Opportunity Score + Market Score per location. All score labels use the trademarked "Padelnomics Market Score" / "Padelnomics Opportunity Score" names. - **API endpoints** — `/api/markets/countries.json` adds `avg_opportunity_score`; `/api/markets//cities.json` adds `opportunity_score`; `/api/opportunity/.json` adds `market_score`. - **Marktpotenzial-Score v3: H3 catchment lens** — addressable market (25pts) and supply gap (30pts) now use a regional H3 catchment (~15-18km radius, res-4 cell + 6 neighbours, ~462km²) instead of local city population and 5km court count. Mid-size cities surrounded by dense Gemeinden (e.g. Oldenburg) now score correctly. New output columns: `catchment_population`, `catchment_padel_courts`, `catchment_venues_per_100k`. Requires one-time `INSTALL h3 FROM community` in DuckDB on each machine. ### Added - **Custom 404/500 error pages** — styled error pages extending `base.html` with i18n support (EN/DE). The 404 page is context-aware: when the URL matches `/markets/{country}/{city}`, it shows a city-specific message with a link back to the country overview instead of a generic "page not found". - **Map: city article indicators** — country overview map bubbles now differentiate cities with/without published articles. All cities retain score-based colors (green/amber/red); non-article cities are visually receded with lower opacity, dashed borders, desaturated color, and default cursor (no click). Tooltips show scores for all cities — article cities get "Click to explore →", non-article cities get "Coming soon". The `/api/markets//cities.json` endpoint includes a `has_article` boolean per city. ### Fixed - **Admin template preview maps** — Leaflet maps rendered blank because `article-maps.js` called `L.divIcon()` at the IIFE top level before Leaflet was dynamically loaded, crashing the script. Moved `VENUE_ICON` creation into the `script.onload` callback so it runs after Leaflet is available. Previous commit's `.card` `overflow: visible` fix remains (clips tile layers otherwise). - **Admin articles page 500** — `/admin/articles` crashed with `BuildError` when an article generation task was running because `article_stats.html` partial referenced `url_for('admin.article_stats')` but the route didn't exist. Added the missing HTMX partial endpoint. - **Silent 500 errors in dev** — `dev_run.sh` used Granian which swallowed Quart's debug error pages, showing generic "Internal Server Error" with no traceback. Switched to `uv run python -m padelnomics.app` for proper debug mode with browser tracebacks. Added `@app.errorhandler(500)` to log exceptions even when running under Granian in production. - **Pipeline diagnostic script** (`scripts/check_pipeline.py`) — handle DuckDB catalog naming quirk where `lakehouse.duckdb` uses catalog `lakehouse` instead of `local`, causing SQLMesh logical views to break. Script now auto-detects the catalog via `USE`, and falls back to querying physical tables (`sqlmesh__.__`) when views fail. - **Eurostat gas prices extractor** — `nrg_pc_203` filter missing `unit` dimension (API returns both KWH and GJ_GCV); now filters to `KWH`. - **Eurostat labour costs extractor** — `lc_lci_lev` used non-existent `currency` filter dimension; corrected to `unit: EUR`. - **Supervisor transform step** — changed `sqlmesh run` to `sqlmesh plan prod --auto-apply` so new/modified models are detected and applied automatically. ### Added - **Pipeline diagnostic script** (`scripts/check_pipeline.py`) — read-only script that reports row counts at every layer of the pricing pipeline (staging → foundation → serving), date range analysis, HAVING filter impact, and join coverage. Run on prod to diagnose empty serving tables. - **Extraction card descriptions** — each workflow card on the admin pipeline page now shows a one-line description explaining what the data source is (e.g. "EU geographic boundaries (NUTS2 polygons) from Eurostat GISCO"). Descriptions defined in `workflows.toml`. - **Running state indicator** — extraction cards show a spinner + "Running" label with a blue-tinted border when an extraction is actively running, replacing the plain Run button. Cards also display the start time with "running..." text. - **Interactive Leaflet maps** — geographic visualization across 4 key placements using self-hosted Leaflet 1.9.4 (GDPR-safe, no CDN): - **Markets hub** (`/markets`): country bubble map with circles sized by total venues, colored by avg market score (green ≥ 60, amber 30-60, red < 30). Click navigates to country overview. - **Country overview articles**: city bubble map loads after article render, auto-fits bounds, click navigates to city page. Bubbles colored by market score. - **City cost articles**: venue dot map centered on city lat/lon (zoom 13), navy dots per venue with tooltip showing name + court breakdown (indoor/outdoor). - **Opportunity map** (`//opportunity-map`): standalone full-width page with country selector. Circles sized by population, colored by opportunity score (green ≥ 70, amber 40-70, blue < 40). Existing venues shown as gray reference dots. - New `/api` blueprint with 4 JSON endpoints (`/api/markets/countries.json`, `/api/markets//cities.json`, `/api/markets///venues.json`, `/api/opportunity/.json`) — 1-hour public cache headers, all queries against `analytics.duckdb` via `fetch_analytics`. - New SQLMesh serving model `city_venue_locations` exposing venue lat/lon + court counts per city. - `pseo_city_costs_de` serving model: added `lat`/`lon` columns for city map data attributes in baked articles. - Leaflet CSS included on all article pages (5KB, cached). JS loaded dynamically only when a map container is present. - **Individualised article financial calculations with real per-country cost data** — ~30 CAPEX/OPEX calculator fields now scale to each country's actual cost level via Eurostat data, eliminating the identical DE-hardcoded numbers shown for every city globally. - **New Eurostat datasets extracted** (8 new landing files): electricity prices (`nrg_pc_205`), gas prices (`nrg_pc_203`), labour costs (`lc_lci_lev`), and 5 price level index categories from `prc_ppp_ind` (construction, housing, services, misc, government). - `extract/padelnomics_extract/src/padelnomics_extract/eurostat.py`: added 8 dataset entries; added `dataset_code` field support so multiple dict entries can share one Eurostat API endpoint (needed for 5 prc_ppp_ind variants). - **4 new staging models**: `stg_electricity_prices`, `stg_gas_prices`, `stg_labour_costs`, `stg_price_levels` — all read from landing zone with ISO code normalisation (EL→GR, UK→GB). - **New `foundation.dim_countries`** — conformed country dimension (grain: `country_code`). Consolidates country names/slugs and income data previously duplicated in `dim_cities` and `dim_locations` as ~50-line CASE blocks. Computes ~29 calculator cost override columns from Eurostat PLI indices and energy prices relative to DE baseline. - **Refactored `dim_cities`** — removed ~50-line CASE blocks and `country_income` CTE; JOIN `dim_countries` for `country_name_en`, `country_slug`, `median_income_pps`, `income_year`. - **Refactored `dim_locations`** — same refactor as `dim_cities`; income cascade still cascades EU NUTS-2 → US state → `dim_countries` country-level. - **Updated `serving.pseo_city_costs_de`** — JOIN `dim_countries`; 29 new camelCase override columns (`electricity`, `heating`, `rentSqm`, `hallCostSqm`, …, `permitsCompliance`) auto-applied by calculator. - **Updated `serving.planner_defaults`** — JOIN `dim_countries`; same 29 cost columns flow through to the planner API `/api/market-data` endpoint. - **Bulk actions for articles and leads** — checkbox selection + floating action bar on admin articles and leads pages (same pattern as suppliers). Articles: publish, unpublish, toggle noindex, rebuild, delete. Leads: set status, set heat. Re-renders results via HTMX after each action. - **Stripe payment provider** — second payment provider alongside Paddle, switchable via `PAYMENT_PROVIDER=stripe` env var. Existing Paddle subscribers keep working regardless of toggle — both webhook endpoints stay active. - `billing/stripe.py`: full Stripe implementation (Checkout Sessions, Billing Portal, subscription cancel, webhook verification + parsing) - `billing/paddle.py`: extracted Paddle-specific logic from routes.py into its own module - `billing/routes.py`: provider-agnostic dispatch layer — checkout, manage, cancel routes call `_provider().xxx()` - `_payment_js.html`: dual-path JS — conditionally loads Paddle.js SDK, universal `startCheckout()` handles both overlay (Paddle) and redirect (Stripe) - `scripts/setup_stripe.py`: mirrors `setup_paddle.py` — creates 17 products + prices in Stripe, registers webhook endpoint - Migration 0028: `payment_products` table generalizing `paddle_products` with `provider` column; existing Paddle rows copied - `get_price_id()` / `get_all_price_ids()` replace `get_paddle_price()` for provider-agnostic lookups - Stripe config vars: `STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET` - Dashboard boost buttons converted from inline `Paddle.Checkout.open()` to server round-trip via `/billing/checkout/item` endpoint - Stripe Tax add-on handles EU VAT (must be enabled in Stripe Dashboard) ### Fixed - **City slug transliteration** — replaced broken inline `REGEXP_REPLACE(LOWER(...), '[^a-z0-9]+', '-')` with new `@slugify` SQLMesh macro that uses `STRIP_ACCENTS` + `ß→ss` pre-replacement. Fixes: `Düsseldorf` → `dusseldorf` (was `d-sseldorf`), `Überlingen` → `uberlingen` (was `-berlingen`). Applied to `dim_venues`, `dim_cities`, `dim_locations`. Python `slugify()` in `core.py` updated to match. - **B2B article market links** — added missing language prefix (`/markets/germany` → `/de/markets/germany` and `/en/markets/germany`). Without the prefix, Quart interpreted `markets` as a language code → 500 error. - **Country overview top-5 city list** — changed ranking from raw `market_score DESC` (which inflated tiny towns with high density scores) to `padel_venue_count DESC` for top cities and `population DESC` for top opportunity cities. Germany now shows Berlin, Hamburg, München instead of Überlingen, Schwaigern. ### Changed - **CRO overhaul — homepage and supplier landing pages** — rewrote all copy from feature-focused ("60+ variables", "6 analysis tabs") to outcome-focused JTBD framing ("Invest in Padel with Confidence, Not Guesswork"). Based on JTBD analysis: the visitor's job is confidence committing €200K+, not "plan faster." - **Homepage hero**: new headline, description, and trust-building bullets (bank-ready metrics, real market data, free/no-signup) - **Proof strip**: live stats bar below hero (business plans created, suppliers, countries, project volume) - **"Sound familiar?" section**: replaces the 5-step journey timeline (3 items said "SOON") with 4 struggling-moment cards from JTBD research - **Feature cards reframed as outcomes**: "60+ Variables" → "Know Your Numbers Inside Out", "6 Analysis Tabs" → "Bank-Ready from Day One", "Sensitivity Analysis" → "Stress-Test Before You Commit", etc. - **"Why Padelnomics" comparison**: 3-column section (DIY Spreadsheet vs. Hired Consultant vs. Padelnomics) from JTBD Competitive Job Map - **FAQ rewritten**: customer-first questions ("How much does it cost to open a padel facility?", "Will a bank accept this?") replace product-internal questions - **Final CTA**: "Your Bank Meeting Is Coming. Be Ready." replaces generic "Start Planning Today" - **Supplier page**: "Is this your sales team?" struggling-moments section, conditional stats display (hides zeros), data-backed proof points replacing anonymous testimonials, ROI math moved above pricing, tier-specific CTAs - **Meta/SEO**: updated page title and description for search intent - All changes in both EN and DE (native-quality German, generisches Maskulinum) ### Fixed - **B2B article CTAs rewritten — all 12 now link to `/quote`** — zero articles previously linked to the quote lead-capture form. Each article's final section has been updated: - `padel-halle-bauen-de` / `padel-hall-build-guide-en`: replaced broken "directory" section (no link) with a contextual light-blue quote CTA block - `padel-halle-kosten-de` / `padel-hall-cost-guide-en`: planner mention linked to `/de/planner` / `/en/planner`; quote CTA block appended - `padel-halle-risiken-de` / `padel-hall-investment-risks-en`: planner sensitivity-tab mention linked; quote CTA block appended - `padel-halle-finanzierung-de` / `padel-hall-financing-germany-en`: quote CTA block appended after scenario card embed - `padel-standort-analyse-de` / `padel-hall-location-guide-en`: fixed broken `[→ Standortanalyse starten]` / `[→ Run a location analysis]` placeholders (no href) to `/de/planner` / `/en/planner`; quote CTA block appended - `padel-business-plan-bank-de` / `padel-business-plan-bank-requirements-en`: fixed broken `[→ Businessplan erstellen]` / `[→ Generate your business plan]` placeholders to `/de/planner` / `/en/planner`; quote CTA block appended - CTA copy is contextual per article (not identical boilerplate); uses the light-blue banner pattern (`.btn` class, `#EFF6FF` background) consistent with other generated articles - **Article editor preview now renders HTML correctly** — replaced the raw `{{ body_html }}` div (which Jinja2 auto-escaped to literal `

...

` text) with a sandboxed `