Commit Graph

789 Commits

Author SHA1 Message Date
Deeman
236f0d1061 fix(markets): map country names, localised dropdown + avg/top score tooltip
- Expand dim_countries.sql CASE to cover 22 missing countries (PL, RO,
  CO, HU, ZA, KE, BR, CZ, QA, NZ, HR, LV, MT, CR, CY, PA, SV, DO,
  PE, VE, EE, ID) that fell through to bare ISO codes
- Add 19 missing entries to COUNTRY_LABELS (i18n.py) + both locale files
  (EN + DE dir_country_* keys) including IE which was in SQL but not i18n
- Localise map tooltips: routes.py injects country_name via
  get_country_name(), JS uses c.country_name instead of c.country_name_en
- Localise dropdown: apply country_name filter to option labels
- Show avg + top score in map tooltip with separate color dots and new
  map_score_avg / map_score_top i18n keys (EN: "Avg. Score" / "Top City",
  DE: "Ø Score" / "Top-Stadt")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:21:59 +01:00
Deeman
301f3b76c3 feat: add scripts/prod_query.py — SSH query tool for prod DuckDB
All checks were successful
CI / test (push) Successful in 56s
CI / tag (push) Successful in 3s
Runs read-only SQL against analytics.duckdb (default) or lakehouse.duckdb
on the prod server over SSH. SQL is base64-encoded to avoid shell escaping.
Supports TSV (default) and JSON output. Blocks mutation keywords.

For lakehouse, works around the DuckDB catalog naming issue (SQLMesh views
reference "local" but the file creates catalog "lakehouse") by attaching
the file as the "local" catalog.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603101516
2026-03-10 16:15:38 +01:00
Deeman
018eacb0f3 feat(analytics): add Microsoft Clarity with consent-gated loading
Some checks failed
CI / tag (push) Has been cancelled
CI / test (push) Has been cancelled
Gate Clarity behind functional cookie consent (TTDSG § 25 + GDPR).
Script loads on page if consent already given, bootstraps immediately
on banner accept without reload. Privacy policy (EN + DE) updated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 16:14:58 +01:00
Deeman
abacaac3f5 docs: add IndexNow integration to CHANGELOG
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 16:04:46 +01:00
Deeman
241e0de78e merge: IndexNow integration for instant Bing/Yandex URL submission 2026-03-10 15:53:46 +01:00
Deeman
fc21c25c82 feat(seo): add IndexNow integration for instant Bing/Yandex URL submission
Push-notify search engines (Bing, Yandex, Seznam, Naver) when content
changes instead of waiting for sitemap crawls. Especially valuable for
batch article publishing and supplier directory updates.

- Add INDEXNOW_KEY config var and key verification route
- New seo/_indexnow.py: async fire-and-forget POST to IndexNow API
- notify_indexnow() wrapper in sitemap.py expands paths to all lang variants
- Integrated at all article publish/unpublish/edit and supplier create points
- Bulk operations batch all URLs into a single IndexNow request
- Skips silently when key is empty (dev environments)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 15:52:45 +01:00
Deeman
bd7fa1ae9a fix(pipeline): stg_playtomic_availability glob reads all files, filters by date range
All checks were successful
CI / test (push) Successful in 1m0s
CI / tag (push) Successful in 3s
The @start_ds in the glob pattern only matched files for the first day
of the batch, so incremental restates only loaded 1 day of data.
Changed to wildcard glob with explicit BETWEEN @start_ds AND @end_ds
filter on the date column.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603101449
2026-03-10 15:48:10 +01:00
Deeman
511a0ebac7 fix(supervisor): always deploy web app on new tag
All checks were successful
CI / test (push) Successful in 1m0s
CI / tag (push) Successful in 3s
The previous approach diffed HEAD~1 vs HEAD to detect web/ changes,
but this missed changes inside merge commits (HEAD~1 IS the merge,
so the diff only saw the follow-up CHANGELOG commit). Result: web
containers never got rebuilt after merge-based pushes.

Simpler and deterministic: always run deploy.sh on every new tag.
Blue/green swap is zero-downtime and Docker layer caching makes
no-op builds fast (~10s). Removes web_code_changed() entirely.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603101049
2026-03-10 11:48:07 +01:00
Deeman
97ba13c42a docs: add SEO audit fixes to CHANGELOG
All checks were successful
CI / test (push) Successful in 56s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603100959
2026-03-10 10:56:19 +01:00
Deeman
1bd5bae90d merge: SEO audit fixes — sitemap, redirects, OG tags, schema markup 2026-03-10 10:53:59 +01:00
Deeman
608f16f578 fix(seo): pre-GSC audit — sitemap, redirects, OG tags, schema markup
- sitemap: replace /market-score with /padelnomics-score, add /opportunity-map,
  remove /billing/pricing (blocked by robots.txt), deduplicate articles query
- app: fix /market-score redirect chain (→ /en/padelnomics-score directly)
- base.html: move default OG tags inside {% block head %} so child overrides
  replace them instead of duplicating
- features, planner, directory: add JSON-LD WebPage + BreadcrumbList schema
- export pages: add meta descriptions and {% block head %} where missing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 10:53:20 +01:00
Deeman
927f77ae5e fix: country_supply column name in location_profiles
All checks were successful
CI / test (push) Successful in 55s
CI / tag (push) Successful in 3s
v202603100913
2026-03-10 10:12:09 +01:00
Deeman
adf6f0c1ef fix(score): country_supply uses dim_cities.padel_venue_count (not city_padel_venue_count)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 10:09:30 +01:00
Deeman
9dc705970e merge: Opportunity Score v8 — better spread/discrimination
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
# Conflicts:
#	CHANGELOG.md
v202603092125
2026-03-09 22:24:43 +01:00
Deeman
9c5bed01f5 docs: add Score v8 entry to CHANGELOG
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 22:15:34 +01:00
Deeman
3ce97cd41b docs(i18n): update methodology weights for Score v8
Addressable Market 20→15, Economic Power 15→10, Supply Deficit 40→50.
Update scaling description (LN/500K → SQRT/1M) and add existence
dampener explanation to supply deficit description.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 22:15:18 +01:00
Deeman
ff6401254a feat(score): Opportunity Score v8 — better spread/discrimination
Reweight: addressable market 20→15, economic power 15→10, supply deficit 40→50.
Supply deficit existence dampener (country_venues/50, floor 0.1): zero-venue
countries drop from ~80 to ~17. Steeper addressable market curve (LN/500K →
SQRT/1M). NULL distance gap → 0.0 (was 0.5). Added country_percentile output
column (PERCENT_RANK within country, 0–100).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 22:14:30 +01:00
Deeman
487722c2f3 chore: changelog + fix stg_population_geonames unicode escapes
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603092016
2026-03-09 20:32:49 +01:00
Deeman
23c7570736 merge: Opportunity Score v7 calibration fix 2026-03-09 18:12:47 +01:00
Deeman
e39dd4ec0b fix(score): Opportunity Score v7 — calibration fix for saturated markets
Two fixes:
1. dim_locations now sources venues from dim_venues (deduplicated OSM + Playtomic)
   instead of stg_padel_courts (OSM only). Playtomic-only venues are no longer
   invisible to spatial lookups.
2. Country-level supply saturation dampener on supply deficit component.
   Saturated countries (Spain 7.4/100k) get dampened supply deficit (x0.30 → 12 pts max).
   Emerging markets (Germany 0.24/100k) nearly unaffected (x0.98 → ~39 pts).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 18:03:14 +01:00
Deeman
cce3c466ba feat: single-score simplification — Padelnomics Score
All checks were successful
CI / test (push) Successful in 56s
CI / tag (push) Successful in 3s
Consolidate Market Score + Opportunity Score into one public-facing
"Padelnomics Score" (internally: opportunity_score column). Market Score
remains as internal pipeline component.

Changes across 7 phases:
- Pipeline: CAST fix, non-Latin city name filter, score clamp
- Maps: single-color markers, 5-tier color scale
- pSEO templates: unified score in city/country articles (EN + DE)
- Methodology page: /padelnomics-score (old URL 301-redirects)
- i18n: mscore_* → pnscore_* keys, map tooltip translations
- Business plan: queries opportunity_score from location_profiles
- Tests: all passing (1534), ruff clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603091323
2026-03-09 14:13:50 +01:00
Deeman
00d2e37934 chore: tests, changelog, project docs (Phase G)
- Rename test_market_score.py → test_padelnomics_score.py
- Test 301 redirects from old /market-score URL
- Update i18n parity allowlist (remove mscore_*, add pnscore brand terms)
- Update CHANGELOG.md with single-score simplification
- Update PROJECT.md: mark single-score done, fix location_profiles refs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:11:28 +01:00
Deeman
8e0dd6af63 fix(data): filter non-Latin city names + score range clamp (Phase F)
- stg_population_geonames: reject CJK/Cyrillic/Arabic city names via regex
  (fixes "Seelow" showing Japanese characters on map)
- dim_locations: filter empty location names after trim
- location_profiles: defensive LEAST/GREATEST clamp on both scores (0-100)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 12:23:50 +01:00
Deeman
eff50aef7d feat(i18n): translate map tooltip strings via __MAP_T (Phase E)
- Add 12 map_* keys to EN and DE locale files
- Inject window.__MAP_T from templates that load map scripts
- article-maps.js already used __MAP_T with fallbacks (no change needed)
- markets.html and opportunity_map.html inline scripts now use T.*
- Admin preview templates get EN fallback __MAP_T
- Fix mkt_legend_color: "Market Score" → "Padelnomics Score"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 12:21:44 +01:00
Deeman
5d0e52ade7 feat(score): methodology page, i18n, and business plan (Phase D)
- Rename /market-score → /padelnomics-score with 301 redirect
- Rewrite methodology page as single Padelnomics Score (pnscore_* keys)
- Replace all mscore_* i18n keys with pnscore_* in both EN and DE
- Business plan: query opportunity_score from location_profiles
- Footer link updated to padelnomics-score route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 12:18:47 +01:00
Deeman
cd95ff7b6c feat(templates): consolidate to single Padelnomics Score in pSEO articles
Phase C: Replace dual Market Score / Opportunity Score display with
single "Padelnomics Score" (backed by opportunity_score column) across
city-cost-de, city-pricing, and country-overview templates.
Remove dual-score FAQ sections, update threshold-based copy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 12:07:29 +01:00
Deeman
6d44c116aa feat(maps): single-color markers with Padelnomics Score
Phase B: Remove dual-ring marker design (ring/core), replace with
single-color markers colored by opportunity_score ("Padelnomics Score").
Simplify CSS, update tooltips to show one score line on all maps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 12:00:13 +01:00
Deeman
bda2f85fd6 fix(pipeline): CAST snapshot_date to DATE in venue_pricing_benchmarks
Phase A: defensive CAST for incremental time_column comparison.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 11:55:44 +01:00
Deeman
a47dfd5535 merge: dual-ring map markers with 5-tier color scale
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
v202603082118
2026-03-08 22:16:54 +01:00
Deeman
116a4272f1 feat(maps): dual-ring markers with 5-tier color scale
Replace flat circle markers with nested dual-ring structure encoding
two scores per marker (core = primary, ring = secondary). Upgrade
from 3-tier to 5-tier colorblind-safe scale. Add pulse animation for
high-opportunity markers (>=75). Extract shared PNMarkers module from
3 duplicated implementations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:16:45 +01:00
Deeman
8ced3a986e merge: geo headers on city/region hubs (opportunity map pre-select, city highlight, color fix)
All checks were successful
CI / test (push) Successful in 57s
CI / tag (push) Successful in 3s
v202603081951
2026-03-08 20:50:43 +01:00
Deeman
291fb2abd9 feat(geo): use CF headers on opportunity map + country overview maps
Pre-select user's country on opportunity map dropdown (CF-IPCountry),
auto-load the map on page load. Highlight user's city on country
overview maps with a blue ring (CF-IPCity best-effort match). Unify
opportunity score color scale to red/amber/green (was using blue for
low scores). Inject window.__GEO global for client-side geo access.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:43:52 +01:00
Deeman
bfb0178615 merge: markets page improvements — score v6, bubble UX, geo-localization 2026-03-08 20:27:01 +01:00
Deeman
40d8c75b81 feat: stash CF-RegionCode and CF-IPCity headers in g for future use
All three Cloudflare geo headers now available:
- g.user_country (CF-IPCountry) — used by geo-sorted article listing
- g.user_region (CF-RegionCode) — available for within-country sorting
- g.user_city (CF-IPCity) — available for city-level proximity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:26:50 +01:00
Deeman
d7bd053dc6 chore: update CHANGELOG for score v6, bubble UX, and geo-localization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:26:28 +01:00
Deeman
d379dc7551 feat(markets): geo-localize article + map sorting via CF-IPCountry
- Read Cloudflare CF-IPCountry header into g.user_country (before_request)
- _filter_articles() sorts user's country first, nearby countries second,
  then rest — falls back to published_at DESC when header is absent
- map_countries sorted so user's country bubble renders on top (Leaflet z-order)
- Nearby-country mapping covers DACH, Iberia, Nordics, Benelux, UK/IE, Americas

Prerequisite: Nginx must forward CF-IPCountry header to Quart (same as Umami).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:25:43 +01:00
Deeman
814e8290a2 fix(markets): add map legend + unify bubble color scales
- Add legend below map: bubble size = venue count, color = Market Score
- Unify opportunity score color to use same green/orange/red scale
  (was using blue for low scores, inconsistent with market score)
- Add mkt_legend_size / mkt_legend_color i18n keys (EN + DE)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:24:21 +01:00
Deeman
67fbfde53d feat(scoring): Opportunity Score v5 → v6 — calibrate for saturated markets
- Lower density ceiling 8→5/100k (Spain at 6-16/100k now hits zero-gap)
- Increase supply deficit weight 35→40 pts (primary differentiator)
- Reduce addressable market 25→20 pts (less weight on population alone)
- Invert market validation → market headroom (high country maturity = less opportunity)

Target: Spain avg opportunity drops from ~78 to ~50-60 range.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:23:08 +01:00
Deeman
bf811444ba merge: Score v6 — World Bank global economic data for non-EU countries
All checks were successful
CI / test (push) Successful in 56s
CI / tag (push) Successful in 3s
v202603081842
2026-03-08 19:40:57 +01:00
Deeman
3c135051fd feat(scoring): Score v6 — World Bank global economic data for non-EU countries
Non-EU countries (AR, MX, AE, AU, etc.) previously got NULL for
median_income_pps and pli_construction, falling back to EU-calibrated
defaults (15K PPS, PLI=100) that produced wrong scores.

New World Bank WDI extractor fetches GNI per capita PPP and price level
ratio for 215 countries. dim_countries uses Germany as calibration anchor
to scale WB values into the Eurostat range (dynamic ratio, self-corrects
as both sources update). EU countries keep exact Eurostat values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:17:33 +01:00
Deeman
c3847bb617 merge: Market Score v4 + Opportunity Score v5
All checks were successful
CI / test (push) Successful in 55s
CI / tag (push) Successful in 2s
v202603081442
2026-03-08 15:32:26 +01:00
Deeman
fcef47cb22 chore: update CHANGELOG + admin dependency graph for score v4/v5
- CHANGELOG.md: document Market Score v4 and Opportunity Score v5 changes
- pipeline_routes.py: add dim_countries to location_profiles dependency list

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:32:06 +01:00
Deeman
118c2c0fc7 feat(scoring): Opportunity Score v4 → v5 — fix correlated components
- Merge supply gap (30pts) + catchment gap (15pts) → supply deficit (35pts, GREATEST)
  Eliminates ~80% correlated double-count on a single signal.
- Add sports culture signal (10pts): tennis court density as racquet-sport adoption proxy.
  Ceiling 50 courts/25km. Harmless when tennis data is zero (contributes 0).
- Add construction affordability (5pts): income relative to PLI construction costs.
  Joins dim_countries.pli_construction. High income + low build cost = high score.
- Reduce economic power from 20 → 15pts to make room.

New weights: addressable market 25, economic power 15, supply deficit 35,
sports culture 10, construction affordability 5, market validation 10.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:30:04 +01:00
Deeman
cd6d950233 feat(scoring): Market Score v3 → v4 — fix Spain underscoring
- Lower count gate threshold: 5 → 3 venues (3 establishes a market pattern)
- Lower density ceiling: LN(21) → LN(11) (10/100k is reachable for mature markets)
- Better demand fallback: 0.4 → 0.65 multiplier + 0.3 floor (venues = demand evidence)
- Fix economic context: income/200 → income/25000 (actual discrimination vs free 10 pts)

Expected: Spain avg market score rises from ~54 to ~65-75.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:22:48 +01:00
Deeman
28e44384ef merge: opportunity map HTMX data islands + remove dead API endpoint
All checks were successful
CI / test (push) Successful in 1m0s
CI / tag (push) Successful in 3s
# Conflicts:
#	transform/sqlmesh_padelnomics/models/serving/location_opportunity_profile.sql
#	web/src/padelnomics/api.py
#	web/src/padelnomics/public/templates/opportunity_map.html
v202603072014
2026-03-07 21:05:52 +01:00
Deeman
b1e008a2a4 refactor(maps): opportunity map → HTMX data islands, remove dead API endpoint
- Delete opportunity() JSON endpoint from api.py (dead after this refactor)
- Add GET /opportunity-map/data route returning HTML partial with two JSON
  data islands (opp_points + ref_points from serving.location_profiles)
- Create partials/opportunity_map_data.html (2-line data island partial)
- Rewrite opportunity_map.html: HTMX attrs on <select>, invisible #map-data
  swap target, htmx:afterSwap listener replaces fetch()-based loadCountry()

city_venues endpoint stays public (article-maps.js calls it on public pages).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 20:56:45 +01:00
Deeman
d556ceecee fix(api): restore public access to country_cities + opportunity endpoints
opportunity_map.html (public page) still fetches these. Only countries.json
and city_venues.json are no longer called from any public page, so those two
keep @login_required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 20:41:12 +01:00
Deeman
f215ea8e3a fix: supply gap inflation + inline map data + guard API endpoints
A. location_profiles.sql: supply gap now uses GREATEST(catchment_padel_courts,
   COALESCE(city_padel_venue_count, 0)) so Playtomic venues prevent cities like
   Murcia/Cordoba/Gijon from receiving a full 30-pt supply gap bonus when their
   OSM catchment count is zero. Expected ~10-15 pt drop for affected ES cities.

B. pseo_country_overview.sql: add population-weighted lat/lon centroid columns
   so the markets map can use accurate country positions from this table.

C/D. content/routes.py + markets.html: query pseo_country_overview in the route
   and pass as map_countries to the template, replacing the fetch('/api/...') call
   with inline JSON. Map scores now match pseo_country_overview (pop-weighted),
   and the page loads without an extra round-trip.

E. api.py: add @login_required to all 4 endpoints. Unauthenticated callers get
   a 302 redirect to login instead of data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 20:33:31 +01:00
Deeman
b2ffad055b fix(supervisor): use file path for export_serving (not -m module syntax)
All checks were successful
CI / test (push) Successful in 1m0s
CI / tag (push) Successful in 3s
-m padelnomics.export_serving resolves to web package, not src/padelnomics.
src/padelnomics is not a uv workspace member so it's not importable by name.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202603071710
2026-03-07 18:08:46 +01:00
Deeman
544891611f feat(transform): opportunity score v4 — market validation + population-weighted aggregation
All checks were successful
CI / test (push) Successful in 57s
CI / tag (push) Successful in 2s
Two targeted fixes for inflated country scores (ES 83, SE 77):

1. pseo_country_overview: replace AVG() with population-weighted averages
   for avg_opportunity_score and avg_market_score. Madrid/Barcelona now
   dominate Spain's average instead of hundreds of 30K-town white-space
   towns. Expected ES drop from ~83 to ~55-65.

2. location_profiles: replace dead sports culture component (10 pts,
   tennis data all zeros) with market validation signal.
   Split scored CTE into: market_scored → country_market → scored.
   country_market aggregates AVG(market_score) per country from cities
   with padel courts (market_score > 0), so zero-court locations don't
   dilute the signal. ES (~60/100) → ~6 pts. SE (~35/100) → ~3.5 pts.
   NULL → 0.5 neutral → 5 pts (untested market, not penalised).

Score budget unchanged: 25+20+30+15+10 = 100 pts.
No new models, no new data sources, no cycles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202603071634
2026-03-07 17:23:11 +01:00