Compare commits
10 Commits
v202603081
...
v202603081
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8ced3a986e | ||
|
|
291fb2abd9 | ||
|
|
bfb0178615 | ||
|
|
40d8c75b81 | ||
|
|
d7bd053dc6 | ||
|
|
d379dc7551 | ||
|
|
814e8290a2 | ||
|
|
67fbfde53d | ||
|
|
bf811444ba | ||
|
|
3c135051fd |
10
CHANGELOG.md
10
CHANGELOG.md
@@ -6,7 +6,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Geo headers on city/region hubs** — Cloudflare geo headers (`CF-IPCountry`, `CF-IPCity`) now used across location-based pages. Opportunity map pre-selects and auto-loads the user's country. Country overview maps highlight the user's city with a blue ring (best-effort CF-IPCity name match). `window.__GEO` JS global injected via `base.html` for client-side map code.
|
||||
|
||||
### Fixed
|
||||
- **Opportunity map color scale** — low-score bubbles used blue (`#3B82F6`) instead of red (`#DC2626`), inconsistent with the unified `scoreColor()` scale used everywhere else. Fixed in `oppColor()`, legend, and `article-maps.js` tooltip colors. Thresholds aligned to ≥60/30/\<30.
|
||||
|
||||
### Changed
|
||||
- **Opportunity Score v5 → v6** — calibrates for saturated markets (Spain avg dropped from ~78 to ~50-60 range). Density ceiling lowered from 8 → 5/100k (Spain at 6-16/100k now hits zero-gap). Supply deficit weight increased from 35 → 40 pts. Addressable market reduced from 25 → 20 pts. Market validation inverted → "market headroom": high country avg maturity now reduces opportunity (saturated market = less room for new entrants).
|
||||
- **Markets page map legend** — bubble map now has a visual legend explaining size = venue count, color = Market Score. Opportunity score tooltip color unified to same green/amber/red scale (was using blue for low scores, inconsistent).
|
||||
- **Geo-localized article sorting** — `/markets` page sorts articles by user proximity using Cloudflare CF-IPCountry header. User's country first, nearby countries second (DACH, Iberia, Nordics, etc.), rest by published_at. Map bubbles re-ordered so user's country renders on top. Falls back to chronological order when header is absent (local dev).
|
||||
- **Score v6: Global economic data** — `dim_countries.median_income_pps` and `pli_construction` now cover all target markets, not just EU. World Bank WDI indicators (GNI per capita PPP + price level ratio) fill gaps for non-EU countries (AR, MX, AE, AU, etc.) with values calibrated to the Eurostat scale using Germany as anchor. EU countries keep exact Eurostat values. New extractor (`worldbank.py`), staging model (`stg_worldbank_income`), and `dim_countries` fallback CTEs. No changes to scoring formulas — the fix is upstream in the data layer.
|
||||
- **Market Score v3 → v4** — fixes Spain averaging 54 (should be 65-80). Four calibration changes: count gate threshold lowered from 5 → 3 venues (3 establishes a market pattern), density ceiling lowered from LN(21) → LN(11) (10/100k is reachable for mature markets), demand evidence fallback raised from 0.4 → 0.65 multiplier with 0.3 floor (existence of venues IS evidence of demand), economic context ceiling changed from income/200 → income/25000 (actual discrimination instead of free 10 pts for everyone).
|
||||
- **Opportunity Score v4 → v5** — fixes structural flaws: supply gap (30pts) + catchment gap (15pts) merged into single supply deficit (35pts, GREATEST of density gap and distance gap) eliminating ~80% correlated double-count. New sports culture signal (10pts) using tennis court density as racquet-sport adoption proxy. New construction affordability signal (5pts) using income relative to PLI construction costs from `dim_countries`. Economic power reduced from 20 → 15pts. New dependency on `foundation.dim_countries` for `pli_construction`.
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@ extract-census-usa-income = "padelnomics_extract.census_usa_income:main"
|
||||
extract-ons-uk = "padelnomics_extract.ons_uk:main"
|
||||
extract-geonames = "padelnomics_extract.geonames:main"
|
||||
extract-gisco = "padelnomics_extract.gisco:main"
|
||||
extract-worldbank = "padelnomics_extract.worldbank:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
|
||||
@@ -7,7 +7,7 @@ A graphlib.TopologicalSorter schedules them: tasks with no unmet dependencies
|
||||
run immediately in parallel; each completion may unlock new tasks.
|
||||
|
||||
Current dependency graph:
|
||||
- All 9 non-availability extractors have no dependencies (run in parallel)
|
||||
- All 10 non-availability extractors have no dependencies (run in parallel)
|
||||
- playtomic_availability depends on playtomic_tenants (starts as soon as
|
||||
tenants finishes, even if other extractors are still running)
|
||||
"""
|
||||
@@ -38,6 +38,8 @@ from .playtomic_availability import EXTRACTOR_NAME as AVAILABILITY_NAME
|
||||
from .playtomic_availability import extract as extract_availability
|
||||
from .playtomic_tenants import EXTRACTOR_NAME as TENANTS_NAME
|
||||
from .playtomic_tenants import extract as extract_tenants
|
||||
from .worldbank import EXTRACTOR_NAME as WORLDBANK_NAME
|
||||
from .worldbank import extract as extract_worldbank
|
||||
|
||||
logger = setup_logging("padelnomics.extract")
|
||||
|
||||
@@ -54,6 +56,7 @@ EXTRACTORS: dict[str, tuple] = {
|
||||
GEONAMES_NAME: (extract_geonames, []),
|
||||
GISCO_NAME: (extract_gisco, []),
|
||||
TENANTS_NAME: (extract_tenants, []),
|
||||
WORLDBANK_NAME: (extract_worldbank, []),
|
||||
AVAILABILITY_NAME: (extract_availability, [TENANTS_NAME]),
|
||||
}
|
||||
|
||||
|
||||
153
extract/padelnomics_extract/src/padelnomics_extract/worldbank.py
Normal file
153
extract/padelnomics_extract/src/padelnomics_extract/worldbank.py
Normal file
@@ -0,0 +1,153 @@
|
||||
"""World Bank WDI extractor — GNI per capita PPP and price level ratio.
|
||||
|
||||
Fetches two indicators (one API call each, no key required):
|
||||
- NY.GNP.PCAP.PP.CD — GNI per capita, PPP (international $)
|
||||
- PA.NUS.PPPC.RF — Price level ratio (PPP conversion factor / exchange rate)
|
||||
|
||||
These provide global fallbacks behind Eurostat for dim_countries.median_income_pps
|
||||
and dim_countries.pli_construction (see dim_countries.sql for calibration logic).
|
||||
|
||||
API: World Bank API v2 — https://api.worldbank.org/v2/
|
||||
No API key required. No env vars.
|
||||
|
||||
Landing: {LANDING_DIR}/worldbank/{year}/{month}/wdi_indicators.json.gz
|
||||
Output: {"rows": [{"country_code": "DE", "indicator": "NY.GNP.PCAP.PP.CD",
|
||||
"ref_year": 2023, "value": 74200.0}, ...], "count": N}
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
import niquests
|
||||
|
||||
from ._shared import HTTP_TIMEOUT_SECONDS, run_extractor, setup_logging
|
||||
from .utils import get_last_cursor, landing_path, write_gzip_atomic
|
||||
|
||||
logger = setup_logging("padelnomics.extract.worldbank")
|
||||
|
||||
EXTRACTOR_NAME = "worldbank"
|
||||
|
||||
INDICATORS = ["NY.GNP.PCAP.PP.CD", "PA.NUS.PPPC.RF"]
|
||||
# 6 years of data — we take the latest non-null per country in staging
|
||||
DATE_RANGE = "2019:2025"
|
||||
MAX_PER_PAGE = 5000
|
||||
MAX_PAGES = 3
|
||||
|
||||
WDI_BASE_URL = "https://api.worldbank.org/v2/country/all/indicator"
|
||||
|
||||
# WB aggregate codes that look like real 2-letter country codes.
|
||||
# These are regional/income-group aggregates, not actual countries.
|
||||
_WB_AGGREGATE_CODES = frozenset({
|
||||
"EU", "OE",
|
||||
"XC", "XD", "XE", "XF", "XG", "XH", "XI", "XJ", "XL", "XM",
|
||||
"XN", "XO", "XP", "XQ", "XR", "XS", "XT", "XU", "XV", "XY",
|
||||
"ZF", "ZG", "ZH", "ZI", "ZJ", "ZQ", "ZT",
|
||||
"V1", "V2", "V3", "V4",
|
||||
})
|
||||
|
||||
|
||||
def _normalize_country_code(wb_code: str) -> str | None:
|
||||
"""Normalize WB country code to ISO alpha-2. Returns None for aggregates."""
|
||||
code = wb_code.strip().upper()
|
||||
if len(code) != 2:
|
||||
return None
|
||||
# Reject codes starting with a digit (e.g. "1W" for World)
|
||||
if code[0].isdigit():
|
||||
return None
|
||||
if code in _WB_AGGREGATE_CODES:
|
||||
return None
|
||||
return code
|
||||
|
||||
|
||||
def _fetch_indicator(
|
||||
session: niquests.Session,
|
||||
indicator: str,
|
||||
) -> list[dict]:
|
||||
"""Fetch all records for one indicator. Returns list of row dicts."""
|
||||
rows: list[dict] = []
|
||||
page = 1
|
||||
|
||||
while page <= MAX_PAGES:
|
||||
url = (
|
||||
f"{WDI_BASE_URL}/{indicator}"
|
||||
f"?format=json&date={DATE_RANGE}&per_page={MAX_PER_PAGE}&page={page}"
|
||||
)
|
||||
logger.info("GET %s page %d", indicator, page)
|
||||
resp = session.get(url, timeout=HTTP_TIMEOUT_SECONDS * 2)
|
||||
resp.raise_for_status()
|
||||
|
||||
data = resp.json()
|
||||
assert isinstance(data, list) and len(data) == 2, (
|
||||
f"unexpected WB response shape for {indicator}: {type(data)}, len={len(data)}"
|
||||
)
|
||||
meta, records = data
|
||||
total_pages = meta.get("pages", 1)
|
||||
|
||||
if records is None:
|
||||
logger.warning("WB returned null data for %s page %d", indicator, page)
|
||||
break
|
||||
|
||||
for record in records:
|
||||
value = record.get("value")
|
||||
if value is None:
|
||||
continue
|
||||
country_code = _normalize_country_code(record["country"]["id"])
|
||||
if country_code is None:
|
||||
continue
|
||||
rows.append({
|
||||
"country_code": country_code,
|
||||
"indicator": indicator,
|
||||
"ref_year": int(record["date"]),
|
||||
"value": float(value),
|
||||
})
|
||||
|
||||
if page >= total_pages:
|
||||
break
|
||||
page += 1
|
||||
|
||||
return rows
|
||||
|
||||
|
||||
def extract(
|
||||
landing_dir: Path,
|
||||
year_month: str,
|
||||
conn: sqlite3.Connection,
|
||||
session: niquests.Session,
|
||||
) -> dict:
|
||||
"""Fetch WDI indicators. Skips if already run this month."""
|
||||
last_cursor = get_last_cursor(conn, EXTRACTOR_NAME)
|
||||
if last_cursor == year_month:
|
||||
logger.info("already have data for %s — skipping", year_month)
|
||||
return {"files_written": 0, "files_skipped": 1, "bytes_written": 0}
|
||||
|
||||
rows: list[dict] = []
|
||||
for indicator in INDICATORS:
|
||||
indicator_rows = _fetch_indicator(session, indicator)
|
||||
logger.info("%s: %d records", indicator, len(indicator_rows))
|
||||
rows.extend(indicator_rows)
|
||||
|
||||
assert len(rows) >= 200, f"expected ≥200 WB records, got {len(rows)} — API may have changed"
|
||||
logger.info("total: %d WDI records", len(rows))
|
||||
|
||||
year, month = year_month.split("/")
|
||||
dest_dir = landing_path(landing_dir, "worldbank", year, month)
|
||||
dest = dest_dir / "wdi_indicators.json.gz"
|
||||
payload = json.dumps({"rows": rows, "count": len(rows)}).encode()
|
||||
bytes_written = write_gzip_atomic(dest, payload)
|
||||
logger.info("written %s bytes compressed", f"{bytes_written:,}")
|
||||
|
||||
return {
|
||||
"files_written": 1,
|
||||
"files_skipped": 0,
|
||||
"bytes_written": bytes_written,
|
||||
"cursor_value": year_month,
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
run_extractor(EXTRACTOR_NAME, extract)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -72,3 +72,8 @@ description = "UK local authority population estimates from ONS"
|
||||
module = "padelnomics_extract.gisco"
|
||||
schedule = "0 0 1 1 *"
|
||||
description = "EU geographic boundaries (NUTS2 polygons) from Eurostat GISCO"
|
||||
|
||||
[worldbank]
|
||||
module = "padelnomics_extract.worldbank"
|
||||
schedule = "monthly"
|
||||
description = "GNI per capita PPP + price level ratio from World Bank WDI"
|
||||
|
||||
@@ -2,10 +2,14 @@
|
||||
--
|
||||
-- Consolidates data previously duplicated across dim_cities and dim_locations:
|
||||
-- - country_name_en / country_slug (was: ~50-line CASE blocks in both models)
|
||||
-- - median_income_pps (was: country_income CTE in both models)
|
||||
-- - energy prices, labour costs, PLI indices (new — from Eurostat datasets)
|
||||
-- - median_income_pps (Eurostat PPS preferred, World Bank GNI PPP fallback)
|
||||
-- - energy prices, labour costs, PLI indices (Eurostat, WB price level ratio fallback)
|
||||
-- - cost override columns for the financial calculator
|
||||
--
|
||||
-- World Bank fallback: for non-EU countries (AR, MX, AE, AU, etc.), income and PLI
|
||||
-- are derived from WB WDI indicators calibrated to the Eurostat scale using Germany
|
||||
-- as anchor. See de_calibration CTE. EU countries keep exact Eurostat values.
|
||||
--
|
||||
-- Used by: dim_cities, dim_locations, pseo_city_costs_de, planner_defaults.
|
||||
-- Grain: country_code (one row per ISO 3166-1 alpha-2 country code).
|
||||
-- Kind: FULL — small table (~40 rows), full refresh daily.
|
||||
@@ -82,6 +86,26 @@ de_elec AS (
|
||||
de_gas AS (
|
||||
SELECT gas_eur_gj FROM latest_gas WHERE country_code = 'DE'
|
||||
),
|
||||
-- Latest World Bank WDI per country (GNI PPP + price level ratio)
|
||||
latest_wb AS (
|
||||
SELECT country_code, gni_ppp, price_level_ratio, ref_year AS wb_year
|
||||
FROM staging.stg_worldbank_income
|
||||
WHERE gni_ppp IS NOT NULL OR price_level_ratio IS NOT NULL
|
||||
QUALIFY ROW_NUMBER() OVER (PARTITION BY country_code ORDER BY ref_year DESC) = 1
|
||||
),
|
||||
-- Germany calibration anchor: Eurostat PPS + WB GNI PPP + WB price ratio + Eurostat PLI construction.
|
||||
-- Used to scale World Bank values into Eurostat-comparable ranges.
|
||||
-- Single row; if DE is missing from any source, that ratio produces NULL (safe fallthrough).
|
||||
de_calibration AS (
|
||||
SELECT
|
||||
i.median_income_pps AS de_eurostat_pps,
|
||||
wb.gni_ppp AS de_gni_ppp,
|
||||
wb.price_level_ratio AS de_price_level_ratio,
|
||||
p.construction AS de_pli_construction
|
||||
FROM (SELECT median_income_pps FROM latest_income WHERE country_code = 'DE') i
|
||||
CROSS JOIN (SELECT gni_ppp, price_level_ratio FROM latest_wb WHERE country_code = 'DE') wb
|
||||
CROSS JOIN (SELECT construction FROM pli_pivoted WHERE country_code = 'DE') p
|
||||
),
|
||||
-- All distinct country codes from any source
|
||||
all_countries AS (
|
||||
SELECT country_code FROM latest_income
|
||||
@@ -93,6 +117,8 @@ all_countries AS (
|
||||
SELECT country_code FROM latest_labour
|
||||
UNION
|
||||
SELECT country_code FROM pli_pivoted
|
||||
UNION
|
||||
SELECT country_code FROM latest_wb
|
||||
-- Ensure known padel markets appear even if Eurostat doesn't cover them yet
|
||||
UNION ALL
|
||||
SELECT unnest(['DE','ES','GB','FR','IT','PT','AT','CH','NL','BE','SE','NO','DK','FI',
|
||||
@@ -149,15 +175,21 @@ SELECT
|
||||
ELSE ac.country_code
|
||||
END, '[^a-zA-Z0-9]+', '-'
|
||||
)) AS country_slug,
|
||||
-- Income data
|
||||
i.median_income_pps,
|
||||
i.income_year,
|
||||
-- Income: Eurostat PPS preferred, World Bank GNI PPP scaled to PPS as fallback
|
||||
COALESCE(
|
||||
i.median_income_pps,
|
||||
ROUND(wb.gni_ppp * (de_cal.de_eurostat_pps / NULLIF(de_cal.de_gni_ppp, 0)), 0)
|
||||
) AS median_income_pps,
|
||||
COALESCE(i.income_year, wb.wb_year) AS income_year,
|
||||
-- Raw energy and labour data (for reference / future staffed-scenario use)
|
||||
e.electricity_eur_kwh,
|
||||
g.gas_eur_gj,
|
||||
la.labour_cost_eur_hour,
|
||||
-- PLI indices per category (EU27=100)
|
||||
p.construction AS pli_construction,
|
||||
-- PLI construction: Eurostat preferred, World Bank price level ratio scaled to PLI as fallback
|
||||
COALESCE(
|
||||
p.construction,
|
||||
ROUND(wb.price_level_ratio / NULLIF(de_cal.de_price_level_ratio, 0) * de_cal.de_pli_construction, 1)
|
||||
) AS pli_construction,
|
||||
p.housing AS pli_housing,
|
||||
p.services AS pli_services,
|
||||
p.misc AS pli_misc,
|
||||
@@ -278,8 +310,10 @@ LEFT JOIN latest_electricity e ON ac.country_code = e.country_code
|
||||
LEFT JOIN latest_gas g ON ac.country_code = g.country_code
|
||||
LEFT JOIN latest_labour la ON ac.country_code = la.country_code
|
||||
LEFT JOIN pli_pivoted p ON ac.country_code = p.country_code
|
||||
LEFT JOIN latest_wb wb ON ac.country_code = wb.country_code
|
||||
CROSS JOIN de_pli de_p
|
||||
CROSS JOIN de_elec de_e
|
||||
CROSS JOIN de_gas de_g
|
||||
CROSS JOIN de_calibration de_cal
|
||||
-- Enforce grain
|
||||
QUALIFY ROW_NUMBER() OVER (PARTITION BY ac.country_code ORDER BY ac.country_code) = 1
|
||||
|
||||
@@ -19,22 +19,22 @@
|
||||
-- 10 pts economic context — income PPS normalised to 25,000 ceiling
|
||||
-- 10 pts data quality — completeness discount
|
||||
--
|
||||
-- Padelnomics Opportunity Score (Marktpotenzial-Score v5, 0–100):
|
||||
-- Padelnomics Opportunity Score (Marktpotenzial-Score v6, 0–100):
|
||||
-- "Where should I build a padel court?"
|
||||
-- Computed for ALL locations — zero-court locations score highest on supply deficit.
|
||||
-- H3 catchment methodology: addressable market and supply deficit use a regional
|
||||
-- H3 catchment (res-5 cell + 6 neighbours, ~24km radius).
|
||||
--
|
||||
-- v5 changes: merge supply gap + catchment gap → single supply deficit (35 pts),
|
||||
-- add sports culture proxy (10 pts, tennis density), add construction affordability (5 pts),
|
||||
-- reduce economic power from 20 → 15 pts.
|
||||
-- v6 changes: lower density ceiling 8→5/100k (saturated markets hit zero-gap sooner),
|
||||
-- increase supply deficit weight 35→40 pts, reduce addressable market 25→20 pts,
|
||||
-- invert market validation (high country maturity = LESS opportunity).
|
||||
--
|
||||
-- 25 pts addressable market — log-scaled catchment population, ceiling 500K
|
||||
-- 20 pts addressable market — log-scaled catchment population, ceiling 500K
|
||||
-- 15 pts economic power — income PPS, normalised to 35,000
|
||||
-- 35 pts supply deficit — max(density gap, distance gap); eliminates double-count
|
||||
-- 40 pts supply deficit — max(density gap, distance gap); eliminates double-count
|
||||
-- 10 pts sports culture — tennis court density as racquet-sport adoption proxy
|
||||
-- 5 pts construction affordability — income relative to construction costs (PLI)
|
||||
-- 10 pts market validation — country-level avg market maturity (from market_scored CTE)
|
||||
-- 10 pts market headroom — inverse country-level avg market maturity
|
||||
--
|
||||
-- Consumers query directly with WHERE filters:
|
||||
-- cities API: WHERE country_slug = ? AND city_slug IS NOT NULL
|
||||
@@ -198,9 +198,9 @@ market_scored AS (
|
||||
END AS market_score
|
||||
FROM with_pricing
|
||||
),
|
||||
-- Step 2: country-level avg market maturity — used as market validation signal (10 pts).
|
||||
-- Step 2: country-level avg market maturity — used as market headroom signal (10 pts).
|
||||
-- Filter to market_score > 0 (cities with padel courts only) so zero-court locations
|
||||
-- don't dilute the country signal. ES proven demand → ~60, SE struggling → ~35.
|
||||
-- don't dilute the country signal. Higher avg = more saturated = less headroom.
|
||||
country_market AS (
|
||||
SELECT
|
||||
country_code,
|
||||
@@ -212,21 +212,21 @@ country_market AS (
|
||||
-- Step 3: add opportunity_score using country market validation signal.
|
||||
scored AS (
|
||||
SELECT ms.*,
|
||||
-- ── Opportunity Score (Marktpotenzial-Score v5, H3 catchment) ──────────
|
||||
-- ── Opportunity Score (Marktpotenzial-Score v6, H3 catchment) ──────────
|
||||
ROUND(
|
||||
-- Addressable market (25 pts): log-scaled catchment population, ceiling 500K
|
||||
25.0 * LEAST(1.0, LN(GREATEST(catchment_population, 1)) / LN(500000))
|
||||
-- Addressable market (20 pts): log-scaled catchment population, ceiling 500K
|
||||
20.0 * LEAST(1.0, LN(GREATEST(catchment_population, 1)) / LN(500000))
|
||||
-- Economic power (15 pts): income PPS normalised to 35,000
|
||||
+ 15.0 * LEAST(1.0, COALESCE(median_income_pps, 15000) / 35000.0)
|
||||
-- Supply deficit (35 pts): max of density gap and distance gap.
|
||||
-- Merges old supply gap (30) + catchment gap (15) which were ~80% correlated.
|
||||
+ 35.0 * GREATEST(
|
||||
-- density-based gap (H3 catchment): 0 courts = 1.0, 8/100k = 0.0
|
||||
-- Supply deficit (40 pts): max of density gap and distance gap.
|
||||
-- Ceiling 5/100k (down from 8): Spain at 6-16/100k now hits zero-gap.
|
||||
+ 40.0 * GREATEST(
|
||||
-- density-based gap (H3 catchment): 0 courts = 1.0, 5/100k = 0.0
|
||||
GREATEST(0.0, 1.0 - COALESCE(
|
||||
CASE WHEN catchment_population > 0
|
||||
THEN GREATEST(catchment_padel_courts, COALESCE(city_padel_venue_count, 0))::DOUBLE / catchment_population * 100000
|
||||
ELSE 0.0
|
||||
END, 0.0) / 8.0),
|
||||
END, 0.0) / 5.0),
|
||||
-- distance-based gap: 30km+ = 1.0, 0km = 0.0; NULL = 0.5
|
||||
COALESCE(LEAST(1.0, nearest_padel_court_km / 30.0), 0.5)
|
||||
)
|
||||
@@ -239,10 +239,11 @@ scored AS (
|
||||
COALESCE(median_income_pps, 15000) / 35000.0
|
||||
/ GREATEST(0.5, COALESCE(pli_construction, 100.0) / 100.0)
|
||||
)
|
||||
-- Market validation (10 pts): country-level avg market maturity.
|
||||
-- ES (~70/100): proven demand → ~7 pts. SE (~35/100): emerging → ~3.5 pts.
|
||||
-- NULL (no courts in country yet): 0.5 neutral → 5 pts (untested, not penalised).
|
||||
+ 10.0 * COALESCE(cm.country_avg_market_score / 100.0, 0.5)
|
||||
-- Market headroom (10 pts): INVERSE country-level avg market maturity.
|
||||
-- High avg market score = saturated market = LESS opportunity for new entrants.
|
||||
-- ES (~46/100): proven demand, less headroom → ~5.4 pts.
|
||||
-- SE (~40/100): emerging → ~6 pts. NULL: 0.5 neutral → 5 pts.
|
||||
+ 10.0 * (1.0 - COALESCE(cm.country_avg_market_score / 100.0, 0.5))
|
||||
, 1) AS opportunity_score
|
||||
FROM market_scored ms
|
||||
LEFT JOIN country_market cm ON ms.country_code = cm.country_code
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
-- World Bank WDI indicators: GNI per capita PPP and price level ratio.
|
||||
-- Pivoted to one row per (country_code, ref_year) with both indicators as columns.
|
||||
--
|
||||
-- Source: data/landing/worldbank/{year}/{month}/wdi_indicators.json.gz
|
||||
-- Extracted by: worldbank.py
|
||||
-- Used by: dim_countries (fallback behind Eurostat for non-EU countries)
|
||||
|
||||
MODEL (
|
||||
name staging.stg_worldbank_income,
|
||||
kind FULL,
|
||||
cron '@daily',
|
||||
grain (country_code, ref_year)
|
||||
);
|
||||
|
||||
WITH parsed AS (
|
||||
SELECT
|
||||
row ->> 'country_code' AS country_code,
|
||||
TRY_CAST(row ->> 'ref_year' AS INTEGER) AS ref_year,
|
||||
row ->> 'indicator' AS indicator,
|
||||
TRY_CAST(row ->> 'value' AS DOUBLE) AS value,
|
||||
CURRENT_DATE AS extracted_date
|
||||
FROM (
|
||||
SELECT UNNEST(rows) AS row
|
||||
FROM read_json(
|
||||
@LANDING_DIR || '/worldbank/*/*/wdi_indicators.json.gz',
|
||||
auto_detect = true
|
||||
)
|
||||
)
|
||||
WHERE (row ->> 'country_code') IS NOT NULL
|
||||
)
|
||||
SELECT
|
||||
country_code,
|
||||
ref_year,
|
||||
MAX(value) FILTER (WHERE indicator = 'NY.GNP.PCAP.PP.CD') AS gni_ppp,
|
||||
MAX(value) FILTER (WHERE indicator = 'PA.NUS.PPPC.RF') AS price_level_ratio,
|
||||
MAX(extracted_date) AS extracted_date
|
||||
FROM parsed
|
||||
WHERE value IS NOT NULL
|
||||
AND value > 0
|
||||
AND LENGTH(country_code) = 2
|
||||
GROUP BY country_code, ref_year
|
||||
@@ -148,6 +148,18 @@ def create_app() -> Quart:
|
||||
# Per-request hooks
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
@app.before_request
|
||||
async def set_user_geo():
|
||||
"""Stash Cloudflare geo headers in g for proximity sorting.
|
||||
|
||||
Requires nginx: proxy_set_header CF-IPCountry $http_cf_ipcountry;
|
||||
proxy_set_header CF-RegionCode $http_cf_regioncode;
|
||||
proxy_set_header CF-IPCity $http_cf_ipcity;
|
||||
"""
|
||||
g.user_country = request.headers.get("CF-IPCountry", "").upper() or ""
|
||||
g.user_region = request.headers.get("CF-RegionCode", "") or ""
|
||||
g.user_city = request.headers.get("CF-IPCity", "") or ""
|
||||
|
||||
@app.before_request
|
||||
async def validate_lang():
|
||||
"""404 unsupported language prefixes (e.g. /fr/terms)."""
|
||||
@@ -234,6 +246,8 @@ def create_app() -> Quart:
|
||||
"csrf_token": get_csrf_token,
|
||||
"ab_variant": getattr(g, "ab_variant", None),
|
||||
"ab_tag": getattr(g, "ab_tag", None),
|
||||
"user_country": g.get("user_country", ""),
|
||||
"user_city": g.get("user_city", ""),
|
||||
"lang": effective_lang,
|
||||
"t": get_translations(effective_lang),
|
||||
"v": _ASSET_VERSION,
|
||||
|
||||
@@ -212,6 +212,13 @@ async def markets():
|
||||
FROM serving.pseo_country_overview
|
||||
ORDER BY total_venues DESC
|
||||
""")
|
||||
# Sort so user's country renders last (on top in Leaflet z-order)
|
||||
user_country = g.get("user_country", "")
|
||||
if user_country and map_countries:
|
||||
map_countries = sorted(
|
||||
map_countries,
|
||||
key=lambda c: 1 if c["country_code"] == user_country else 0,
|
||||
)
|
||||
|
||||
return await render_template(
|
||||
"markets.html",
|
||||
@@ -237,9 +244,46 @@ async def market_results():
|
||||
return await render_template("partials/market_results.html", articles=articles)
|
||||
|
||||
|
||||
_NEARBY_COUNTRIES: dict[str, tuple[str, ...]] = {
|
||||
"DE": ("AT", "CH"), "AT": ("DE", "CH"), "CH": ("DE", "AT"),
|
||||
"ES": ("PT",), "PT": ("ES",),
|
||||
"GB": ("IE",), "IE": ("GB",),
|
||||
"US": ("CA",), "CA": ("US",),
|
||||
"IT": ("CH",), "FR": ("BE", "CH"), "BE": ("FR", "NL", "DE"),
|
||||
"NL": ("BE", "DE"), "SE": ("NO", "DK", "FI"), "NO": ("SE", "DK"),
|
||||
"DK": ("SE", "NO", "DE"), "FI": ("SE",),
|
||||
"MX": ("US",), "BR": ("AR",), "AR": ("BR",),
|
||||
}
|
||||
|
||||
|
||||
def _geo_order_clause(user_country: str) -> tuple[str, list]:
|
||||
"""Build ORDER BY clause that sorts user's country first, nearby second.
|
||||
|
||||
Returns (order_sql, params) where order_sql starts with the geo CASE
|
||||
followed by published_at DESC. Caller prepends 'ORDER BY'.
|
||||
"""
|
||||
if not user_country:
|
||||
return "published_at DESC", []
|
||||
|
||||
nearby = _NEARBY_COUNTRIES.get(user_country, ())
|
||||
if nearby:
|
||||
placeholders = ",".join("?" * len(nearby))
|
||||
geo_case = f"""CASE WHEN country = ? THEN 0
|
||||
WHEN country IN ({placeholders}) THEN 1
|
||||
ELSE 2 END,
|
||||
published_at DESC"""
|
||||
return geo_case, [user_country, *nearby]
|
||||
|
||||
return """CASE WHEN country = ? THEN 0 ELSE 1 END,
|
||||
published_at DESC""", [user_country]
|
||||
|
||||
|
||||
async def _filter_articles(q: str, country: str, region: str) -> list[dict]:
|
||||
"""Query published articles for the current language."""
|
||||
"""Query published articles for the current language, geo-sorted."""
|
||||
lang = g.get("lang", "en")
|
||||
user_country = g.get("user_country", "")
|
||||
geo_order, geo_params = _geo_order_clause(user_country)
|
||||
|
||||
if q:
|
||||
# FTS query
|
||||
wheres = ["articles_fts MATCH ?"]
|
||||
@@ -253,14 +297,16 @@ async def _filter_articles(q: str, country: str, region: str) -> list[dict]:
|
||||
wheres.append("a.region = ?")
|
||||
params.append(region)
|
||||
where = " AND ".join(wheres)
|
||||
# Geo-sort references a.country
|
||||
order = geo_order.replace("country", "a.country")
|
||||
return await fetch_all(
|
||||
f"""SELECT a.* FROM articles a
|
||||
JOIN articles_fts ON articles_fts.rowid = a.id
|
||||
WHERE {where}
|
||||
AND a.status = 'published' AND a.published_at <= datetime('now')
|
||||
ORDER BY a.published_at DESC
|
||||
ORDER BY {order}
|
||||
LIMIT 100""",
|
||||
tuple(params),
|
||||
tuple(params + geo_params),
|
||||
)
|
||||
else:
|
||||
wheres = ["status = 'published'", "published_at <= datetime('now')", "language = ?"]
|
||||
@@ -274,8 +320,8 @@ async def _filter_articles(q: str, country: str, region: str) -> list[dict]:
|
||||
where = " AND ".join(wheres)
|
||||
return await fetch_all(
|
||||
f"""SELECT * FROM articles WHERE {where}
|
||||
ORDER BY published_at DESC LIMIT 100""",
|
||||
tuple(params),
|
||||
ORDER BY {geo_order} LIMIT 100""",
|
||||
tuple(params + geo_params),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -16,7 +16,22 @@
|
||||
<p class="text-slate">{{ t.mkt_subheading }}</p>
|
||||
</header>
|
||||
|
||||
<div id="markets-map" style="height:420px; border-radius:12px;" class="mb-6"></div>
|
||||
<div id="markets-map" style="height:420px; border-radius:12px;" class="mb-4"></div>
|
||||
|
||||
<!-- Map legend -->
|
||||
<div class="mb-6" style="display:flex; gap:1.5rem; align-items:center; font-size:0.82rem; color:#64748B;">
|
||||
<span style="display:flex; align-items:center; gap:0.35rem;">
|
||||
<span style="display:inline-block; width:12px; height:12px; border-radius:50%; background:#16A34A; border:2px solid white; box-shadow:0 1px 3px rgba(0,0,0,0.2);"></span>
|
||||
<span style="display:inline-block; width:18px; height:18px; border-radius:50%; background:#16A34A; border:2px solid white; box-shadow:0 1px 3px rgba(0,0,0,0.2);"></span>
|
||||
{{ t.mkt_legend_size }}
|
||||
</span>
|
||||
<span style="display:flex; align-items:center; gap:0.35rem;">
|
||||
<span style="display:inline-block; width:14px; height:14px; border-radius:50%; background:#16A34A; border:2px solid white; box-shadow:0 1px 3px rgba(0,0,0,0.2);"></span>
|
||||
<span style="display:inline-block; width:14px; height:14px; border-radius:50%; background:#D97706; border:2px solid white; box-shadow:0 1px 3px rgba(0,0,0,0.2);"></span>
|
||||
<span style="display:inline-block; width:14px; height:14px; border-radius:50%; background:#DC2626; border:2px solid white; box-shadow:0 1px 3px rgba(0,0,0,0.2);"></span>
|
||||
{{ t.mkt_legend_color }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<!-- Filters -->
|
||||
<div class="card mb-8">
|
||||
@@ -100,7 +115,7 @@
|
||||
if (!c.lat || !c.lon) return;
|
||||
var size = 12 + 44 * Math.sqrt(c.total_venues / maxV);
|
||||
var color = scoreColor(c.avg_market_score);
|
||||
var oppColor = c.avg_opportunity_score >= 60 ? '#16A34A' : (c.avg_opportunity_score >= 30 ? '#D97706' : '#3B82F6');
|
||||
var oppColor = scoreColor(c.avg_opportunity_score || 0);
|
||||
var tip = '<strong>' + c.country_name_en + '</strong><br>'
|
||||
+ c.total_venues + ' venues · ' + c.city_count + ' cities<br>'
|
||||
+ '<span style="color:' + color + ';font-weight:600;">Padelnomics Market Score: ' + c.avg_market_score + '/100</span><br>'
|
||||
|
||||
@@ -606,6 +606,8 @@
|
||||
"mkt_all_countries": "Alle Länder",
|
||||
"mkt_all_regions": "Alle Regionen",
|
||||
"mkt_no_results": "Keine Märkte gefunden. Passe Deine Filter an.",
|
||||
"mkt_legend_size": "Kreisgröße = Anzahl Anlagen",
|
||||
"mkt_legend_color": "Farbe = Market Score",
|
||||
"waitlist_markets_title": "Marktdaten — Demnächst verfügbar",
|
||||
"waitlist_markets_sub": "Detaillierte Marktberichte für Padel-Investoren: Baukosten, Umsatz-Benchmarks, Auslastungsdaten und ROI-Analysen nach Stadt und Region.",
|
||||
"waitlist_markets_feature1": "Echte Kostendaten aus laufenden Anlagen in über 30 Ländern",
|
||||
|
||||
@@ -606,6 +606,8 @@
|
||||
"mkt_all_countries": "All Countries",
|
||||
"mkt_all_regions": "All Regions",
|
||||
"mkt_no_results": "No markets found. Try adjusting your filters.",
|
||||
"mkt_legend_size": "Bubble size = venue count",
|
||||
"mkt_legend_color": "Color = Market Score",
|
||||
"waitlist_markets_title": "Markets Intelligence — Coming Soon",
|
||||
"waitlist_markets_sub": "Deep-dive market reports for padel investors: construction costs, revenue benchmarks, occupancy data, and ROI analysis by city and region.",
|
||||
"waitlist_markets_feature1": "Real cost data from operating venues across 30+ countries",
|
||||
|
||||
@@ -79,12 +79,23 @@ async def opportunity_map():
|
||||
if not await is_flag_enabled("maps", default=True):
|
||||
abort(404)
|
||||
countries = await fetch_analytics("""
|
||||
SELECT DISTINCT country_slug, country_name_en
|
||||
SELECT DISTINCT country_slug, country_name_en, country_code
|
||||
FROM serving.location_profiles
|
||||
WHERE city_slug IS NOT NULL
|
||||
ORDER BY country_name_en
|
||||
""")
|
||||
return await render_template("opportunity_map.html", countries=countries)
|
||||
user_cc = g.get("user_country", "")
|
||||
selected_slug = ""
|
||||
if user_cc:
|
||||
for c in countries:
|
||||
if c["country_code"] == user_cc:
|
||||
selected_slug = c["country_slug"]
|
||||
break
|
||||
countries = sorted(
|
||||
countries,
|
||||
key=lambda c: (0 if c["country_code"] == user_cc else 1, c["country_name_en"]),
|
||||
)
|
||||
return await render_template("opportunity_map.html", countries=countries, selected_slug=selected_slug)
|
||||
|
||||
|
||||
@bp.route("/opportunity-map/data")
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
hx-trigger="change">
|
||||
<option value="">— choose country —</option>
|
||||
{% for c in countries %}
|
||||
<option value="{{ c.country_slug }}">{{ c.country_name_en }}</option>
|
||||
<option value="{{ c.country_slug }}" {% if c.country_slug == selected_slug %}selected{% endif %}>{{ c.country_name_en }}</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
@@ -41,9 +41,9 @@
|
||||
<div class="mt-4 text-sm text-slate">
|
||||
<strong>Circle size:</strong> population |
|
||||
<strong>Color:</strong>
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#16A34A;vertical-align:middle;margin:0 4px"></span>High (≥70)
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#D97706;vertical-align:middle;margin:0 4px"></span>Mid (40–70)
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#3B82F6;vertical-align:middle;margin:0 4px"></span>Low (<40)
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#16A34A;vertical-align:middle;margin:0 4px"></span>High (≥60)
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#D97706;vertical-align:middle;margin:0 4px"></span>Mid (30–60)
|
||||
<span style="display:inline-block;width:10px;height:10px;border-radius:50%;background:#DC2626;vertical-align:middle;margin:0 4px"></span>Low (<30)
|
||||
</div>
|
||||
</main>
|
||||
{% endblock %}
|
||||
@@ -62,9 +62,9 @@
|
||||
var refLayer = L.layerGroup().addTo(map);
|
||||
|
||||
function oppColor(score) {
|
||||
if (score >= 70) return '#16A34A';
|
||||
if (score >= 40) return '#D97706';
|
||||
return '#3B82F6';
|
||||
if (score >= 60) return '#16A34A';
|
||||
if (score >= 30) return '#D97706';
|
||||
return '#DC2626';
|
||||
}
|
||||
|
||||
function makeIcon(size, color) {
|
||||
@@ -133,6 +133,10 @@
|
||||
document.body.addEventListener('htmx:afterSwap', function(e) {
|
||||
if (e.detail.target.id === 'map-data') renderMap();
|
||||
});
|
||||
|
||||
// Auto-load if country pre-selected via geo header
|
||||
var sel = document.getElementById('opp-country-select');
|
||||
if (sel.value) htmx.trigger(sel, 'change');
|
||||
})();
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@@ -892,6 +892,12 @@
|
||||
transform: scale(1.1);
|
||||
}
|
||||
|
||||
/* User's city highlight — blue ring on top of score-colored bubble */
|
||||
.pn-marker--highlight {
|
||||
border: 3px solid #3B82F6;
|
||||
box-shadow: 0 0 0 2px rgba(59, 130, 246, 0.3), 0 2px 8px rgba(0,0,0,0.28);
|
||||
}
|
||||
|
||||
/* Non-article city markers: faded + dashed border, no click affordance */
|
||||
.pn-marker--muted {
|
||||
opacity: 0.45;
|
||||
|
||||
@@ -49,7 +49,7 @@
|
||||
var pop = c.population >= 1000000
|
||||
? (c.population / 1000000).toFixed(1) + 'M'
|
||||
: (c.population >= 1000 ? Math.round(c.population / 1000) + 'K' : (c.population || ''));
|
||||
var oppColor = c.opportunity_score >= 60 ? '#16A34A' : (c.opportunity_score >= 30 ? '#D97706' : '#3B82F6');
|
||||
var oppColor = c.opportunity_score >= 60 ? '#16A34A' : (c.opportunity_score >= 30 ? '#D97706' : '#DC2626');
|
||||
var tip = '<strong>' + c.city_name + '</strong><br>'
|
||||
+ (c.padel_venue_count || 0) + ' venues'
|
||||
+ (pop ? ' · ' + pop : '')
|
||||
@@ -69,6 +69,26 @@
|
||||
bounds.push([c.lat, c.lon]);
|
||||
});
|
||||
if (bounds.length) map.fitBounds(bounds, { padding: [24, 24] });
|
||||
|
||||
// Highlight user's city (best-effort name match via CF-IPCity)
|
||||
var uc = (window.__GEO || {}).city || '';
|
||||
if (uc) {
|
||||
var match = data.find(function(c) {
|
||||
return c.city_name && c.city_name.toLowerCase() === uc.toLowerCase();
|
||||
});
|
||||
if (match && match.lat && match.lon) {
|
||||
var hSize = 10 + 36 * Math.sqrt((match.padel_venue_count || 1) / maxV);
|
||||
var hs = Math.round(hSize);
|
||||
var hColor = scoreColor(match.market_score);
|
||||
var hIcon = L.divIcon({
|
||||
className: '',
|
||||
html: '<div class="pn-marker pn-marker--highlight" style="width:' + hs + 'px;height:' + hs + 'px;background:' + hColor + ';"></div>',
|
||||
iconSize: [hs, hs],
|
||||
iconAnchor: [hs / 2, hs / 2],
|
||||
});
|
||||
L.marker([match.lat, match.lon], { icon: hIcon }).addTo(map);
|
||||
}
|
||||
}
|
||||
})
|
||||
.catch(function(err) { console.error('Country map fetch failed:', err); });
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@
|
||||
<meta property="og:image" content="{{ url_for('static', filename='images/logo.png', _external=True) }}">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
|
||||
<script>window.__GEO = {country: "{{ user_country }}", city: "{{ user_city }}"};</script>
|
||||
{% block head %}{% endblock %}
|
||||
</head>
|
||||
<body>
|
||||
|
||||
Reference in New Issue
Block a user