feat(transform): individualise article costs with per-country Eurostat data

Add real per-country cost data to ~30 calculator fields so pSEO articles
show country-specific CAPEX/OPEX instead of hardcoded DE defaults.

Extractor:
- eurostat.py: add 8 new datasets (nrg_pc_205, nrg_pc_203, lc_lci_lev,
  5×prc_ppp_ind variants); add optional `dataset_code` field so multiple
  dict entries can share one Eurostat API endpoint

Staging (4 new models):
- stg_electricity_prices — EUR/kWh by country, semi-annual
- stg_gas_prices         — EUR/GJ by country, semi-annual
- stg_labour_costs       — EUR/hour by country, annual (future staffed scenario)
- stg_price_levels       — PLI indices (EU27=100) for 5 categories, annual

Foundation:
- dim_countries (new) — conformed country dimension; eliminates ~50-line CASE
  blocks duplicated in dim_cities/dim_locations; computes ~29 calculator cost
  override columns from PLI ratios and energy price ratios vs DE baseline;
  NULL for DE so calculator falls through to DEFAULTS unchanged
- dim_cities — replace country_name/slug CASE blocks + country_income CTE
  with JOIN dim_countries
- dim_locations — same refactor as dim_cities

Serving:
- pseo_city_costs_de — JOIN dim_countries; add 29 camelCase override columns
  auto-applied by calculator (electricity, heating, rentSqm, hallCostSqm, …)
- planner_defaults — JOIN dim_countries; same 29 cost columns flow through
  to /api/market-data endpoint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-03-04 10:09:48 +01:00
parent ce466e3f7f
commit 2e68cfbe4f
12 changed files with 681 additions and 128 deletions

View File

@@ -7,6 +7,10 @@
-- 2. Country-level: median across cities in same country
-- 3. Hardcoded fallback: market research estimates (only when no Playtomic data)
--
-- Cost override columns from dim_countries (Eurostat PLI + energy price indices) are
-- included so the planner API pre-fills country-adjusted CAPEX/OPEX for all cities.
-- NULL = fall through to calculator.py DEFAULTS. DE always NULL (baseline preserved).
--
-- Units are explicit in column names. Monetary values in local currency.
MODEL (
@@ -125,6 +129,37 @@ SELECT
ELSE 0.2
END AS data_confidence,
COALESCE(cb.price_currency, ctb.price_currency, hf.currency, 'EUR') AS price_currency,
-- Cost override columns (Eurostat PLI + energy prices via dim_countries).
-- NULL = fall through to calculator.py DEFAULTS. DE always NULL (baseline).
dc.electricity,
dc.heating,
dc.rent_sqm,
dc.insurance,
dc.cleaning,
dc.maintenance,
dc.marketing,
dc.water,
dc.property_tax,
dc.outdoor_rent,
dc.hall_cost_sqm,
dc.foundation_sqm,
dc.land_price_sqm,
dc.hvac,
dc.electrical,
dc.sanitary,
dc.parking,
dc.fitout,
dc.planning,
dc.fire_protection,
dc.floor_prep,
dc.hvac_upgrade,
dc.lighting_upgrade,
dc.outdoor_foundation,
dc.outdoor_site_work,
dc.outdoor_lighting,
dc.outdoor_fencing,
dc.working_capital,
dc.permits_compliance,
CURRENT_DATE AS refreshed_date
FROM city_profiles cp
LEFT JOIN city_benchmarks cb
@@ -134,3 +169,5 @@ LEFT JOIN country_benchmarks ctb
ON cp.country_code = ctb.country_code
LEFT JOIN hardcoded_fallbacks hf
ON cp.country_code = hf.country_code
LEFT JOIN foundation.dim_countries dc
ON cp.country_code = dc.country_code

View File

@@ -4,6 +4,10 @@
--
-- Calculator override columns use camelCase to match the DEFAULTS keys in
-- planner/calculator.py, so they are auto-applied as calc pre-fills.
--
-- Cost override columns come from foundation.dim_countries (Eurostat PLI and energy
-- price indices). NULL = fall through to calculator.py DEFAULTS (safe: auto-mapping
-- filters None). DE always produces NULL overrides — preserves exact DEFAULTS behaviour.
MODEL (
name serving.pseo_city_costs_de,
@@ -44,6 +48,39 @@ SELECT
FLOOR(p.courts_typical) AS "dblCourts",
-- 'country' drives currency formatting in the calculator
c.country_code AS "country",
-- Cost override columns from dim_countries (Eurostat PLI + energy price indices).
-- NULL = fall through to calculator.py DEFAULTS. DE always NULL (baseline preserved).
-- OPEX overrides
cc.electricity AS "electricity",
cc.heating AS "heating",
cc.rent_sqm AS "rentSqm",
cc.insurance AS "insurance",
cc.cleaning AS "cleaning",
cc.maintenance AS "maintenance",
cc.marketing AS "marketing",
cc.water AS "water",
cc.property_tax AS "propertyTax",
cc.outdoor_rent AS "outdoorRent",
-- CAPEX overrides
cc.hall_cost_sqm AS "hallCostSqm",
cc.foundation_sqm AS "foundationSqm",
cc.land_price_sqm AS "landPriceSqm",
cc.hvac AS "hvac",
cc.electrical AS "electrical",
cc.sanitary AS "sanitary",
cc.parking AS "parking",
cc.fitout AS "fitout",
cc.planning AS "planning",
cc.fire_protection AS "fireProtection",
cc.floor_prep AS "floorPrep",
cc.hvac_upgrade AS "hvacUpgrade",
cc.lighting_upgrade AS "lightingUpgrade",
cc.outdoor_foundation AS "outdoorFoundation",
cc.outdoor_site_work AS "outdoorSiteWork",
cc.outdoor_lighting AS "outdoorLighting",
cc.outdoor_fencing AS "outdoorFencing",
cc.working_capital AS "workingCapital",
cc.permits_compliance AS "permitsCompliance",
CURRENT_DATE AS refreshed_date
FROM serving.city_market_profile c
LEFT JOIN serving.planner_defaults p
@@ -52,6 +89,8 @@ LEFT JOIN serving.planner_defaults p
LEFT JOIN serving.location_opportunity_profile lop
ON c.country_code = lop.country_code
AND c.geoname_id = lop.geoname_id
LEFT JOIN foundation.dim_countries cc
ON c.country_code = cc.country_code
-- Only cities with actual padel presence and at least some rate data
WHERE c.padel_venue_count > 0
AND (p.rate_peak IS NOT NULL OR c.median_peak_rate IS NOT NULL)