Add real per-country cost data to ~30 calculator fields so pSEO articles
show country-specific CAPEX/OPEX instead of hardcoded DE defaults.
Extractor:
- eurostat.py: add 8 new datasets (nrg_pc_205, nrg_pc_203, lc_lci_lev,
5×prc_ppp_ind variants); add optional `dataset_code` field so multiple
dict entries can share one Eurostat API endpoint
Staging (4 new models):
- stg_electricity_prices — EUR/kWh by country, semi-annual
- stg_gas_prices — EUR/GJ by country, semi-annual
- stg_labour_costs — EUR/hour by country, annual (future staffed scenario)
- stg_price_levels — PLI indices (EU27=100) for 5 categories, annual
Foundation:
- dim_countries (new) — conformed country dimension; eliminates ~50-line CASE
blocks duplicated in dim_cities/dim_locations; computes ~29 calculator cost
override columns from PLI ratios and energy price ratios vs DE baseline;
NULL for DE so calculator falls through to DEFAULTS unchanged
- dim_cities — replace country_name/slug CASE blocks + country_income CTE
with JOIN dim_countries
- dim_locations — same refactor as dim_cities
Serving:
- pseo_city_costs_de — JOIN dim_countries; add 29 camelCase override columns
auto-applied by calculator (electricity, heating, rentSqm, hallCostSqm, …)
- planner_defaults — JOIN dim_countries; same 29 cost columns flow through
to /api/market-data endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`-m padelnomics.export_serving` doesn't resolve because src/ is not
installed as a package in the workspace. Use the direct script path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three deviations from the quart_saas_boilerplate methodology corrected:
1. Fix dim_cities LIKE join (data quality bug)
- Old: FROM eurostat_cities LEFT JOIN venue_counts LIKE '%country_code%'
→ cartesian product (2.6M rows vs ~5500 expected)
- New: FROM venue_cities (dim_venues) as primary table, Eurostat for
enrichment only. grain (country_code, city_slug).
- Also fixes REGEXP_REPLACE to LOWER() before regex so uppercase city
names aren't stripped to '-'
2. Rename fct_venue_capacity → dim_venue_capacity
- Static venue attributes with no time key are a dimension, not a fact
- No SQL logic changes; update fct_daily_availability reference
3. Add fct_availability_slot at event grain
- New: grain (snapshot_date, tenant_id, resource_id, slot_start_time)
- Recheck dedup logic moves here from fct_daily_availability
- fct_daily_availability now reads fct_availability_slot (cleaner DAG)
Downstream fixes:
- city_market_profile, planner_defaults grain → (country_code, city_slug)
- pseo_city_costs_de, pseo_city_pricing add city_key composite natural key
(country_slug || '-' || city_slug) to avoid URL collisions across countries
- planner_defaults join in pseo_city_costs_de uses both country_code + city_slug
- Templates updated: natural_key city_slug → city_key
Added transform/sqlmesh_padelnomics/CLAUDE.md documenting data modeling rules,
conformed dimension map, and source integration architecture.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
