- Add _get_article_list() with filters: status, template, language, search
- Add _get_article_stats() for header stats strip (total/live/scheduled/draft)
- Add /articles/results HTMX partial endpoint
- Add filter bar: search input + status/template/language dropdowns
- Paginate at 50 articles per page
- Add "View" link to live articles (opens public URL in new tab)
- Remove URL column (redundant), add Language column
Subtasks 2+3 of CMS admin improvement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Compute display_status (live/scheduled/draft) in SQL instead of broken
Jinja string comparison against undefined `now` variable
- Replace template_data_id (dropped in migration 0018) with template_slug
Subtask 1/8 of CMS admin improvement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add section 9 to data-sources-inventory.md covering live API quirks:
Eurostat SDMX city labels response shape, ONS CSV download path (observations
API 404s), US Census ACS place endpoint, GeoNames cities15000 bulk format
- Add population coverage summary table and DuckDB glob limitation note
- fix(extract): census_usa + geonames write empty placeholder when credentials
absent so SQLMesh staging models don't fail with "no files found"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
eurostat_city_labels: API returns compact dimension JSON (category.label dict),
not SDMX 2.1 nested codelists structure. Fixed parser to read from
data["category"]["label"]. 1771 city codes fetched successfully.
ons_uk: observations endpoint (TS007A) is 404. Switched to CSV download via
/datasets/mid-year-pop-est/editions/mid-2022-england-wales — fetches ~68MB CSV,
filters to sex='all' + target year, aggregates population per LAD. 316 LADs ≥50K.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DuckDB rows can have NULL columns (e.g. market_score, median_peak_rate).
Replace None with 0 in render context so numeric Jinja2 filters like
round() and int don't crash with "NoneType doesn't define __round__".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add _datetimeformat Jinja2 filter to _render_pattern() — templates
use {{ 'now' | datetimeformat('%Y') }} but the filter was never
registered, causing "No filter named 'datetimeformat'" on preview.
- Move Preview button to first column in template detail data table
so it's visible without scrolling on wide tables.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Part B: Calculator improvements
Fix 1 — annualRevGrowth was dead code. Now applied as compound multiplier
from Year 2 onwards across all revenue streams (court, ancillary, membership,
F&B, coaching, retail).
Fix 2 — IRR initial outflow bug (HIGH). Was using -capex but NCFs are
post-debt-service (levered). Using capex as denominator while using levered
cash flows understates equity returns by the leverage ratio. Fix: use -equity
as outflow for equity IRR, add separate projectIrr (unlevered, uses -capex
with EBITDA flows).
Fix 3 — NPV at hurdle rate. Discounts equity NCFs and exit proceeds at
hurdleRate (default 12%). Reports npv and npvPositive. NPV > 0 iff equity
IRR > hurdle rate. Added hurdleRate slider (5–35%) to Exit settings.
Fix 4 — Remaining loan: replaces heuristic with correct amortization math
(PV of remaining payments: monthlyPayment × (1 - (1+r)^-(n-k)) / r).
Fix 5 — Exit EBITDA: uses terminal year EBITDA (holdYears - 1) instead of
hardcoded Year 3. exitValue now reflects actual exit year, not always Y3.
Fix 6 — MOIC: moic is now equity MOIC (total equity CFs / equity invested).
projectMoic is the project-level multiple. Waterfall updated to show both.
Fix 7 — Return decomposition / value bridge. Standard PE attribution:
EBITDA growth value (operational alpha) + debt paydown (financial leverage).
Displayed in tab_returns.html as an attribution table.
Fix 8 — OPEX growth rate. annualOpexGrowth (default 2%) inflates utilities,
staff, insurance from Year 2 onwards. Without this Y4-Y5 EBITDA was
systematically overstated. Added annualOpexGrowth slider to Exit settings.
Fix 9 — LTV and DSCR warnings. ltvWarning (>75%) and dscrWarning (<1.25x)
with inline warnings in tab_metrics.html.
Fix 10 — Interest-only period. interestOnlyMonths (0–24) reduces early NCF
drag. Added slider to Financing section.
Updated test: test_stab_ebitda_is_year3 → test_stab_ebitda_is_exit_year.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Part A: Data Layer — Sprints 1-5
Sprint 1 — Eurostat SDMX city labels (unblocks EU population):
- New extractor: eurostat_city_labels.py — fetches ESTAT/CITIES codelist
(city_code → city_name mapping) with ETag dedup
- New staging model: stg_city_labels.sql — grain city_code
- Updated dim_cities.sql — joins Eurostat population via city code lookup;
replaces hardcoded 0::BIGINT population
Sprint 2 — Market score formula v2:
- city_market_profile.sql: 30pt population (LN/1M), 25pt income PPS (/200),
30pt demand (occupancy or density), 15pt data confidence
- Moved venue_pricing_benchmarks join into base CTE so median_occupancy_rate
is available to the scoring formula
Sprint 3 — US Census ACS extractor:
- New extractor: census_usa.py — ACS 5-year place population (vintage 2023)
- New staging model: stg_population_usa.sql — grain (place_fips, ref_year)
Sprint 4 — ONS UK extractor:
- New extractor: ons_uk.py — 2021 Census LAD population via ONS beta API
- New staging model: stg_population_uk.sql — grain (lad_code, ref_year)
Sprint 5 — GeoNames global extractor:
- New extractor: geonames.py — cities15000.zip bulk download, filtered to ≥50K pop
- New staging model: stg_population_geonames.sql — grain geoname_id
- dim_cities: 5-source population cascade (Eurostat > Census > ONS > GeoNames > 0)
with case/whitespace-insensitive city name matching
Registered all 4 new CLI entrypoints in pyproject.toml and all.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the admin role was only granted during dev-login via
ensure_admin_role(), but dev_run.sh resets the DB on each start,
so the role was never present when first visiting /admin.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The multi-line python3 -c heredoc in the Makefile caused
"missing separator" errors since Make runs each recipe line
in a separate shell. Moved to web/scripts/init_landing_seeds.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add NTFY_TOKEN support to send_alert() — sends Authorization header
when token is set, backwards-compatible with plain webhook URLs
- Set ALERT_WEBHOOK_URL and NTFY_TOKEN in .env.prod.sops
- Add NTFY_TOKEN= placeholder in .env.dev.sops
- Topic: gWMeiHxj8ZqLbbqT (hard-to-guess, token-gated)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three deviations from the quart_saas_boilerplate methodology corrected:
1. Fix dim_cities LIKE join (data quality bug)
- Old: FROM eurostat_cities LEFT JOIN venue_counts LIKE '%country_code%'
→ cartesian product (2.6M rows vs ~5500 expected)
- New: FROM venue_cities (dim_venues) as primary table, Eurostat for
enrichment only. grain (country_code, city_slug).
- Also fixes REGEXP_REPLACE to LOWER() before regex so uppercase city
names aren't stripped to '-'
2. Rename fct_venue_capacity → dim_venue_capacity
- Static venue attributes with no time key are a dimension, not a fact
- No SQL logic changes; update fct_daily_availability reference
3. Add fct_availability_slot at event grain
- New: grain (snapshot_date, tenant_id, resource_id, slot_start_time)
- Recheck dedup logic moves here from fct_daily_availability
- fct_daily_availability now reads fct_availability_slot (cleaner DAG)
Downstream fixes:
- city_market_profile, planner_defaults grain → (country_code, city_slug)
- pseo_city_costs_de, pseo_city_pricing add city_key composite natural key
(country_slug || '-' || city_slug) to avoid URL collisions across countries
- planner_defaults join in pseo_city_costs_de uses both country_code + city_slug
- Templates updated: natural_key city_slug → city_key
Added transform/sqlmesh_padelnomics/CLAUDE.md documenting data modeling rules,
conformed dimension map, and source integration architecture.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 18 new E2E tests from master: pricing, checkout, supplier signup,
supplier dashboard, and business plan export (sections J-N)
- Force WAITLIST_MODE=false in visual server subprocess — the root .env
sets WAITLIST_MODE=true, and since Config class attributes evaluate at
import time (before fork), the subprocess inherits the parent's value.
Patching both os.environ and core.config directly ensures feature pages
render instead of waitlist templates.
- All 77 visual tests now pass in ~59 seconds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
data/ is gitignored so seed files are not tracked. The recheck glob in
stg_playtomic_availability requires at least one matching file or DuckDB
throws IOException. Run 'make init-landing-seeds' after a fresh clone.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
stg_playtomic_availability:
- Add maximum_object_size = 134217728 (128 MB) to both read_json calls;
daily files exceed the 16 MB default as venue count grows
- Add seed recheck file (1970-01-01_recheck_00.json.gz, gitignored with data/)
to avoid READ_JSON IOException when no recheck files exist
pseo_city_costs_de + pseo_city_pricing:
- Add QUALIFY ROW_NUMBER() OVER (PARTITION BY city_slug ...) = 1 to
deduplicate rows caused by dim_cities' loose LIKE join; reduces
pseo_city_costs_de from 2.6M → 222 rows (one per unique city)
content/__init__.py:
- DuckDB lowercases all column names at rest ("ratePeak" → "ratepeak"),
so calc_overrides dict comprehension never matched DEFAULTS keys.
Fix: build case-insensitive reverse map {k.lower(): k} and normalise
row keys before lookup. Applied in both generate_articles() and
preview_article().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Consolidate 3 duplicate server processes into 1 session-scoped
live_server fixture in conftest.py (port 5111, shared across all
visual test modules). Reduces startup overhead from ~3× to 1×.
- Fix init_db mock: patch padelnomics.app.init_db (where it's used)
instead of core.init_db (where it's defined). The before_serving
hook imported init_db locally — patching core alone didn't prevent
the real init_db from replacing the in-memory test DB.
- Keep patches active through app.run_task() so before_serving hooks
can't replace the test DB during the server's lifetime.
- Force RESEND_API_KEY="" in the visual test server subprocess to
prevent real email sends (dev mode: prints to stdout, returns "dev").
- Remove 4 screenshot-only no-op tests, replace with single
test_capture_screenshots that grabs all pages in one pass.
- Fix test_planner_tab_switching: remove nonexistent "metrics" tab.
- Delete ~200 lines of duplicated boilerplate from 3 test files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
city-cost-de.md.jinja:
- Lead with market score hook instead of raw venue count
- Stats strip hero (venues, market score, peak rate, population)
- Better section headings ("What Does a Padel Investment Cost in X?")
- Mid-body planner CTA after financial cards
- Expanded FAQ (6 questions incl. ROI and country comparison)
- Footer cross-link to country overview page
- Fixed url_pattern to use country_slug directly
country-overview.md.jinja:
- Country hub page at /markets/{country_slug}
- Aggregates: total venues, cities, avg market score, pricing
- Top-5 cities table with internal links to city pages
- Hub-and-spoke internal linking architecture
city-pricing.md.jinja:
- Per-city pricing deep-dive at /markets/{country_slug}/{city_slug}/court-prices
- Stats strip: peak rate, off-peak, P25-P75 range, occupancy
- Pricing table + market context (above/below national median)
- Occupancy-driven pricing explanation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Grid layout (2-col mobile, 4-col sm+) with label/value/unit slots.
Baked into static HTML at article generation time — no JS needed.
output.css is git-ignored (rebuild with: bin/tailwindcss -i ... -o ...).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prerequisite for all pSEO serving models. Adds CASE-based country_name_en
and URL-safe country_slug to foundation.dim_cities, then selects them through
serving.city_market_profile so downstream models inherit them automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pricing page (EN/DE, plan cards, no-auth access)
- Checkout success (auth required, renders for authed user)
- Supplier signup wizard (step 1, plan cards, DE variant, success page)
- Supplier dashboard (overview stats, boosts/credit packs, listing, leads tabs)
- Business plan export (auth required, form renders)
Also fixes:
- E2e server init_db mock scope — before_serving was calling real init_db
outside the patch context, overwriting the in-memory DB (fixes 3
pre-existing failures: markets_hub, markets_results, signup_page)
- Add _seed_billing_data() for supplier + feature flags in e2e server
- Mock RESEND_API_KEY="" in conftest + e2e server to prevent real emails
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- deploy.sh installs sops/age to ./bin/ (no root/sudo needed)
- Remove CI deploy stage — supervisor auto-pulls and deploys
(zero CI secrets: no SSH keys, no deploy credentials)
- Supervisor sends alert on deploy success/failure via webhook
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On first deploy to a new server, deploy.sh:
1. Installs age and sops binaries if missing
2. Generates an age keypair if missing
3. Prints the public key and exits with instructions
All checks are idempotent — subsequent deploys skip to decryption.
Removed duplicate sops/age setup from setup_server.sh (deploy.sh handles it).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
.env.example replaced by .env.dev.sops. Added comment explaining that
.env.*.sops files are encrypted and committed, while .env is a
decrypted artifact that stays gitignored.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Installs age and sops binaries, generates an age keypair at
/opt/padelnomics/age-key.txt, and prints the public key in next
steps so it can be added to .sops.yaml.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
deploy.sh handles decryption on the server. CI only needs SSH credentials
(SSH_PRIVATE_KEY, SSH_KNOWN_HOSTS, DEPLOY_USER, DEPLOY_HOST). All app
secrets removed from GitLab CI variables. Dead ADMIN_PASSWORD removed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reads age key from /opt/padelnomics/age-key.txt (overridable via
SOPS_AGE_KEY_FILE env var). Decrypts .env.prod.sops → .env with
chmod 600.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Placeholder values (CHANGE_ME) for all secrets — fill via `make secrets-edit-prod`.
Includes all new supervisor, extraction, and SEO vars. Removes dead ADMIN_PASSWORD
and deprecated WAITLIST_MODE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-existing test failures after merge: enqueue payloads now include
'lang' key, and audience names changed from 'waitlist-auth' to
'newsletter' and 'waitlist-suppliers' to 'suppliers'.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_supervisor.py: 28 tests covering load_workflows, resolve_schedule,
is_due, topological_waves, and proxy round-robin / sticky selection
- test_feature_flags.py: 31 tests covering migration 0019, is_flag_enabled,
feature_gate decorator, admin toggle routes, and full toggle e2e flows
- conftest.py: seed feature flags with production defaults (markets=1,
others=0) so all routes behave consistently in tests
- Fix is_flag_enabled bug: replace non-existent db.execute_fetchone()
with fetch_one() helper
- Update 4 test_waitlist / test_businessplan tests that relied on
WAITLIST_MODE patches — now enable the relevant DB flag instead
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Landing files (append-only JSON.gz) synced to R2 every 30 min via
systemd timer + rclone. Extraction state DB (.state.sqlite) continuously
replicated via Litestream (second DB entry). Auto-restore on container
startup for both app.db and .state.sqlite. Reuses existing R2 bucket
and credentials — no new env vars needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
