Three deviations from the quart_saas_boilerplate methodology corrected:
1. Fix dim_cities LIKE join (data quality bug)
- Old: FROM eurostat_cities LEFT JOIN venue_counts LIKE '%country_code%'
→ cartesian product (2.6M rows vs ~5500 expected)
- New: FROM venue_cities (dim_venues) as primary table, Eurostat for
enrichment only. grain (country_code, city_slug).
- Also fixes REGEXP_REPLACE to LOWER() before regex so uppercase city
names aren't stripped to '-'
2. Rename fct_venue_capacity → dim_venue_capacity
- Static venue attributes with no time key are a dimension, not a fact
- No SQL logic changes; update fct_daily_availability reference
3. Add fct_availability_slot at event grain
- New: grain (snapshot_date, tenant_id, resource_id, slot_start_time)
- Recheck dedup logic moves here from fct_daily_availability
- fct_daily_availability now reads fct_availability_slot (cleaner DAG)
Downstream fixes:
- city_market_profile, planner_defaults grain → (country_code, city_slug)
- pseo_city_costs_de, pseo_city_pricing add city_key composite natural key
(country_slug || '-' || city_slug) to avoid URL collisions across countries
- planner_defaults join in pseo_city_costs_de uses both country_code + city_slug
- Templates updated: natural_key city_slug → city_key
Added transform/sqlmesh_padelnomics/CLAUDE.md documenting data modeling rules,
conformed dimension map, and source integration architecture.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 18 new E2E tests from master: pricing, checkout, supplier signup,
supplier dashboard, and business plan export (sections J-N)
- Force WAITLIST_MODE=false in visual server subprocess — the root .env
sets WAITLIST_MODE=true, and since Config class attributes evaluate at
import time (before fork), the subprocess inherits the parent's value.
Patching both os.environ and core.config directly ensures feature pages
render instead of waitlist templates.
- All 77 visual tests now pass in ~59 seconds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
data/ is gitignored so seed files are not tracked. The recheck glob in
stg_playtomic_availability requires at least one matching file or DuckDB
throws IOException. Run 'make init-landing-seeds' after a fresh clone.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
stg_playtomic_availability:
- Add maximum_object_size = 134217728 (128 MB) to both read_json calls;
daily files exceed the 16 MB default as venue count grows
- Add seed recheck file (1970-01-01_recheck_00.json.gz, gitignored with data/)
to avoid READ_JSON IOException when no recheck files exist
pseo_city_costs_de + pseo_city_pricing:
- Add QUALIFY ROW_NUMBER() OVER (PARTITION BY city_slug ...) = 1 to
deduplicate rows caused by dim_cities' loose LIKE join; reduces
pseo_city_costs_de from 2.6M → 222 rows (one per unique city)
content/__init__.py:
- DuckDB lowercases all column names at rest ("ratePeak" → "ratepeak"),
so calc_overrides dict comprehension never matched DEFAULTS keys.
Fix: build case-insensitive reverse map {k.lower(): k} and normalise
row keys before lookup. Applied in both generate_articles() and
preview_article().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Consolidate 3 duplicate server processes into 1 session-scoped
live_server fixture in conftest.py (port 5111, shared across all
visual test modules). Reduces startup overhead from ~3× to 1×.
- Fix init_db mock: patch padelnomics.app.init_db (where it's used)
instead of core.init_db (where it's defined). The before_serving
hook imported init_db locally — patching core alone didn't prevent
the real init_db from replacing the in-memory test DB.
- Keep patches active through app.run_task() so before_serving hooks
can't replace the test DB during the server's lifetime.
- Force RESEND_API_KEY="" in the visual test server subprocess to
prevent real email sends (dev mode: prints to stdout, returns "dev").
- Remove 4 screenshot-only no-op tests, replace with single
test_capture_screenshots that grabs all pages in one pass.
- Fix test_planner_tab_switching: remove nonexistent "metrics" tab.
- Delete ~200 lines of duplicated boilerplate from 3 test files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
city-cost-de.md.jinja:
- Lead with market score hook instead of raw venue count
- Stats strip hero (venues, market score, peak rate, population)
- Better section headings ("What Does a Padel Investment Cost in X?")
- Mid-body planner CTA after financial cards
- Expanded FAQ (6 questions incl. ROI and country comparison)
- Footer cross-link to country overview page
- Fixed url_pattern to use country_slug directly
country-overview.md.jinja:
- Country hub page at /markets/{country_slug}
- Aggregates: total venues, cities, avg market score, pricing
- Top-5 cities table with internal links to city pages
- Hub-and-spoke internal linking architecture
city-pricing.md.jinja:
- Per-city pricing deep-dive at /markets/{country_slug}/{city_slug}/court-prices
- Stats strip: peak rate, off-peak, P25-P75 range, occupancy
- Pricing table + market context (above/below national median)
- Occupancy-driven pricing explanation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Grid layout (2-col mobile, 4-col sm+) with label/value/unit slots.
Baked into static HTML at article generation time — no JS needed.
output.css is git-ignored (rebuild with: bin/tailwindcss -i ... -o ...).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prerequisite for all pSEO serving models. Adds CASE-based country_name_en
and URL-safe country_slug to foundation.dim_cities, then selects them through
serving.city_market_profile so downstream models inherit them automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pricing page (EN/DE, plan cards, no-auth access)
- Checkout success (auth required, renders for authed user)
- Supplier signup wizard (step 1, plan cards, DE variant, success page)
- Supplier dashboard (overview stats, boosts/credit packs, listing, leads tabs)
- Business plan export (auth required, form renders)
Also fixes:
- E2e server init_db mock scope — before_serving was calling real init_db
outside the patch context, overwriting the in-memory DB (fixes 3
pre-existing failures: markets_hub, markets_results, signup_page)
- Add _seed_billing_data() for supplier + feature flags in e2e server
- Mock RESEND_API_KEY="" in conftest + e2e server to prevent real emails
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- deploy.sh installs sops/age to ./bin/ (no root/sudo needed)
- Remove CI deploy stage — supervisor auto-pulls and deploys
(zero CI secrets: no SSH keys, no deploy credentials)
- Supervisor sends alert on deploy success/failure via webhook
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On first deploy to a new server, deploy.sh:
1. Installs age and sops binaries if missing
2. Generates an age keypair if missing
3. Prints the public key and exits with instructions
All checks are idempotent — subsequent deploys skip to decryption.
Removed duplicate sops/age setup from setup_server.sh (deploy.sh handles it).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
.env.example replaced by .env.dev.sops. Added comment explaining that
.env.*.sops files are encrypted and committed, while .env is a
decrypted artifact that stays gitignored.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Installs age and sops binaries, generates an age keypair at
/opt/padelnomics/age-key.txt, and prints the public key in next
steps so it can be added to .sops.yaml.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
deploy.sh handles decryption on the server. CI only needs SSH credentials
(SSH_PRIVATE_KEY, SSH_KNOWN_HOSTS, DEPLOY_USER, DEPLOY_HOST). All app
secrets removed from GitLab CI variables. Dead ADMIN_PASSWORD removed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reads age key from /opt/padelnomics/age-key.txt (overridable via
SOPS_AGE_KEY_FILE env var). Decrypts .env.prod.sops → .env with
chmod 600.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Placeholder values (CHANGE_ME) for all secrets — fill via `make secrets-edit-prod`.
Includes all new supervisor, extraction, and SEO vars. Removes dead ADMIN_PASSWORD
and deprecated WAITLIST_MODE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-existing test failures after merge: enqueue payloads now include
'lang' key, and audience names changed from 'waitlist-auth' to
'newsletter' and 'waitlist-suppliers' to 'suppliers'.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_supervisor.py: 28 tests covering load_workflows, resolve_schedule,
is_due, topological_waves, and proxy round-robin / sticky selection
- test_feature_flags.py: 31 tests covering migration 0019, is_flag_enabled,
feature_gate decorator, admin toggle routes, and full toggle e2e flows
- conftest.py: seed feature flags with production defaults (markets=1,
others=0) so all routes behave consistently in tests
- Fix is_flag_enabled bug: replace non-existent db.execute_fetchone()
with fetch_one() helper
- Update 4 test_waitlist / test_businessplan tests that relied on
WAITLIST_MODE patches — now enable the relevant DB flag instead
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Landing files (append-only JSON.gz) synced to R2 every 30 min via
systemd timer + rclone. Extraction state DB (.state.sqlite) continuously
replicated via Litestream (second DB entry). Auto-restore on container
startup for both app.db and .state.sqlite. Reuses existing R2 bucket
and credentials — no new env vars needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload CMS documentation with the new SSG system:
git templates, DuckDB data sources, generation pipeline, SEO pipeline
(hreflang, JSON-LD, canonical, OG), admin routes, and step-by-step
guide for adding new pSEO ideas.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add full email management at /admin/emails with:
- email_log table tracking all outgoing emails with resend_id + delivery events
- inbound_emails table for Resend webhook-received messages
- Resend webhook handler (/webhooks/resend) updating delivery status in real-time
- send_email() returns resend_id (str|None) instead of bool; all 9 worker
handlers pass email_type= for per-type filtering
- Admin UI: sent log with HTMX filters, email detail with API-enriched HTML
preview, inbox with unread badges + reply, compose with branded wrapping,
audience management with contact list/remove
- Sidebar Email section with unread badge via blueprint context processor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload-based CMS with an SSG architecture where
templates live in git as .md.jinja files with YAML frontmatter and
data comes directly from DuckDB serving tables. Only articles and
published_scenarios remain in SQLite for routing/state.
- Content module: discover, load, generate, preview functions
- Migration 0018: drop article_templates + template_data, recreate
articles + published_scenarios without FK references, add
template_slug/language/date_modified/seo_head columns
- Admin routes: read-only template views with generate/regenerate/preview
- SEO pipeline: canonical URLs, hreflang (EN+DE), JSON-LD (Article,
FAQPage, BreadcrumbList), Open Graph tags baked at generation time
- Example template: city-cost-de.md.jinja for German city market data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9 tests exercise the full handler→wrap→send_email→Resend API path
using Resend's @resend.dev test addresses. Skipped when RESEND_API_KEY
is not set.
With a full_access API key, tests also retrieve the sent email via
resend.Emails.get() and assert on the rendered HTML (wordmark, links,
project details, heat badges). With a sending_access key, send is
verified but HTML assertions are skipped gracefully.
Includes bounce handling test and 0.6s inter-test delay for Resend's
2 req/sec rate limit.
Run with: RESEND_API_KEY=re_xxx pytest -k resend_live
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mock send_email and call each handler directly. Covers recipient,
subject content, HTML design elements (wordmark, preheader, heat
badges), from_addr, skip-on-missing-data guards, and email_sent_at
timestamp updates.
Also fixes IndexError in handle_send_welcome when payload has no name
("".split()[0] → safe fallback).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesigned _email_wrap(): lowercase wordmark header matching website,
3px blue accent border, preheader text support, HR separators.
_email_button() now full-width block for mobile tap targets.
Rewrote copy: improved subject lines, urgency cues, quick-start links
in welcome, styled project recap in quote verify, heat badges on lead
forward, "what happens next" in lead matched, secondary CTAs.
~30 new/updated translation keys in both EN and DE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add maximum_object_size=128MB to read_json for 14K-venue tenants file
- Rewrite opening_hours to use UNION ALL unpivot (DuckDB struct dynamic access)
- Add seed file guard for availability model (empty result on first run)
- Fix snapshot_date VARCHAR→DATE comparison in venue_pricing_benchmarks
- Fix export_serving to resolve SQLMesh physical tables from view definitions
(SQLMesh views reference "local" catalog unavailable outside its context)
- Add pyarrow dependency for Arrow-based cross-connection data transfer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Playtomic API ignores bbox params (min_latitude, etc.) and offset param.
Discovered that `page` param works correctly for global enumeration.
Result: 14,202 venues across 82 countries (was 100 with bbox approach).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
