Reads age key from /opt/padelnomics/age-key.txt (overridable via
SOPS_AGE_KEY_FILE env var). Decrypts .env.prod.sops → .env with
chmod 600.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Placeholder values (CHANGE_ME) for all secrets — fill via `make secrets-edit-prod`.
Includes all new supervisor, extraction, and SEO vars. Removes dead ADMIN_PASSWORD
and deprecated WAITLIST_MODE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-existing test failures after merge: enqueue payloads now include
'lang' key, and audience names changed from 'waitlist-auth' to
'newsletter' and 'waitlist-suppliers' to 'suppliers'.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_supervisor.py: 28 tests covering load_workflows, resolve_schedule,
is_due, topological_waves, and proxy round-robin / sticky selection
- test_feature_flags.py: 31 tests covering migration 0019, is_flag_enabled,
feature_gate decorator, admin toggle routes, and full toggle e2e flows
- conftest.py: seed feature flags with production defaults (markets=1,
others=0) so all routes behave consistently in tests
- Fix is_flag_enabled bug: replace non-existent db.execute_fetchone()
with fetch_one() helper
- Update 4 test_waitlist / test_businessplan tests that relied on
WAITLIST_MODE patches — now enable the relevant DB flag instead
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Landing files (append-only JSON.gz) synced to R2 every 30 min via
systemd timer + rclone. Extraction state DB (.state.sqlite) continuously
replicated via Litestream (second DB entry). Auto-restore on container
startup for both app.db and .state.sqlite. Reuses existing R2 bucket
and credentials — no new env vars needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload CMS documentation with the new SSG system:
git templates, DuckDB data sources, generation pipeline, SEO pipeline
(hreflang, JSON-LD, canonical, OG), admin routes, and step-by-step
guide for adding new pSEO ideas.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add full email management at /admin/emails with:
- email_log table tracking all outgoing emails with resend_id + delivery events
- inbound_emails table for Resend webhook-received messages
- Resend webhook handler (/webhooks/resend) updating delivery status in real-time
- send_email() returns resend_id (str|None) instead of bool; all 9 worker
handlers pass email_type= for per-type filtering
- Admin UI: sent log with HTMX filters, email detail with API-enriched HTML
preview, inbox with unread badges + reply, compose with branded wrapping,
audience management with contact list/remove
- Sidebar Email section with unread badge via blueprint context processor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload-based CMS with an SSG architecture where
templates live in git as .md.jinja files with YAML frontmatter and
data comes directly from DuckDB serving tables. Only articles and
published_scenarios remain in SQLite for routing/state.
- Content module: discover, load, generate, preview functions
- Migration 0018: drop article_templates + template_data, recreate
articles + published_scenarios without FK references, add
template_slug/language/date_modified/seo_head columns
- Admin routes: read-only template views with generate/regenerate/preview
- SEO pipeline: canonical URLs, hreflang (EN+DE), JSON-LD (Article,
FAQPage, BreadcrumbList), Open Graph tags baked at generation time
- Example template: city-cost-de.md.jinja for German city market data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9 tests exercise the full handler→wrap→send_email→Resend API path
using Resend's @resend.dev test addresses. Skipped when RESEND_API_KEY
is not set.
With a full_access API key, tests also retrieve the sent email via
resend.Emails.get() and assert on the rendered HTML (wordmark, links,
project details, heat badges). With a sending_access key, send is
verified but HTML assertions are skipped gracefully.
Includes bounce handling test and 0.6s inter-test delay for Resend's
2 req/sec rate limit.
Run with: RESEND_API_KEY=re_xxx pytest -k resend_live
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mock send_email and call each handler directly. Covers recipient,
subject content, HTML design elements (wordmark, preheader, heat
badges), from_addr, skip-on-missing-data guards, and email_sent_at
timestamp updates.
Also fixes IndexError in handle_send_welcome when payload has no name
("".split()[0] → safe fallback).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesigned _email_wrap(): lowercase wordmark header matching website,
3px blue accent border, preheader text support, HR separators.
_email_button() now full-width block for mobile tap targets.
Rewrote copy: improved subject lines, urgency cues, quick-start links
in welcome, styled project recap in quote verify, heat badges on lead
forward, "what happens next" in lead matched, secondary CTAs.
~30 new/updated translation keys in both EN and DE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add maximum_object_size=128MB to read_json for 14K-venue tenants file
- Rewrite opening_hours to use UNION ALL unpivot (DuckDB struct dynamic access)
- Add seed file guard for availability model (empty result on first run)
- Fix snapshot_date VARCHAR→DATE comparison in venue_pricing_benchmarks
- Fix export_serving to resolve SQLMesh physical tables from view definitions
(SQLMesh views reference "local" catalog unavailable outside its context)
- Add pyarrow dependency for Arrow-based cross-connection data transfer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Playtomic API ignores bbox params (min_latitude, etc.) and offset param.
Discovered that `page` param works correctly for global enumeration.
Result: 14,202 venues across 82 countries (was 100 with bbox approach).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add lang parameter to all enqueue() calls for email internationalization.
Restructure Resend audiences to 3 named audiences (owners, suppliers, waitlist).
Use _t() translation function in all email template handlers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Markets now sits left of logo with Planner and Quotes (investor/demand
side). Mobile section headers use i18n keys instead of hardcoded English.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three workstreams:
1. Playtomic full data extraction & transform pipeline:
- Expand venue bounding boxes from 4 to 23 regions (global coverage)
- New staging models for court resources, opening hours, and slot-level
availability with real prices from the Playtomic API
- Foundation fact tables for venue capacity and daily occupancy/revenue
- City-level pricing benchmarks replacing hardcoded country estimates
- Planner defaults now use 3-tier cascade: city data → country → fallback
2. Transactional email i18n:
- _t() helper in worker.py with ~70 translation keys (EN + DE)
- All 8 email handlers translated, lang passed in task payloads
3. Resend audiences restructured to 3 named audiences (free plan limit)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract sitemap generation to sitemap.py with xhtml:link hreflang
alternates (en/de/x-default) on every URL entry. Add 1-hour in-memory
TTL cache with Cache-Control header. Include supplier pages in both
languages (were EN-only). Drop misleading "today" lastmod from static
pages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add requires_weasyprint marker to TestGenerateBusinessPlan and TestWorkerHandler
(these need libgobject/pango/cairo which CI python:3.12-slim lacks)
- Fix export route tests: use opaque tokens instead of integer IDs
- Replace deprecated datetime.utcnow() with datetime.now(UTC)
- Add missing jsonify/Response imports to admin routes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add /scenarios/<id>/pdf admin route for direct PDF generation via WeasyPrint.
Fix plan.html Jinja template: .items → ['items'] to avoid dict method collision.
Add scenario fixture in conftest.py and comprehensive test suite for business
plan sections, PDF generation, worker handler, and export routes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move historical docs from docs/ and .claude/ to research/. Add superseded
notice to research/PLAN.md. Add CHANGELOG entries for previous fixes.
New: PROJECT.md (task tracker), docs/MARKETING.md (marketing strategy).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eurostat JSON-stat format (4-7 dimension sparse dict with 583K values)
causes DuckDB OOM — pre-process in extractor to flat records.
Also fix dim_cities unused CTE bug and playtomic venue lat/lon path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Playtomic tenants API recycles results past its internal limit —
stop after 3 consecutive pages with zero new unique IDs.
Calculator tests: replace hardcoded default values (6 courts, specific
sqm/capex) with DEFAULTS references so tests don't break when
defaults change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove raw/ layer — staging models now read landing JSON directly.
Rename all model schemas from padelnomics.* to staging.*/foundation.*/serving.*.
Web app queries updated to serving.planner_defaults via SERVING_DUCKDB_PATH.
Supervisor gets daily sleep interval between pipeline runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split monolithic execute.py into per-source modules with separate CLI
entry points. Each extractor now uses the framework from utils.py:
- SQLite state tracking (start_run / end_run per extractor)
- Proper logging (replace print() with logger)
- Atomic gzip writes (write_gzip_atomic)
- Connection pooling (niquests.Session)
- Bounded pagination (MAX_PAGES_PER_BBOX = 500)
New entry points:
extract — run all 4 extractors sequentially
extract-overpass — OSM padel courts
extract-eurostat — city demographics (etag dedup)
extract-playtomic-tenants — venue listings
extract-playtomic-availability — booking slots + pricing (NEW)
The availability extractor reads tenant IDs from the latest tenants.json.gz,
queries next-day slots for each venue, and stores daily consolidated snapshots.
Supports resumability via cursor and retry with backoff.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pulls in template changes: export_serving.py for atomic DuckDB swap,
supervisor export step, SQLMesh glob macro, server provisioning script,
imprint template, and formatting improvements.
Template scaffold SQL models excluded (padelnomics has real models).
Web app routes/analytics unchanged (padelnomics-specific customizations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix quote sidebar z-index (behind tab nav) and align top with tab content
- Fix bottom nav sticky positioning (move outside .planner-app)
- Fix wizard footer fixed positioning and width on mobile
- Fix bottom nav active state visibility (hardcoded colors outside CSS var scope)
- Fix country pills overflow with flex-wrap
- Fix tooltip clipping in collapsible sections
- Hide feedback button on mobile planner
- Add cache busting for static assets (_ASSET_VERSION)
- Convert export CTA to full clickable button
- Add CAPEX table section header, sort doughnut chart by size
- Cap data tables at 640px centered, horizontal scroll for wide tables
- Replace CAPEX jargon with plain German (Gesamtinvestition, Kostenaufschlüsselung)
- Update FAQ/landing copy to global language (not Europe-specific)
- Update default court sizes to realistic values (court + walkway only)
- Add missing planner_export_inline translation key (en + de)
- Revert wizard nav to client-side (HTMX broke on lang-prefixed routes)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sequential IDs in /planner/export/<id> and /leads/<id>/unlock leaked
business volume (e.g. export_id=47 reveals ~47 PDFs sold). Replace with
22-char URL-safe tokens that carry no countable information.
- Migration 0017: adds `token TEXT` to business_plan_exports and
lead_requests, backfills existing rows with secrets.token_urlsafe(16),
creates unique indexes for fast lookups
- billing/routes.py: INSERT into business_plan_exports includes token
- leads/routes.py: INSERT into lead_requests includes token; enqueue
payload includes lead_token; verify_quote() looks up by token
- planner/routes.py: /export/<token> route (was /export/<int:export_id>)
- suppliers/routes.py: /leads/<token>/unlock (was /leads/<int:lead_id>)
- worker.py: email links use token for both export and verify URLs
- Templates: url_for() calls use token= param
- test_phase0.py: _submit_guest_quote() returns (lead_id, auth_token,
lead_token); verify URL tests use opaque lead token
Integer PKs unchanged; admin routes unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sync template from 29ac25b → v0.9.0 (29 template commits). Due to
template's _subdirectory migration, new files were manually rendered
rather than auto-merged by copier.
New files:
- .claude/CLAUDE.md + coding_philosophy.md (agent instructions)
- extract utils.py: SQLite state tracking for extraction runs
- extract/transform READMEs: architecture & pattern documentation
- infra/supervisor: systemd service + orchestration script
- Per-layer model READMEs (raw, staging, foundation, serving)
Also fixes copier-answers.yml (adds 4 feature toggles, removes stale
payment_provider key) and scopes CLAUDE.md gitignore to root only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add sticky bottom tab bar on mobile (<768px) with 5 tabs (Setup, CAPEX, P&L, Cash, Returns)
- Merge Metrics tab into Returns as collapsible <details> section
- Wrap wizard input groups in collapsible <details> elements to reduce scroll fatigue
- Add contextual CTA bar above bottom nav showing CAPEX estimate + "Get Quotes" button
- Simplify desktop sidebar CTA (remove checklist, add text export link)
- Convert loadScenario/resetToDefaults/saveScenario from client-side JS to HTMX/navigation
- Convert wizard nav buttons to server-rendered partial (removes i18n from JS)
- Remove 3 unused window.__*__ globals, reduce planner.js from 208 to 131 lines
- Increase slider thumb size to 20px on mobile for better touch targets
- Add bottom padding to main content for bottom nav clearance
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add enable_daas, enable_cms, enable_directory, enable_i18n (all true)
and remove stale payment_provider key before running copier update.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
