Landing files (append-only JSON.gz) synced to R2 every 30 min via
systemd timer + rclone. Extraction state DB (.state.sqlite) continuously
replicated via Litestream (second DB entry). Auto-restore on container
startup for both app.db and .state.sqlite. Reuses existing R2 bucket
and credentials — no new env vars needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload CMS documentation with the new SSG system:
git templates, DuckDB data sources, generation pipeline, SEO pipeline
(hreflang, JSON-LD, canonical, OG), admin routes, and step-by-step
guide for adding new pSEO ideas.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add full email management at /admin/emails with:
- email_log table tracking all outgoing emails with resend_id + delivery events
- inbound_emails table for Resend webhook-received messages
- Resend webhook handler (/webhooks/resend) updating delivery status in real-time
- send_email() returns resend_id (str|None) instead of bool; all 9 worker
handlers pass email_type= for per-type filtering
- Admin UI: sent log with HTMX filters, email detail with API-enriched HTML
preview, inbox with unread badges + reply, compose with branded wrapping,
audience management with contact list/remove
- Sidebar Email section with unread badge via blueprint context processor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the old CSV-upload-based CMS with an SSG architecture where
templates live in git as .md.jinja files with YAML frontmatter and
data comes directly from DuckDB serving tables. Only articles and
published_scenarios remain in SQLite for routing/state.
- Content module: discover, load, generate, preview functions
- Migration 0018: drop article_templates + template_data, recreate
articles + published_scenarios without FK references, add
template_slug/language/date_modified/seo_head columns
- Admin routes: read-only template views with generate/regenerate/preview
- SEO pipeline: canonical URLs, hreflang (EN+DE), JSON-LD (Article,
FAQPage, BreadcrumbList), Open Graph tags baked at generation time
- Example template: city-cost-de.md.jinja for German city market data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9 tests exercise the full handler→wrap→send_email→Resend API path
using Resend's @resend.dev test addresses. Skipped when RESEND_API_KEY
is not set.
With a full_access API key, tests also retrieve the sent email via
resend.Emails.get() and assert on the rendered HTML (wordmark, links,
project details, heat badges). With a sending_access key, send is
verified but HTML assertions are skipped gracefully.
Includes bounce handling test and 0.6s inter-test delay for Resend's
2 req/sec rate limit.
Run with: RESEND_API_KEY=re_xxx pytest -k resend_live
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mock send_email and call each handler directly. Covers recipient,
subject content, HTML design elements (wordmark, preheader, heat
badges), from_addr, skip-on-missing-data guards, and email_sent_at
timestamp updates.
Also fixes IndexError in handle_send_welcome when payload has no name
("".split()[0] → safe fallback).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesigned _email_wrap(): lowercase wordmark header matching website,
3px blue accent border, preheader text support, HR separators.
_email_button() now full-width block for mobile tap targets.
Rewrote copy: improved subject lines, urgency cues, quick-start links
in welcome, styled project recap in quote verify, heat badges on lead
forward, "what happens next" in lead matched, secondary CTAs.
~30 new/updated translation keys in both EN and DE.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add maximum_object_size=128MB to read_json for 14K-venue tenants file
- Rewrite opening_hours to use UNION ALL unpivot (DuckDB struct dynamic access)
- Add seed file guard for availability model (empty result on first run)
- Fix snapshot_date VARCHAR→DATE comparison in venue_pricing_benchmarks
- Fix export_serving to resolve SQLMesh physical tables from view definitions
(SQLMesh views reference "local" catalog unavailable outside its context)
- Add pyarrow dependency for Arrow-based cross-connection data transfer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Playtomic API ignores bbox params (min_latitude, etc.) and offset param.
Discovered that `page` param works correctly for global enumeration.
Result: 14,202 venues across 82 countries (was 100 with bbox approach).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add lang parameter to all enqueue() calls for email internationalization.
Restructure Resend audiences to 3 named audiences (owners, suppliers, waitlist).
Use _t() translation function in all email template handlers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Markets now sits left of logo with Planner and Quotes (investor/demand
side). Mobile section headers use i18n keys instead of hardcoded English.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three workstreams:
1. Playtomic full data extraction & transform pipeline:
- Expand venue bounding boxes from 4 to 23 regions (global coverage)
- New staging models for court resources, opening hours, and slot-level
availability with real prices from the Playtomic API
- Foundation fact tables for venue capacity and daily occupancy/revenue
- City-level pricing benchmarks replacing hardcoded country estimates
- Planner defaults now use 3-tier cascade: city data → country → fallback
2. Transactional email i18n:
- _t() helper in worker.py with ~70 translation keys (EN + DE)
- All 8 email handlers translated, lang passed in task payloads
3. Resend audiences restructured to 3 named audiences (free plan limit)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract sitemap generation to sitemap.py with xhtml:link hreflang
alternates (en/de/x-default) on every URL entry. Add 1-hour in-memory
TTL cache with Cache-Control header. Include supplier pages in both
languages (were EN-only). Drop misleading "today" lastmod from static
pages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add requires_weasyprint marker to TestGenerateBusinessPlan and TestWorkerHandler
(these need libgobject/pango/cairo which CI python:3.12-slim lacks)
- Fix export route tests: use opaque tokens instead of integer IDs
- Replace deprecated datetime.utcnow() with datetime.now(UTC)
- Add missing jsonify/Response imports to admin routes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add /scenarios/<id>/pdf admin route for direct PDF generation via WeasyPrint.
Fix plan.html Jinja template: .items → ['items'] to avoid dict method collision.
Add scenario fixture in conftest.py and comprehensive test suite for business
plan sections, PDF generation, worker handler, and export routes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move historical docs from docs/ and .claude/ to research/. Add superseded
notice to research/PLAN.md. Add CHANGELOG entries for previous fixes.
New: PROJECT.md (task tracker), docs/MARKETING.md (marketing strategy).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eurostat JSON-stat format (4-7 dimension sparse dict with 583K values)
causes DuckDB OOM — pre-process in extractor to flat records.
Also fix dim_cities unused CTE bug and playtomic venue lat/lon path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Playtomic tenants API recycles results past its internal limit —
stop after 3 consecutive pages with zero new unique IDs.
Calculator tests: replace hardcoded default values (6 courts, specific
sqm/capex) with DEFAULTS references so tests don't break when
defaults change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove raw/ layer — staging models now read landing JSON directly.
Rename all model schemas from padelnomics.* to staging.*/foundation.*/serving.*.
Web app queries updated to serving.planner_defaults via SERVING_DUCKDB_PATH.
Supervisor gets daily sleep interval between pipeline runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split monolithic execute.py into per-source modules with separate CLI
entry points. Each extractor now uses the framework from utils.py:
- SQLite state tracking (start_run / end_run per extractor)
- Proper logging (replace print() with logger)
- Atomic gzip writes (write_gzip_atomic)
- Connection pooling (niquests.Session)
- Bounded pagination (MAX_PAGES_PER_BBOX = 500)
New entry points:
extract — run all 4 extractors sequentially
extract-overpass — OSM padel courts
extract-eurostat — city demographics (etag dedup)
extract-playtomic-tenants — venue listings
extract-playtomic-availability — booking slots + pricing (NEW)
The availability extractor reads tenant IDs from the latest tenants.json.gz,
queries next-day slots for each venue, and stores daily consolidated snapshots.
Supports resumability via cursor and retry with backoff.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pulls in template changes: export_serving.py for atomic DuckDB swap,
supervisor export step, SQLMesh glob macro, server provisioning script,
imprint template, and formatting improvements.
Template scaffold SQL models excluded (padelnomics has real models).
Web app routes/analytics unchanged (padelnomics-specific customizations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix quote sidebar z-index (behind tab nav) and align top with tab content
- Fix bottom nav sticky positioning (move outside .planner-app)
- Fix wizard footer fixed positioning and width on mobile
- Fix bottom nav active state visibility (hardcoded colors outside CSS var scope)
- Fix country pills overflow with flex-wrap
- Fix tooltip clipping in collapsible sections
- Hide feedback button on mobile planner
- Add cache busting for static assets (_ASSET_VERSION)
- Convert export CTA to full clickable button
- Add CAPEX table section header, sort doughnut chart by size
- Cap data tables at 640px centered, horizontal scroll for wide tables
- Replace CAPEX jargon with plain German (Gesamtinvestition, Kostenaufschlüsselung)
- Update FAQ/landing copy to global language (not Europe-specific)
- Update default court sizes to realistic values (court + walkway only)
- Add missing planner_export_inline translation key (en + de)
- Revert wizard nav to client-side (HTMX broke on lang-prefixed routes)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sequential IDs in /planner/export/<id> and /leads/<id>/unlock leaked
business volume (e.g. export_id=47 reveals ~47 PDFs sold). Replace with
22-char URL-safe tokens that carry no countable information.
- Migration 0017: adds `token TEXT` to business_plan_exports and
lead_requests, backfills existing rows with secrets.token_urlsafe(16),
creates unique indexes for fast lookups
- billing/routes.py: INSERT into business_plan_exports includes token
- leads/routes.py: INSERT into lead_requests includes token; enqueue
payload includes lead_token; verify_quote() looks up by token
- planner/routes.py: /export/<token> route (was /export/<int:export_id>)
- suppliers/routes.py: /leads/<token>/unlock (was /leads/<int:lead_id>)
- worker.py: email links use token for both export and verify URLs
- Templates: url_for() calls use token= param
- test_phase0.py: _submit_guest_quote() returns (lead_id, auth_token,
lead_token); verify URL tests use opaque lead token
Integer PKs unchanged; admin routes unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sync template from 29ac25b → v0.9.0 (29 template commits). Due to
template's _subdirectory migration, new files were manually rendered
rather than auto-merged by copier.
New files:
- .claude/CLAUDE.md + coding_philosophy.md (agent instructions)
- extract utils.py: SQLite state tracking for extraction runs
- extract/transform READMEs: architecture & pattern documentation
- infra/supervisor: systemd service + orchestration script
- Per-layer model READMEs (raw, staging, foundation, serving)
Also fixes copier-answers.yml (adds 4 feature toggles, removes stale
payment_provider key) and scopes CLAUDE.md gitignore to root only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add sticky bottom tab bar on mobile (<768px) with 5 tabs (Setup, CAPEX, P&L, Cash, Returns)
- Merge Metrics tab into Returns as collapsible <details> section
- Wrap wizard input groups in collapsible <details> elements to reduce scroll fatigue
- Add contextual CTA bar above bottom nav showing CAPEX estimate + "Get Quotes" button
- Simplify desktop sidebar CTA (remove checklist, add text export link)
- Convert loadScenario/resetToDefaults/saveScenario from client-side JS to HTMX/navigation
- Convert wizard nav buttons to server-rendered partial (removes i18n from JS)
- Remove 3 unused window.__*__ globals, reduce planner.js from 208 to 131 lines
- Increase slider thumb size to 20px on mobile for better touch targets
- Add bottom padding to main content for bottom nav clearance
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add enable_daas, enable_cms, enable_directory, enable_i18n (all true)
and remove stale payment_provider key before running copier update.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nginx -t resolves upstream hostnames — if the config points to a stopped
slot from a previous failed deploy, the health check fails and the router
stays unhealthy indefinitely, blocking all future deploys.
Before up -d --wait, write the router config to point to the CURRENT live
slot (which is still running) and restart the router. This clears the
stale unhealthy state. After the new slot passes health checks, switch
the router config to the new slot and reload.
Also extracted _write_router_conf() to avoid duplicating the nginx config
template.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker compose requires --profile to access profiled services even for
the logs command. Without it, blue-app logs were empty in the failure
dump, hiding the actual crash reason.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The 100-line combined log dump was entirely filled by litestream R2
errors, hiding the actual blue-app crash output. Now dumps blue-app
(60 lines), router (10 lines), and litestream (10 lines) separately.
Revert litestream image tag to latest — the R2 errors were caused by
misconfigured endpoint/bucket CI variables, not a litestream version
bug. The v0.5.8 tag may not exist on Docker Hub (tags omit 'v' prefix).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
latest tag may resolve to an older version that treats Cloudflare R2's
NoSuchKey response on empty-prefix ListObjectsV2 as a hard error instead
of an empty list, causing the replica sync to stall on first deployment.
v0.5.8 is the current stable release (2026-02-12).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
analytics.py imports duckdb at the top level. The Dockerfile runs
`uv sync --package padelnomics` which only installs padelnomics deps —
duckdb was missing, so hypercorn failed to import padelnomics.app
entirely and never bound to port 5000. The health check timed out and
the container was marked unhealthy. Tests passed because uv sync in CI
syncs all workspace members (including transform/ which has duckdb).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Router had no profile so it was always included in `up -d --wait`.
Writing the new target's config BEFORE the wait caused the router to become
unhealthy if the new slot failed — leaving it in a broken state for the next
deploy attempt.
Now: router keeps its old config (pointing to the still-running old slot)
during the health check wait, so it stays healthy throughout. Config is only
written and nginx -s reload triggered after the new slot passes its health
check. This is the correct blue-green pattern.
Also add `retries: 3` and `start_period: 10s` to the router health check
for resilience against transient startup failures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
