padelnomics/PROJECT.md

# Padelnomics — Project Tracker

> Move tasks across columns as you work. Add new tasks at the top of the relevant column.
> Last updated: 2026-02-25.

---

## Done ✅

> Confirmed by full code audit (2026-02-22). More is built than the original strategy docs assumed.

### Infrastructure & Deploy
- [x] UV workspace monorepo structure (web/, transform/, extract/ members)
- [x] Docker + docker-compose production deploy
- [x] Litestream R2 backup (1-year retention, auto-restore on startup)
- [x] CI pipeline (GitLab, health check gated deploys)
- [x] SOPS + age encrypted secrets (`.env.dev.sops` / `.env.prod.sops`; `deploy.sh` auto-decrypts; `setup_server.sh` installs sops+age)
- [x] Pre-migration DB backup + auto-restore on failed deploy
- [x] Nginx router config

### Auth & Users
- [x] Magic link auth (signup + login)
- [x] User accounts + sessions
- [x] RBAC (admin role, supplier role)
- [x] User dashboard (stats, settings, account deletion)
- [x] Admin impersonation (login as any user, return to admin)

### Planner / Calculator
- [x] Full 60-var financial planner — **fully open, no login required**
- [x] HTMX refactor (server-rendered tabs, ~200 line JS)
- [x] Planner i18n (EN + DE, all strings translated)
- [x] Currency formatting by country (€ / £ / $)
- [x] Market data endpoint (`/planner/api/market-data`) — city-specific defaults from DuckDB
- [x] Scenario save/load (requires login)
- [x] "Get Supplier Quotes" button linking to quote wizard

### Leads / Quote Flow
- [x] 9-step quote wizard — no login required, heat scoring, email verification
- [x] Lead heat scoring (hot 35 credits / warm 20 / cool 8)
- [x] Guest email verification (magic link sent before lead activates)
- [x] Admin lead management (list, detail, status transitions, manual forwarding)
- [x] Supplier credit-based lead unlock (atomic credit spend + email notification to both sides)
- [x] i18n for full quote flow

### Supplier Directory & Dashboard
- [x] Directory — fully public, search/FTS/filter/pagination/detail/enquiry
- [x] Tier-based ordering (sticky > pro > growth > basic > free)
- [x] Boost badges on directory cards
- [x] Supplier signup wizard (4-step, plan selection, Paddle checkout, waitlist-gatable)
- [x] Supplier dashboard (4 tabs: overview, leads feed, listing editor, boosts)
- [x] Listing editor with live HTMX preview, logo/cover upload
- [x] Boost purchase from dashboard (Paddle checkout)
- [x] Credit pack purchase from dashboard
- [x] i18n for directory + supplier pages

### Billing & Credits
- [x] Paddle billing (SDK, 18 products, webhooks, checkout, subscription lifecycle)
- [x] Credit system (balance, ledger, atomic unlock, monthly refill on 1st of month)
- [x] Business Plan PDF purchase flow (Paddle one-time → webhook → async generation)
- [x] Boost purchases (logo, highlight, verified, card color, sticky week/month)
- [x] Credit pack purchases (25/50/100/250)
- [x] Supplier subscription tiers (Basic free / Growth €149 / Pro €399, monthly + annual)
- [x] **Feature flags** (DB-backed, migration 0019) — `is_flag_enabled()` + `feature_gate()` decorator replace `WAITLIST_MODE`; 5 flags (markets, payments, planner_export, supplier_signup, lead_unlock); admin UI at `/admin/flags` with toggle
- [x] **Python supervisor** (`src/padelnomics/supervisor.py`) + `workflows.toml` — replaces `supervisor.sh`; topological wave scheduling; croniter-based `is_due()`; systemd service updated
- [x] **Proxy rotation** (`extract/padelnomics_extract/proxy.py`) — round-robin + sticky hash-based selector via `PROXY_URLS` env var
- [x] Resend email integration (transactional: magic link, welcome, quote verify, lead forward, enquiry)
- [x] Auto-create Resend audiences per blueprint (waitlist, planner nurture)

### Business Plan PDF Export
- [x] Full WeasyPrint PDF pipeline (not just a CTA — actually generates PDFs)
- [x] Paddle checkout for PDF purchase
- [x] Async generation via worker queue, email on completion
- [x] PDF content: executive summary, CAPEX, OPEX, 5-year P&L, financing, IRR/MOIC/payback/DSCR
- [x] EN + DE PDF localization

### CMS / Programmatic SEO
- [x] Article template engine (Jinja2 Markdown + `[scenario:slug:section]` markers)
- [x] Seed script (`seed_content.py`) — 40 cities × EN + DE = 80 articles
- [x] City coverage: DE (8), US (6), UK (4), ES (5), FR (3), IT (2), NL, AT, CH, SE, PT (2), BE, AE, AU (2), IE
- [x] Per-city financial model overrides (rates, rent, utilities, permits, court config)
- [x] Admin CMS (template CRUD, data row management, bulk CSV upload, bulk generate, publish toggle, rebuild)
- [x] Admin CMS v2: HTMX filter/search/pagination, background generation, inline actions, sitemap invalidation, markdown editing
- [x] pSEO template improvements: bilingual DE+EN (all 3 templates), expanded content depth (~1500 words), cross-template links, scenario cross-references, FAQPage schema fix, 2 extra FAQs per template, second CTAs
- [x] URL prefix fix: articles stored without lang prefix (was causing `/en/en/markets/...`), all consumers updated
- [x] Markets hub (`/<lang>/markets`) — article listing with FTS + country/region filters
- [x] DuckDB refresh script (`refresh_from_daas.py`)

### Data Pipeline (DaaS)
- [x] Overpass API extractor (OSM padel courts)
- [x] Eurostat extractor (city demographics)
- [x] Playtomic unauthenticated tenant search extractor
- [x] SQLMesh 3-layer DuckDB pipeline (staging → foundation → serving)
- [x] `dim_venues` (OSM + Playtomic deduped), `dim_cities` (Eurostat population)
- [x] `city_market_profile` (market score OBT), `planner_defaults` (per-city calculator pre-fill)
- [x] DuckDB analytics reader in app lifecycle
- [x] **JSONL streaming landing format** — extractors write `.jsonl.gz` (one record per line); constant-memory compression via `compress_jsonl_atomic()`; eliminates `maximum_object_size` workarounds; all modified staging models use UNION ALL transition to support both formats
- [x] **Regional Overpass tennis splitting** — 10 regional bbox queries replace the single global 150K-element query that timed out; crash recovery via `working.jsonl` accumulation
- [x] **`init_landing_seeds.py`** — creates minimal seed files for both JSONL and blob formats so SQLMesh can run before real data arrives

### i18n
- [x] Full i18n across entire app (EN + DE)
- [x] URL prefixes (`/en/`, `/de/`) on all public blueprints
- [x] Language detection (cookie + Accept-Language header)
- [x] `tformat` Jinja2 filter for parameterized translations
- [x] German copy: informal "Du/Dein" throughout
- [x] hreflang tags + `x-default`

### Admin Panel
- [x] Comprehensive admin: users, tasks, leads, suppliers, CMS templates, scenarios, articles, feedback
- [x] Task queue management (list, retry, delete)
- [x] Lead funnel stats on admin dashboard
- [x] Email hub (`/admin/emails`) — sent log, inbox, compose, audiences, delivery event tracking via Resend webhooks
- [x] **Email template system** — 11 transactional emails as Jinja2 templates (`emails/*.html`); standalone `render_email_template()` renderer works in worker + admin; `_base.html` + `_macros.html` shared shell; `EMAIL_TEMPLATE_REGISTRY` with sample data for gallery previews; `_email_wrap()` / `_email_button()` helpers removed
- [x] **Admin email gallery** (`/admin/emails/gallery`) — card grid of all templates, EN/DE preview in sandboxed iframe, "View in sent log" cross-link; compose page now has HTMX live preview pane
- [x] **pSEO Engine tab** (`/admin/pseo`) — content gap detection, data freshness signals, article health checks (hreflang orphans, missing build files, broken scenario refs), generation job monitoring with live progress bars
- [x] **Marketplace admin dashboard** (`/admin/marketplace`) — lead funnel, credit economy, supplier engagement, live activity stream, inline feature flag toggles
- [x] **Pipeline Console** (`/admin/pipeline`) — 4-tab operational dashboard: extraction status grid per source, filterable run history with stale-run management ("Mark Failed"), data catalog with column schema + 10-row sample, SQL query editor with dark-themed textarea + schema sidebar + read-only security sandboxing (keyword blocklist, 10s timeout, 1,000-row cap)
- [x] **Lead matching notifications** — `notify_matching_suppliers` task on quote verification + `send_weekly_lead_digest` every Monday; one-click CTA token in forward emails
- [x] **Migration 0022** — `status_updated_at`, `supplier_note`, `cta_token` on `lead_forwards`; supplier respond endpoint; inline HTMX lead detail actions; extended quote form fields
- [x] **Outreach pipeline** (`/admin/outreach`) — cold B2B supplier outreach with separate sending domain (`hello.padelnomics.io`), 6-stage pipeline cards, HTMX inline status + note editing, CSV import, bulk add-to-pipeline from supplier list, compose integration (auto-updates pipeline on send); migration 0024 adds 4 outreach columns to suppliers; 44 tests
- [x] **Outreach follow-up scheduling + activity timeline** — `follow_up_at` date column on suppliers (migration 0025), HTMX date picker on outreach rows, amber "follow-ups due" banner with `?follow_up=due` filter, activity timeline on supplier detail merging sent + received emails by contact email; 29 tests
- [x] **pSEO article noindex** — `noindex` column on articles (migration 0025), `NOINDEX_THRESHOLDS` per-template lambdas in `content/__init__.py`, robots meta tag in `article_detail.html`, sitemap exclusion, pSEO dashboard count card + article row badge; 20 tests

### SEO & Legal
- [x] Sitemap (both language variants, `<lastmod>` on all entries)
- [x] robots.txt
- [x] JSON-LD schemas (Organization, FAQPage, Article)
- [x] OG tags + canonical on all pages
- [x] German legal pages (Impressum, Datenschutz, AGB — DSGVO compliant)
- [x] English legal pages (GDPR, proper controller identity)
- [x] Cookie consent banner (functional/A/B categories, 1-year cookie)
- [x] Virtual office address on imprint
- [x] SEO/GEO admin hub — GSC + Bing + Umami sync, search/funnel/scorecard views, daily background sync
- [x] Market Score methodology page (`/{lang}/market-score`) — Zillow-style explanation of the padelnomics Market Score; EN + DE; JSON-LD (WebPage + FAQPage + BreadcrumbList); hub-and-spoke internal linking from all article templates

### Testing
- [x] Playwright visual/E2E test suite — 77 tests across 3 files (visual, e2e flows, quote wizard); single session-scoped server + browser; mocked emails + waitlist mode; ~59s runtime
- [x] Unit test suite — auth, billing webhooks, planner calculator, PDF export, RBAC, scenarios, lead scoring

### Other
- [x] A/B testing framework (`@ab_test` decorator + Umami `data-tag`)
- [x] Mobile nav (hamburger < 900px, full overlay panel)
- [x] Padel racket SVG logo/favicon
- [x] Feedback widget (HTMX POST, rate-limited)
- [x] Interactive ROI calculator widget on landing page (JS sliders, no server call)

---

---

## Next Up 📋

> Two independent tracks — pick from either at any time, no sequencing between them.
> Tech tasks can be shipped in hours. Business tasks depend on other people and run in parallel.

### Go-Live (config, not code)

| 🛠 Tech | 📣 Business |
|--------|------------|
| Paddle: set production env vars + run `setup_paddle` against prod | First 3–5 supplier outreach emails |
| Publish SEO articles: run `seed_content --generate` on prod (or trigger from admin) | First LinkedIn post |
| Wipe 5 test suppliers (`example.com` entries from `seed_dev_data.py`) | |
| Verify Resend production API key — test magic link email | |
| Submit sitemap to Google Search Console | Set up Google Search Console + Bing Webmaster Tools (SEO hub ready — just add env vars) |
| Verify Litestream R2 backup running on prod | |

### Gemeinde-level pSEO (follow-up from dual score work)

| 🛠 Tech |
|--------|
| Gemeinde-level pSEO article template — consumes `location_opportunity_profile` data, targets "Padel in [Ort]" + "Padel bauen in [Ort]" queries (zero SERP competition confirmed) |
| "Top 50 underserved locations" ranking page — high-value SEO content, fully programmatic from `location_opportunity_profile` ORDER BY opportunity_score DESC |

### Week 1–2 — First Revenue

| 🛠 Tech | 📣 Business |
|--------|------------|
| Email nurture sequence (3-email drip for planner users who save scenarios — Resend infra ready, just need content + scheduling) | 30–50 supplier outreach emails |
| | 2–3 founding member deals (free leads for 3 months) |
| | "State of Padel Q1 2026" report written + published |
| | First 3 priority SEO articles (see `docs/MARKETING.md` for titles) |
| | LinkedIn: 5 posts published |

### Week 2–4 — Market Map

| 🛠 Tech | 📣 Business |
|--------|------------|
| Market map UI (geographic visualization over DuckDB city data — no map exists yet) | Follow up on founding member outreach |
| | More SEO articles |

### Month 2 — Market Intelligence

| 🛠 Tech | 📣 Business |
|--------|------------|
| Market Intelligence Dashboard (city analytics, occupancy estimates, demand map) | Explorer tier (€79/mo) promoted to email list |
| Explorer tier paywall (€79/mo subscription gate) | Email drip running |

---

## Backlog 🗂️

> Validated ideas not yet scheduled. Pick up when capacity allows.

### Product
- [ ] Market Intelligence Pro tier (€149/mo — hall-level data, competitor tracking, historical)
- [ ] Location Scorer (mechanical turk first: form → manual PDF delivery → automate if demand)
- [ ] Operational analytics for running venues (€49–99/mo, Phase 3 product)
- [ ] Business Plan Pro subscription (€39/mo, saved scenarios + auto-updates)
- [ ] Site Selection Reports (€499–999 high-ticket, productized)
- [ ] "State of Padel" quarterly report product (€299–499, gated)
- [ ] Enterprise / API tier (custom pricing)
- [ ] Padel Hall Accelerator (€999 — report + call + supplier intros)

### Data & Intelligence
- [ ] Sports centre Overpass extract (`leisure=sports_centre`) — additional market signal for `dim_locations`
- [ ] City-level income enrichment (Eurostat NUTS-3 regional income — replaces country-level PPS proxy, higher granularity)
- [ ] Interactive opportunity map / explorer in web app (map UI over `location_opportunity_profile` — bounding box queries via ST_Distance_Sphere)
- [ ] Multi-source data aggregation (add booking platforms beyond Playtomic)
- [ ] Google Maps signals (reviews, ratings)
- [ ] Weather + demographic overlays
- [ ] Voluntary data sharing from operating venues (benchmarking network effects)
- [ ] ML/forecasting layer (demand forecasting, pricing optimization)
- [ ] Scraping risk mitigation: rotate sources, voluntary sharing fallback

### Bugs / Tech Debt
- [ ] Resend audiences: two segments both using "waitlist-auth" — review audience/segment model and fix duplication
- [ ] Transactional emails not all translated to German — some emails still sent in English regardless of user language
- [x] ~~Resend inbound emails enabled~~ — integrated: webhook handler + admin inbox with reply (done in email hub)
- [ ] Extraction: Playtomic API only returns ~20 venues per bbox — investigate smaller/targeted bboxes

### Marketing & Content
- [ ] LinkedIn presence (ongoing — founder posts, thought leadership)
- [ ] "Wirecutter for padel" affiliate site (racket reviews, gear guides)
- [ ] "The Padel Business Report" newsletter
- [ ] Equipment supplier affiliate partnerships (€500–1,000/lead or 5%)
- [ ] Padel podcasts (guest appearances)
- [ ] Sports business media outreach
- [ ] National padel associations (DTB, LTA — co-distribution)
- [ ] Franchise partnerships (market data / leads)
- [ ] Lender distribution (banks recommending Padelnomics plans)

### Geographic Expansion
- [ ] Austria + Switzerland (language done, cities seeded — just outreach + supplier onboarding)
- [ ] France (cities seeded in CMS)
- [ ] Italy, Netherlands, Sweden (cities seeded)
- [ ] UAE / Middle East (cities seeded)
- [ ] Pan-European supplier directory

---

## Human Tasks 🧑

> Tasks that require manual setup, external accounts, or human judgement.

- [x] Set up ntfy.sh for supervisor notifications — topic `gWMeiHxj8ZqLbbqT`, token in `.env.prod.sops`

---

## Decisions Log

| Date | Decision | Rationale |
|------|----------|-----------|
| 2026-02-22 | Two-sided marketplace framing (Side A = aspiring owners, Side B = suppliers) | Suppliers are the main revenue engine; calling them "secondary" was wrong |
| 2026-02-22 | Tennis/sports club owner as beachhead | Faster decision cycle, talk to each other, ~500–1,000 targets, small enough to dominate |
| 2026-02-22 | Credit system over pay-per-lead blast | Suppliers self-select → higher quality perception; scales without manual intervention |
| 2026-02-22 | No soft email gate on planner | Planner already captures emails at natural points (scenario save → login, quote wizard step 9). Gate would add friction without meaningful list value. Revisit if data shows a gap. |
| 2026-02-22 | Wipe test suppliers before launch | 5 `example.com` entries from seed_dev_data.py — empty directory with "Be the first" CTA is better than obviously fake data |
| 2026-02-24 | Split market score into two branded scores | Marktreife-Score (existing market maturity, cities with ≥1 venue) vs Marktpotenzial-Score (greenfield opportunity, all GeoNames locations globally). SERP analysis confirmed zero competition for hyperlocal Gemeinde-level market intelligence pages. |