Commit Graph

6 Commits

Author SHA1 Message Date
Deeman
a1faddbed6 feat: Python supervisor + feature flags
Supervisor (replaces supervisor.sh):
- supervisor.py — cron-based pipeline orchestration, reads workflows.toml
  on every tick, runs due extractors in topological waves with parallel
  execution, then SQLMesh transform + serving export
- workflows.toml — workflow registry: overpass (monthly), eurostat (monthly),
  playtomic_tenants (weekly), playtomic_availability (daily),
  playtomic_recheck (hourly 6–23)
- padelnomics-supervisor.service — updated ExecStart to Python supervisor

Extraction enhancements:
- proxy.py — optional round-robin/sticky proxy rotation via PROXY_URLS env
- playtomic_availability.py — parallel fetch (EXTRACT_WORKERS), recheck mode
  (main_recheck) re-queries imminent slots for accurate occupancy measurement
- _shared.py — realistic browser User-Agent on all extractor sessions
- stg_playtomic_availability.sql — reads morning + recheck snapshots, tags each
- fct_daily_availability.sql — prefers recheck over morning for same slot

Feature flags (replaces WAITLIST_MODE env var):
- migration 0019 — feature_flags table, 5 initial flags:
  markets (on), payments/planner_export/supplier_signup/lead_unlock (off)
- core.py — is_flag_enabled() + feature_gate() decorator
- routes — payments, markets, planner_export, supplier_signup, lead_unlock gated
- admin flags UI — /admin/flags toggle page + nav link
- app.py — flag() injected as Jinja2 global

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:53:45 +01:00
Deeman
1a6eae20d5 feat: pSEO CMS — SSG architecture with git templates + DuckDB
# Conflicts:
#	web/pyproject.toml
2026-02-23 12:51:30 +01:00
Deeman
f1181342ad feat: SSG-inspired pSEO CMS — git templates + DuckDB direct reads
Replace the old CSV-upload-based CMS with an SSG architecture where
templates live in git as .md.jinja files with YAML frontmatter and
data comes directly from DuckDB serving tables. Only articles and
published_scenarios remain in SQLite for routing/state.

- Content module: discover, load, generate, preview functions
- Migration 0018: drop article_templates + template_data, recreate
  articles + published_scenarios without FK references, add
  template_slug/language/date_modified/seo_head columns
- Admin routes: read-only template views with generate/regenerate/preview
- SEO pipeline: canonical URLs, hreflang (EN+DE), JSON-LD (Article,
  FAQPage, BreadcrumbList), Open Graph tags baked at generation time
- Example template: city-cost-de.md.jinja for German city market data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 12:25:44 +01:00
Deeman
7737b79230 fix: DuckDB compat issues in Playtomic pipeline + export_serving
- Add maximum_object_size=128MB to read_json for 14K-venue tenants file
- Rewrite opening_hours to use UNION ALL unpivot (DuckDB struct dynamic access)
- Add seed file guard for availability model (empty result on first run)
- Fix snapshot_date VARCHAR→DATE comparison in venue_pricing_benchmarks
- Fix export_serving to resolve SQLMesh physical tables from view definitions
  (SQLMesh views reference "local" catalog unavailable outside its context)
- Add pyarrow dependency for Arrow-based cross-connection data transfer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 01:27:51 +01:00
Deeman
044dfd836b fix(deps): add duckdb to padelnomics production dependencies
analytics.py imports duckdb at the top level. The Dockerfile runs
`uv sync --package padelnomics` which only installs padelnomics deps —
duckdb was missing, so hypercorn failed to import padelnomics.app
entirely and never bound to port 5000. The health check timed out and
the container was marked unhealthy. Tests passed because uv sync in CI
syncs all workspace members (including transform/ which has duckdb).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 13:43:34 +01:00
Deeman
4ae00b35d1 refactor: flatten padelnomics/padelnomics/ → repo root
git mv all tracked files from the nested padelnomics/ workspace
directory to the git repo root. Merged .gitignore files.
No code changes — pure path rename.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 00:44:40 +01:00