Commit Graph

7 Commits

Author SHA1 Message Date
Deeman
6c1dc90a8d fix(ci): gate deploys on passing tags + fix markets feature flag test
- CI now creates v<pipeline_iid> tag after tests pass on master
- Supervisor fetches tags and only deploys when a newer tag is available;
  skips if already on latest or no tags exist
- Fix test_seeds_markets_enabled: markets is seeded disabled (enabled=0),
  test was asserting the wrong value

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 03:36:59 +01:00
Deeman
b5de3e0681 feat: integrate ntfy.sh push notifications for supervisor
- Add NTFY_TOKEN support to send_alert() — sends Authorization header
  when token is set, backwards-compatible with plain webhook URLs
- Set ALERT_WEBHOOK_URL and NTFY_TOKEN in .env.prod.sops
- Add NTFY_TOKEN= placeholder in .env.dev.sops
- Topic: gWMeiHxj8ZqLbbqT (hard-to-guess, token-gated)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 22:02:57 +01:00
Deeman
1c195f3c05 fix: run_shell returns bool, not tuple
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 18:36:46 +01:00
Deeman
8558fd6b40 feat: no-root deploy, remove CI deploy stage, deploy alerts
- deploy.sh installs sops/age to ./bin/ (no root/sudo needed)
- Remove CI deploy stage — supervisor auto-pulls and deploys
  (zero CI secrets: no SSH keys, no deploy credentials)
- Supervisor sends alert on deploy success/failure via webhook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 18:23:19 +01:00
Deeman
a1faddbed6 feat: Python supervisor + feature flags
Supervisor (replaces supervisor.sh):
- supervisor.py — cron-based pipeline orchestration, reads workflows.toml
  on every tick, runs due extractors in topological waves with parallel
  execution, then SQLMesh transform + serving export
- workflows.toml — workflow registry: overpass (monthly), eurostat (monthly),
  playtomic_tenants (weekly), playtomic_availability (daily),
  playtomic_recheck (hourly 6–23)
- padelnomics-supervisor.service — updated ExecStart to Python supervisor

Extraction enhancements:
- proxy.py — optional round-robin/sticky proxy rotation via PROXY_URLS env
- playtomic_availability.py — parallel fetch (EXTRACT_WORKERS), recheck mode
  (main_recheck) re-queries imminent slots for accurate occupancy measurement
- _shared.py — realistic browser User-Agent on all extractor sessions
- stg_playtomic_availability.sql — reads morning + recheck snapshots, tags each
- fct_daily_availability.sql — prefers recheck over morning for same slot

Feature flags (replaces WAITLIST_MODE env var):
- migration 0019 — feature_flags table, 5 initial flags:
  markets (on), payments/planner_export/supplier_signup/lead_unlock (off)
- core.py — is_flag_enabled() + feature_gate() decorator
- routes — payments, markets, planner_export, supplier_signup, lead_unlock gated
- admin flags UI — /admin/flags toggle page + nav link
- app.py — flag() injected as Jinja2 global

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:53:45 +01:00
Deeman
7737b79230 fix: DuckDB compat issues in Playtomic pipeline + export_serving
- Add maximum_object_size=128MB to read_json for 14K-venue tenants file
- Rewrite opening_hours to use UNION ALL unpivot (DuckDB struct dynamic access)
- Add seed file guard for availability model (empty result on first run)
- Fix snapshot_date VARCHAR→DATE comparison in venue_pricing_benchmarks
- Fix export_serving to resolve SQLMesh physical tables from view definitions
  (SQLMesh views reference "local" catalog unavailable outside its context)
- Add pyarrow dependency for Arrow-based cross-connection data transfer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 01:27:51 +01:00
Deeman
ea86940b78 feat: copier update v0.9.0 → v0.10.0
Pulls in template changes: export_serving.py for atomic DuckDB swap,
supervisor export step, SQLMesh glob macro, server provisioning script,
imprint template, and formatting improvements.

Template scaffold SQL models excluded (padelnomics has real models).
Web app routes/analytics unchanged (padelnomics-specific customizations).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 17:50:36 +01:00