Commit Graph

6 Commits

Author SHA1 Message Date
Deeman
79d1b0e672 feat(extract): tiered proxy with circuit breaker + proxy provider research
- playtomic_tenants.py: simplify proxy cycler call (cycler() instead of
  cycler["next_proxy"]()) — matches refactored proxy API
- docs/proxy-provider-inventory.md: proxy provider comparison table for
  Playtomic scraping (~14k req/day, residential IPs, pay-per-GB)
- .env.*.sops: updated encrypted secrets (re-encrypted)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 22:15:11 +01:00
Deeman
d3db830c98 Merge branch 'master' into worktree-dual-market-score
# Conflicts:
#	.env.dev.sops
2026-02-24 16:38:25 +01:00
Deeman
edf1e30444 feat(data): add spatial extension + extraction API keys
- Add GEONAMES_USERNAME=padelnomics and CENSUS_API_KEY to .env.dev.sops and .env.prod.sops
- Enable DuckDB spatial extension in SQLMesh config.yaml (ST_Distance_Sphere for distance calcs + future map features)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 16:05:46 +01:00
Deeman
0b472e1a32 feat(extract): tiered proxy with circuit breaker for Playtomic availability
Adds a two-tier proxy system for the Playtomic availability extractor:
- Primary tier (PROXY_URLS): datacenter proxies, cheap and fast
- Fallback tier (PROXY_URLS_FALLBACK): residential rotating gateway, reliable

Circuit breaker opens after CIRCUIT_BREAKER_THRESHOLD (default: 10) consecutive
failures, permanently switching to the fallback tier for the rest of the run.
No auto-recovery — avoids flapping. If circuit opens with no fallback configured,
logs an error and writes partial results rather than continuing on a dead proxy pool.

Parallel mode submits futures in PARALLEL_BATCH_SIZE=100 batches so the circuit
breaker can stop new submissions after it opens.

New env vars added to .env.dev.sops (blank defaults):
  PROXY_URLS_FALLBACK          — residential/rotating gateway URL
  CIRCUIT_BREAKER_THRESHOLD    — consecutive failures before switching (default 10)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 16:01:50 +01:00
Deeman
b5de3e0681 feat: integrate ntfy.sh push notifications for supervisor
- Add NTFY_TOKEN support to send_alert() — sends Authorization header
  when token is set, backwards-compatible with plain webhook URLs
- Set ALERT_WEBHOOK_URL and NTFY_TOKEN in .env.prod.sops
- Add NTFY_TOKEN= placeholder in .env.dev.sops
- Topic: gWMeiHxj8ZqLbbqT (hard-to-guess, token-gated)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 22:02:57 +01:00
Deeman
04ef8deaa1 feat: add encrypted dev secrets (.env.dev.sops) and Makefile targets
Dev env file replaces .env.example — decrypt with `make secrets-decrypt-dev`.
Makefile provides secrets-decrypt-dev, secrets-decrypt-prod, secrets-edit-dev,
secrets-edit-prod targets (wraps sops with --input-type dotenv).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 17:01:28 +01:00