Deeman 0b472e1a32 feat(extract): tiered proxy with circuit breaker for Playtomic availability ...

Adds a two-tier proxy system for the Playtomic availability extractor:
- Primary tier (PROXY_URLS): datacenter proxies, cheap and fast
- Fallback tier (PROXY_URLS_FALLBACK): residential rotating gateway, reliable

Circuit breaker opens after CIRCUIT_BREAKER_THRESHOLD (default: 10) consecutive
failures, permanently switching to the fallback tier for the rest of the run.
No auto-recovery — avoids flapping. If circuit opens with no fallback configured,
logs an error and writes partial results rather than continuing on a dead proxy pool.

Parallel mode submits futures in PARALLEL_BATCH_SIZE=100 batches so the circuit
breaker can stop new submissions after it opens.

New env vars added to .env.dev.sops (blank defaults):
  PROXY_URLS_FALLBACK          — residential/rotating gateway URL
  CIRCUIT_BREAKER_THRESHOLD    — consecutive failures before switching (default 10)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-24 16:01:50 +01:00

8.7 KiB

Raw Blame History

View Raw