deemanone/padelnomics
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(proxy): skip URLs without scheme in load_proxy_tiers()

Browse Source 
Validates each URL in PROXY_URLS_DATACENTER / PROXY_URLS_RESIDENTIAL:
logs a warning and skips any entry missing an http:// or https:// scheme
instead of passing malformed URLs that cause SSL or connection errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-03-01 14:26:41 +01:00
parent f92d863781
commit 81a57db272
1 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
extract/padelnomics_extract/src/padelnomics_extract/proxy.py
View File

@@ -88,8 +88,14 @@ def load_proxy_tiers() -> list[list[str]]:
for var in ("PROXY_URLS_DATACENTER", "PROXY_URLS_RESIDENTIAL"): for var in ("PROXY_URLS_DATACENTER", "PROXY_URLS_RESIDENTIAL"):
raw = os.environ.get(var, "") raw = os.environ.get(var, "")
urls = [u.strip() for u in raw.split(",") if u.strip()] urls = [u.strip() for u in raw.split(",") if u.strip()]
if urls: valid = []
tiers.append(urls) for url in urls:
if not url.startswith(("http://", "https://")):
logger.warning("%s contains URL without scheme, skipping: %s", var, url[:60])
continue
valid.append(url)
if valid:
tiers.append(valid)
return tiers return tiers