08e74665bb
Adds extract/openweathermap package with daily weather extraction for 8
coffee-growing regions (Brazil, Vietnam, Colombia, Ethiopia, Honduras,
Guatemala, Indonesia). Feeds crop stress signal for commodity sentiment score.
Extractor:
- OWM One Call API 3.0 / Day Summary — one JSON.gz per (location, date)
- extract_weather: daily, fetches yesterday + today (16 calls max)
- extract_weather_backfill: fills 2020-01-01 to yesterday, capped at 500
calls/run with resume cursor '{location_id}:{date}' for crash safety
- Full idempotency via file existence check; state tracking via extract_core
SQLMesh:
- seeds.weather_locations (8 regions with lat/lon/variety)
- foundation.fct_weather_daily: INCREMENTAL_BY_TIME_RANGE, grain
(location_id, observation_date), dedup via hash key, crop stress flags:
is_frost (<2°C), is_heat_stress (>35°C), is_drought (<1mm), in_growing_season
Landing path: LANDING_DIR/weather/{location_id}/{year}/{date}.json.gz
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
480 B
480 B
| 1 | location_id | name | country | lat | lon | variety |
|---|---|---|---|---|---|---|
| 2 | brazil_minas_gerais | Minas Gerais | BR | -19.9167 | -43.9345 | Arabica |
| 3 | brazil_parana | Paraná | BR | -23.4205 | -51.9330 | Arabica |
| 4 | vietnam_highlands | Central Highlands | VN | 12.6667 | 108.0500 | Robusta |
| 5 | colombia_huila | Huila | CO | 2.5359 | -75.5277 | Arabica |
| 6 | ethiopia_sidama | Sidama | ET | 6.7612 | 38.4721 | Arabica |
| 7 | honduras_copan | Copán | HN | 14.8333 | -89.1500 | Arabica |
| 8 | guatemala_antigua | Antigua | GT | 14.5586 | -90.7295 | Arabica |
| 9 | indonesia_sumatra | Sumatra | ID | 3.5952 | 98.6722 | Robusta |