08e74665bb
Adds extract/openweathermap package with daily weather extraction for 8
coffee-growing regions (Brazil, Vietnam, Colombia, Ethiopia, Honduras,
Guatemala, Indonesia). Feeds crop stress signal for commodity sentiment score.
Extractor:
- OWM One Call API 3.0 / Day Summary — one JSON.gz per (location, date)
- extract_weather: daily, fetches yesterday + today (16 calls max)
- extract_weather_backfill: fills 2020-01-01 to yesterday, capped at 500
calls/run with resume cursor '{location_id}:{date}' for crash safety
- Full idempotency via file existence check; state tracking via extract_core
SQLMesh:
- seeds.weather_locations (8 regions with lat/lon/variety)
- foundation.fct_weather_daily: INCREMENTAL_BY_TIME_RANGE, grain
(location_id, observation_date), dedup via hash key, crop stress flags:
is_frost (<2°C), is_heat_stress (>35°C), is_drought (<1mm), in_growing_season
Landing path: LANDING_DIR/weather/{location_id}/{year}/{date}.json.gz
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
10 lines
480 B
CSV
10 lines
480 B
CSV
location_id;name;country;lat;lon;variety
|
|
brazil_minas_gerais;Minas Gerais;BR;-19.9167;-43.9345;Arabica
|
|
brazil_parana;Paraná;BR;-23.4205;-51.9330;Arabica
|
|
vietnam_highlands;Central Highlands;VN;12.6667;108.0500;Robusta
|
|
colombia_huila;Huila;CO;2.5359;-75.5277;Arabica
|
|
ethiopia_sidama;Sidama;ET;6.7612;38.4721;Arabica
|
|
honduras_copan;Copán;HN;14.8333;-89.1500;Arabica
|
|
guatemala_antigua;Antigua;GT;14.5586;-90.7295;Arabica
|
|
indonesia_sumatra;Sumatra;ID;3.5952;98.6722;Robusta
|