From c469f585eb926658e4bea15c298f83bcc62fa5d4 Mon Sep 17 00:00:00 2001 From: Deeman Date: Thu, 26 Feb 2026 20:08:12 +0100 Subject: [PATCH] docs: add PROJECT.md with backlog (retry/backoff for ICE + yfinance) --- PROJECT.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 PROJECT.md diff --git a/PROJECT.md b/PROJECT.md new file mode 100644 index 0000000..b3445ab --- /dev/null +++ b/PROJECT.md @@ -0,0 +1,29 @@ +# BeanFlows Project Tracker + +## In Progress + +_nothing active right now_ + +## Next Up + +- **Retry/backoff for ICE and yfinance extractors** — Add a 3-attempt exponential backoff wrapper (30s/60s/120s) to `extract/ice_stocks/` and `extract/coffee_prices/`. These are the two sources most exposed to transient rate-limiting (ICE commercial exchange, Yahoo Finance IP throttling). No proxies needed unless active blocking is observed. Government sources (USDA, CFTC) and Open-Meteo don't need this. + +## Backlog + +- Expand to additional commodities beyond coffee (cocoa, sugar, cotton — all on ICE US; COT + warehouse + prices already modelled generically) +- WASDE report integration (USDA monthly supply/demand estimates, official balance sheet) +- FAO trade statistics (export flow by origin country pair) +- FRED macro overlay (USD index, interest rates) on price chart +- Options positioning — Barchart API or CBOE put/call ratios + +## Done + +- USDA PSD Online extractor + SQLMesh pipeline (supply/demand, stock-to-use) +- CFTC COT disaggregated futures-only extractor + foundation + serving models +- CFTC COT combined (futures+options) extractor + foundation union + serving model + web toggle +- KC=F daily price extractor (yfinance) + SQLMesh pipeline +- ICE certified warehouse stocks — daily rolling, aging by port, historical by port +- Open-Meteo ERA5 weather extractor — 12 coffee-growing regions, daily +- Dashboard pages: Pulse, Supply, Positioning (with F+O toggle), Warehouse, Weather, Origins +- REST API v1: commodities, pricing, positioning (?type=fut|combined), stocks, weather +- Chart.js resize fix — global htmx:beforeSwap chart destroy + requestAnimationFrame resize on all pages