diff --git a/vision.md b/vision.md index 6025490..1d4f47d 100644 --- a/vision.md +++ b/vision.md @@ -95,18 +95,23 @@ We move fast, ship incrementally, and prioritize value over vanity metrics. - USDA PSD Online extraction + full SQLMesh pipeline (raw→staging→cleaned→serving) - CFTC COT disaggregated futures: weekly positioning, COT index, managed money net - KC=F Coffee C futures prices: daily OHLCV, 20d/50d SMA, 52-week range (1971–present) -- ICE certified warehouse stocks: extractor ready, awaiting URL confirmation -- Web app (Quart + HTMX): dashboard with supply/demand + COT + price + ICE charts -- REST API with key auth + rate limiting: /metrics, /positioning, /prices, /stocks +- ICE certified warehouse stocks: daily rolling + aging report + historical by-port (Nov 1996–present) + — API-based URL discovery (no env var needed), XLS + CSV parsing, full SQLMesh pipeline + — Dashboard: daily stocks chart, aging stacked bar, 30-year by-port stacked area + — API endpoints: /stocks, /stocks/aging, /stocks/by-port +- Web app (Quart + HTMX): dashboard with supply/demand + COT + price + all ICE charts +- Origin Intelligence page: HATEOAS country comparison, click-to-update via HTMX +- REST API with key auth + rate limiting - Paddle billing (Starter/Pro plans), magic-link auth, admin panel - /methodology page with full data source documentation -- Automated supervisor: all extractors + webhook alerting on failure +- Automated supervisor: all extractors (extract_all meta-pipeline) + webhook alerting - 23 passing tests, GitLab CI pipeline -### What's Missing -- ICE stocks URL confirmed and backfill running (URL needs manual discovery at theice.com/report-center) -- Python SDK -- Public API documentation +### What's Missing (Phase 1 remainder) +- Python SDK (`pip install beanflows`) +- Deploy to Hetzner production +- Cloudflare R2 raw data backup +- Example Jupyter notebooks ## Roadmap @@ -117,7 +122,7 @@ We move fast, ship incrementally, and prioritize value over vanity metrics. - ✅ USDA PSD Online (production, stocks, consumption) - ✅ CFTC COT data (trader positioning, COT index) - ✅ KC=F Coffee futures prices (daily OHLCV, moving averages) -- ✅ ICE warehouse stocks (extractor built, seed models deployed) +- ✅ ICE warehouse stocks (daily + aging + historical by-port, full pipeline + API + dashboard) - ⬜ ICO (International Coffee Organization) — future **Features:**