Update vision.md: reflect Phase 1 completion as of Feb 2026

All Phase 1 data sources shipped. Mark ready for outreach.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-02-21 11:42:26 +01:00
parent 67c048485b
commit c9e9562030

View File

@@ -89,47 +89,49 @@ We move fast, ship incrementally, and prioritize value over vanity metrics.
- Avoid full table scans - Avoid full table scans
- Pay only for what changed - Pay only for what changed
## Current State (October 2025) ## Current State (February 2026)
### What's Working ### What's Shipped
- USDA PSD Online extraction (2006-present, monthly archives) - USDA PSD Online extraction + full SQLMesh pipeline (raw→staging→cleaned→serving)
- 4-layer SQLMesh pipeline (raw → staging → cleaned → serving) - CFTC COT disaggregated futures: weekly positioning, COT index, managed money net
- DuckDB backend with 13GB dev database - KC=F Coffee C futures prices: daily OHLCV, 20d/50d SMA, 52-week range (1971present)
- Incremental-by-time-range models with deduplication - ICE certified warehouse stocks: extractor ready, awaiting URL confirmation
- Development environment with pre-commit hooks, linting, formatting - Web app (Quart + HTMX): dashboard with supply/demand + COT + price + ICE charts
- REST API with key auth + rate limiting: /metrics, /positioning, /prices, /stocks
- Paddle billing (Starter/Pro plans), magic-link auth, admin panel
- /methodology page with full data source documentation
- Automated supervisor: all extractors + webhook alerting on failure
- 23 passing tests, GitLab CI pipeline
### What We Have ### What's Missing
- Comprehensive commodity supply/demand data (USDA PSD) - ICE stocks URL confirmed and backfill running (URL needs manual discovery at theice.com/report-center)
- Established naming conventions and data quality patterns - Python SDK
- GitLab CI pipeline (lint, test, build) - Public API documentation
- Documentation (CLAUDE.md, layer conventions)
## Roadmap ## Roadmap
### Phase 1: Coffee Market Foundation (Current) ### Phase 1: Coffee Market Foundation (COMPLETE — ready for outreach)
**Goal:** Build complete coffee analytics from supply to price **Goal:** Build complete coffee analytics from supply to price
**Data Sources to Integrate:** **Data Sources:**
- ✅ USDA PSD Online (production, stocks, consumption) - ✅ USDA PSD Online (production, stocks, consumption)
- ⬜ ICO (International Coffee Organization) data - ✅ CFTC COT data (trader positioning, COT index)
- ⬜ Yahoo Finance / Alpha Vantage (coffee futures prices - KC=F) - ✅ KC=F Coffee futures prices (daily OHLCV, moving averages)
- ⬜ Weather data for coffee-growing regions (OpenWeatherMap, NOAA) - ✅ ICE warehouse stocks (extractor built, seed models deployed)
-CFTC COT data (trader positioning) -ICO (International Coffee Organization) — future
- ⬜ ICE warehouse stocks (web scraping)
**Features to Build:** **Features:**
- ⬜ Historical price correlation analysis - ✅ Dashboard: supply/demand + COT + price + ICE warehouse charts
- ⬜ Supply/demand balance modeling - ✅ REST API: all 4 data sources
- ⬜ Weather impact scoring - ✅ Data methodology page
- ⬜ Trader sentiment indicators (COT) - ✅ Automated daily pipeline with alerting
-Simple web dashboard (read-only analytics) -Python SDK
-Data export APIs (JSON, CSV, Parquet) -Historical correlation analysis
**Infrastructure:** **Infrastructure:**
- ⬜ Move to Cloudflare R2 for raw data storage - ✅ Supervisor loop with all extractors
-Deploy SQLMesh to Hetzner production environment -Move to Cloudflare R2 for raw data backup
-Set up automated daily extraction + transformation pipeline -Deploy to Hetzner production
- ⬜ Implement monitoring and alerting
### Phase 2: Product Market Fit ### Phase 2: Product Market Fit
**Goal:** Validate with real traders, iterate on feedback **Goal:** Validate with real traders, iterate on feedback