changelog: bring up to date through Feb 2026
- [Unreleased]: ICE overhaul (aging + by-port + API discovery + XLS parsing), extract_all meta-pipeline, Origin Intelligence redesign + HTMX, axis labels - [0.2.0]: CFTC COT, KC=F prices, ICE daily stocks, methodology page, supervisor - [0.1.0]: initial dashboard, country comparison, REST API, plan tiers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
50
CHANGELOG.md
50
CHANGELOG.md
@@ -4,6 +4,46 @@ All notable changes to BeanFlows are documented here.
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **ICE certified stock aging report** — Monthly age-bucket × port breakdown extracted via ICE API, stored as gzip CSV, modelled through raw→foundation→serving, exposed at `GET /api/v1/commodities/<code>/stocks/aging`
|
||||||
|
- **ICE historical warehouse stocks by port** — End-of-month data from Nov 1996 to present, downloaded from static ICE URL, full SQLMesh pipeline, exposed at `GET /api/v1/commodities/<code>/stocks/by-port`
|
||||||
|
- **ICE API-based URL discovery** — Replaces brittle `ICE_STOCKS_URL` env var; queries ICE Report Center JSON API, finds latest report by label, falls back automatically on 404
|
||||||
|
- **XLS parsing** (`ice_stocks/xls_parse.py`) — Magic-byte format detection (`OLE2_MAGIC`) and xlrd-based row extraction; handles both CSV and XLS responses from ICE
|
||||||
|
- **`extract_ice_all`** CLI command — Runs all three ICE extractors (daily stocks, aging, by-port) in one command
|
||||||
|
- **`extract_all`** meta-pipeline — Sequences all four data source extractors (`extract`, `extract_cot`, `extract_prices`, `extract_ice_all`), stops on first failure
|
||||||
|
- **ICE aging dashboard section** — Metric cards (total bags, youngest %, oldest %, bucket count) + stacked horizontal bar chart (age buckets × ports)
|
||||||
|
- **ICE by-port dashboard section** — Metric cards (total, MoM change, 12m avg, 30yr history) + stacked area chart (30-year view by delivery port)
|
||||||
|
- **Origin Intelligence page** — Full redesign of country comparison: dark espresso selector panel + metric segment tabs + colored chips + Chart.js line chart + live-built rankings table with proportional bars
|
||||||
|
- **HATEOAS country comparison** — Clicking an origin immediately fetches an HTMX partial and swaps the canvas; URL updates via `history.pushState()`; no Apply button needed
|
||||||
|
- **Axis labels on all charts** — Y-axis titles on every overview and Origins chart; column unit header in YoY and rankings tables
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **ICE XLS date parsing** — Handle `"Feb 20, 2026 1:35:39PM"` format (previously only `"1/30/2026"` worked); tries multiple `strptime` formats across single-token and three-token date candidates
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **Countries nav item renamed to "Origins"** — Matches "Origin Intelligence" page heading in both sidebar and mobile bottom nav
|
||||||
|
- **`dim_commodity`** — Added `ticker` and `ice_stock_report_code` columns (`KC=F`, `COFFEE-C`)
|
||||||
|
- **SQLMesh macros** — Added `ice_aging_glob()` and `ice_stocks_by_port_glob()` alongside existing `ice_stocks_glob()`
|
||||||
|
|
||||||
|
## [0.2.0] - 2026-02-01
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **CFTC COT disaggregated futures** — Weekly trader positioning (managed money net, COT index 26w/52w, % of open interest); full SQLMesh pipeline; `GET /api/v1/commodities/<code>/positioning`
|
||||||
|
- **KC=F Coffee C futures prices** — Daily OHLCV from Yahoo Finance (1971–present), 20d/50d SMA, 52-week range; `GET /api/v1/commodities/<code>/prices`
|
||||||
|
- **ICE certified warehouse stocks (daily rolling)** — Certified Arabica bags, WoW change, 30d average, 52-week drawdown; `GET /api/v1/commodities/<code>/stocks`
|
||||||
|
- **COT positioning dashboard section** — Metric cards + dual-axis line chart (managed money net + COT index)
|
||||||
|
- **Price dashboard section** — Metric cards + three-line chart (close, 20d MA, 50d MA)
|
||||||
|
- **ICE daily stocks dashboard section** — Metric cards + area chart with 30d MA
|
||||||
|
- **`/methodology` page** — Full data source documentation: lineage, field definitions, update cadence for every source
|
||||||
|
- **Automated supervisor pipeline** — Systemd service runs all extractors daily; Discord/webhook alerting on failure
|
||||||
|
- **`extract_all` meta-pipeline precursor** — Individual pipeline entries for all four data sources
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **COT pipeline nulls** — `TRY_CAST` instead of `CAST` in foundation model; leading zeros in CFTC commodity codes preserved
|
||||||
|
- **Concurrent DuckDB queries** — `cursor()` per thread prevents connection state corruption
|
||||||
|
|
||||||
|
## [0.1.0] - 2025-12-01
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- **Coffee analytics dashboard** — Chart.js visualizations for global supply/demand time series, stock-to-use ratio trend, top producing countries bar chart, and YoY production change table
|
- **Coffee analytics dashboard** — Chart.js visualizations for global supply/demand time series, stock-to-use ratio trend, top producing countries bar chart, and YoY production change table
|
||||||
- **Country comparison page** — Multi-select country picker with metric selector and overlay line chart at `/dashboard/countries`
|
- **Country comparison page** — Multi-select country picker with metric selector and overlay line chart at `/dashboard/countries`
|
||||||
@@ -13,20 +53,16 @@ All notable changes to BeanFlows are documented here.
|
|||||||
- **Landing, features, and pricing pages** rewritten for coffee market intelligence positioning
|
- **Landing, features, and pricing pages** rewritten for coffee market intelligence positioning
|
||||||
- **Health check** now verifies both SQLite and DuckDB connectivity
|
- **Health check** now verifies both SQLite and DuckDB connectivity
|
||||||
- **Admin panel** shows commodity count and data year range from DuckDB
|
- **Admin panel** shows commodity count and data year range from DuckDB
|
||||||
- **Docker** config updated with DuckDB volume mount
|
|
||||||
- **Tests** — `test_dashboard.py` (8 tests), `test_api_commodities.py` (8 tests), analytics mock fixture in conftest
|
- **Tests** — `test_dashboard.py` (8 tests), `test_api_commodities.py` (8 tests), analytics mock fixture in conftest
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- **Pipeline time granularity** — Added `market_year` to GROUP BY in `cleaned.psdalldata__commodity_pivoted` and carried through to `serving.commodity_metrics`. Previously summed across all market years, making per-year metrics meaningless.
|
- **Pipeline time granularity** — Added `market_year` to GROUP BY in `cleaned.psdalldata__commodity_pivoted`; previously summed across all market years making per-year metrics meaningless
|
||||||
|
|
||||||
### Removed
|
### Removed
|
||||||
- `items` table, FTS virtual table, and triggers from schema (boilerplate domain entity)
|
- `items` table, FTS virtual table, and triggers from schema (boilerplate)
|
||||||
- Items CRUD from API routes
|
- Items CRUD from API routes
|
||||||
- Items count from dashboard stats
|
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- `PLAN_FEATURES` updated from generic (`basic`, `export`) to domain-specific (`dashboard`, `coffee_only`, `all_commodities`, `full_history`, `export`, `api`)
|
- `PLAN_FEATURES` updated from generic to domain-specific scopes
|
||||||
- `PLAN_LIMITS` changed from `items`/`api_calls` to `commodities`/`history_years`/`api_calls`
|
- `PLAN_LIMITS` changed from `items`/`api_calls` to `commodities`/`history_years`/`api_calls`
|
||||||
- API now requires Starter or Pro plan (free plan gets 403)
|
- API now requires Starter or Pro plan (free plan gets 403)
|
||||||
- Dashboard routes and API routes import analytics module (not individual functions) for testability
|
|
||||||
- Billing tests updated to match new feature and limit names
|
|
||||||
|
|||||||
Reference in New Issue
Block a user