Add CFTC COT data integration with foundation data model layer
- New extraction package (cftc_cot): downloads yearly Disaggregated Futures ZIPs from CFTC, etag-based dedup, dynamic inner filename discovery, gzip normalization - SQLMesh 3-layer architecture: raw (technical) → foundation (business model) → serving (mart) - dim_commodity seed: conformed dimension mapping USDA ↔ CFTC codes — the commodity ontology - fct_cot_positioning: typed, deduplicated weekly positioning facts for all commodities - obt_cot_positioning: Coffee C mart with COT Index (26w/52w), WoW delta, OI ratios - Analytics functions + REST API endpoints: /commodities/<code>/positioning[/latest] - Dashboard widget: Managed Money net, COT Index card, dual-axis Chart.js chart - 23 passing tests (10 unit + 2 SQLMesh model + existing regression suite) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
-- Commodity dimension: conforms identifiers across source systems.
|
||||
--
|
||||
-- This is the ontology seed. Each row is a commodity tracked by BeanFlows.
|
||||
-- As new sources are added (ICO, futures prices, satellite), their
|
||||
-- commodity identifiers are added as columns here — not as separate tables.
|
||||
-- As new commodities are added (cocoa, sugar), rows are added here.
|
||||
--
|
||||
-- References:
|
||||
-- usda_commodity_code → raw.psd_alldata.commodity_code
|
||||
-- cftc_commodity_code → raw.cot_disaggregated.cftc_commodity_code
|
||||
|
||||
MODEL (
|
||||
name foundation.dim_commodity,
|
||||
kind SEED (
|
||||
path '$root/seeds/dim_commodity.csv',
|
||||
csv_settings (delimiter = ';')
|
||||
),
|
||||
columns (
|
||||
usda_commodity_code varchar,
|
||||
cftc_commodity_code varchar,
|
||||
commodity_name varchar,
|
||||
commodity_group varchar
|
||||
)
|
||||
);
|
||||
Reference in New Issue
Block a user