c3c8333407
- Delete 6 data raw models (coffee_prices, cot_disaggregated, ice_*, psd_data) — pure read_csv passthroughs with no added value - Move 3 PSD seed models raw/ → seeds/, rename schema raw.* → seeds.* - Update staging.psdalldata__commodity: read_csv(@psd_glob()) directly, join seeds.psd_* instead of raw.psd_* - Update 5 foundation models: inline read_csv() with src CTE, removing raw.* dependency (fct_coffee_prices, fct_cot_positioning, fct_ice_*) - Remove fixture-based SQLMesh test that depended on raw.cot_disaggregated (unit tests incompatible with inline read_csv; integration run covers this) - Update readme.md: 3-layer architecture (staging/foundation → serving) Landing files are immutable and content-addressed — the landing directory is the audit trail. A raw SQL layer duplicated file bytes into DuckDB with no added value. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
24 lines
1.1 KiB
SQL
24 lines
1.1 KiB
SQL
-- Commodity dimension: conforms identifiers across source systems.
|
|
--
|
|
-- This is the ontology. Each row is a commodity tracked by BeanFlows.
|
|
-- As new sources are added (ICO, futures prices, satellite), their
|
|
-- commodity identifiers are added as columns here — not as separate tables.
|
|
-- As new commodities are added (cocoa, sugar), rows are added here.
|
|
--
|
|
-- References:
|
|
-- usda_commodity_code → staging.psdalldata__commodity.commodity_code (numeric string, e.g. '0711100')
|
|
-- cftc_commodity_code → foundation.fct_cot_positioning.cftc_commodity_code (3-char, e.g. '083')
|
|
--
|
|
-- NOTE: Defined as FULL model (not SEED) to guarantee leading-zero preservation.
|
|
-- Pandas CSV loading converts '083' → 83 even with varchar column declarations.
|
|
|
|
MODEL (
|
|
name foundation.dim_commodity,
|
|
kind FULL
|
|
);
|
|
|
|
SELECT usda_commodity_code, cftc_commodity_code, ticker, ice_stock_report_code, commodity_name, commodity_group
|
|
FROM (VALUES
|
|
('0711100', '083', 'KC=F', 'COFFEE-C', 'Coffee, Green', 'Softs')
|
|
) AS t(usda_commodity_code, cftc_commodity_code, ticker, ice_stock_report_code, commodity_name, commodity_group)
|