Fix SQLMesh config to use correct Pulumi ESC env var names

- Update secret token: CLOUDFLARE_API_TOKEN → R2_ADMIN_API_TOKEN
- Update warehouse name: R2_WAREHOUSE_NAME → ICEBERG_WAREHOUSE_NAME
- Update endpoint: ICEBERG_REST_URI → ICEBERG_CATALOG_URI

- Remove CREATE SCHEMA and USE statements
  - DuckDB has bug with Iceberg REST: missing Content-Type header
  - Schema creation via SQL currently not supported
  - Models will use fully-qualified table names instead

Successfully tested with real R2 credentials:
- Iceberg catalog attachment works ✓
- Plan dry-run executes ✓
- Only fails on missing source data (expected) ✓

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Deeman
2025-10-13 22:21:27 +02:00
parent 05ef15bfdf
commit 2d248a2eef

View File

@@ -12,7 +12,7 @@ gateways:
secrets:
r2_secret:
type: iceberg
token: "{{ env_var('CLOUDFLARE_API_TOKEN') }}"
token: "{{ env_var('R2_ADMIN_API_TOKEN') }}"
default_gateway: prod
@@ -22,9 +22,10 @@ default_gateway: prod
# https://developers.cloudflare.com/r2/data-catalog/config-examples/duckdb/
before_all:
- "ATTACH '{{ env_var('R2_WAREHOUSE_NAME', 'materia') }}' AS catalog (TYPE ICEBERG, ENDPOINT '{{ env_var('ICEBERG_REST_URI') }}', SECRET r2_secret);"
- CREATE SCHEMA IF NOT EXISTS catalog.materia;
- USE catalog.materia;
- "ATTACH '{{ env_var('ICEBERG_WAREHOUSE_NAME') }}' AS catalog (TYPE ICEBERG, ENDPOINT '{{ env_var('ICEBERG_CATALOG_URI') }}', SECRET r2_secret);"
# Note: CREATE SCHEMA has a DuckDB/Iceberg bug (missing Content-Type header)
# Schema must be pre-created in R2 Data Catalog via Cloudflare dashboard or API
# For now, skip USE statement and rely on fully-qualified table names in models
# --- Model Defaults ---
# https://sqlmesh.readthedocs.io/en/stable/reference/model_configuration/#model-defaults