fix(pipeline): make availability chain incremental + fix supervisor

Convert the availability chain (stg_playtomic_availability →
fct_availability_slot → fct_daily_availability) from FULL to
INCREMENTAL_BY_TIME_RANGE so sqlmesh run processes only new daily
intervals instead of re-reading all files.

Supervisor changes:
- run_transform(): plan prod --auto-apply → run prod (evaluates
  missing cron intervals, picks up new data)
- git_pull_and_sync(): add plan prod --auto-apply before re-exec
  so model code changes are applied on deploy
- supervisor.sh: same plan → run change

Staging model uses a date-scoped glob (@start_ds) to read only
the current interval's files. snapshot_date cast to DATE (was
VARCHAR) as required by time_column.

Clean up redundant TRY_CAST(snapshot_date AS DATE) in
venue_pricing_benchmarks since it's already DATE from foundation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-03-05 21:34:02 +01:00
parent c54eb50004
commit 2f47d1e589
6 changed files with 30 additions and 41 deletions

View File

@@ -14,7 +14,10 @@
MODEL (
name foundation.fct_availability_slot,
kind FULL,
kind INCREMENTAL_BY_TIME_RANGE (
time_column snapshot_date
),
start '2026-03-01',
cron '@daily',
grain (snapshot_date, tenant_id, resource_id, slot_start_time)
);
@@ -37,7 +40,8 @@ WITH deduped AS (
captured_at_utc DESC
) AS rn
FROM staging.stg_playtomic_availability
WHERE price_amount IS NOT NULL
WHERE snapshot_date BETWEEN @start_ds AND @end_ds
AND price_amount IS NOT NULL
AND price_amount > 0
)
SELECT