Convert the availability chain (stg_playtomic_availability →
fct_availability_slot → fct_daily_availability) from FULL to
INCREMENTAL_BY_TIME_RANGE so sqlmesh run processes only new daily
intervals instead of re-reading all files.
Supervisor changes:
- run_transform(): plan prod --auto-apply → run prod (evaluates
missing cron intervals, picks up new data)
- git_pull_and_sync(): add plan prod --auto-apply before re-exec
so model code changes are applied on deploy
- supervisor.sh: same plan → run change
Staging model uses a date-scoped glob (@start_ds) to read only
the current interval's files. snapshot_date cast to DATE (was
VARCHAR) as required by time_column.
Clean up redundant TRY_CAST(snapshot_date AS DATE) in
venue_pricing_benchmarks since it's already DATE from foundation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add scripts/check_pipeline.py: read-only diagnostic for pricing pipeline
row counts, date range analysis, HAVING filter impact, join coverage
- Add description field to all 12 workflows in workflows.toml
- Parse and display descriptions on extraction status cards
- Show spinner + "Running" state with blue-tinted card border
- Display start time with "running..." text for active extractions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
stg_population_geonames → dim_locations → location_opportunity_profile
were all 0 rows in prod because the GeoNames extractor was never
scheduled. First run will backfill cities1000 to landing zone.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- playtomic_tenants: partition by YYYY/MM/DD instead of ISO week;
schedule changed from weekly to daily in workflows.toml
- playtomic_availability: _load_tenant_ids now tries 3-level glob
(*/*/*/tenants.jsonl.gz) first for daily files, falls back to
2-level for old monthly/weekly data
Alphabetical sort would rank old monthly files above daily ones
('t' > '2' in ASCII), so the explicit fallback chain is required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tennis extraction was missing from workflows.toml — only ran via the combined
`uv run extract` command, not automatically in production.
Schedule: monthly (same cadence as padel courts, OSM tennis data updates slowly).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each slot is now rechecked once, at most 30 min before it starts.
Worst-case miss: a booking made 29 min before start.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- setup_server.sh now requires root, creates padelnomics_service user,
adds to docker group, generates deploy key in service user's home,
owns /opt/padelnomics and /data/padelnomics to service user
- supervisor service: User=padelnomics_service, updated PATH
- landing-backup service: User=padelnomics_service
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove raw/ layer — staging models now read landing JSON directly.
Rename all model schemas from padelnomics.* to staging.*/foundation.*/serving.*.
Web app queries updated to serving.planner_defaults via SERVING_DUCKDB_PATH.
Supervisor gets daily sleep interval between pipeline runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pulls in template changes: export_serving.py for atomic DuckDB swap,
supervisor export step, SQLMesh glob macro, server provisioning script,
imprint template, and formatting improvements.
Template scaffold SQL models excluded (padelnomics has real models).
Web app routes/analytics unchanged (padelnomics-specific customizations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync template from 29ac25b → v0.9.0 (29 template commits). Due to
template's _subdirectory migration, new files were manually rendered
rather than auto-merged by copier.
New files:
- .claude/CLAUDE.md + coding_philosophy.md (agent instructions)
- extract utils.py: SQLite state tracking for extraction runs
- extract/transform READMEs: architecture & pattern documentation
- infra/supervisor: systemd service + orchestration script
- Per-layer model READMEs (raw, staging, foundation, serving)
Also fixes copier-answers.yml (adds 4 feature toggles, removes stale
payment_provider key) and scopes CLAUDE.md gitignore to root only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
