fix(transform+content): unblock SQLMesh plan — three pipeline fixes

stg_playtomic_availability:
- Add maximum_object_size = 134217728 (128 MB) to both read_json calls;
  daily files exceed the 16 MB default as venue count grows
- Add seed recheck file (1970-01-01_recheck_00.json.gz, gitignored with data/)
  to avoid READ_JSON IOException when no recheck files exist

pseo_city_costs_de + pseo_city_pricing:
- Add QUALIFY ROW_NUMBER() OVER (PARTITION BY city_slug ...) = 1 to
  deduplicate rows caused by dim_cities' loose LIKE join; reduces
  pseo_city_costs_de from 2.6M → 222 rows (one per unique city)

content/__init__.py:
- DuckDB lowercases all column names at rest ("ratePeak" → "ratepeak"),
  so calc_overrides dict comprehension never matched DEFAULTS keys.
  Fix: build case-insensitive reverse map {k.lower(): k} and normalise
  row keys before lookup. Applied in both generate_articles() and
  preview_article().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-02-23 18:51:53 +01:00
parent dd1daaad1e
commit e3a6b91bc0
4 changed files with 26 additions and 4 deletions

View File

@@ -42,3 +42,6 @@ INNER JOIN serving.city_market_profile c
AND LOWER(TRIM(vpb.city)) = LOWER(TRIM(c.city_name))
-- Only cities with enough venues for meaningful pricing statistics
WHERE vpb.venue_count >= 2
-- city_market_profile inherits duplicates from dim_cities' loose LIKE join;
-- take the highest market_score row as the canonical city record.
QUALIFY ROW_NUMBER() OVER (PARTITION BY c.city_slug ORDER BY c.market_score DESC NULLS LAST) = 1