diff --git a/docs/CMS.md b/docs/CMS.md index 970ea24..a2c42d6 100644 --- a/docs/CMS.md +++ b/docs/CMS.md @@ -1,181 +1,294 @@ # CMS & Programmatic SEO -How the content management system works: template definitions, bulk article generation, scenario embedding, the build pipeline, and how content is discovered and served. +How the content management system works: git-based template definitions, DuckDB data sources, bulk article generation, scenario embedding, SEO pipeline, and how content is served. --- ## Overview -The CMS is a **template-driven, programmatic content system**. The core idea: +The CMS is an **SSG-inspired programmatic content system**. The core idea: -1. Define a **template** — URL pattern, title pattern, meta description pattern, and a Jinja2 body -2. Upload **data rows** (CSV or single entry via admin UI) that fill the template variables -3. **Generate** — the system runs the financial calculator, writes HTML to disk, and inserts `articles` rows -4. Articles are **served** by a catch-all route that looks up the URL path in the database +1. **Templates live in git** — `.md.jinja` files with YAML frontmatter define how articles look +2. **Data lives in DuckDB** — one serving table per pSEO idea provides what articles say +3. **Generation** renders templates with data, bakes in SEO metadata, writes HTML to disk +4. **SQLite** stores routing state only — `articles` and `published_scenarios` tables +5. Articles are **served** by a catch-all route reading pre-built HTML from disk -This is designed for scale: one template definition can produce hundreds of SEO-targeted articles. +``` + Git repo (content shape) DuckDB (content data) SQLite (routing/state) + ──────────────────────── ───────────────────── ────────────────────── + content/templates/ serving.pseo_city_costs_de articles + city-cost-de.md.jinja → city_slug, population, → url_path, slug + market-compare.md.jinja venue_count, rates... → status, published_at + → template_slug + serving.pseo_market_compare + "How articles look" "What articles say" published_scenarios + → slug, state_json + → calc_json + "Where things live" +``` + +--- + +## Template file format + +Templates are `.md.jinja` files in `web/src/padelnomics/content/templates/`. Each file has YAML frontmatter delimited by `---`, followed by a Jinja2 Markdown body. + +### Example + +```markdown +--- +name: "DE City Padel Costs" +slug: city-cost-de +content_type: calculator +data_table: serving.pseo_city_costs_de +natural_key: city_slug +languages: [de, en] +url_pattern: "/markets/{{ country_name_en | lower | slugify }}/{{ city_slug }}" +title_pattern: "Padel in {{ city_name }} — Market Analysis & Costs" +meta_description_pattern: "How much does it cost to build a padel center in {{ city_name }}? {{ padel_venue_count }} venues, pricing data & financial model." +schema_type: [Article, FAQPage] +priority_column: population +--- +# Padel in {{ city_name }} + +{{ city_name }} has {{ padel_venue_count }} padel venues... + +[scenario:{{ scenario_slug }}:capex] + +## FAQ + +**How much does a padel court cost in {{ city_name }}?** +Based on our financial model, a {{ courts_typical }}-court center requires... +``` + +### Frontmatter fields + +| Field | Required | Description | +|-------|----------|-------------| +| `name` | Yes | Human-readable template name (shown in admin) | +| `slug` | Yes | URL-safe identifier, also the filename stem | +| `content_type` | Yes | `calculator` (has financial scenario) or `editorial` | +| `data_table` | Yes | DuckDB serving table, schema-qualified (e.g. `serving.pseo_city_costs_de`) | +| `natural_key` | Yes | Column used as unique row identifier (e.g. `city_slug`) | +| `languages` | Yes | List of language codes to generate (e.g. `[de, en]`) | +| `url_pattern` | Yes | Jinja2 pattern for article URL path | +| `title_pattern` | Yes | Jinja2 pattern for `` and `og:title` | +| `meta_description_pattern` | Yes | Jinja2 pattern for meta description (aim for 120-155 chars) | +| `schema_type` | No | JSON-LD type(s): `Article` (default), `FAQPage`, or list. See [Structured data](#structured-data) | +| `priority_column` | No | Column to sort by for publish order (highest value first) | +| `related_template_slugs` | No | Other template slugs for cross-linking (future) | + +### Body template + +Everything after the closing `---` is the body. It's Jinja2 Markdown with access to: + +- **All columns** from the DuckDB row (e.g. `{{ city_name }}`, `{{ population }}`) +- **`language`** — current language code (`en`, `de`) +- **`scenario_slug`** — auto-derived slug for `[scenario:]` markers (calculator type only) + +The `slugify` filter is available: `{{ country_name_en | slugify }}`. --- ## Database schema -``` -articles — Published articles (the SEO output) -article_templates — Template definitions (patterns + Jinja2 body) -template_data — One row per article to be generated; links template → article -published_scenarios — Calculator results embedded in article bodies -articles_fts — FTS5 virtual table for full-text search over articles -``` +After migration 0018, only two content tables remain in SQLite: ### articles | Column | Type | Notes | |--------|------|-------| -| `id` | INTEGER PK | | -| `url_path` | TEXT UNIQUE | Matches `request.path`, e.g. `/markets/de/berlin` | -| `slug` | TEXT UNIQUE | Used for the build HTML filename | -| `title` | TEXT | | -| `meta_description` | TEXT | | -| `country` | TEXT | Used for grouping in the markets hub | -| `region` | TEXT | Used for grouping in the markets hub | -| `og_image_url` | TEXT | | +| `url_path` | TEXT UNIQUE | e.g. `/en/markets/germany/berlin` | +| `slug` | TEXT UNIQUE | e.g. `city-cost-de-en-berlin` | +| `title` | TEXT | Rendered from `title_pattern` | +| `meta_description` | TEXT | Rendered from `meta_description_pattern` | +| `country` | TEXT | For markets hub grouping | +| `region` | TEXT | For markets hub grouping | | `status` | TEXT | `'published'` or `'draft'` | -| `published_at` | DATETIME | Future dates suppress the article until then | -| `template_data_id` | INTEGER FK | Back-reference to `template_data.id` | +| `published_at` | DATETIME | Future dates suppress the article | +| `template_slug` | TEXT | Links back to git template for regeneration | +| `language` | TEXT | Language code (`en`, `de`) for hreflang | +| `date_modified` | TEXT | ISO timestamp, updated on regeneration | +| `seo_head` | TEXT | Pre-computed HTML: canonical, hreflang, JSON-LD, OG tags | -FTS5 virtual table `articles_fts` is kept in sync with `title`, `meta_description`, `country`, `region`. - -### article_templates - -| Column | Type | Notes | -|--------|------|-------| -| `id` | INTEGER PK | | -| `name` | TEXT | Human label shown in admin | -| `slug` | TEXT UNIQUE | | -| `content_type` | TEXT | e.g. `'market'` | -| `input_schema` | JSON | Array of field definitions; drives the admin form | -| `url_pattern` | TEXT | Jinja2, e.g. `/markets/{{ country \| lower }}/{{ city_slug }}` | -| `title_pattern` | TEXT | Jinja2, e.g. `{{ city }} Padel Court Economics` | -| `meta_description_pattern` | TEXT | Jinja2 | -| `body_template` | TEXT | Jinja2 Markdown with optional `[scenario:slug]` markers | - -### template_data - -| Column | Type | Notes | -|--------|------|-------| -| `id` | INTEGER PK | | -| `template_id` | INTEGER FK | → `article_templates.id` | -| `data_json` | JSON | The variables that fill the template | -| `scenario_id` | INTEGER FK | → `published_scenarios.id` (set after generation) | -| `article_id` | INTEGER FK | → `articles.id` (set after generation) | -| `created_at` | DATETIME | | -| `updated_at` | DATETIME | | +FTS5 virtual table `articles_fts` syncs `title`, `meta_description`, `country`, `region` via triggers. ### published_scenarios -Pre-calculated financial scenarios embedded in articles. - | Column | Type | Notes | |--------|------|-------| -| `id` | INTEGER PK | | | `slug` | TEXT UNIQUE | Referenced in `[scenario:slug]` markers | -| `title` | TEXT | | -| `subtitle` | TEXT | | +| `title` | TEXT | City name | | `location` | TEXT | City name | | `country` | TEXT | | -| `venue_type` | TEXT | | -| `ownership` | TEXT | | -| `court_config` | TEXT | | +| `venue_type` | TEXT | `indoor` / `outdoor` | +| `ownership` | TEXT | `rent` / `own` | +| `court_config` | TEXT | e.g. `4 double + 2 single` | | `state_json` | JSON | Calculator input state | | `calc_json` | JSON | Calculator output (all sections) | -| `template_data_id` | INTEGER FK | | --- -## Programmatic SEO pipeline +## Generation pipeline -### Step 1 — Define a template +### Full flow -In admin → Templates → New Template. +``` + 1. Create serving model in SQLMesh + models/serving/pseo_city_costs_de.sql → sqlmesh plan prod → analytics.duckdb -**URL pattern** uses Jinja2 and must produce a unique, clean path: + 2. Create template file in git + content/templates/city-cost-de.md.jinja -```jinja2 -/markets/{{ country | lower }}/{{ city_slug }} + 3. Preview locally + Admin → Templates → pick template → pick city → preview rendered article + + 4. Commit + push + deploy + + 5. Admin → Templates → "DE City Padel Costs" + sees data_table, row count from DuckDB, generated count + + 6. Click "Generate" + start_date, articles_per_day → for each DuckDB row × language: + • render URL, title, meta patterns + • create/update published_scenario (calculator type) + • render body.md.jinja → Markdown → HTML + • bake [scenario:] markers into HTML + • inject SEO head (canonical, hreflang, JSON-LD, OG) + • write HTML to data/content/_build/{lang}/{slug}.html + • upsert article row in SQLite + • stagger published_at dates + + 7. Articles live at /en/markets/germany/berlin + with links to /planner?scenario=city-cost-de-berlin + + 8. Data refresh → "Regenerate" in admin + reads fresh DuckDB data, updates HTML + scenarios in-place ``` -**Title and meta description patterns** are also Jinja2: +### What happens per row -```jinja2 -{{ city }} Padel Court Business Plan — {{ court_count }}-Court Indoor Venue -Annual revenue and ROI analysis for a {{ court_count }}-court padel venue in {{ city }}, {{ country }}. -``` +For each DuckDB row × each language in `config.languages`: -**Body template** is Jinja2 Markdown. Access any field from `data_json`, plus the generated scenario's slug via `scenario_slug`. Use `[scenario:slug]` markers to embed calculator sections: +1. Render `url_pattern`, `title_pattern`, `meta_description_pattern` with row data +2. If `content_type == "calculator"`: extract calc fields → `validate_state()` → `calc()` → upsert `published_scenarios` +3. Render body template with row data + `scenario_slug` +4. Convert Markdown → HTML via `mistune` +5. Replace `[scenario:slug:section]` markers with rendered HTML cards +6. Build SEO head: canonical, hreflang links, JSON-LD, OG tags +7. Write HTML to `data/content/_build/{lang}/{article_slug}.html` +8. Upsert `articles` row in SQLite +9. Stagger `published_at` based on `articles_per_day` -```markdown -## {{ city }} Market Overview +### Scenario markers -A {{ court_count }}-court indoor venue in {{ city }} has the following economics: - -[scenario:{{ scenario_slug }}] - -### Capital Expenditure - -[scenario:{{ scenario_slug }}:capex] - -### Annual Cash Flow - -[scenario:{{ scenario_slug }}:cashflow] -``` - -**Scenario section options** for the `[scenario:slug:section]` syntax: +Use `[scenario:slug]` or `[scenario:slug:section]` in the body template: | Section | Shows | |---------|-------| -| _(none)_ | Default summary card | +| _(none)_ | Default summary card with CTA link to planner | | `capex` | Capital expenditure breakdown | | `operating` | Operating cost breakdown | -| `cashflow` | Annual cash flow | +| `cashflow` | Annual cash flow projection | | `returns` | ROI / returns summary | -| `full` | All sections | +| `full` | All sections combined | -### Step 2 — Upload data +Always use `{{ scenario_slug }}` — never hardcode a slug. -Admin → Templates → (template) → Data → Upload CSV, or add single rows via the form. +### Scenario slug derivation -The CSV columns must match the template's `input_schema`. Each row becomes one `template_data` record and will generate one article. - -### Step 3 — Generate - -Admin → Templates → (template) → Generate. - -**What happens for each data row:** - -1. Merge row data with calculator DEFAULTS -2. Run financial calculator → produce `state_json` + `calc_json` -3. Create a `published_scenarios` record with an auto-derived slug -4. Render Jinja2 patterns (URL, title, meta description, body) using row data + scenario slug -5. Convert body Markdown → HTML via `mistune` -6. Replace `[scenario:slug(:section)?]` markers with rendered HTML partials -7. Write final HTML to `data/content/_build/{slug}.html` -8. Back up Markdown source to `data/content/articles/{slug}.md` -9. Create an `articles` record -10. Update `template_data` row with `scenario_id` and `article_id` - -**Staggered publishing:** the `articles_per_day` parameter spreads `published_at` dates so articles drip out over time rather than all appearing at once. - -### Step 4 — Serve - -The `content` blueprint registers a catch-all route **last**, so it only fires if no other blueprint matched: - -```python -@bp.route("/<path:url_path>") -async def article_page(url_path: str): - ... +``` +scenario_slug = template_slug + "-" + natural_key_value ``` -Reserved prefixes are short-circuited to avoid conflicts: +Example: template `city-cost-de` + row `city_slug=berlin` → `city-cost-de-berlin` + +### Article slug derivation + +``` +article_slug = template_slug + "-" + language + "-" + natural_key_value +``` + +Example: `city-cost-de-en-berlin`, `city-cost-de-de-berlin` + +--- + +## SEO pipeline + +All SEO metadata is baked into the `seo_head` column at generation time. No runtime computation. + +### URL structure + +``` +/{lang}/markets/{country_name}/{city_slug} +``` + +- Language prefix for hreflang (`/de/...`, `/en/...`) +- Full country name in path (`/germany/`, not `/de/`) — avoids confusion with lang prefix +- `slugify` filter converts to lowercase hyphenated form + +### Hreflang + +For each article, links to all language variants + `x-default` (points to English): + +```html +<link rel="alternate" hreflang="de" href="https://padelnomics.io/de/markets/germany/berlin" /> +<link rel="alternate" hreflang="en" href="https://padelnomics.io/en/markets/germany/berlin" /> +<link rel="alternate" hreflang="x-default" href="https://padelnomics.io/en/markets/germany/berlin" /> +``` + +### Canonical URLs + +Self-referencing canonical on every article: + +```html +<link rel="canonical" href="https://padelnomics.io/en/markets/germany/berlin" /> +``` + +### Structured data + +JSON-LD `<script>` blocks injected in `seo_head`. The `schema_type` frontmatter controls which types are generated: + +| Schema | When | Data source | +|--------|------|-------------| +| `BreadcrumbList` | Always | Auto-generated from URL segments | +| `Article` | Default (or `schema_type: Article`) | Template patterns + row data | +| `FAQPage` | `schema_type: FAQPage` | Parsed from `## FAQ` section — bold questions + answer paragraphs | + +**`Article`** includes: `headline`, `datePublished`, `dateModified`, `author` (Padelnomics), `publisher`, `description`, `inLanguage`. + +**`FAQPage`** wraps bold-question/answer pairs from the `## FAQ` heading into `Question`/`acceptedAnswer` pairs. + +`dateModified` updates on every regeneration (freshness signal for Google). + +### Open Graph + +```html +<meta property="og:title" content="..." /> +<meta property="og:description" content="..." /> +<meta property="og:url" content="..." /> +<meta property="og:type" content="article" /> +``` + +### Drip publishing + +- `articles_per_day` parameter (default 3) staggers `published_at` dates +- `priority_column` sorts rows so high-value cities publish first +- Articles with future `published_at` are invisible to the catch-all route + +--- + +## Article serving + +The `content` blueprint registers a catch-all route: + +```python +@bp.route("/<lang>/<path:url_path>") +async def article_page(lang, url_path): +``` + +Reserved prefixes are short-circuited: ```python RESERVED_PREFIXES = ( @@ -185,261 +298,133 @@ RESERVED_PREFIXES = ( ) ``` -Lookup query: - -```sql -SELECT * FROM articles -WHERE url_path = ? AND status = 'published' AND published_at <= datetime('now') -``` - -If found, the pre-built HTML is read from `data/content/_build/{slug}.html` and injected into `article_detail.html`. No runtime Markdown rendering. - ---- - -## SEO meta tags - -`article_detail.html` overrides the base template's defaults with article-specific values: - -```html -<title>{{ article.title }} | Padelnomics - - - - - - - -``` - -**JSON-LD structured data** is also injected per article: - -```json -{ - "@context": "https://schema.org", - "@type": "Article", - "headline": "...", - "datePublished": "2025-01-15", - "author": { "@type": "Organization", "name": "Padelnomics" }, - "publisher": { "@type": "Organization", "name": "Padelnomics" } -} -``` +Lookup: find the article by `url_path`, check `status = 'published'` and `published_at <= now`, then read the pre-built HTML from `data/content/_build/{lang}/{slug}.html`. --- ## Markets hub -`/markets` is the discovery index for all published articles. +`/{lang}/markets` is the discovery index for all published articles. - Groups articles by `country` and `region` -- Full-text search via FTS5 (`articles_fts`) across `title`, `meta_description`, `country`, `region` -- HTMX partial at `/markets/results` handles live filtering (no page reload) +- Full-text search via FTS5 across `title`, `meta_description`, `country`, `region` +- HTMX partial at `/{lang}/markets/results` handles live filtering -FTS5 query (simplified): +--- -```sql -SELECT a.* FROM articles a -JOIN articles_fts ON articles_fts.rowid = a.id -WHERE articles_fts MATCH ? - AND a.status = 'published' - AND a.published_at <= datetime('now') -ORDER BY a.published_at DESC -LIMIT 100 -``` +## Admin interface -Filter fields: `q` (full-text), `country`, `region`. +Templates are **read-only** in the admin UI — edit them in git, preview and generate in admin. + +### Routes + +| Route | Method | Purpose | +|-------|--------|---------| +| `/admin/templates` | GET | List templates scanned from disk | +| `/admin/templates/` | GET | Template detail: config, DuckDB columns, sample data | +| `/admin/templates//preview/` | GET | Preview one article rendered in-memory | +| `/admin/templates//generate` | GET/POST | Generate form + action | +| `/admin/templates//regenerate` | POST | Re-generate all articles with fresh DuckDB data | + +Scenario and article management routes are unchanged (create, edit, delete, publish toggle, rebuild). --- ## Directory structure ``` -src/padelnomics/ +web/src/padelnomics/ ├── content/ -│ ├── routes.py # Catch-all article serving + markets hub -│ └── templates/ -│ ├── article_detail.html # SEO meta tags, JSON-LD, article body injection -│ ├── markets.html # Discovery index -│ └── partials/ -│ ├── market_results.html # HTMX search result rows -│ └── scenario_*.html # Scenario cards (capex, cashflow, etc.) +│ ├── __init__.py # Core engine: discover, load, generate, preview, SEO +│ ├── routes.py # Catch-all serving, markets hub, scenario baking +│ └── templates/ # Git-based .md.jinja template files +│ └── city-cost-de.md.jinja │ ├── admin/ -│ ├── routes.py # Full CMS admin (templates, data, generate, articles) +│ ├── routes.py # Read-only template views + generate/regenerate │ └── templates/admin/ -│ ├── templates.html -│ ├── template_form.html -│ ├── template_data.html -│ ├── generate_form.html +│ ├── templates.html # Template list (scanned from disk) +│ ├── template_detail.html # Config view, columns, sample data +│ ├── template_preview.html # Article preview +│ ├── generate_form.html # Schedule form │ ├── articles.html │ └── article_form.html │ -└── data/ - └── content/ - ├── _build/ # Generated HTML (served at runtime) - │ └── {slug}.html - └── articles/ # Markdown source backup - └── {slug}.md +└── migrations/versions/ + └── 0018_pseo_cms_refactor.py # Drops old tables, recreates articles + scenarios + +data/content/ +├── _build/ # Generated HTML (served at runtime) +│ ├── en/ +│ │ └── {slug}.html +│ └── de/ +│ └── {slug}.html +└── articles/ # Markdown source backup (manual articles) + └── {slug}.md ``` --- -## Adding a new content type +## Adding a new pSEO idea -### Step 1 — Define the template in admin +### Step 1 — Create the DuckDB serving model -Go to **Admin → Templates → New Template** and fill in: +Write a SQLMesh model at `transform/sqlmesh_padelnomics/models/serving/pseo_your_idea.sql` that produces one row per article to generate. Must include a `natural_key` column (e.g. `city_slug`). -| Field | Notes | -|-------|-------| -| Name | Human label, e.g. `"Indoor Market Report"` | -| Slug | Auto-generated; used to derive scenario slugs (cannot be changed after creation) | -| Content type | `"calculator"` (the only current option) | -| Input schema | JSON — see below | -| URL pattern | Jinja2 — see below | -| Title pattern | Jinja2 | -| Meta description pattern | Jinja2 (optional) | -| Body template | Jinja2 Markdown | - -### Step 2 — Write the input_schema - -`input_schema` is a JSON array of field definitions. Each field becomes a column in the CSV and an input in the single-row add form. - -```json -[ - { "name": "city", "label": "City", "field_type": "text", "required": true }, - { "name": "city_slug", "label": "City slug", "field_type": "text", "required": true }, - { "name": "country", "label": "Country (ISO 2)", "field_type": "text", "required": true }, - { "name": "region", "label": "Region", "field_type": "text", "required": false }, - { "name": "court_count", "label": "Number of courts", "field_type": "number", "required": false }, - { "name": "ratePeak", "label": "Peak rate (€/hr)", "field_type": "float", "required": false } -] +```bash +uv run sqlmesh -p transform/sqlmesh_padelnomics plan prod ``` -**Field keys:** +### Step 2 — Create the template file -| Key | Required | Values | Effect | -|-----|----------|--------|--------| -| `name` | yes | any string | Variable name in Jinja2 patterns; CSV column header | -| `label` | yes | any string | Human label in the admin form | -| `field_type` | yes | `"text"`, `"number"`, `"float"` | Controls input type and coercion during generation | -| `required` | no | `true` / `false` | Whether the admin form enforces a value | - -**`city_slug` is special** — the system looks for a field named exactly `city_slug` to derive the scenario slug. If it's absent, the `template_data` row ID is used as a fallback. - -**Calculator field names** — any field whose name matches a key in the calculator `DEFAULTS` dict (e.g. `ratePeak`, `rateOffPeak`, `dblCourts`, `loanPct`) is automatically passed to the financial model during generation. You don't need to do anything extra; just name the field correctly and the generator merges it into the calc state. - -### Step 3 — Write the Jinja2 patterns - -All patterns are rendered with the same context: every field from the data row, plus `scenario_slug` (injected by the generator). - -**URL pattern** — must be unique across all articles and must not start with a reserved prefix: - -```jinja2 -/markets/{{ country | lower }}/{{ city_slug }} -``` - -The article slug is derived by stripping the leading `/` and replacing remaining `/` with `-`, so `/markets/de/berlin` → slug `markets-de-berlin`. - -**Title pattern:** - -```jinja2 -{{ city }} Padel Court Business Plan — {{ court_count }}-Court {{ venue_type | title }} Venue -``` - -**Meta description pattern** (aim for ≤ 160 characters): - -```jinja2 -Revenue and ROI analysis for a {{ court_count }}-court padel venue in {{ city }}, {{ country }}. Includes CapEx, operating costs, and cash flow projections. -``` - -**Body template** — Jinja2 Markdown. Use any data row field, plus `scenario_slug` to embed calculator results: +Create `web/src/padelnomics/content/templates/your-idea.md.jinja`: ```markdown -## {{ city }} Market Overview +--- +name: "Your Idea Name" +slug: your-idea +content_type: calculator +data_table: serving.pseo_your_idea +natural_key: city_slug +languages: [en, de] +url_pattern: "/markets/{{ country_name_en | lower | slugify }}/{{ city_slug }}" +title_pattern: "Your Title for {{ city_name }}" +meta_description_pattern: "Your description for {{ city_name }}. Max 155 chars." +schema_type: [Article, FAQPage] +priority_column: population +--- +# Your Title for {{ city_name }} -A {{ court_count }}-court indoor venue in {{ city }} ({{ country }}) projects the following economics: - -[scenario:{{ scenario_slug }}] - -### Capital Expenditure +Article body with {{ template_variables }}... [scenario:{{ scenario_slug }}:capex] -### Annual Cash Flow +## FAQ -[scenario:{{ scenario_slug }}:cashflow] - -### Returns - -[scenario:{{ scenario_slug }}:returns] +**Your question about {{ city_name }}?** +Your answer with data from the row. ``` -### Step 4 — How scenario_slug is derived +### Step 3 — Preview and iterate -``` -scenario_slug = template_slug + "-" + city_slug -``` +Start the dev server, go to Admin → Templates → your template → pick a row → Preview. Iterate on the body until it looks right. -Example: template slug `indoor-market` + city_slug `berlin` → `indoor-market-berlin`. +### Step 4 — Commit, deploy, generate -This slug identifies the `published_scenarios` row. It's what you reference in `[scenario:slug]` markers. Always use `{{ scenario_slug }}` in the body template — never hardcode a slug. +Push the template file. In production admin, click Generate with a start date and articles/day cadence. -### Step 5 — Upload data +### Calculator field overrides -**CSV upload** (Admin → Templates → (template) → Data → Upload CSV): - -- Column headers must exactly match the `name` fields in `input_schema` -- One row per article to be generated -- Values are coerced to the declared `field_type` (`text` → string, `number` → int, `float` → float) - -Example CSV: - -```csv -city,city_slug,country,region,court_count,ratePeak,rateOffPeak -Berlin,berlin,DE,Brandenburg,6,55,35 -Munich,munich,DE,Bavaria,8,60,40 -Hamburg,hamburg,DE,Hamburg,4,50,30 -``` - -**Single-row add** — use the form in Admin → Templates → (template) → Data → Add Row for one-off entries. - -### Step 6 — Generate - -Admin → Templates → (template) → Generate. Set: - -- **Start date** — first article's `published_at` -- **Articles per day** — how many articles share each calendar day (stagger cadence) - -The generator processes all data rows that don't yet have an `article_id` (i.e. not previously generated). For each row it: - -1. Merges calculator fields from the row into `DEFAULTS`, validates and runs `calc(state)` -2. Creates a `published_scenarios` row -3. Injects `scenario_slug` into the render context -4. Renders URL, title, meta, and body via Jinja2 -5. Converts body Markdown → HTML, replaces `[scenario:…]` markers with rendered cards -6. Writes `data/content/_build/{slug}.html` -7. Inserts the `articles` row (status `'draft'` until publish date passes) - -### Step 7 — Review and publish - -Articles with a `published_at` in the future are invisible to the catch-all route. Check a few via Admin → Articles → (article) → Preview before the first date arrives. If anything looks wrong, fix the template and use Rebuild (see below) — no need to delete and re-generate. - ---- - -## Rebuilding an article - -If you edit a template body or fix an error in a data row: - -1. Admin → Articles → (article) → Rebuild -2. The system re-renders the body template with the current data, regenerates the scenario card HTML, and overwrites the `_build/{slug}.html` file -3. The `articles` row metadata (title, meta description, URL) is also updated +Any DuckDB column whose name matches a key in `DEFAULTS` (e.g. `electricity`, `ratePeak`, `dblCourts`) is automatically used as a calculator override. Name your serving model columns accordingly. --- ## Gotchas -- **Reserved prefixes** — if a new blueprint is added at a path that matches a content article's URL, the blueprint wins. Keep all content under `/markets/` or add a new reserved prefix. -- **Slug uniqueness** — article slugs drive the build filenames. The slug is derived from the URL path; collisions are caught as a DB UNIQUE constraint violation during generation. -- **Future publish dates** — articles with `published_at` in the future are invisible to the catch-all route and the markets hub. They still exist in the DB and can be previewed via admin. -- **FTS5 sync** — the FTS5 table is populated by triggers. If you manually insert into `articles` without going through the admin, run `INSERT INTO articles_fts(articles_fts) VALUES('rebuild')` to resync. -- **Scenario slugs in templates** — use `{{ scenario_slug }}` (injected at generation time) rather than hardcoding a slug in the body template. Hardcoded slugs break if you regenerate with different data. +- **Reserved prefixes** — content URLs must not collide with blueprint routes. The `is_reserved_path()` check rejects them during generation. +- **Slug uniqueness** — article slugs include template slug + language + natural key. Collisions are caught as DB UNIQUE constraint violations. +- **Future publish dates** — articles with `published_at` in the future are invisible to the catch-all route and markets hub. They exist in the DB and can be previewed via admin. +- **FTS5 sync** — triggers keep FTS in sync. If you manually insert into `articles`, run `INSERT INTO articles_fts(articles_fts) VALUES('rebuild')`. +- **Template edits** — editing a `.md.jinja` file in git doesn't automatically update existing articles. Use "Regenerate" in admin after deploying template changes. +- **DuckDB read-only** — all DuckDB access uses `read_only=True`. No write risk. +- **Table name validation** — `data_table` is validated against `^[a-z_][a-z0-9_.]*$` to prevent SQL injection.