docs: rewrite CMS.md for SSG architecture
Replace the old CSV-upload CMS documentation with the new SSG system: git templates, DuckDB data sources, generation pipeline, SEO pipeline (hreflang, JSON-LD, canonical, OG), admin routes, and step-by-step guide for adding new pSEO ideas. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
637
docs/CMS.md
637
docs/CMS.md
@@ -1,181 +1,294 @@
|
|||||||
# CMS & Programmatic SEO
|
# CMS & Programmatic SEO
|
||||||
|
|
||||||
How the content management system works: template definitions, bulk article generation, scenario embedding, the build pipeline, and how content is discovered and served.
|
How the content management system works: git-based template definitions, DuckDB data sources, bulk article generation, scenario embedding, SEO pipeline, and how content is served.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The CMS is a **template-driven, programmatic content system**. The core idea:
|
The CMS is an **SSG-inspired programmatic content system**. The core idea:
|
||||||
|
|
||||||
1. Define a **template** — URL pattern, title pattern, meta description pattern, and a Jinja2 body
|
1. **Templates live in git** — `.md.jinja` files with YAML frontmatter define how articles look
|
||||||
2. Upload **data rows** (CSV or single entry via admin UI) that fill the template variables
|
2. **Data lives in DuckDB** — one serving table per pSEO idea provides what articles say
|
||||||
3. **Generate** — the system runs the financial calculator, writes HTML to disk, and inserts `articles` rows
|
3. **Generation** renders templates with data, bakes in SEO metadata, writes HTML to disk
|
||||||
4. Articles are **served** by a catch-all route that looks up the URL path in the database
|
4. **SQLite** stores routing state only — `articles` and `published_scenarios` tables
|
||||||
|
5. Articles are **served** by a catch-all route reading pre-built HTML from disk
|
||||||
|
|
||||||
This is designed for scale: one template definition can produce hundreds of SEO-targeted articles.
|
```
|
||||||
|
Git repo (content shape) DuckDB (content data) SQLite (routing/state)
|
||||||
|
──────────────────────── ───────────────────── ──────────────────────
|
||||||
|
content/templates/ serving.pseo_city_costs_de articles
|
||||||
|
city-cost-de.md.jinja → city_slug, population, → url_path, slug
|
||||||
|
market-compare.md.jinja venue_count, rates... → status, published_at
|
||||||
|
→ template_slug
|
||||||
|
serving.pseo_market_compare
|
||||||
|
"How articles look" "What articles say" published_scenarios
|
||||||
|
→ slug, state_json
|
||||||
|
→ calc_json
|
||||||
|
"Where things live"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Template file format
|
||||||
|
|
||||||
|
Templates are `.md.jinja` files in `web/src/padelnomics/content/templates/`. Each file has YAML frontmatter delimited by `---`, followed by a Jinja2 Markdown body.
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
---
|
||||||
|
name: "DE City Padel Costs"
|
||||||
|
slug: city-cost-de
|
||||||
|
content_type: calculator
|
||||||
|
data_table: serving.pseo_city_costs_de
|
||||||
|
natural_key: city_slug
|
||||||
|
languages: [de, en]
|
||||||
|
url_pattern: "/markets/{{ country_name_en | lower | slugify }}/{{ city_slug }}"
|
||||||
|
title_pattern: "Padel in {{ city_name }} — Market Analysis & Costs"
|
||||||
|
meta_description_pattern: "How much does it cost to build a padel center in {{ city_name }}? {{ padel_venue_count }} venues, pricing data & financial model."
|
||||||
|
schema_type: [Article, FAQPage]
|
||||||
|
priority_column: population
|
||||||
|
---
|
||||||
|
# Padel in {{ city_name }}
|
||||||
|
|
||||||
|
{{ city_name }} has {{ padel_venue_count }} padel venues...
|
||||||
|
|
||||||
|
[scenario:{{ scenario_slug }}:capex]
|
||||||
|
|
||||||
|
## FAQ
|
||||||
|
|
||||||
|
**How much does a padel court cost in {{ city_name }}?**
|
||||||
|
Based on our financial model, a {{ courts_typical }}-court center requires...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontmatter fields
|
||||||
|
|
||||||
|
| Field | Required | Description |
|
||||||
|
|-------|----------|-------------|
|
||||||
|
| `name` | Yes | Human-readable template name (shown in admin) |
|
||||||
|
| `slug` | Yes | URL-safe identifier, also the filename stem |
|
||||||
|
| `content_type` | Yes | `calculator` (has financial scenario) or `editorial` |
|
||||||
|
| `data_table` | Yes | DuckDB serving table, schema-qualified (e.g. `serving.pseo_city_costs_de`) |
|
||||||
|
| `natural_key` | Yes | Column used as unique row identifier (e.g. `city_slug`) |
|
||||||
|
| `languages` | Yes | List of language codes to generate (e.g. `[de, en]`) |
|
||||||
|
| `url_pattern` | Yes | Jinja2 pattern for article URL path |
|
||||||
|
| `title_pattern` | Yes | Jinja2 pattern for `<title>` and `og:title` |
|
||||||
|
| `meta_description_pattern` | Yes | Jinja2 pattern for meta description (aim for 120-155 chars) |
|
||||||
|
| `schema_type` | No | JSON-LD type(s): `Article` (default), `FAQPage`, or list. See [Structured data](#structured-data) |
|
||||||
|
| `priority_column` | No | Column to sort by for publish order (highest value first) |
|
||||||
|
| `related_template_slugs` | No | Other template slugs for cross-linking (future) |
|
||||||
|
|
||||||
|
### Body template
|
||||||
|
|
||||||
|
Everything after the closing `---` is the body. It's Jinja2 Markdown with access to:
|
||||||
|
|
||||||
|
- **All columns** from the DuckDB row (e.g. `{{ city_name }}`, `{{ population }}`)
|
||||||
|
- **`language`** — current language code (`en`, `de`)
|
||||||
|
- **`scenario_slug`** — auto-derived slug for `[scenario:]` markers (calculator type only)
|
||||||
|
|
||||||
|
The `slugify` filter is available: `{{ country_name_en | slugify }}`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Database schema
|
## Database schema
|
||||||
|
|
||||||
```
|
After migration 0018, only two content tables remain in SQLite:
|
||||||
articles — Published articles (the SEO output)
|
|
||||||
article_templates — Template definitions (patterns + Jinja2 body)
|
|
||||||
template_data — One row per article to be generated; links template → article
|
|
||||||
published_scenarios — Calculator results embedded in article bodies
|
|
||||||
articles_fts — FTS5 virtual table for full-text search over articles
|
|
||||||
```
|
|
||||||
|
|
||||||
### articles
|
### articles
|
||||||
|
|
||||||
| Column | Type | Notes |
|
| Column | Type | Notes |
|
||||||
|--------|------|-------|
|
|--------|------|-------|
|
||||||
| `id` | INTEGER PK | |
|
| `url_path` | TEXT UNIQUE | e.g. `/en/markets/germany/berlin` |
|
||||||
| `url_path` | TEXT UNIQUE | Matches `request.path`, e.g. `/markets/de/berlin` |
|
| `slug` | TEXT UNIQUE | e.g. `city-cost-de-en-berlin` |
|
||||||
| `slug` | TEXT UNIQUE | Used for the build HTML filename |
|
| `title` | TEXT | Rendered from `title_pattern` |
|
||||||
| `title` | TEXT | |
|
| `meta_description` | TEXT | Rendered from `meta_description_pattern` |
|
||||||
| `meta_description` | TEXT | |
|
| `country` | TEXT | For markets hub grouping |
|
||||||
| `country` | TEXT | Used for grouping in the markets hub |
|
| `region` | TEXT | For markets hub grouping |
|
||||||
| `region` | TEXT | Used for grouping in the markets hub |
|
|
||||||
| `og_image_url` | TEXT | |
|
|
||||||
| `status` | TEXT | `'published'` or `'draft'` |
|
| `status` | TEXT | `'published'` or `'draft'` |
|
||||||
| `published_at` | DATETIME | Future dates suppress the article until then |
|
| `published_at` | DATETIME | Future dates suppress the article |
|
||||||
| `template_data_id` | INTEGER FK | Back-reference to `template_data.id` |
|
| `template_slug` | TEXT | Links back to git template for regeneration |
|
||||||
|
| `language` | TEXT | Language code (`en`, `de`) for hreflang |
|
||||||
|
| `date_modified` | TEXT | ISO timestamp, updated on regeneration |
|
||||||
|
| `seo_head` | TEXT | Pre-computed HTML: canonical, hreflang, JSON-LD, OG tags |
|
||||||
|
|
||||||
FTS5 virtual table `articles_fts` is kept in sync with `title`, `meta_description`, `country`, `region`.
|
FTS5 virtual table `articles_fts` syncs `title`, `meta_description`, `country`, `region` via triggers.
|
||||||
|
|
||||||
### article_templates
|
|
||||||
|
|
||||||
| Column | Type | Notes |
|
|
||||||
|--------|------|-------|
|
|
||||||
| `id` | INTEGER PK | |
|
|
||||||
| `name` | TEXT | Human label shown in admin |
|
|
||||||
| `slug` | TEXT UNIQUE | |
|
|
||||||
| `content_type` | TEXT | e.g. `'market'` |
|
|
||||||
| `input_schema` | JSON | Array of field definitions; drives the admin form |
|
|
||||||
| `url_pattern` | TEXT | Jinja2, e.g. `/markets/{{ country \| lower }}/{{ city_slug }}` |
|
|
||||||
| `title_pattern` | TEXT | Jinja2, e.g. `{{ city }} Padel Court Economics` |
|
|
||||||
| `meta_description_pattern` | TEXT | Jinja2 |
|
|
||||||
| `body_template` | TEXT | Jinja2 Markdown with optional `[scenario:slug]` markers |
|
|
||||||
|
|
||||||
### template_data
|
|
||||||
|
|
||||||
| Column | Type | Notes |
|
|
||||||
|--------|------|-------|
|
|
||||||
| `id` | INTEGER PK | |
|
|
||||||
| `template_id` | INTEGER FK | → `article_templates.id` |
|
|
||||||
| `data_json` | JSON | The variables that fill the template |
|
|
||||||
| `scenario_id` | INTEGER FK | → `published_scenarios.id` (set after generation) |
|
|
||||||
| `article_id` | INTEGER FK | → `articles.id` (set after generation) |
|
|
||||||
| `created_at` | DATETIME | |
|
|
||||||
| `updated_at` | DATETIME | |
|
|
||||||
|
|
||||||
### published_scenarios
|
### published_scenarios
|
||||||
|
|
||||||
Pre-calculated financial scenarios embedded in articles.
|
|
||||||
|
|
||||||
| Column | Type | Notes |
|
| Column | Type | Notes |
|
||||||
|--------|------|-------|
|
|--------|------|-------|
|
||||||
| `id` | INTEGER PK | |
|
|
||||||
| `slug` | TEXT UNIQUE | Referenced in `[scenario:slug]` markers |
|
| `slug` | TEXT UNIQUE | Referenced in `[scenario:slug]` markers |
|
||||||
| `title` | TEXT | |
|
| `title` | TEXT | City name |
|
||||||
| `subtitle` | TEXT | |
|
|
||||||
| `location` | TEXT | City name |
|
| `location` | TEXT | City name |
|
||||||
| `country` | TEXT | |
|
| `country` | TEXT | |
|
||||||
| `venue_type` | TEXT | |
|
| `venue_type` | TEXT | `indoor` / `outdoor` |
|
||||||
| `ownership` | TEXT | |
|
| `ownership` | TEXT | `rent` / `own` |
|
||||||
| `court_config` | TEXT | |
|
| `court_config` | TEXT | e.g. `4 double + 2 single` |
|
||||||
| `state_json` | JSON | Calculator input state |
|
| `state_json` | JSON | Calculator input state |
|
||||||
| `calc_json` | JSON | Calculator output (all sections) |
|
| `calc_json` | JSON | Calculator output (all sections) |
|
||||||
| `template_data_id` | INTEGER FK | |
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Programmatic SEO pipeline
|
## Generation pipeline
|
||||||
|
|
||||||
### Step 1 — Define a template
|
### Full flow
|
||||||
|
|
||||||
In admin → Templates → New Template.
|
```
|
||||||
|
1. Create serving model in SQLMesh
|
||||||
|
models/serving/pseo_city_costs_de.sql → sqlmesh plan prod → analytics.duckdb
|
||||||
|
|
||||||
**URL pattern** uses Jinja2 and must produce a unique, clean path:
|
2. Create template file in git
|
||||||
|
content/templates/city-cost-de.md.jinja
|
||||||
|
|
||||||
```jinja2
|
3. Preview locally
|
||||||
/markets/{{ country | lower }}/{{ city_slug }}
|
Admin → Templates → pick template → pick city → preview rendered article
|
||||||
|
|
||||||
|
4. Commit + push + deploy
|
||||||
|
|
||||||
|
5. Admin → Templates → "DE City Padel Costs"
|
||||||
|
sees data_table, row count from DuckDB, generated count
|
||||||
|
|
||||||
|
6. Click "Generate"
|
||||||
|
start_date, articles_per_day → for each DuckDB row × language:
|
||||||
|
• render URL, title, meta patterns
|
||||||
|
• create/update published_scenario (calculator type)
|
||||||
|
• render body.md.jinja → Markdown → HTML
|
||||||
|
• bake [scenario:] markers into HTML
|
||||||
|
• inject SEO head (canonical, hreflang, JSON-LD, OG)
|
||||||
|
• write HTML to data/content/_build/{lang}/{slug}.html
|
||||||
|
• upsert article row in SQLite
|
||||||
|
• stagger published_at dates
|
||||||
|
|
||||||
|
7. Articles live at /en/markets/germany/berlin
|
||||||
|
with links to /planner?scenario=city-cost-de-berlin
|
||||||
|
|
||||||
|
8. Data refresh → "Regenerate" in admin
|
||||||
|
reads fresh DuckDB data, updates HTML + scenarios in-place
|
||||||
```
|
```
|
||||||
|
|
||||||
**Title and meta description patterns** are also Jinja2:
|
### What happens per row
|
||||||
|
|
||||||
```jinja2
|
For each DuckDB row × each language in `config.languages`:
|
||||||
{{ city }} Padel Court Business Plan — {{ court_count }}-Court Indoor Venue
|
|
||||||
Annual revenue and ROI analysis for a {{ court_count }}-court padel venue in {{ city }}, {{ country }}.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Body template** is Jinja2 Markdown. Access any field from `data_json`, plus the generated scenario's slug via `scenario_slug`. Use `[scenario:slug]` markers to embed calculator sections:
|
1. Render `url_pattern`, `title_pattern`, `meta_description_pattern` with row data
|
||||||
|
2. If `content_type == "calculator"`: extract calc fields → `validate_state()` → `calc()` → upsert `published_scenarios`
|
||||||
|
3. Render body template with row data + `scenario_slug`
|
||||||
|
4. Convert Markdown → HTML via `mistune`
|
||||||
|
5. Replace `[scenario:slug:section]` markers with rendered HTML cards
|
||||||
|
6. Build SEO head: canonical, hreflang links, JSON-LD, OG tags
|
||||||
|
7. Write HTML to `data/content/_build/{lang}/{article_slug}.html`
|
||||||
|
8. Upsert `articles` row in SQLite
|
||||||
|
9. Stagger `published_at` based on `articles_per_day`
|
||||||
|
|
||||||
```markdown
|
### Scenario markers
|
||||||
## {{ city }} Market Overview
|
|
||||||
|
|
||||||
A {{ court_count }}-court indoor venue in {{ city }} has the following economics:
|
Use `[scenario:slug]` or `[scenario:slug:section]` in the body template:
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}]
|
|
||||||
|
|
||||||
### Capital Expenditure
|
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}:capex]
|
|
||||||
|
|
||||||
### Annual Cash Flow
|
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}:cashflow]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Scenario section options** for the `[scenario:slug:section]` syntax:
|
|
||||||
|
|
||||||
| Section | Shows |
|
| Section | Shows |
|
||||||
|---------|-------|
|
|---------|-------|
|
||||||
| _(none)_ | Default summary card |
|
| _(none)_ | Default summary card with CTA link to planner |
|
||||||
| `capex` | Capital expenditure breakdown |
|
| `capex` | Capital expenditure breakdown |
|
||||||
| `operating` | Operating cost breakdown |
|
| `operating` | Operating cost breakdown |
|
||||||
| `cashflow` | Annual cash flow |
|
| `cashflow` | Annual cash flow projection |
|
||||||
| `returns` | ROI / returns summary |
|
| `returns` | ROI / returns summary |
|
||||||
| `full` | All sections |
|
| `full` | All sections combined |
|
||||||
|
|
||||||
### Step 2 — Upload data
|
Always use `{{ scenario_slug }}` — never hardcode a slug.
|
||||||
|
|
||||||
Admin → Templates → (template) → Data → Upload CSV, or add single rows via the form.
|
### Scenario slug derivation
|
||||||
|
|
||||||
The CSV columns must match the template's `input_schema`. Each row becomes one `template_data` record and will generate one article.
|
```
|
||||||
|
scenario_slug = template_slug + "-" + natural_key_value
|
||||||
### Step 3 — Generate
|
|
||||||
|
|
||||||
Admin → Templates → (template) → Generate.
|
|
||||||
|
|
||||||
**What happens for each data row:**
|
|
||||||
|
|
||||||
1. Merge row data with calculator DEFAULTS
|
|
||||||
2. Run financial calculator → produce `state_json` + `calc_json`
|
|
||||||
3. Create a `published_scenarios` record with an auto-derived slug
|
|
||||||
4. Render Jinja2 patterns (URL, title, meta description, body) using row data + scenario slug
|
|
||||||
5. Convert body Markdown → HTML via `mistune`
|
|
||||||
6. Replace `[scenario:slug(:section)?]` markers with rendered HTML partials
|
|
||||||
7. Write final HTML to `data/content/_build/{slug}.html`
|
|
||||||
8. Back up Markdown source to `data/content/articles/{slug}.md`
|
|
||||||
9. Create an `articles` record
|
|
||||||
10. Update `template_data` row with `scenario_id` and `article_id`
|
|
||||||
|
|
||||||
**Staggered publishing:** the `articles_per_day` parameter spreads `published_at` dates so articles drip out over time rather than all appearing at once.
|
|
||||||
|
|
||||||
### Step 4 — Serve
|
|
||||||
|
|
||||||
The `content` blueprint registers a catch-all route **last**, so it only fires if no other blueprint matched:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@bp.route("/<path:url_path>")
|
|
||||||
async def article_page(url_path: str):
|
|
||||||
...
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Reserved prefixes are short-circuited to avoid conflicts:
|
Example: template `city-cost-de` + row `city_slug=berlin` → `city-cost-de-berlin`
|
||||||
|
|
||||||
|
### Article slug derivation
|
||||||
|
|
||||||
|
```
|
||||||
|
article_slug = template_slug + "-" + language + "-" + natural_key_value
|
||||||
|
```
|
||||||
|
|
||||||
|
Example: `city-cost-de-en-berlin`, `city-cost-de-de-berlin`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## SEO pipeline
|
||||||
|
|
||||||
|
All SEO metadata is baked into the `seo_head` column at generation time. No runtime computation.
|
||||||
|
|
||||||
|
### URL structure
|
||||||
|
|
||||||
|
```
|
||||||
|
/{lang}/markets/{country_name}/{city_slug}
|
||||||
|
```
|
||||||
|
|
||||||
|
- Language prefix for hreflang (`/de/...`, `/en/...`)
|
||||||
|
- Full country name in path (`/germany/`, not `/de/`) — avoids confusion with lang prefix
|
||||||
|
- `slugify` filter converts to lowercase hyphenated form
|
||||||
|
|
||||||
|
### Hreflang
|
||||||
|
|
||||||
|
For each article, links to all language variants + `x-default` (points to English):
|
||||||
|
|
||||||
|
```html
|
||||||
|
<link rel="alternate" hreflang="de" href="https://padelnomics.io/de/markets/germany/berlin" />
|
||||||
|
<link rel="alternate" hreflang="en" href="https://padelnomics.io/en/markets/germany/berlin" />
|
||||||
|
<link rel="alternate" hreflang="x-default" href="https://padelnomics.io/en/markets/germany/berlin" />
|
||||||
|
```
|
||||||
|
|
||||||
|
### Canonical URLs
|
||||||
|
|
||||||
|
Self-referencing canonical on every article:
|
||||||
|
|
||||||
|
```html
|
||||||
|
<link rel="canonical" href="https://padelnomics.io/en/markets/germany/berlin" />
|
||||||
|
```
|
||||||
|
|
||||||
|
### Structured data
|
||||||
|
|
||||||
|
JSON-LD `<script>` blocks injected in `seo_head`. The `schema_type` frontmatter controls which types are generated:
|
||||||
|
|
||||||
|
| Schema | When | Data source |
|
||||||
|
|--------|------|-------------|
|
||||||
|
| `BreadcrumbList` | Always | Auto-generated from URL segments |
|
||||||
|
| `Article` | Default (or `schema_type: Article`) | Template patterns + row data |
|
||||||
|
| `FAQPage` | `schema_type: FAQPage` | Parsed from `## FAQ` section — bold questions + answer paragraphs |
|
||||||
|
|
||||||
|
**`Article`** includes: `headline`, `datePublished`, `dateModified`, `author` (Padelnomics), `publisher`, `description`, `inLanguage`.
|
||||||
|
|
||||||
|
**`FAQPage`** wraps bold-question/answer pairs from the `## FAQ` heading into `Question`/`acceptedAnswer` pairs.
|
||||||
|
|
||||||
|
`dateModified` updates on every regeneration (freshness signal for Google).
|
||||||
|
|
||||||
|
### Open Graph
|
||||||
|
|
||||||
|
```html
|
||||||
|
<meta property="og:title" content="..." />
|
||||||
|
<meta property="og:description" content="..." />
|
||||||
|
<meta property="og:url" content="..." />
|
||||||
|
<meta property="og:type" content="article" />
|
||||||
|
```
|
||||||
|
|
||||||
|
### Drip publishing
|
||||||
|
|
||||||
|
- `articles_per_day` parameter (default 3) staggers `published_at` dates
|
||||||
|
- `priority_column` sorts rows so high-value cities publish first
|
||||||
|
- Articles with future `published_at` are invisible to the catch-all route
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Article serving
|
||||||
|
|
||||||
|
The `content` blueprint registers a catch-all route:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@bp.route("/<lang>/<path:url_path>")
|
||||||
|
async def article_page(lang, url_path):
|
||||||
|
```
|
||||||
|
|
||||||
|
Reserved prefixes are short-circuited:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
RESERVED_PREFIXES = (
|
RESERVED_PREFIXES = (
|
||||||
@@ -185,261 +298,133 @@ RESERVED_PREFIXES = (
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
Lookup query:
|
Lookup: find the article by `url_path`, check `status = 'published'` and `published_at <= now`, then read the pre-built HTML from `data/content/_build/{lang}/{slug}.html`.
|
||||||
|
|
||||||
```sql
|
|
||||||
SELECT * FROM articles
|
|
||||||
WHERE url_path = ? AND status = 'published' AND published_at <= datetime('now')
|
|
||||||
```
|
|
||||||
|
|
||||||
If found, the pre-built HTML is read from `data/content/_build/{slug}.html` and injected into `article_detail.html`. No runtime Markdown rendering.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## SEO meta tags
|
|
||||||
|
|
||||||
`article_detail.html` overrides the base template's defaults with article-specific values:
|
|
||||||
|
|
||||||
```html
|
|
||||||
<title>{{ article.title }} | Padelnomics</title>
|
|
||||||
<meta name="description" content="{{ article.meta_description }}">
|
|
||||||
<link rel="canonical" href="{{ config.BASE_URL }}{{ request.path }}">
|
|
||||||
|
|
||||||
<meta property="og:title" content="{{ article.title }}">
|
|
||||||
<meta property="og:description" content="{{ article.meta_description }}">
|
|
||||||
<meta property="og:image" content="{{ article.og_image_url }}">
|
|
||||||
<meta property="og:url" content="{{ config.BASE_URL }}{{ request.path }}">
|
|
||||||
```
|
|
||||||
|
|
||||||
**JSON-LD structured data** is also injected per article:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"@context": "https://schema.org",
|
|
||||||
"@type": "Article",
|
|
||||||
"headline": "...",
|
|
||||||
"datePublished": "2025-01-15",
|
|
||||||
"author": { "@type": "Organization", "name": "Padelnomics" },
|
|
||||||
"publisher": { "@type": "Organization", "name": "Padelnomics" }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Markets hub
|
## Markets hub
|
||||||
|
|
||||||
`/markets` is the discovery index for all published articles.
|
`/{lang}/markets` is the discovery index for all published articles.
|
||||||
|
|
||||||
- Groups articles by `country` and `region`
|
- Groups articles by `country` and `region`
|
||||||
- Full-text search via FTS5 (`articles_fts`) across `title`, `meta_description`, `country`, `region`
|
- Full-text search via FTS5 across `title`, `meta_description`, `country`, `region`
|
||||||
- HTMX partial at `/markets/results` handles live filtering (no page reload)
|
- HTMX partial at `/{lang}/markets/results` handles live filtering
|
||||||
|
|
||||||
FTS5 query (simplified):
|
---
|
||||||
|
|
||||||
```sql
|
## Admin interface
|
||||||
SELECT a.* FROM articles a
|
|
||||||
JOIN articles_fts ON articles_fts.rowid = a.id
|
|
||||||
WHERE articles_fts MATCH ?
|
|
||||||
AND a.status = 'published'
|
|
||||||
AND a.published_at <= datetime('now')
|
|
||||||
ORDER BY a.published_at DESC
|
|
||||||
LIMIT 100
|
|
||||||
```
|
|
||||||
|
|
||||||
Filter fields: `q` (full-text), `country`, `region`.
|
Templates are **read-only** in the admin UI — edit them in git, preview and generate in admin.
|
||||||
|
|
||||||
|
### Routes
|
||||||
|
|
||||||
|
| Route | Method | Purpose |
|
||||||
|
|-------|--------|---------|
|
||||||
|
| `/admin/templates` | GET | List templates scanned from disk |
|
||||||
|
| `/admin/templates/<slug>` | GET | Template detail: config, DuckDB columns, sample data |
|
||||||
|
| `/admin/templates/<slug>/preview/<row_key>` | GET | Preview one article rendered in-memory |
|
||||||
|
| `/admin/templates/<slug>/generate` | GET/POST | Generate form + action |
|
||||||
|
| `/admin/templates/<slug>/regenerate` | POST | Re-generate all articles with fresh DuckDB data |
|
||||||
|
|
||||||
|
Scenario and article management routes are unchanged (create, edit, delete, publish toggle, rebuild).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Directory structure
|
## Directory structure
|
||||||
|
|
||||||
```
|
```
|
||||||
src/padelnomics/
|
web/src/padelnomics/
|
||||||
├── content/
|
├── content/
|
||||||
│ ├── routes.py # Catch-all article serving + markets hub
|
│ ├── __init__.py # Core engine: discover, load, generate, preview, SEO
|
||||||
│ └── templates/
|
│ ├── routes.py # Catch-all serving, markets hub, scenario baking
|
||||||
│ ├── article_detail.html # SEO meta tags, JSON-LD, article body injection
|
│ └── templates/ # Git-based .md.jinja template files
|
||||||
│ ├── markets.html # Discovery index
|
│ └── city-cost-de.md.jinja
|
||||||
│ └── partials/
|
|
||||||
│ ├── market_results.html # HTMX search result rows
|
|
||||||
│ └── scenario_*.html # Scenario cards (capex, cashflow, etc.)
|
|
||||||
│
|
│
|
||||||
├── admin/
|
├── admin/
|
||||||
│ ├── routes.py # Full CMS admin (templates, data, generate, articles)
|
│ ├── routes.py # Read-only template views + generate/regenerate
|
||||||
│ └── templates/admin/
|
│ └── templates/admin/
|
||||||
│ ├── templates.html
|
│ ├── templates.html # Template list (scanned from disk)
|
||||||
│ ├── template_form.html
|
│ ├── template_detail.html # Config view, columns, sample data
|
||||||
│ ├── template_data.html
|
│ ├── template_preview.html # Article preview
|
||||||
│ ├── generate_form.html
|
│ ├── generate_form.html # Schedule form
|
||||||
│ ├── articles.html
|
│ ├── articles.html
|
||||||
│ └── article_form.html
|
│ └── article_form.html
|
||||||
│
|
│
|
||||||
└── data/
|
└── migrations/versions/
|
||||||
└── content/
|
└── 0018_pseo_cms_refactor.py # Drops old tables, recreates articles + scenarios
|
||||||
|
|
||||||
|
data/content/
|
||||||
├── _build/ # Generated HTML (served at runtime)
|
├── _build/ # Generated HTML (served at runtime)
|
||||||
|
│ ├── en/
|
||||||
|
│ │ └── {slug}.html
|
||||||
|
│ └── de/
|
||||||
│ └── {slug}.html
|
│ └── {slug}.html
|
||||||
└── articles/ # Markdown source backup
|
└── articles/ # Markdown source backup (manual articles)
|
||||||
└── {slug}.md
|
└── {slug}.md
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Adding a new content type
|
## Adding a new pSEO idea
|
||||||
|
|
||||||
### Step 1 — Define the template in admin
|
### Step 1 — Create the DuckDB serving model
|
||||||
|
|
||||||
Go to **Admin → Templates → New Template** and fill in:
|
Write a SQLMesh model at `transform/sqlmesh_padelnomics/models/serving/pseo_your_idea.sql` that produces one row per article to generate. Must include a `natural_key` column (e.g. `city_slug`).
|
||||||
|
|
||||||
| Field | Notes |
|
```bash
|
||||||
|-------|-------|
|
uv run sqlmesh -p transform/sqlmesh_padelnomics plan prod
|
||||||
| Name | Human label, e.g. `"Indoor Market Report"` |
|
|
||||||
| Slug | Auto-generated; used to derive scenario slugs (cannot be changed after creation) |
|
|
||||||
| Content type | `"calculator"` (the only current option) |
|
|
||||||
| Input schema | JSON — see below |
|
|
||||||
| URL pattern | Jinja2 — see below |
|
|
||||||
| Title pattern | Jinja2 |
|
|
||||||
| Meta description pattern | Jinja2 (optional) |
|
|
||||||
| Body template | Jinja2 Markdown |
|
|
||||||
|
|
||||||
### Step 2 — Write the input_schema
|
|
||||||
|
|
||||||
`input_schema` is a JSON array of field definitions. Each field becomes a column in the CSV and an input in the single-row add form.
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{ "name": "city", "label": "City", "field_type": "text", "required": true },
|
|
||||||
{ "name": "city_slug", "label": "City slug", "field_type": "text", "required": true },
|
|
||||||
{ "name": "country", "label": "Country (ISO 2)", "field_type": "text", "required": true },
|
|
||||||
{ "name": "region", "label": "Region", "field_type": "text", "required": false },
|
|
||||||
{ "name": "court_count", "label": "Number of courts", "field_type": "number", "required": false },
|
|
||||||
{ "name": "ratePeak", "label": "Peak rate (€/hr)", "field_type": "float", "required": false }
|
|
||||||
]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Field keys:**
|
### Step 2 — Create the template file
|
||||||
|
|
||||||
| Key | Required | Values | Effect |
|
Create `web/src/padelnomics/content/templates/your-idea.md.jinja`:
|
||||||
|-----|----------|--------|--------|
|
|
||||||
| `name` | yes | any string | Variable name in Jinja2 patterns; CSV column header |
|
|
||||||
| `label` | yes | any string | Human label in the admin form |
|
|
||||||
| `field_type` | yes | `"text"`, `"number"`, `"float"` | Controls input type and coercion during generation |
|
|
||||||
| `required` | no | `true` / `false` | Whether the admin form enforces a value |
|
|
||||||
|
|
||||||
**`city_slug` is special** — the system looks for a field named exactly `city_slug` to derive the scenario slug. If it's absent, the `template_data` row ID is used as a fallback.
|
|
||||||
|
|
||||||
**Calculator field names** — any field whose name matches a key in the calculator `DEFAULTS` dict (e.g. `ratePeak`, `rateOffPeak`, `dblCourts`, `loanPct`) is automatically passed to the financial model during generation. You don't need to do anything extra; just name the field correctly and the generator merges it into the calc state.
|
|
||||||
|
|
||||||
### Step 3 — Write the Jinja2 patterns
|
|
||||||
|
|
||||||
All patterns are rendered with the same context: every field from the data row, plus `scenario_slug` (injected by the generator).
|
|
||||||
|
|
||||||
**URL pattern** — must be unique across all articles and must not start with a reserved prefix:
|
|
||||||
|
|
||||||
```jinja2
|
|
||||||
/markets/{{ country | lower }}/{{ city_slug }}
|
|
||||||
```
|
|
||||||
|
|
||||||
The article slug is derived by stripping the leading `/` and replacing remaining `/` with `-`, so `/markets/de/berlin` → slug `markets-de-berlin`.
|
|
||||||
|
|
||||||
**Title pattern:**
|
|
||||||
|
|
||||||
```jinja2
|
|
||||||
{{ city }} Padel Court Business Plan — {{ court_count }}-Court {{ venue_type | title }} Venue
|
|
||||||
```
|
|
||||||
|
|
||||||
**Meta description pattern** (aim for ≤ 160 characters):
|
|
||||||
|
|
||||||
```jinja2
|
|
||||||
Revenue and ROI analysis for a {{ court_count }}-court padel venue in {{ city }}, {{ country }}. Includes CapEx, operating costs, and cash flow projections.
|
|
||||||
```
|
|
||||||
|
|
||||||
**Body template** — Jinja2 Markdown. Use any data row field, plus `scenario_slug` to embed calculator results:
|
|
||||||
|
|
||||||
```markdown
|
```markdown
|
||||||
## {{ city }} Market Overview
|
---
|
||||||
|
name: "Your Idea Name"
|
||||||
|
slug: your-idea
|
||||||
|
content_type: calculator
|
||||||
|
data_table: serving.pseo_your_idea
|
||||||
|
natural_key: city_slug
|
||||||
|
languages: [en, de]
|
||||||
|
url_pattern: "/markets/{{ country_name_en | lower | slugify }}/{{ city_slug }}"
|
||||||
|
title_pattern: "Your Title for {{ city_name }}"
|
||||||
|
meta_description_pattern: "Your description for {{ city_name }}. Max 155 chars."
|
||||||
|
schema_type: [Article, FAQPage]
|
||||||
|
priority_column: population
|
||||||
|
---
|
||||||
|
# Your Title for {{ city_name }}
|
||||||
|
|
||||||
A {{ court_count }}-court indoor venue in {{ city }} ({{ country }}) projects the following economics:
|
Article body with {{ template_variables }}...
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}]
|
|
||||||
|
|
||||||
### Capital Expenditure
|
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}:capex]
|
[scenario:{{ scenario_slug }}:capex]
|
||||||
|
|
||||||
### Annual Cash Flow
|
## FAQ
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}:cashflow]
|
**Your question about {{ city_name }}?**
|
||||||
|
Your answer with data from the row.
|
||||||
### Returns
|
|
||||||
|
|
||||||
[scenario:{{ scenario_slug }}:returns]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 4 — How scenario_slug is derived
|
### Step 3 — Preview and iterate
|
||||||
|
|
||||||
```
|
Start the dev server, go to Admin → Templates → your template → pick a row → Preview. Iterate on the body until it looks right.
|
||||||
scenario_slug = template_slug + "-" + city_slug
|
|
||||||
```
|
|
||||||
|
|
||||||
Example: template slug `indoor-market` + city_slug `berlin` → `indoor-market-berlin`.
|
### Step 4 — Commit, deploy, generate
|
||||||
|
|
||||||
This slug identifies the `published_scenarios` row. It's what you reference in `[scenario:slug]` markers. Always use `{{ scenario_slug }}` in the body template — never hardcode a slug.
|
Push the template file. In production admin, click Generate with a start date and articles/day cadence.
|
||||||
|
|
||||||
### Step 5 — Upload data
|
### Calculator field overrides
|
||||||
|
|
||||||
**CSV upload** (Admin → Templates → (template) → Data → Upload CSV):
|
Any DuckDB column whose name matches a key in `DEFAULTS` (e.g. `electricity`, `ratePeak`, `dblCourts`) is automatically used as a calculator override. Name your serving model columns accordingly.
|
||||||
|
|
||||||
- Column headers must exactly match the `name` fields in `input_schema`
|
|
||||||
- One row per article to be generated
|
|
||||||
- Values are coerced to the declared `field_type` (`text` → string, `number` → int, `float` → float)
|
|
||||||
|
|
||||||
Example CSV:
|
|
||||||
|
|
||||||
```csv
|
|
||||||
city,city_slug,country,region,court_count,ratePeak,rateOffPeak
|
|
||||||
Berlin,berlin,DE,Brandenburg,6,55,35
|
|
||||||
Munich,munich,DE,Bavaria,8,60,40
|
|
||||||
Hamburg,hamburg,DE,Hamburg,4,50,30
|
|
||||||
```
|
|
||||||
|
|
||||||
**Single-row add** — use the form in Admin → Templates → (template) → Data → Add Row for one-off entries.
|
|
||||||
|
|
||||||
### Step 6 — Generate
|
|
||||||
|
|
||||||
Admin → Templates → (template) → Generate. Set:
|
|
||||||
|
|
||||||
- **Start date** — first article's `published_at`
|
|
||||||
- **Articles per day** — how many articles share each calendar day (stagger cadence)
|
|
||||||
|
|
||||||
The generator processes all data rows that don't yet have an `article_id` (i.e. not previously generated). For each row it:
|
|
||||||
|
|
||||||
1. Merges calculator fields from the row into `DEFAULTS`, validates and runs `calc(state)`
|
|
||||||
2. Creates a `published_scenarios` row
|
|
||||||
3. Injects `scenario_slug` into the render context
|
|
||||||
4. Renders URL, title, meta, and body via Jinja2
|
|
||||||
5. Converts body Markdown → HTML, replaces `[scenario:…]` markers with rendered cards
|
|
||||||
6. Writes `data/content/_build/{slug}.html`
|
|
||||||
7. Inserts the `articles` row (status `'draft'` until publish date passes)
|
|
||||||
|
|
||||||
### Step 7 — Review and publish
|
|
||||||
|
|
||||||
Articles with a `published_at` in the future are invisible to the catch-all route. Check a few via Admin → Articles → (article) → Preview before the first date arrives. If anything looks wrong, fix the template and use Rebuild (see below) — no need to delete and re-generate.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Rebuilding an article
|
|
||||||
|
|
||||||
If you edit a template body or fix an error in a data row:
|
|
||||||
|
|
||||||
1. Admin → Articles → (article) → Rebuild
|
|
||||||
2. The system re-renders the body template with the current data, regenerates the scenario card HTML, and overwrites the `_build/{slug}.html` file
|
|
||||||
3. The `articles` row metadata (title, meta description, URL) is also updated
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- **Reserved prefixes** — if a new blueprint is added at a path that matches a content article's URL, the blueprint wins. Keep all content under `/markets/` or add a new reserved prefix.
|
- **Reserved prefixes** — content URLs must not collide with blueprint routes. The `is_reserved_path()` check rejects them during generation.
|
||||||
- **Slug uniqueness** — article slugs drive the build filenames. The slug is derived from the URL path; collisions are caught as a DB UNIQUE constraint violation during generation.
|
- **Slug uniqueness** — article slugs include template slug + language + natural key. Collisions are caught as DB UNIQUE constraint violations.
|
||||||
- **Future publish dates** — articles with `published_at` in the future are invisible to the catch-all route and the markets hub. They still exist in the DB and can be previewed via admin.
|
- **Future publish dates** — articles with `published_at` in the future are invisible to the catch-all route and markets hub. They exist in the DB and can be previewed via admin.
|
||||||
- **FTS5 sync** — the FTS5 table is populated by triggers. If you manually insert into `articles` without going through the admin, run `INSERT INTO articles_fts(articles_fts) VALUES('rebuild')` to resync.
|
- **FTS5 sync** — triggers keep FTS in sync. If you manually insert into `articles`, run `INSERT INTO articles_fts(articles_fts) VALUES('rebuild')`.
|
||||||
- **Scenario slugs in templates** — use `{{ scenario_slug }}` (injected at generation time) rather than hardcoding a slug in the body template. Hardcoded slugs break if you regenerate with different data.
|
- **Template edits** — editing a `.md.jinja` file in git doesn't automatically update existing articles. Use "Regenerate" in admin after deploying template changes.
|
||||||
|
- **DuckDB read-only** — all DuckDB access uses `read_only=True`. No write risk.
|
||||||
|
- **Table name validation** — `data_table` is validated against `^[a-z_][a-z0-9_.]*$` to prevent SQL injection.
|
||||||
|
|||||||
Reference in New Issue
Block a user