Compare commits

...

5 Commits

Author SHA1 Message Date
Deeman
bed07974cb fix(pseo): error details collapse + UNIQUE constraint on slug
Some checks failed
CI / test (push) Failing after 32s
CI / tag (push) Has been skipped
# Conflicts:
#	CHANGELOG.md
2026-03-10 21:47:38 +01:00
Deeman
207fa18fda fix(pseo): error details collapse + UNIQUE constraint on slug
1. Stop HTMX polling on job rows that have an error set, so the
   <details> element stays open when clicked (was being replaced
   every 2s by the poll cycle).

2. Migration 0030: drop redundant single-column UNIQUE on
   articles.slug — the real uniqueness key is (url_path, language).
   The slug index is kept for lookups.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:33:38 +01:00
Deeman
8cc1cef780 merge: map country names + localised dropdown + avg/top score tooltip
Some checks failed
CI / test (push) Failing after 33s
CI / tag (push) Has been skipped
2026-03-10 17:22:53 +01:00
Deeman
236f0d1061 fix(markets): map country names, localised dropdown + avg/top score tooltip
- Expand dim_countries.sql CASE to cover 22 missing countries (PL, RO,
  CO, HU, ZA, KE, BR, CZ, QA, NZ, HR, LV, MT, CR, CY, PA, SV, DO,
  PE, VE, EE, ID) that fell through to bare ISO codes
- Add 19 missing entries to COUNTRY_LABELS (i18n.py) + both locale files
  (EN + DE dir_country_* keys) including IE which was in SQL but not i18n
- Localise map tooltips: routes.py injects country_name via
  get_country_name(), JS uses c.country_name instead of c.country_name_en
- Localise dropdown: apply country_name filter to option labels
- Show avg + top score in map tooltip with separate color dots and new
  map_score_avg / map_score_top i18n keys (EN: "Avg. Score" / "Top City",
  DE: "Ø Score" / "Top-Stadt")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:21:59 +01:00
Deeman
44617ea783 docs: document prod_query.py in CLAUDE.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:16:34 +01:00
10 changed files with 236 additions and 6 deletions

View File

@@ -74,6 +74,24 @@ DUCKDB_PATH=local.duckdb SERVING_DUCKDB_PATH=analytics.duckdb \
```
## Production queries
Use `scripts/prod_query.py` to query the production DuckDB over SSH. **Always prefer this over raw SSH commands** — it handles escaping, enforces read-only, and blocks mutation keywords.
```bash
# Query analytics.duckdb (serving tables — default)
uv run python scripts/prod_query.py "SELECT COUNT(*) FROM serving.location_profiles"
# Query lakehouse.duckdb (foundation/staging tables)
uv run python scripts/prod_query.py --db lakehouse "SELECT * FROM foundation.dim_countries LIMIT 5"
# JSON output
uv run python scripts/prod_query.py --json "SELECT COUNT(*) FROM serving.location_profiles"
# Limit rows (default 500)
uv run python scripts/prod_query.py --max-rows 1000 "SELECT ..."
```
## Architecture documentation
| Topic | File |

View File

@@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Fixed
- **pSEO error details collapse** — clicking "Error" on a job row expanded the details, but they collapsed after ~2s because HTMX polling replaced the `<tr>`. Jobs with errors now stop polling, keeping the `<details>` element stable.
- **UNIQUE constraint on article slug** — `ON CONFLICT(url_path, language)` upsert failed because a separate single-column `UNIQUE` on `slug` fired first. Migration 0030 drops the redundant `UNIQUE` from `slug` (keeps the index for lookups and the composite `UNIQUE(url_path, language)`).
- **Map country names** — 22 countries (PL, RO, CO, HU, ZA, KE, BR, CZ, QA, NZ, HR, LV, MT, CR, CY, PA, SV, DO, PE, VE, EE, ID) that appeared as bare ISO codes on the markets map and dropdown now show proper English/German names. Added country names to `dim_countries.sql`, `COUNTRY_LABELS` (i18n.py), and both locale files. Map tooltips and dropdown are now fully localised via `get_country_name()`.
- **Map score tooltip clarity** — tooltip now shows both "Avg. Score" (country average) and "Top City" (highest location score) with separate color dots, making clear the map bubble color represents the country average — not a cap.
### Added
- **Microsoft Clarity integration** — consent-gated heatmaps and session recordings (project ID via `CLARITY_PROJECT_ID` env var). Script only loads when the user has accepted functional cookies; bootstraps immediately on consent without requiring a reload. Privacy policy (EN + DE) updated with Clarity disclosure: data collection, sub-processor, cookies (`_clck`, `_clsk`), and international transfers.
- **IndexNow integration** — push-notify Bing, Yandex, Seznam, and Naver when articles are published/unpublished/edited or suppliers are created. Bulk operations batch all URLs into a single request. Skips silently in dev (no key configured). Serves key verification file at `/{key}.txt`.

View File

@@ -148,6 +148,28 @@ SELECT
WHEN 'AE' THEN 'UAE'
WHEN 'AU' THEN 'Australia'
WHEN 'IE' THEN 'Ireland'
WHEN 'PL' THEN 'Poland'
WHEN 'RO' THEN 'Romania'
WHEN 'CO' THEN 'Colombia'
WHEN 'HU' THEN 'Hungary'
WHEN 'ZA' THEN 'South Africa'
WHEN 'KE' THEN 'Kenya'
WHEN 'BR' THEN 'Brazil'
WHEN 'CZ' THEN 'Czech Republic'
WHEN 'QA' THEN 'Qatar'
WHEN 'NZ' THEN 'New Zealand'
WHEN 'HR' THEN 'Croatia'
WHEN 'LV' THEN 'Latvia'
WHEN 'MT' THEN 'Malta'
WHEN 'CR' THEN 'Costa Rica'
WHEN 'CY' THEN 'Cyprus'
WHEN 'PA' THEN 'Panama'
WHEN 'SV' THEN 'El Salvador'
WHEN 'DO' THEN 'Dominican Republic'
WHEN 'PE' THEN 'Peru'
WHEN 'VE' THEN 'Venezuela'
WHEN 'EE' THEN 'Estonia'
WHEN 'ID' THEN 'Indonesia'
ELSE ac.country_code
END AS country_name_en,
LOWER(REGEXP_REPLACE(
@@ -172,6 +194,28 @@ SELECT
WHEN 'AE' THEN 'UAE'
WHEN 'AU' THEN 'Australia'
WHEN 'IE' THEN 'Ireland'
WHEN 'PL' THEN 'Poland'
WHEN 'RO' THEN 'Romania'
WHEN 'CO' THEN 'Colombia'
WHEN 'HU' THEN 'Hungary'
WHEN 'ZA' THEN 'South Africa'
WHEN 'KE' THEN 'Kenya'
WHEN 'BR' THEN 'Brazil'
WHEN 'CZ' THEN 'Czech Republic'
WHEN 'QA' THEN 'Qatar'
WHEN 'NZ' THEN 'New Zealand'
WHEN 'HR' THEN 'Croatia'
WHEN 'LV' THEN 'Latvia'
WHEN 'MT' THEN 'Malta'
WHEN 'CR' THEN 'Costa Rica'
WHEN 'CY' THEN 'Cyprus'
WHEN 'PA' THEN 'Panama'
WHEN 'SV' THEN 'El Salvador'
WHEN 'DO' THEN 'Dominican Republic'
WHEN 'PE' THEN 'Peru'
WHEN 'VE' THEN 'Venezuela'
WHEN 'EE' THEN 'Estonia'
WHEN 'ID' THEN 'Indonesia'
ELSE ac.country_code
END, '[^a-zA-Z0-9]+', '-'
)) AS country_slug,

View File

@@ -5,7 +5,7 @@
{% set pct = [((job.progress_current / job.progress_total) * 100) | int, 100] | min if job.progress_total else 0 %}
<tr id="job-{{ job.id }}"
{% if job.status == 'pending' %}
{% if job.status == 'pending' and not job.error %}
hx-get="{{ url_for('pseo.pseo_job_status', job_id=job.id) }}"
hx-trigger="every 2s"
hx-target="this"

View File

@@ -18,7 +18,7 @@ from ..core import (
fetch_all,
fetch_one,
)
from ..i18n import get_translations
from ..i18n import get_country_name, get_translations
bp = Blueprint(
"content",
@@ -208,10 +208,14 @@ async def markets():
SELECT country_code, country_name_en, country_slug,
city_count, total_venues,
avg_market_score, avg_opportunity_score,
top_opportunity_score,
lat, lon
FROM serving.pseo_country_overview
ORDER BY total_venues DESC
""")
lang = g.get("lang", "en")
for c in map_countries:
c["country_name"] = get_country_name(c["country_code"], lang)
# Sort so user's country renders last (on top in Leaflet z-order)
user_country = g.get("user_country", "")
if user_country and map_countries:

View File

@@ -55,7 +55,7 @@
hx-include="#market-q, #market-region">
<option value="">{{ t.mkt_all_countries }}</option>
{% for c in countries %}
<option value="{{ c }}" {% if c == current_country %}selected{% endif %}>{{ c }}</option>
<option value="{{ c }}" {% if c == current_country %}selected{% endif %}>{{ c | country_name(lang) }}</option>
{% endfor %}
</select>
</div>
@@ -86,7 +86,7 @@
<script src="{{ url_for('static', filename='vendor/leaflet/leaflet.min.js') }}"></script>
<script src="{{ url_for('static', filename='js/map-markers.js') }}"></script>
<script>
window.__MAP_T = {score_label:"{{ t.map_score_label }}",venues:"{{ t.map_venues }}",cities:"{{ t.map_cities }}"};
window.__MAP_T = {score_label:"{{ t.map_score_label }}",venues:"{{ t.map_venues }}",cities:"{{ t.map_cities }}",score_avg:"{{ t.map_score_avg }}",score_top:"{{ t.map_score_top }}"};
(function() {
var sc = PNMarkers.scoreColor;
var T = window.__MAP_T;
@@ -105,9 +105,13 @@ window.__MAP_T = {score_label:"{{ t.map_score_label }}",venues:"{{ t.map_venues
var size = 12 + 44 * Math.sqrt(c.total_venues / maxV);
var score = c.avg_opportunity_score || 0;
var hex = sc(score);
var tip = '<strong>' + c.country_name_en + '</strong><br>'
var topScore = c.top_opportunity_score || 0;
var topHex = sc(topScore);
var tip = '<strong>' + c.country_name + '</strong><br>'
+ '<span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + hex + ';vertical-align:middle;margin-right:4px;"></span>'
+ '<span style="color:' + hex + ';font-weight:600;">' + T.score_label + ': ' + score + '/100</span><br>'
+ '<span style="color:' + hex + ';font-weight:600;">' + T.score_avg + ': ' + score + '/100</span><br>'
+ '<span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + topHex + ';vertical-align:middle;margin-right:4px;"></span>'
+ '<span style="color:' + topHex + ';font-weight:600;">' + T.score_top + ': ' + topScore + '/100</span><br>'
+ '<span style="color:#94A3B8;font-size:0.75rem;">' + c.total_venues + ' ' + T.venues + ' · ' + c.city_count + ' ' + T.cities + '</span>';
L.marker([c.lat, c.lon], { icon: PNMarkers.makeIcon({ size: size, color: hex }) })
.bindTooltip(tip, { className: 'map-tooltip', direction: 'top', offset: [0, -Math.round(size / 2)] })

View File

@@ -49,6 +49,25 @@ COUNTRY_LABELS: dict[str, str] = {
"AU": "Australia",
"ZA": "South Africa",
"EG": "Egypt",
"PL": "Poland",
"RO": "Romania",
"CO": "Colombia",
"HU": "Hungary",
"KE": "Kenya",
"CZ": "Czech Republic",
"QA": "Qatar",
"NZ": "New Zealand",
"HR": "Croatia",
"LV": "Latvia",
"MT": "Malta",
"CR": "Costa Rica",
"CY": "Cyprus",
"PA": "Panama",
"SV": "El Salvador",
"DO": "Dominican Republic",
"PE": "Peru",
"VE": "Venezuela",
"IE": "Ireland",
}
_LOCALES_DIR = Path(__file__).parent / "locales"

View File

@@ -345,6 +345,25 @@
"dir_country_AU": "Australien",
"dir_country_ZA": "Südafrika",
"dir_country_EG": "Ägypten",
"dir_country_PL": "Polen",
"dir_country_RO": "Rumänien",
"dir_country_CO": "Kolumbien",
"dir_country_HU": "Ungarn",
"dir_country_KE": "Kenia",
"dir_country_CZ": "Tschechien",
"dir_country_QA": "Katar",
"dir_country_NZ": "Neuseeland",
"dir_country_HR": "Kroatien",
"dir_country_LV": "Lettland",
"dir_country_MT": "Malta",
"dir_country_CR": "Costa Rica",
"dir_country_CY": "Zypern",
"dir_country_PA": "Panama",
"dir_country_SV": "El Salvador",
"dir_country_DO": "Dominikanische Republik",
"dir_country_PE": "Peru",
"dir_country_VE": "Venezuela",
"dir_country_IE": "Irland",
"sp_back": "Zurück zum Verzeichnis",
"sp_verified": "Verifiziert ✓",
"sp_request_quote": "Angebot anfragen →",
@@ -620,6 +639,8 @@
"map_existing_venues": "bestehende Anlagen",
"map_km_nearest": "km zur nächsten Anlage",
"map_no_nearby": "Keine Anlagen in der Nähe",
"map_score_avg": "Ø Score",
"map_score_top": "Top-Stadt",
"waitlist_markets_title": "Marktdaten — Demnächst verfügbar",
"waitlist_markets_sub": "Detaillierte Marktberichte für Padel-Investoren: Baukosten, Umsatz-Benchmarks, Auslastungsdaten und ROI-Analysen nach Stadt und Region.",
"waitlist_markets_feature1": "Echte Kostendaten aus laufenden Anlagen in über 30 Ländern",

View File

@@ -345,6 +345,25 @@
"dir_country_AU": "Australia",
"dir_country_ZA": "South Africa",
"dir_country_EG": "Egypt",
"dir_country_PL": "Poland",
"dir_country_RO": "Romania",
"dir_country_CO": "Colombia",
"dir_country_HU": "Hungary",
"dir_country_KE": "Kenya",
"dir_country_CZ": "Czech Republic",
"dir_country_QA": "Qatar",
"dir_country_NZ": "New Zealand",
"dir_country_HR": "Croatia",
"dir_country_LV": "Latvia",
"dir_country_MT": "Malta",
"dir_country_CR": "Costa Rica",
"dir_country_CY": "Cyprus",
"dir_country_PA": "Panama",
"dir_country_SV": "El Salvador",
"dir_country_DO": "Dominican Republic",
"dir_country_PE": "Peru",
"dir_country_VE": "Venezuela",
"dir_country_IE": "Ireland",
"sp_back": "Back to Directory",
"sp_verified": "Verified ✓",
"sp_request_quote": "Request Quote →",
@@ -620,6 +639,8 @@
"map_existing_venues": "existing venues",
"map_km_nearest": "km to nearest court",
"map_no_nearby": "No nearby courts",
"map_score_avg": "Avg. Score",
"map_score_top": "Top City",
"waitlist_markets_title": "Markets Intelligence — Coming Soon",
"waitlist_markets_sub": "Deep-dive market reports for padel investors: construction costs, revenue benchmarks, occupancy data, and ROI analysis by city and region.",
"waitlist_markets_feature1": "Real cost data from operating venues across 30+ countries",

View File

@@ -0,0 +1,93 @@
"""Drop UNIQUE constraint from articles.slug column.
The single-column UNIQUE on slug conflicts with the ON CONFLICT(url_path, language)
upsert in pSEO generation, causing 'UNIQUE constraint failed: articles.slug' errors
when re-running generation for the same template.
The slug is already unique by construction ({template_slug}-{lang}-{natural_key}),
and the real uniqueness key is (url_path, language). The idx_articles_slug index
is kept for fast lookups.
"""
def up(conn) -> None:
# ── 1. Drop FTS triggers + virtual table ──────────────────────────────────
conn.execute("DROP TRIGGER IF EXISTS articles_ai")
conn.execute("DROP TRIGGER IF EXISTS articles_ad")
conn.execute("DROP TRIGGER IF EXISTS articles_au")
conn.execute("DROP TABLE IF EXISTS articles_fts")
# ── 2. Recreate articles without UNIQUE on slug ───────────────────────────
conn.execute("""
CREATE TABLE articles_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
url_path TEXT NOT NULL,
slug TEXT NOT NULL,
title TEXT NOT NULL,
meta_description TEXT,
country TEXT,
region TEXT,
og_image_url TEXT,
status TEXT NOT NULL DEFAULT 'draft',
published_at TEXT,
template_slug TEXT,
language TEXT NOT NULL DEFAULT 'en',
date_modified TEXT,
seo_head TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT,
group_key TEXT DEFAULT NULL,
noindex INTEGER NOT NULL DEFAULT 0,
article_type TEXT NOT NULL DEFAULT 'editorial',
UNIQUE(url_path, language)
)
""")
conn.execute("""
INSERT INTO articles_new
(id, url_path, slug, title, meta_description, country, region,
og_image_url, status, published_at, template_slug, language,
date_modified, seo_head, created_at, updated_at, group_key,
noindex, article_type)
SELECT id, url_path, slug, title, meta_description, country, region,
og_image_url, status, published_at, template_slug, language,
date_modified, seo_head, created_at, updated_at, group_key,
noindex, article_type
FROM articles
""")
conn.execute("DROP TABLE articles")
conn.execute("ALTER TABLE articles_new RENAME TO articles")
# ── 3. Recreate indexes ───────────────────────────────────────────────────
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_url_path ON articles(url_path)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_url_lang ON articles(url_path, language)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_slug ON articles(slug)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_status ON articles(status, published_at)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_articles_article_type ON articles(article_type)")
# ── 4. Recreate FTS + triggers ────────────────────────────────────────────
conn.execute("""
CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(
title, meta_description, country, region,
content='articles', content_rowid='id'
)
""")
conn.execute("""
CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN
INSERT INTO articles_fts(rowid, title, meta_description, country, region)
VALUES (new.id, new.title, new.meta_description, new.country, new.region);
END
""")
conn.execute("""
CREATE TRIGGER IF NOT EXISTS articles_ad AFTER DELETE ON articles BEGIN
INSERT INTO articles_fts(articles_fts, rowid, title, meta_description, country, region)
VALUES ('delete', old.id, old.title, old.meta_description, old.country, old.region);
END
""")
conn.execute("""
CREATE TRIGGER IF NOT EXISTS articles_au AFTER UPDATE ON articles BEGIN
INSERT INTO articles_fts(articles_fts, rowid, title, meta_description, country, region)
VALUES ('delete', old.id, old.title, old.meta_description, old.country, old.region);
INSERT INTO articles_fts(rowid, title, meta_description, country, region)
VALUES (new.id, new.title, new.meta_description, new.country, new.region);
END
""")