Commit Graph

680 Commits

Author SHA1 Message Date
Deeman
9515ec8ae9 fix(admin): load Leaflet maps in template preview page
The /admin/templates/<slug>/preview/<key> page renders article HTML
directly but never loaded Leaflet CSS/JS, so country-map and city-map
divs appeared empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 22:29:00 +01:00
Deeman
4ee80603ef fix(articles): load Leaflet maps in article editor preview
The admin article preview iframe was missing Leaflet CSS/JS and had
scripts blocked by the sandbox policy, so map shortcodes rendered as
empty divs.

- Extract inline map script to static/js/article-maps.js (shared
  between article_detail.html and admin preview)
- Replace f-string preview doc with a proper Jinja template that
  includes Leaflet assets
- Add allow-scripts to iframe sandbox on both initial load and HTMX
  preview updates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 22:03:50 +01:00
Deeman
2e42245ad5 fix(worker): use sqlmesh run prod instead of plan prod --auto-apply
`plan --auto-apply` only detects SQL model changes and won't re-run
for new data. `run prod` evaluates missing cron intervals and picks
up newly extracted data — matching the fix already applied to the
supervisor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 21:49:51 +01:00
Deeman
2f47d1e589 fix(pipeline): make availability chain incremental + fix supervisor
Convert the availability chain (stg_playtomic_availability →
fct_availability_slot → fct_daily_availability) from FULL to
INCREMENTAL_BY_TIME_RANGE so sqlmesh run processes only new daily
intervals instead of re-reading all files.

Supervisor changes:
- run_transform(): plan prod --auto-apply → run prod (evaluates
  missing cron intervals, picks up new data)
- git_pull_and_sync(): add plan prod --auto-apply before re-exec
  so model code changes are applied on deploy
- supervisor.sh: same plan → run change

Staging model uses a date-scoped glob (@start_ds) to read only
the current interval's files. snapshot_date cast to DATE (was
VARCHAR) as required by time_column.

Clean up redundant TRY_CAST(snapshot_date AS DATE) in
venue_pricing_benchmarks since it's already DATE from foundation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 21:34:02 +01:00
Deeman
c54eb50004 fix(planner): prevent chart containers from overflowing on small screens
Grid children default to min-width:auto, letting the Chart.js canvas
push the container wider than its grid track. Adding min-width:0 and
overflow:hidden constrains charts to their column width.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 18:27:27 +01:00
Deeman
f7faf7ab57 chore: change GISCO extraction schedule from monthly to yearly
NUTS2 boundaries rarely change; yearly (Jan 1) is sufficient.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:50:14 +01:00
Deeman
15ca316682 fix(extract): correct lc_lci_lev lcstruct filter value
D1_D2_A_HW doesn't exist in the API; use D1_D4_MD5 (total labour cost
= compensation + taxes - subsidies).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:32:49 +01:00
Deeman
aa27f14f3c fix(pipeline): eurostat filter bugs + supervisor uses sqlmesh plan
- nrg_pc_203: add missing unit=KWH filter (API returns 2 units)
- lc_lci_lev: fix currency→unit filter dimension name
- supervisor: use `sqlmesh plan prod --auto-apply` instead of
  `sqlmesh run` so new/changed models are detected automatically

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:19:12 +01:00
Deeman
c3f15535b8 fix(pipeline): handle DuckDB catalog naming in diagnostic script
The lakehouse.duckdb file uses catalog "lakehouse" not "local", causing
SQLMesh logical views to break. Script now auto-detects the catalog via
USE and falls back to physical tables when views fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:06:44 +01:00
Deeman
6b7fa45bce feat(admin): add pipeline diagnostic script + extraction card UX improvements
- Add scripts/check_pipeline.py: read-only diagnostic for pricing pipeline
  row counts, date range analysis, HAVING filter impact, join coverage
- Add description field to all 12 workflows in workflows.toml
- Parse and display descriptions on extraction status cards
- Show spinner + "Running" state with blue-tinted card border
- Display start time with "running..." text for active extractions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 15:40:12 +01:00
Deeman
0d8687859d fix(docker): copy workflows.toml into container for admin pipeline view
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
The admin Extraction Status page reads infra/supervisor/workflows.toml
but the Dockerfile only copied web/ into the image. Adding the COPY
so the file exists at /app/infra/supervisor/workflows.toml in the
container.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051417
2026-03-05 15:16:07 +01:00
Deeman
b064e18aa1 fix(admin): resolve workflows.toml path via CWD instead of __file__
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
In prod the package is installed in a venv, so __file__.parents[4] doesn't
reach the repo root. Use CWD (repo root in both dev and prod via systemd
WorkingDirectory) with REPO_ROOT env var override.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051340
2026-03-05 14:39:30 +01:00
Deeman
dc68976148 docs(marketing): add GTM, social posts, Reddit plan, and SEO calendar
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051044
2026-03-05 11:43:11 +01:00
Deeman
60fa2bc720 test(billing): add Stripe E2E test scripts for sandbox validation
- test_stripe_sandbox.py: API-only validation of all 17 products (67 tests)
- stripe_e2e_setup.py: webhook endpoint registration via ngrok
- stripe_e2e_test.py: live webhook tests with real DB verification (67 tests)
- stripe_e2e_checkout_test.py: checkout webhook tests for credit packs,
  sticky boosts, and business plan PDF purchases (40 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:50:26 +01:00
Deeman
66c2dfce66 fix(billing): fetch line items for checkout.session.completed webhooks
_extract_line_items() was returning [] for all checkout sessions, which
meant _handle_transaction_completed never processed credit packs, sticky
boosts, or business plan PDF purchases. Now fetches line items from the
Stripe API using the session ID, with a fallback to embedded line_items.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:49:41 +01:00
Deeman
6e3c5554aa fix(admin): enable bulk actions in grouped articles view
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
- dev_run.sh: also remove app.db-shm and app.db-wal on reset to fix
  SQLite disk I/O error from stale WAL/SHM files
- articles bulk: add checkboxes to grouped rows (data-ids holds all
  variant IDs); checking a group selects EN+DE together
- restore select-all checkbox in grouped <th>
- add toggleArticleGroupSelect() JS function
- fix htmx:afterSwap to re-check group checkboxes correctly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202603050849
2026-03-05 09:48:54 +01:00
Deeman
ad02140594 fix(quote): add missing required asterisk and error hint to step 4
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
Step 4 (Project Phase) required location_status server-side but had no
visual "*" indicator and no error message when submitting without a
selection. All other steps already had both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603042122
2026-03-04 22:21:57 +01:00
Deeman
5bcd87d7e5 fix(ci): replace non-existent quote.wizard endpoint with leads.quote_request
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
The CRO homepage overhaul (f4f8a45) introduced url_for('quote.wizard')
in landing.html, but that endpoint never existed — the actual route is
leads.quote_request. This broke CI runs #99–#109.

Also adds landing_vs_col_us to i18n allowlist (brand name, same in both
languages).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603042113
2026-03-04 22:12:45 +01:00
Deeman
77772b7ea4 feat(maps): beanflows-style divIcon bubbles + feature flag gate
Replace L.circleMarker with L.divIcon + .pn-marker CSS class (white
border, box-shadow, hover scale) matching the beanflows growing
conditions map pattern. Dark .map-tooltip CSS override (no arrow,
dark navy background). Small venue dots use .pn-venue class.

Add _require_maps_flag() to all 4 API endpoints (default=True so
dev works without seeding the flag row). Gate /opportunity-map route
the same way.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 20:51:00 +01:00
Deeman
59f1f0d699 merge(worktree): interactive maps for market pages
Self-hosted Leaflet 1.9.4 maps across 4 placements: markets hub
country bubbles, country overview city bubbles, city venue dots, and
a standalone opportunity map. New /api blueprint with 4 JSON endpoints.
New city_venue_locations SQLMesh serving model. No CDN — GDPR-safe.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
2026-03-04 15:36:41 +01:00
Deeman
0a89ba2213 docs: update CHANGELOG with interactive maps feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:33:35 +01:00
Deeman
6e936dbb95 feat(maps): Phase 5 — standalone opportunity map page
New route GET /<lang>/opportunity-map renders a full-width Leaflet map
with a country selector. On country change, fetches
/api/opportunity/{slug}.json and renders opportunity circles
(color-coded by score, sized by population) plus existing-venue gray
reference dots from /api/markets/{country}/cities.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:32:56 +01:00
Deeman
edf678ac4e feat(maps): Phase 4 — city venue dot map
New serving model: city_venue_locations joins dim_venues + dim_cities
to expose lat/lon/court_count per venue for the city dot map endpoint.

pseo_city_costs_de.sql: add c.lat, c.lon so city-cost articles have
city coordinates for the #city-map data attributes.

city-cost-de.md.jinja: add #city-map div (both DE and EN sections)
after the stats strip. Leaflet init handled by article_detail.html.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:07:06 +01:00
Deeman
0eef455543 feat(maps): Phase 3 — country overview city bubble map + article_detail Leaflet loader
Add #country-map div to country-overview.md.jinja (both DE/EN).
article_detail.html: always include Leaflet CSS, conditionally load
Leaflet JS only when #country-map or #city-map divs are present.
Initializes country city-bubble map and city venue-dot map from
/api/markets/{slug}/cities.json and /api/markets/{country}/{city}/venues.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:15:41 +01:00
Deeman
8e53fda283 feat(maps): Phase 2 — markets hub country bubble map
Add Leaflet map to /markets with country-level bubbles sized by
total_venues and colored by avg_market_score. Click navigates to
country overview page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:04:40 +01:00
Deeman
db0d7cfee9 feat(maps): Phase 1 — Leaflet vendor files, API blueprint, app registration
Self-host Leaflet 1.9.4 JS/CSS/images in static/vendor/leaflet/.
Create api.py blueprint with 4 JSON endpoints for map data.
Register api_bp at /api in app.py (before content catch-all).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 12:46:13 +01:00
Deeman
61c197d233 merge(worktree): individualise article costs with per-country Eurostat data + tiered proxy tenant work
# Conflicts:
#	CHANGELOG.md
#	transform/sqlmesh_padelnomics/models/foundation/dim_cities.sql
#	transform/sqlmesh_padelnomics/models/foundation/dim_locations.sql
2026-03-04 12:44:56 +01:00
Deeman
2e68cfbe4f feat(transform): individualise article costs with per-country Eurostat data
Add real per-country cost data to ~30 calculator fields so pSEO articles
show country-specific CAPEX/OPEX instead of hardcoded DE defaults.

Extractor:
- eurostat.py: add 8 new datasets (nrg_pc_205, nrg_pc_203, lc_lci_lev,
  5×prc_ppp_ind variants); add optional `dataset_code` field so multiple
  dict entries can share one Eurostat API endpoint

Staging (4 new models):
- stg_electricity_prices — EUR/kWh by country, semi-annual
- stg_gas_prices         — EUR/GJ by country, semi-annual
- stg_labour_costs       — EUR/hour by country, annual (future staffed scenario)
- stg_price_levels       — PLI indices (EU27=100) for 5 categories, annual

Foundation:
- dim_countries (new) — conformed country dimension; eliminates ~50-line CASE
  blocks duplicated in dim_cities/dim_locations; computes ~29 calculator cost
  override columns from PLI ratios and energy price ratios vs DE baseline;
  NULL for DE so calculator falls through to DEFAULTS unchanged
- dim_cities — replace country_name/slug CASE blocks + country_income CTE
  with JOIN dim_countries
- dim_locations — same refactor as dim_cities

Serving:
- pseo_city_costs_de — JOIN dim_countries; add 29 camelCase override columns
  auto-applied by calculator (electricity, heating, rentSqm, hallCostSqm, …)
- planner_defaults — JOIN dim_countries; same 29 cost columns flow through
  to /api/market-data endpoint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 10:09:48 +01:00
Deeman
7af6f32a2b merge: bulk actions for articles and leads
Some checks failed
CI / test (push) Failing after 33s
CI / tag (push) Has been skipped
2026-03-04 09:55:19 +01:00
Deeman
53fdbd9fd5 docs: update CHANGELOG with bulk actions feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 09:55:04 +01:00
Deeman
81487d6f01 feat(admin): bulk actions for articles and leads
Add bulk selection checkboxes and action bars to the articles and leads
admin pages, replicating the existing supplier bulk pattern.

Articles: publish, unpublish, toggle noindex, rebuild, delete (with
confirmation dialog). Leads: set status, set heat. Both re-render the
results partial after action via HTMX, preserving current filters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 09:40:26 +01:00
Deeman
477f635bc5 test(billing): Stripe E2E webhook lifecycle tests
Some checks failed
CI / test (push) Failing after 29s
CI / tag (push) Has been skipped
2026-03-03 18:17:10 +01:00
Deeman
4dbded74ca test(billing): add Stripe E2E webhook lifecycle tests
16 tests covering the full Stripe webhook flow through /billing/webhook/stripe:
- Subscription creation (customer.subscription.created → DB row)
- Period end extraction from items (Stripe API 2026-02+ compatibility)
- Billing customer creation
- Status updates (active, past_due, trialing)
- Cancellation (customer.subscription.deleted → cancelled)
- Payment failure (invoice.payment_failed → past_due)
- One-time payments (checkout.session.completed mode=payment)
- Full lifecycle: create → update → recover → cancel
- Edge cases: missing metadata, empty items, invalid JSON, bad signatures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:17:05 +01:00
Deeman
230406f34f fix(billing): period_end from Stripe items + test 2026-03-03 18:06:01 +01:00
Deeman
7da6a4737d fix(billing): extract current_period_end from Stripe subscription items
Stripe API 2026-02+ moved current_period_end from subscription to
subscription items. Add _get_period_end() helper that falls back to
items[0].current_period_end when the subscription-level field is None.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:05:55 +01:00
Deeman
710e21a186 fix(billing): handle customer.subscription.created + test isolation 2026-03-03 17:58:15 +01:00
Deeman
72c4de91b0 fix(billing): handle customer.subscription.created webhook + test isolation
- Add customer.subscription.created → subscription.activated mapping in
  stripe.parse_webhook so direct API subscription creation also creates DB rows
- Add customer.subscription.created to setup_stripe.py enabled_events
- Pin PAYMENT_PROVIDER=paddle and STRIPE_WEBHOOK_SECRET="" in test conftest
  so billing tests don't hit real Stripe API when env has Stripe keys
- Add 8 unit tests for stripe.parse_webhook covering all event types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 17:29:13 +01:00
Deeman
046be665db merge: fix remaining request_options in stripe.py 2026-03-03 16:46:48 +01:00
Deeman
7c5fa86fb8 fix(billing): remove remaining request_options from Price.retrieve calls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:46:25 +01:00
Deeman
0a9f980813 merge: fix Stripe SDK request_options + webhook endpoint graceful failure 2026-03-03 16:36:58 +01:00
Deeman
2682e810fa fix(billing): remove invalid request_options from Stripe SDK calls
Stripe Python SDK doesn't accept request_options as a kwarg to create/retrieve/modify.
Timeouts are handled by the global max_network_retries setting.
Also gracefully handle webhook endpoint creation failure for localhost URLs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:36:47 +01:00
Deeman
10af6a284c fix(content): slug transliteration, article links, country overview ranking
Some checks failed
CI / test (push) Failing after 30s
CI / tag (push) Has been skipped
# Conflicts:
#	CHANGELOG.md
2026-03-03 16:29:41 +01:00
Deeman
68f354ac2b docs: update CHANGELOG for slug fix + country overview ranking
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:26:55 +01:00
Deeman
0b74156ef7 merge: accept alternative Stripe env var names 2026-03-03 16:24:25 +01:00
Deeman
fab16cb48f fix(billing): accept STRIPE_API_PRIVATE_KEY / STRIPE_API_PUBLIC_KEY env var names
Also normalise PAYMENT_PROVIDER to lowercase so STRIPE/stripe both work.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:24:03 +01:00
Deeman
062a6d2766 merge: Stripe payment provider (dispatch-by-config alongside Paddle) 2026-03-03 16:07:52 +01:00
Deeman
80c2f111d2 feat(billing): B4-B5 — tests, lint fixes, CHANGELOG + PROJECT.md
- Fix unused imports in stripe.py (hashlib, hmac, time)
- Update test_billing_routes.py: insert into payment_products table,
  fix mock paths for extracted paddle.py, add Stripe webhook 404 test
- Update CHANGELOG.md with Stripe provider feature
- Update PROJECT.md Done section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:07:30 +01:00
Deeman
7ae8334d7a feat(billing): B3 — setup_stripe.py product/price creation script
Mirrors setup_paddle.py structure:
- Creates 17 products + prices in Stripe (same keys, same prices)
- Writes to payment_products table with provider='stripe'
- Registers webhook endpoint at /billing/webhook/stripe
- tax_behavior='exclusive' (price + VAT on top, EU standard)
- Supports --sync flag to re-populate from existing Stripe products

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 15:53:38 +01:00
Deeman
032fe8d86c feat(billing): B2 — Stripe payment provider implementation
billing/stripe.py exports the same interface as paddle.py:
- build_checkout_payload() → Stripe Checkout Session with automatic_tax
- build_multi_item_checkout_payload() → multi-line-item sessions
- cancel_subscription() → cancel_at_period_end=True
- get_management_url() → Stripe Billing Portal session
- verify_webhook() → Stripe-Signature header verification
- parse_webhook() → maps Stripe events to shared format:
  checkout.session.completed → subscription.activated / transaction.completed
  customer.subscription.updated → subscription.updated
  customer.subscription.deleted → subscription.canceled
  invoice.payment_failed → subscription.past_due

All API calls have 10s timeout and max 2 retries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 15:48:08 +01:00
Deeman
4907bc8b64 feat(billing): B1 — add stripe SDK dependency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 15:37:29 +01:00