Commit Graph

687 Commits

Author SHA1 Message Date
Deeman
f81d5f19da fix(transform): tighten H3 catchment to res 5 (~24km radius)
Res 4 + k_ring(1) gave ~50-60km effective radius, causing Oldenburg to
absorb Bremen (40km away) and destroying score differentiation.

Res 5 + k_ring(1) gives ~24km — captures adjacent Gemeinden (Delmenhorst
at 15km) without bleeding into unrelated cities at 40km+.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 14:34:56 +01:00
Deeman
159d1b5b9a fix(transform): use community repository for h3 extension install
SQLMesh's extensions config supports dict form with 'repository' key,
which runs INSTALL h3 FROM community + LOAD h3 automatically at connect
time. No manual one-time install needed per machine.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 10:26:56 +01:00
Deeman
4e4ff61699 feat(transform): H3 catchment index for Marktpotenzial-Score v3
Add H3 res-4 regional catchment metrics (~15-18km radius, cell + 6
neighbours) to both the addressable market (25pts) and supply gap
(30pts) components of location_opportunity_profile.

Changes:
- config.yaml: add h3 to DuckDB extensions (requires one-time
  INSTALL h3 FROM community on each machine)
- dim_locations: add h3_cell_res4 column via h3_latlng_to_cell()
- location_opportunity_profile: add hex_stats + catchment CTEs;
  update score formula to use catchment_population and
  catchment_padel_courts; expose catchment_population,
  catchment_padel_courts, catchment_venues_per_100k as output cols

Motivation: local population underestimates functional market for
mid-size cities (e.g. Oldenburg ~170K misses surrounding Gemeinden).
H3 k_ring(1) captures the realistic driving-distance catchment
(~462km²) consistently across both score components.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 10:19:43 +01:00
Deeman
ead12c4552 fix(planner): prevent chart containers from overflowing on small screens
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 2s
v202603051728
2026-03-05 18:27:33 +01:00
Deeman
c54eb50004 fix(planner): prevent chart containers from overflowing on small screens
Grid children default to min-width:auto, letting the Chart.js canvas
push the container wider than its grid track. Adding min-width:0 and
overflow:hidden constrains charts to their column width.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 18:27:27 +01:00
Deeman
5d7fcec17a chore: change GISCO extraction schedule from monthly to yearly
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
v202603051651
2026-03-05 17:50:19 +01:00
Deeman
f7faf7ab57 chore: change GISCO extraction schedule from monthly to yearly
NUTS2 boundaries rarely change; yearly (Jan 1) is sufficient.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:50:14 +01:00
Deeman
add5f8ddfa fix(extract): correct lc_lci_lev lcstruct filter value
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
v202603051640
2026-03-05 17:39:37 +01:00
Deeman
15ca316682 fix(extract): correct lc_lci_lev lcstruct filter value
D1_D2_A_HW doesn't exist in the API; use D1_D4_MD5 (total labour cost
= compensation + taxes - subsidies).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:32:49 +01:00
Deeman
103ef73cf5 fix(pipeline): eurostat filter bugs + supervisor uses sqlmesh plan
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
v202603051620
2026-03-05 17:19:21 +01:00
Deeman
aa27f14f3c fix(pipeline): eurostat filter bugs + supervisor uses sqlmesh plan
- nrg_pc_203: add missing unit=KWH filter (API returns 2 units)
- lc_lci_lev: fix currency→unit filter dimension name
- supervisor: use `sqlmesh plan prod --auto-apply` instead of
  `sqlmesh run` so new/changed models are detected automatically

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:19:12 +01:00
Deeman
8205744444 chore: remove accidentally committed .claire/ worktree directory
All checks were successful
CI / test (push) Successful in 56s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202603051612
2026-03-05 17:10:48 +01:00
Deeman
1cbefe349c add env var 2026-03-05 17:08:52 +01:00
Deeman
003f19e071 fix(pipeline): handle DuckDB catalog naming in diagnostic script 2026-03-05 17:07:52 +01:00
Deeman
c3f15535b8 fix(pipeline): handle DuckDB catalog naming in diagnostic script
The lakehouse.duckdb file uses catalog "lakehouse" not "local", causing
SQLMesh logical views to break. Script now auto-detects the catalog via
USE and falls back to physical tables when views fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 17:06:44 +01:00
Deeman
fcb8ec4227 merge: pipeline diagnostic script + extraction card UX improvements
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
v202603051445
2026-03-05 15:40:16 +01:00
Deeman
6b7fa45bce feat(admin): add pipeline diagnostic script + extraction card UX improvements
- Add scripts/check_pipeline.py: read-only diagnostic for pricing pipeline
  row counts, date range analysis, HAVING filter impact, join coverage
- Add description field to all 12 workflows in workflows.toml
- Parse and display descriptions on extraction status cards
- Show spinner + "Running" state with blue-tinted card border
- Display start time with "running..." text for active extractions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 15:40:12 +01:00
Deeman
0d8687859d fix(docker): copy workflows.toml into container for admin pipeline view
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
The admin Extraction Status page reads infra/supervisor/workflows.toml
but the Dockerfile only copied web/ into the image. Adding the COPY
so the file exists at /app/infra/supervisor/workflows.toml in the
container.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051417
2026-03-05 15:16:07 +01:00
Deeman
b064e18aa1 fix(admin): resolve workflows.toml path via CWD instead of __file__
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
In prod the package is installed in a venv, so __file__.parents[4] doesn't
reach the repo root. Use CWD (repo root in both dev and prod via systemd
WorkingDirectory) with REPO_ROOT env var override.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051340
2026-03-05 14:39:30 +01:00
Deeman
dc68976148 docs(marketing): add GTM, social posts, Reddit plan, and SEO calendar
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603051044
2026-03-05 11:43:11 +01:00
Deeman
60fa2bc720 test(billing): add Stripe E2E test scripts for sandbox validation
- test_stripe_sandbox.py: API-only validation of all 17 products (67 tests)
- stripe_e2e_setup.py: webhook endpoint registration via ngrok
- stripe_e2e_test.py: live webhook tests with real DB verification (67 tests)
- stripe_e2e_checkout_test.py: checkout webhook tests for credit packs,
  sticky boosts, and business plan PDF purchases (40 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:50:26 +01:00
Deeman
66c2dfce66 fix(billing): fetch line items for checkout.session.completed webhooks
_extract_line_items() was returning [] for all checkout sessions, which
meant _handle_transaction_completed never processed credit packs, sticky
boosts, or business plan PDF purchases. Now fetches line items from the
Stripe API using the session ID, with a fallback to embedded line_items.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:49:41 +01:00
Deeman
6e3c5554aa fix(admin): enable bulk actions in grouped articles view
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
- dev_run.sh: also remove app.db-shm and app.db-wal on reset to fix
  SQLite disk I/O error from stale WAL/SHM files
- articles bulk: add checkboxes to grouped rows (data-ids holds all
  variant IDs); checking a group selects EN+DE together
- restore select-all checkbox in grouped <th>
- add toggleArticleGroupSelect() JS function
- fix htmx:afterSwap to re-check group checkboxes correctly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202603050849
2026-03-05 09:48:54 +01:00
Deeman
ad02140594 fix(quote): add missing required asterisk and error hint to step 4
All checks were successful
CI / test (push) Successful in 54s
CI / tag (push) Successful in 3s
Step 4 (Project Phase) required location_status server-side but had no
visual "*" indicator and no error message when submitting without a
selection. All other steps already had both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603042122
2026-03-04 22:21:57 +01:00
Deeman
5bcd87d7e5 fix(ci): replace non-existent quote.wizard endpoint with leads.quote_request
All checks were successful
CI / test (push) Successful in 53s
CI / tag (push) Successful in 3s
The CRO homepage overhaul (f4f8a45) introduced url_for('quote.wizard')
in landing.html, but that endpoint never existed — the actual route is
leads.quote_request. This broke CI runs #99–#109.

Also adds landing_vs_col_us to i18n allowlist (brand name, same in both
languages).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v202603042113
2026-03-04 22:12:45 +01:00
Deeman
77772b7ea4 feat(maps): beanflows-style divIcon bubbles + feature flag gate
Replace L.circleMarker with L.divIcon + .pn-marker CSS class (white
border, box-shadow, hover scale) matching the beanflows growing
conditions map pattern. Dark .map-tooltip CSS override (no arrow,
dark navy background). Small venue dots use .pn-venue class.

Add _require_maps_flag() to all 4 API endpoints (default=True so
dev works without seeding the flag row). Gate /opportunity-map route
the same way.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 20:51:00 +01:00
Deeman
59f1f0d699 merge(worktree): interactive maps for market pages
Self-hosted Leaflet 1.9.4 maps across 4 placements: markets hub
country bubbles, country overview city bubbles, city venue dots, and
a standalone opportunity map. New /api blueprint with 4 JSON endpoints.
New city_venue_locations SQLMesh serving model. No CDN — GDPR-safe.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
2026-03-04 15:36:41 +01:00
Deeman
0a89ba2213 docs: update CHANGELOG with interactive maps feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:33:35 +01:00
Deeman
6e936dbb95 feat(maps): Phase 5 — standalone opportunity map page
New route GET /<lang>/opportunity-map renders a full-width Leaflet map
with a country selector. On country change, fetches
/api/opportunity/{slug}.json and renders opportunity circles
(color-coded by score, sized by population) plus existing-venue gray
reference dots from /api/markets/{country}/cities.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:32:56 +01:00
Deeman
edf678ac4e feat(maps): Phase 4 — city venue dot map
New serving model: city_venue_locations joins dim_venues + dim_cities
to expose lat/lon/court_count per venue for the city dot map endpoint.

pseo_city_costs_de.sql: add c.lat, c.lon so city-cost articles have
city coordinates for the #city-map data attributes.

city-cost-de.md.jinja: add #city-map div (both DE and EN sections)
after the stats strip. Leaflet init handled by article_detail.html.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:07:06 +01:00
Deeman
0eef455543 feat(maps): Phase 3 — country overview city bubble map + article_detail Leaflet loader
Add #country-map div to country-overview.md.jinja (both DE/EN).
article_detail.html: always include Leaflet CSS, conditionally load
Leaflet JS only when #country-map or #city-map divs are present.
Initializes country city-bubble map and city venue-dot map from
/api/markets/{slug}/cities.json and /api/markets/{country}/{city}/venues.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:15:41 +01:00
Deeman
8e53fda283 feat(maps): Phase 2 — markets hub country bubble map
Add Leaflet map to /markets with country-level bubbles sized by
total_venues and colored by avg_market_score. Click navigates to
country overview page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:04:40 +01:00
Deeman
db0d7cfee9 feat(maps): Phase 1 — Leaflet vendor files, API blueprint, app registration
Self-host Leaflet 1.9.4 JS/CSS/images in static/vendor/leaflet/.
Create api.py blueprint with 4 JSON endpoints for map data.
Register api_bp at /api in app.py (before content catch-all).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 12:46:13 +01:00
Deeman
61c197d233 merge(worktree): individualise article costs with per-country Eurostat data + tiered proxy tenant work
# Conflicts:
#	CHANGELOG.md
#	transform/sqlmesh_padelnomics/models/foundation/dim_cities.sql
#	transform/sqlmesh_padelnomics/models/foundation/dim_locations.sql
2026-03-04 12:44:56 +01:00
Deeman
2e68cfbe4f feat(transform): individualise article costs with per-country Eurostat data
Add real per-country cost data to ~30 calculator fields so pSEO articles
show country-specific CAPEX/OPEX instead of hardcoded DE defaults.

Extractor:
- eurostat.py: add 8 new datasets (nrg_pc_205, nrg_pc_203, lc_lci_lev,
  5×prc_ppp_ind variants); add optional `dataset_code` field so multiple
  dict entries can share one Eurostat API endpoint

Staging (4 new models):
- stg_electricity_prices — EUR/kWh by country, semi-annual
- stg_gas_prices         — EUR/GJ by country, semi-annual
- stg_labour_costs       — EUR/hour by country, annual (future staffed scenario)
- stg_price_levels       — PLI indices (EU27=100) for 5 categories, annual

Foundation:
- dim_countries (new) — conformed country dimension; eliminates ~50-line CASE
  blocks duplicated in dim_cities/dim_locations; computes ~29 calculator cost
  override columns from PLI ratios and energy price ratios vs DE baseline;
  NULL for DE so calculator falls through to DEFAULTS unchanged
- dim_cities — replace country_name/slug CASE blocks + country_income CTE
  with JOIN dim_countries
- dim_locations — same refactor as dim_cities

Serving:
- pseo_city_costs_de — JOIN dim_countries; add 29 camelCase override columns
  auto-applied by calculator (electricity, heating, rentSqm, hallCostSqm, …)
- planner_defaults — JOIN dim_countries; same 29 cost columns flow through
  to /api/market-data endpoint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 10:09:48 +01:00
Deeman
7af6f32a2b merge: bulk actions for articles and leads
Some checks failed
CI / test (push) Failing after 33s
CI / tag (push) Has been skipped
2026-03-04 09:55:19 +01:00
Deeman
53fdbd9fd5 docs: update CHANGELOG with bulk actions feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 09:55:04 +01:00
Deeman
81487d6f01 feat(admin): bulk actions for articles and leads
Add bulk selection checkboxes and action bars to the articles and leads
admin pages, replicating the existing supplier bulk pattern.

Articles: publish, unpublish, toggle noindex, rebuild, delete (with
confirmation dialog). Leads: set status, set heat. Both re-render the
results partial after action via HTMX, preserving current filters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 09:40:26 +01:00
Deeman
477f635bc5 test(billing): Stripe E2E webhook lifecycle tests
Some checks failed
CI / test (push) Failing after 29s
CI / tag (push) Has been skipped
2026-03-03 18:17:10 +01:00
Deeman
4dbded74ca test(billing): add Stripe E2E webhook lifecycle tests
16 tests covering the full Stripe webhook flow through /billing/webhook/stripe:
- Subscription creation (customer.subscription.created → DB row)
- Period end extraction from items (Stripe API 2026-02+ compatibility)
- Billing customer creation
- Status updates (active, past_due, trialing)
- Cancellation (customer.subscription.deleted → cancelled)
- Payment failure (invoice.payment_failed → past_due)
- One-time payments (checkout.session.completed mode=payment)
- Full lifecycle: create → update → recover → cancel
- Edge cases: missing metadata, empty items, invalid JSON, bad signatures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:17:05 +01:00
Deeman
230406f34f fix(billing): period_end from Stripe items + test 2026-03-03 18:06:01 +01:00
Deeman
7da6a4737d fix(billing): extract current_period_end from Stripe subscription items
Stripe API 2026-02+ moved current_period_end from subscription to
subscription items. Add _get_period_end() helper that falls back to
items[0].current_period_end when the subscription-level field is None.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:05:55 +01:00
Deeman
710e21a186 fix(billing): handle customer.subscription.created + test isolation 2026-03-03 17:58:15 +01:00
Deeman
72c4de91b0 fix(billing): handle customer.subscription.created webhook + test isolation
- Add customer.subscription.created → subscription.activated mapping in
  stripe.parse_webhook so direct API subscription creation also creates DB rows
- Add customer.subscription.created to setup_stripe.py enabled_events
- Pin PAYMENT_PROVIDER=paddle and STRIPE_WEBHOOK_SECRET="" in test conftest
  so billing tests don't hit real Stripe API when env has Stripe keys
- Add 8 unit tests for stripe.parse_webhook covering all event types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 17:29:13 +01:00
Deeman
046be665db merge: fix remaining request_options in stripe.py 2026-03-03 16:46:48 +01:00
Deeman
7c5fa86fb8 fix(billing): remove remaining request_options from Price.retrieve calls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:46:25 +01:00
Deeman
0a9f980813 merge: fix Stripe SDK request_options + webhook endpoint graceful failure 2026-03-03 16:36:58 +01:00
Deeman
2682e810fa fix(billing): remove invalid request_options from Stripe SDK calls
Stripe Python SDK doesn't accept request_options as a kwarg to create/retrieve/modify.
Timeouts are handled by the global max_network_retries setting.
Also gracefully handle webhook endpoint creation failure for localhost URLs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:36:47 +01:00
Deeman
10af6a284c fix(content): slug transliteration, article links, country overview ranking
Some checks failed
CI / test (push) Failing after 30s
CI / tag (push) Has been skipped
# Conflicts:
#	CHANGELOG.md
2026-03-03 16:29:41 +01:00
Deeman
68f354ac2b docs: update CHANGELOG for slug fix + country overview ranking
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:26:55 +01:00