- Add global htmx:confirm handler in base_admin.html that intercepts
hx-confirm attributes and shows #confirm-dialog instead of window.confirm()
- Convert 4 pipeline HTMX buttons (Run Transform, Run Export, Run Full
Pipeline, Run extractor) from onclick+confirm() to hx-confirm
- Convert 4 affiliate form/list delete buttons from onclick+confirm()
to confirmAction() via event.preventDefault()
- Add scrollbar-width:none + ::-webkit-scrollbar{display:none} to
.pipeline-tabs to suppress spurious horizontal scrollbar
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add proxy_failure_limit param to make_tiered_cycler (default 3).
Individual proxies hitting the limit are marked dead and permanently
skipped. next_proxy() auto-escalates when all proxies in the active
tier are dead. Both mechanisms coexist: per-proxy dead tracking removes
broken individuals; tier-level threshold catches systemic failure.
- proxy.py: dead_proxies set + proxy_failure_counts dict in state;
next_proxy skips dead proxies with bounded loop; record_failure/
record_success accept optional proxy_url; dead_proxy_count() added
- playtomic_tenants.py: pass proxy_url to record_success/record_failure
- playtomic_availability.py: _worker returns (proxy_url, result);
serial loops in extract + extract_recheck capture proxy_url
- test_supervisor.py: 11 new tests in TestTieredCyclerDeadProxyTracking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docker-compose.prod.yml: fix volume mount for all 6 web containers
from /opt/padelnomics/data (stale) → /data/padelnomics (live supervisor output);
add LANDING_DIR=/app/data/pipeline/landing so extraction/landing stats work
- pipeline_routes.py: fix _REPO_ROOT parents[5] → parents[4] so workflows.toml
is found in dev and pipeline overview shows workflow schedules
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Centralises retailer config in affiliate_programs table (URL template,
tracking tag, commission %). Products now use program dropdown + product
identifier instead of manual URL baking. URL assembled at redirect time
via build_affiliate_url() — changing a tag propagates to all products
instantly. Backward compatible: legacy baked-URL products fall through
unchanged. Amazon OneLink (configured in Associates dashboard) handles
geo-redirect to local marketplaces with no additional programs needed.
Also fixes _rebuild_article() frontmatter rendering bug.
Commits: fix frontmatter, migration 0027, program CRUD functions,
redirect update, admin CRUD + templates, product form update, tests.
41 tests, all passing. Ruff clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the manual affiliate URL field with a program selector and
product identifier input. JS toggles visibility between program mode and
manual (custom URL) mode. retailer field is auto-populated from the
program name on save. INSERT/UPDATE statements include new program_id
and product_identifier columns. Validation accepts program+ID or manual
URL as the URL source.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds program list, create, edit, delete routes with appropriate guards
(delete blocked if products reference the program). Adds "Programs" tab
to the affiliate subnav. New templates: affiliate_programs.html,
affiliate_program_form.html, partials/affiliate_program_results.html.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Program-based products now get URLs assembled from the template at
redirect time. Changing a program's tracking_tag propagates instantly
to all its products without rebuilding. Legacy products (no program_id)
still use their baked affiliate_url via fallback.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds get_all_programs(), get_program(), get_program_by_slug() for admin
CRUD. Adds build_affiliate_url() that assembles URLs from program template
+ product identifier, with fallback to baked affiliate_url for legacy
products. Updates get_product() to JOIN affiliate_programs so _program
dict is available at redirect time. _parse_product() extracts program
fields into nested _program key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes a bug where manual article previews rendered raw frontmatter
(title:, slug:, etc.) as visible text. Now strips the --- block using
the existing _FRONTMATTER_RE before passing the body to mistune.html().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Live chips now open the article in a new tab. Draft/scheduled chips are
non-clickable spans (informational only). The Edit button is the sole
path to the edit page, removing the redundant double-link.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EN: replace cliché phrase "pointing to pockets of underserved demand"
→ "leaving genuine supply gaps even in established markets" (more precise)
DE version already had a cleaner equivalent — no change needed there.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EN prose: tighten intro paragraph — "The question investors actually need
answered is:" → "The question that matters:" (DE version already had the
cleaner formulation; now aligned)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docker-compose.prod.yml: replace file bind mount for analytics.duckdb
with directory bind mount (/opt/padelnomics/data:/app/data/pipeline:ro)
so os.rename() on the host is visible inside the container
- Override SERVING_DUCKDB_PATH to /app/data/pipeline/analytics.duckdb in
all 6 blue/green services (removes dependency on .env value)
- analytics.py: track file inode; call _check_and_reopen() at start of
each query — transparently picks up new analytics.duckdb without restart
when export_serving.py atomically replaces it after each pipeline run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The invisible trigger div was inside the CSS grid, occupying the first cell
(1fr) and pushing the form into the 380px column and the preview below it.
Moved it before the grid with display:none so it has no layout impact.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
hx-trigger="load, input from:..." fires the preview POST as soon as the page
opens, so editing an existing product shows its card without needing to
touch any field first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The form was posting to the save route on every input change (which would
save the product on every keystroke). Added a dedicated POST
/admin/affiliate/preview route that renders the product_card.html partial
from form data without touching the database.
Form now keeps action pointing to the save route; an invisible hx-div
triggers preview-only POSTs via hx-include="#affiliate-form".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added in both en.json and de.json. German uses generisches Maskulinum per
project standards. tformat-compatible {retailer} placeholder in at_retailer key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pure CSS bar chart (div heights via inline %). Stats computed server-side in SQL.
Days filter (7d/30d/90d). Estimated revenue shown as rough indicator (~3% CR × €80).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old script wrote blob json.gz seeds; staging models now only read jsonl.gz.
Seeds are empty JSONL gzip files — zero rows, satisfies DuckDB file-not-found check.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
302 redirect (not 301) so every click is tracked. Extracts lang/article_slug
from Referer header best-effort. Rate-limited to 60/min per IP; clicks
above limit still redirect but are not logged to prevent amplification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pure async functions: get_product(), get_products_by_category(), log_click(),
hash_ip() with daily-rotating GDPR salt, get_click_stats() with SQL aggregation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`credit_ledger cl` joined with `suppliers s` — both have `id`, so
SQLite raised OperationalError. Qualify as `cl.id` and `cl.supplier_id`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New route GET /admin/pipeline/lineage/schema/<model> — returns JSON
with columns+types (from information_schema for serving models),
row count, upstream and downstream model lists. Validates model
against _DAG to prevent arbitrary table access.
- Precomputes _DOWNSTREAM map at import time from _DAG.
- Lineage template: replaces minimal edge-highlight JS with full UX —
hover triggers schema prefetch + floating tooltip (layer badge, top 4
columns, "+N more" note); click opens 320px slide-in panel showing
row count, full schema table, upstream/downstream dep lists.
Dep items in panel are clickable to navigate between models.
Schema responses are cached client-side to avoid repeat fetches.
Staging/foundation models show "schema in lakehouse.duckdb only".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Propagates the conformed city key (city_slug) from dim_venues through the
full pricing pipeline, eliminating 3 fragile LOWER(TRIM(...)) fuzzy string
joins with deterministic key joins.
Changes (cascading, task-by-task):
- dim_venues: add city_slug computed column (REGEXP_REPLACE slug derivation)
- dim_venue_capacity: join foundation.dim_venues instead of stg_playtomic_venues;
carry city_slug alongside country_code/city
- fct_daily_availability: carry city_slug from dim_venue_capacity
- venue_pricing_benchmarks: carry city_slug from fct_daily_availability;
add to venue_stats GROUP BY and final SELECT/GROUP BY
- city_market_profile: join vpb on city_slug = city_slug (was LOWER(TRIM))
- planner_defaults: add city_slug to city_benchmarks CTE; join on city_slug
- pseo_city_pricing: join city_market_profile on city_slug (was LOWER(TRIM))
- pipeline_routes._DAG: dim_venue_capacity now depends on dim_venues, not stg_playtomic_venues
Result: dim_venues.city_slug → dim_cities.(country_code, city_slug) forms a
fully conformed geographic hierarchy with no fuzzy string comparisons.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a 5th tab to the admin pipeline page showing the full 3-layer
SQLMesh data lineage: 28 models, 35 edges across staging / foundation /
serving swim lanes.
- _DAG: canonical model dependency dict in pipeline_routes.py;
update when models are added/removed
- _classify_layer(): derives layer from name prefix (stg_/dim_fct_/rest)
- _render_lineage_svg(): pure Python SVG generator — 3-column swim lane
layout, bezier edges, color-coded per layer (green/blue/amber),
no external dependencies
- /lineage route: HTMX tab handler
- pipeline_lineage.html: partial with SVG embed + vanilla JS hover
effects (highlight connected edges, dim unrelated)
- pipeline.html: 5th "Lineage" tab button
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
