Commit Graph

579 Commits

Author SHA1 Message Date
Deeman
8c4a4078f9 fix(affiliate): live preview uses dedicated /affiliate/preview endpoint
The form was posting to the save route on every input change (which would
save the product on every keystroke). Added a dedicated POST
/admin/affiliate/preview route that renders the product_card.html partial
from form data without touching the database.

Form now keeps action pointing to the save route; an invisible hx-div
triggers preview-only POSTs via hx-include="#affiliate-form".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:34:07 +01:00
Deeman
5f756a2ba5 editorial: review + improve padel-hall-financing-germany-en (C6)
- Add bridge sentence before Personal Guarantee section — this key topic
  was abrupt without introduction; now connects cleanly from the debt
  structure discussion above

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:34:06 +01:00
Deeman
4ac17af503 merge: affiliate sidebar/nav fixes + dev seed data 2026-02-28 21:32:28 +01:00
Deeman
0984657e72 fix(affiliate): sidebar active state, subnav order, dev seed data
- base_admin.html: add 'affiliate_dashboard' to _section_map so Dashboard
  page stays under the Affiliate section (was falling through to 'overview')
- base_admin.html: sidebar Affiliate link now points to dashboard (first tab)
- base_admin.html: subnav order Dashboard | Products (was Products | Dashboard)
- seed_dev_data.py: add 10 affiliate products (4 rackets, 2 shoes, 1 ball,
  1 grip, 1 bag) + 236 click events spread over 30 days for dashboard charts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:32:20 +01:00
Deeman
73547ec876 editorial: propagate C7 improvements to padel-halle-risiken-de
- Tightened competitive risk advice opener (Rechnen Sie das durch.)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:31:15 +01:00
Deeman
129ca26143 editorial: review + improve padel-hall-investment-risks-en (C7)
- Fixed Even so: colon to em dash (punctuation)
- Tightened Risk 5 advice opener (Model this explicitly.)
- Removed double pronoun in F&B note (before committing)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:29:55 +01:00
Deeman
9ea4ff55fa editorial: propagate C8 improvements to padel-halle-bauen-de
- Lender reference: made active sentence
- Fixed grammar: Ihr persoenlicher Track Record (nominative)
- Added closing thought before Was-erfolgreiche-Bauprojekte section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:27:11 +01:00
Deeman
8a91fc752b editorial: review + improve padel-hall-build-guide-en (C8)
- Tightened Phase 1 intro (removed embedded clause, sharper)
- Nail the concept: simplified phrase
- Lender requirements: passive link sentence made active
- Added two-sentence conclusion to final section (solved problem framing)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:25:19 +01:00
Deeman
4783067c6e editorial: propagate C2 improvements to padel-halle-kosten-de
- Tightened opening sentence (native German equivalent)
- Added Munich/Leipzig rent gap qualifier (vergleichbare Marktsegmente)
- Added bridging transition before Hallenmiete section
- Improved court hire rates opener (Ertragspotenzial folgt Standortlogik)
- Extended OPEX rent note: adjust for Munich/Berlin
- Sharpened lease signal sentence (planbarer Cashflow im Kreditbescheid)
- Expanded lender section intro with insider framing
- Tightened Fazit opening (Richtig aufgesetzt...)
- Updated CTA (Die Zahlen in diesem Artikel sind Ihr Ausgangspunkt)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:22:55 +01:00
Deeman
c1e1f42aad fix(supervisor): redeploy web app when .env.prod.sops changes
All checks were successful
CI / test (push) Successful in 50s
CI / tag (push) Successful in 3s
web_code_changed() only checked web/ and Dockerfile, so secret rotations
(updated RESEND_API_KEY, etc.) didn't trigger a container redeploy.
Added .env.prod.sops to the diff so any committed secret change
automatically causes the new .env to be baked into the containers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602282019
2026-02-28 21:18:26 +01:00
Deeman
ecd1cdd27a editorial: review + improve padel-hall-cost-guide-en (C2)
- Tightened opening sentence and intro paragraph
- Added Munich/Leipzig rent gap qualifier (across comparable market tiers)
- Added bridging transition before Commercial Rent section
- Improved Court Hire Rates section opener for better flow
- Added OPEX note: rent line is mid-tier city calibrated; adjust for Munich/Berlin
- Expanded lender section intro with insider framing
- Sharpened lease signal sentence (converts uncertain future revenue...)
- Fixed cashflow to cash flow
- Strengthened Bottom Line and CTA

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:15:20 +01:00
Deeman
24ec7060b3 merge: affiliate product system (racket/gear editorial cards, click tracking, admin CRUD) 2026-02-28 21:07:03 +01:00
Deeman
5c22ea9780 feat(affiliate): tests, ruff cleanup, CHANGELOG + PROJECT.md (commit 9/9)
- 26 tests in web/tests/test_affiliate.py covering hash_ip determinism,
  daily rotation, product CRUD, bake_product_cards marker replacement,
  click redirect (302 + logged), inactive/unknown 404, multi-retailer
- ruff: fix E741 ambiguous var (l → line in _form_to_product), F401 unused
  import, I001 import sort in admin/routes.py
- CHANGELOG: affiliate product system entry
- PROJECT.md: affiliate system moved to Done, Wirecutter backlog item removed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 21:06:01 +01:00
Deeman
aee3733b49 fix(supervisor+ci): self-restart on deploy, CI creates date-based tags
All checks were successful
CI / test (push) Successful in 48s
CI / tag (push) Successful in 2s
supervisor: after git checkout + uv sync, os.execv replaces the running
process so new code takes effect immediately without a manual systemd
restart. systemd sees the same PID, so the unit stays "active".

ci: changed tag format from v{run_number} to v{YYYYMMDDHHMM}, matching
the supervisor's deploy tag convention. Sequential v<N> tags conflicted
with manual date-based tags causing an infinite redeploy loop.
No more manual tagging needed — CI tags automatically after green tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602282003 v202602282004
2026-02-28 21:02:30 +01:00
Deeman
51d9aab4a0 fix(supervisor): use version-sorted tag list for current_deployed_tag
All checks were successful
CI / test (push) Successful in 48s
CI / tag (push) Successful in 2s
git describe --exact-match returns the first tag alphabetically when multiple
tags point to the same commit. This caused an infinite redeploy loop when
Gitea CI created a sequential tag (v11) on the same commit as our date-based
tag (v202602281745) — v11 < v202602281745 alphabetically but the deploy check
uses version sort where v202602281745 > v11.

Fix: use git tag --points-at HEAD --sort=-version:refname to pick the
highest-version tag at HEAD, matching the sort order of latest_remote_tag().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602281955 v13
2026-02-28 20:55:44 +01:00
Deeman
1fdd2d07a4 feat(affiliate): 10 German equipment review article scaffolds
Topics: bester Schläger, Anfänger, defensiv, Fortgeschrittene, unter 100€,
Bälle, Schuhe, Ausrüstung-Guide, Zubehör, Geschenke. Each includes
[product:slug] and [product-group:category] markers, German headings,
placeholder prose, and <details> FAQ sections. Ready for editorial fill-in.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 20:51:05 +01:00
Deeman
2214d7a58f feat(affiliate): i18n strings — affiliate_cta_buy, disclosure, pros/cons labels
Added in both en.json and de.json. German uses generisches Maskulinum per
project standards. tformat-compatible {retailer} placeholder in at_retailer key.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:52:43 +01:00
Deeman
0f360fd230 feat(affiliate): admin dashboard — click stats, daily bar chart, top products/articles
Pure CSS bar chart (div heights via inline %). Stats computed server-side in SQL.
Days filter (7d/30d/90d). Estimated revenue shown as rough indicator (~3% CR × €80).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:51:15 +01:00
Deeman
85b6aa0d0a fix(seeds): update init_landing_seeds.py to write JSONL format
All checks were successful
CI / test (push) Successful in 48s
CI / tag (push) Successful in 2s
Old script wrote blob json.gz seeds; staging models now only read jsonl.gz.
Seeds are empty JSONL gzip files — zero rows, satisfies DuckDB file-not-found check.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v12
2026-02-28 18:50:51 +01:00
Deeman
bc7e40b531 feat(affiliate): admin CRUD — routes, list/form templates, sidebar entry
Routes: GET/POST affiliate, affiliate/results (HTMX), affiliate/new,
affiliate/<id>/edit, affiliate/<id>/delete, affiliate/<id>/toggle.
Templates: affiliate_products.html (filterable list), affiliate_form.html
(two-column with live preview slot), partials/affiliate_row.html,
partials/affiliate_results.html. Affiliate added to base_admin.html sidebar
and subnav (Products | Dashboard).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:50:25 +01:00
Deeman
ef85d3bb36 feat(affiliate): /go/<slug> click redirect with rate limiting + click logging
302 redirect (not 301) so every click is tracked. Extracts lang/article_slug
from Referer header best-effort. Rate-limited to 60/min per IP; clicks
above limit still redirect but are not logged to prevent amplification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:41:04 +01:00
Deeman
4d45b99cd8 feat(affiliate): product card baking — PRODUCT_RE, bake_product_cards(), templates
Adds [product:slug] and [product-group:category] marker replacement.
Templates: product_card.html (horizontal editorial callout) and
product_group.html (responsive comparison grid). Chained after
bake_scenario_cards() in generate_articles(), preview_article(),
article_new(), article_edit(), and _rebuild_article().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:40:27 +01:00
Deeman
e62aad148b fix(transform): remove blob CTE from stg_population_geonames
All checks were successful
CI / test (push) Successful in 49s
CI / tag (push) Successful in 2s
Server has cities_global.jsonl.gz (JSONL), not cities_global.json.gz (blob).
TigerStyle clean break — removed blob_rows CTE and UNION ALL.
Simplified to a single SELECT directly from read_json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v11 v202602281745
2026-02-28 18:40:15 +01:00
Deeman
b5db9d16b9 feat(affiliate): core affiliate module — product lookup, click logging, stats
Pure async functions: get_product(), get_products_by_category(), log_click(),
hash_ip() with daily-rotating GDPR salt, get_click_stats() with SQL aggregation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:36:31 +01:00
Deeman
2e149fc1db feat(affiliate): migration 0026 — affiliate_products + affiliate_clicks tables
Adds affiliate product catalog and click tracking tables.
UNIQUE(slug, language) mirrors articles schema for multi-language support.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:35:27 +01:00
Deeman
6fb1e990e3 merge: three-tier proxy + daily tenants + staging model cleanup
All checks were successful
CI / test (push) Successful in 48s
CI / tag (push) Successful in 3s
v10
2026-02-28 18:26:50 +01:00
Deeman
6edf8ba65e fix(transform): remove blob fallback CTEs, update tenants glob to daily partition depth
TigerStyle clean break — no backwards-compat shims for old file formats:

- stg_playtomic_{venues,opening_hours,resources}: glob updated from
  */*/tenants.jsonl.gz (2-level, old weekly) to */*/*/tenants.jsonl.gz
  (3-level, new daily YYYY/MM/DD partition); blob tenants.json.gz CTE removed
- stg_playtomic_availability: morning_blob and recheck_blob CTEs removed;
  only JSONL format (availability_*.jsonl.gz) is read going forward

Verified locally: stg_playtomic_venues evaluates to 14231 venues from
2026/02/28/tenants.jsonl.gz with 0 errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 18:26:44 +01:00
Deeman
ed0a578050 add concurrency var
All checks were successful
CI / test (push) Successful in 49s
CI / tag (push) Successful in 3s
v9
2026-02-28 18:20:52 +01:00
Deeman
c1cdeec6be fix(extract): default worker count to 200 when proxies configured
All checks were successful
CI / test (push) Successful in 49s
CI / tag (push) Successful in 3s
Previously fell back to len(tiers[0]) (e.g. 10 for Webshare) when
PROXY_CONCURRENCY was unset. Default is now MAX_PROXY_CONCURRENCY=200
so single-URL rotating proxies (DC/residential) run at full concurrency
without needing an explicit env var.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602281706 v8
2026-02-28 18:06:55 +01:00
Deeman
710624f417 fix(supervisor): re-decrypt .env.prod.sops on tag deploy
All checks were successful
CI / test (push) Successful in 49s
CI / tag (push) Successful in 3s
git_pull_and_sync() was missing the sops decrypt step, so .env on the
server was never updated when secrets changed. Now decrypts after
checkout, before uv sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602281657 v7
2026-02-28 17:57:32 +01:00
Deeman
6cf98f44d4 fix(transform): remove blob compat CTE from stg_tennis_courts
All checks were successful
CI / test (push) Successful in 49s
CI / tag (push) Successful in 3s
The overpass_tennis extractor has written JSONL-only since it was added.
The dual-format UNION ALL was backwards-compat debt that broke the
transform once no courts.json.gz files exist on the server:

  IO Error: No files found that match the pattern
  "data/landing/overpass_tennis/*/*/courts.json.gz"

Remove blob_elements CTE and the UNION ALL. Only read JSONL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602281640 v6
2026-02-28 17:39:11 +01:00
Deeman
60659a5ec5 merge: daily tenant snapshots with date-based partition 2026-02-28 17:30:33 +01:00
Deeman
beb4195f16 feat(extract): daily tenant snapshots with date-based partition
- playtomic_tenants: partition by YYYY/MM/DD instead of ISO week;
  schedule changed from weekly to daily in workflows.toml
- playtomic_availability: _load_tenant_ids now tries 3-level glob
  (*/*/*/tenants.jsonl.gz) first for daily files, falls back to
  2-level for old monthly/weekly data

Alphabetical sort would rank old monthly files above daily ones
('t' > '2' in ASCII), so the explicit fallback chain is required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 17:27:16 +01:00
Deeman
88cc857f3a merge: weekly tenant snapshots via ISO week partition 2026-02-28 17:19:25 +01:00
Deeman
9116625884 feat(extract): weekly tenant snapshots via ISO week partition
Tenants extractor now partitions by ISO week (e.g. 2026/W09) instead of
month (2026/02), so each weekly run writes a fresh file rather than
skipping for the rest of the month.

_load_tenant_ids() in playtomic_availability already globs */*/tenants.jsonl.gz
and sorts reverse — 'W09' > '02' alphabetically so weekly files take priority
over old monthly ones automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 17:19:19 +01:00
Deeman
1af65bb46f feat(extract): add PROXY_CONCURRENCY override for rotating single-URL proxies
When DC/residential tiers have a single rotating endpoint, worker_count
defaulted to 1 (one URL = one worker). PROXY_CONCURRENCY lets you set
an explicit thread count (e.g. 100) for providers that handle concurrent
connections on a single URL.

Capped at MAX_PROXY_CONCURRENCY=200 to avoid overloading the endpoint.
Falls back to len(tiers[0]) when unset (existing behaviour).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 17:06:53 +01:00
Deeman
9b0bfc478d merge: three-tier proxy system with Webshare auto-fetch 2026-02-28 17:00:10 +01:00
Deeman
adf22924f6 feat(extract): three-tier proxy system with Webshare auto-fetch
Replace two-tier proxy setup (PROXY_URLS / PROXY_URLS_FALLBACK) with
N-tier escalation: free → datacenter → residential.

- proxy.py: fetch_webshare_proxies() auto-fetches the Webshare download
  API on each run (no more stale manually-copied lists). load_proxy_tiers()
  assembles tiers from WEBSHARE_DOWNLOAD_URL, PROXY_URLS_DATACENTER,
  PROXY_URLS_RESIDENTIAL. make_tiered_cycler() generalised to list[list[str]]
  with N-level escalation; is_fallback_active() replaced by is_exhausted().
  Old load_proxy_urls() / load_fallback_proxy_urls() deleted.

- playtomic_availability.py: both extract() and extract_recheck() use
  load_proxy_tiers() + generalised cycler. _fetch_venues_parallel fallback_urls
  param removed. All is_fallback_active() checks → is_exhausted().

- playtomic_tenants.py: flattens tiers for simple round-robin.

- test_supervisor.py: TestLoadProxyUrls removed (function deleted).
  Added TestFetchWebshareProxies, TestLoadProxyTiers, TestTieredCyclerNTier
  (11 tests covering parse format, error handling, escalation, thread safety).

47 tests pass, ruff clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 16:57:07 +01:00
Deeman
09665b7786 update proxies 2026-02-28 16:51:40 +01:00
Deeman
93349923bd merge(better-alerts): improve supervisor alert messages 2026-02-28 12:27:14 +01:00
Deeman
642041b32b fix(supervisor): improve alert messages with category prefix and error snippet
Each alert now includes a neutral category tag ([extract], [transform],
[export], [deploy], [supervisor]) and the first line of the error, so
notifications are actionable without revealing tech stack details on the
public free ntfy tier.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:27:11 +01:00
Deeman
bb70a5372b docs: replace GitLab CI/CD section with Gitea pull-based deployment
All checks were successful
CI / test (push) Successful in 48s
CI / tag (push) Successful in 3s
Remove outdated SSH-push model referencing GitLab variables. Document
the actual pull-based flow: Gitea Actions → tag → supervisor polls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v5
2026-02-28 01:58:11 +01:00
Deeman
bc28d93662 fix: remove duplicate age key in .sops.yaml
All checks were successful
CI / test (push) Successful in 47s
CI / tag (push) Successful in 3s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v202602271834 v4
2026-02-27 18:30:31 +01:00
Deeman
81ce1d277a update key
Some checks failed
CI / test (push) Has been cancelled
CI / tag (push) Has been cancelled
2026-02-27 18:26:14 +01:00
Deeman
2012894eeb chore: migrate from GitLab to self-hosted Gitea
Some checks failed
CI / test (push) Has been cancelled
CI / tag (push) Has been cancelled
Update bootstrap_supervisor.sh and setup_server.sh to use
git.padelnomics.io:2222 instead of gitlab.com.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 18:23:20 +01:00
Deeman
143ad28854 fix(supervisor): use sqlmesh plan --auto-apply instead of run
Some checks failed
CI / test (push) Has been cancelled
CI / tag (push) Has been cancelled
'run' requires the prod environment to already exist. 'plan --auto-apply'
initializes the environment on first run and applies pending changes on
subsequent runs — fully self-healing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v1
2026-02-27 15:40:37 +01:00
Deeman
415d28afa9 fix(supervisor): run sqlmesh against prod environment
Without the 'prod' argument sqlmesh defaults to dev_<username>, which
doesn't exist on the server (padelnomics_service user).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:39:55 +01:00
Deeman
66d7cdea21 update 2026-02-27 15:39:39 +01:00
Deeman
9c2bf51c73 fix(infra): chown -R APP_DIR so service user owns full tree
Without -R, a manual uv sync or git operation run as root would create
files under /opt/padelnomics owned by root, breaking uv for the service
user (Permission denied on .venv/bin/python3).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-27 15:23:12 +01:00
Deeman
7e0b06a2ad prototype 2026-02-27 14:03:40 +01:00