b899bcbad4
Split the single lakehouse.duckdb into two files to eliminate the exclusive write-lock conflict between SQLMesh (pipeline) and the Quart web app (reader): lakehouse.duckdb — SQLMesh exclusive (all pipeline layers) serving.duckdb — web app reads (serving tables only, atomically swapped) Changes: web/src/beanflows/analytics.py - Replace persistent global _conn with per-thread connections (threading.local) - Add _get_conn(): opens read_only=True on first call per thread, reopens automatically on inode change (~1μs os.stat) to pick up atomic file swaps - Switch env var from DUCKDB_PATH → SERVING_DUCKDB_PATH - Add module docstring documenting architecture + DuckLake migration path web/src/beanflows/app.py - Startup check: use SERVING_DUCKDB_PATH - Health check: use _db_path instead of _conn src/materia/export_serving.py (new) - Reads all serving.* tables from lakehouse.duckdb (read_only) - Writes to serving_new.duckdb, then os.rename → serving.duckdb (atomic) - ~50 lines; runs after each SQLMesh transform src/materia/pipelines.py - Add export_serving pipeline entry (uv run python -c ...) infra/supervisor/supervisor.sh - Add SERVING_DUCKDB_PATH env var comment - Add export step: uv run materia pipeline run export_serving infra/supervisor/materia-supervisor.service - Add Environment=SERVING_DUCKDB_PATH=/data/materia/serving.duckdb infra/bootstrap_supervisor.sh - Add SERVING_DUCKDB_PATH to .env template web/.env.example + web/docker-compose.yml - Document both env vars; switch web service to SERVING_DUCKDB_PATH web/src/beanflows/dashboard/templates/settings.html - Minor settings page fix from prior session Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
39 lines
939 B
Plaintext
39 lines
939 B
Plaintext
# App
|
|
APP_NAME=BeanFlows
|
|
SECRET_KEY=change-me-generate-a-real-secret
|
|
BASE_URL=http://localhost:5001
|
|
DEBUG=true
|
|
ADMIN_EMAILS=admin@beanflows.coffee
|
|
|
|
# Database
|
|
DATABASE_PATH=data/app.db
|
|
# DUCKDB_PATH points to the full pipeline DB (lakehouse.duckdb) — used by SQLMesh and export_serving.
|
|
# SERVING_DUCKDB_PATH points to the serving-only export (serving.duckdb) — used by the web app.
|
|
# Run `uv run materia pipeline run export_serving` after each SQLMesh transform to populate it.
|
|
DUCKDB_PATH=../local.duckdb
|
|
SERVING_DUCKDB_PATH=../serving.duckdb
|
|
|
|
# Auth
|
|
MAGIC_LINK_EXPIRY_MINUTES=15
|
|
SESSION_LIFETIME_DAYS=30
|
|
|
|
# Email (Resend)
|
|
RESEND_API_KEY=
|
|
EMAIL_FROM=hello@example.com
|
|
|
|
|
|
# Paddle
|
|
PADDLE_API_KEY=
|
|
PADDLE_WEBHOOK_SECRET=
|
|
PADDLE_PRICE_STARTER=
|
|
PADDLE_PRICE_PRO=
|
|
|
|
|
|
# Rate limiting
|
|
RATE_LIMIT_REQUESTS=100
|
|
RATE_LIMIT_WINDOW=60
|
|
|
|
# Waitlist (set to true to enable waitlist gate on /auth/signup)
|
|
WAITLIST_MODE=false
|
|
RESEND_AUDIENCE_WAITLIST=
|