Update SQLMesh for R2 data access & Convert psd data to gzip
This commit is contained in:
24
CLAUDE.md
24
CLAUDE.md
@@ -8,9 +8,25 @@ Materia is a commodity data analytics platform built on a modern data engineerin
|
||||
|
||||
**Tech Stack:**
|
||||
- Python 3.13 with `uv` package manager
|
||||
- SQLMesh for SQL transformation and orchestration
|
||||
- DuckDB as the analytical database
|
||||
- Workspace structure with separate extract and transform packages
|
||||
Build for simplicity and performance. Data oriented design.
|
||||
Data Platform:
|
||||
- duckdb
|
||||
- cloudflare r2 iceberg catalog
|
||||
- sqlmesh
|
||||
- ephemeral workers on hetzner/oracle/ovh
|
||||
- custom simple scheduler cli
|
||||
|
||||
Saas
|
||||
- Quart framework - async flask api implementation
|
||||
- No Js frameworks
|
||||
- As simple as possible, as performant as possible
|
||||
- DaisyUI
|
||||
- htmx if interactivity needed (use sparingly)
|
||||
- embedded dashboards with evidence.dev, static site BI generator
|
||||
- Paddle as payment provider
|
||||
|
||||
Manager CLI
|
||||
Materia cli is used to manage the whole project, like running pipelines, starting servers etc. and will be continuously extended.
|
||||
|
||||
## Environment Setup
|
||||
|
||||
@@ -374,3 +390,5 @@ All data is stored in Cloudflare R2 Data Catalog (Apache Iceberg) via REST API:
|
||||
- NEVER hardcode secrets in plaintext
|
||||
- Never add ssh keys to the git repo!
|
||||
- If there is a simpler more direct solution and there is no other tradeoff, always choose the simpler solution
|
||||
- Logo and wordmark of beanflows.coffee are stored in assets/
|
||||
- Architecture change/feature plans of the architecture agent and their progress/status are stored in .claude/plans
|
||||
|
||||
Reference in New Issue
Block a user