"""Pipeline execution via local subprocess.""" import subprocess from dataclasses import dataclass @dataclass class PipelineResult: success: bool output: str error: str | None = None PIPELINES = { "extract": { "command": ["uv", "run", "--package", "psdonline", "extract_psd"], "timeout_seconds": 1800, }, "extract_cot": { "command": ["uv", "run", "--package", "cftc_cot", "extract_cot"], "timeout_seconds": 1800, }, "transform": { "command": ["uv", "run", "--package", "sqlmesh_materia", "sqlmesh", "-p", "transform/sqlmesh_materia", "plan", "prod", "--no-prompts", "--auto-apply"], "timeout_seconds": 3600, }, } def run_pipeline(pipeline_name: str) -> PipelineResult: assert pipeline_name, "pipeline_name must not be empty" if pipeline_name not in PIPELINES: return PipelineResult( success=False, output="", error=f"Unknown pipeline: {pipeline_name}. Available: {', '.join(PIPELINES.keys())}", ) pipeline = PIPELINES[pipeline_name] timeout_seconds = pipeline["timeout_seconds"] try: result = subprocess.run( pipeline["command"], capture_output=True, text=True, timeout=timeout_seconds, ) return PipelineResult( success=result.returncode == 0, output=result.stdout, error=result.stderr if result.returncode != 0 else None, ) except subprocess.TimeoutExpired: return PipelineResult( success=False, output="", error=f"Pipeline '{pipeline_name}' timed out after {timeout_seconds} seconds", )