# Pilot Documentation Self-hosting, product, operator, integration, and security documentation for Pilot. Canonical host: https://pilot.docs.mindburn.org Access tier: public Generated: 2026-06-05T19:34:06.694Z --- title: "Pilot Architecture" canonical: "https://pilot.docs.mindburn.org/architecture" source: "pilot/docs/architecture.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/architecture.md" section: "architecture" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:95e199c321f16680b6720318e106177b57678bf85ae2d7cb9f723f812aa1d2d2" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot Architecture Pilot is a self-hostable founder operating system built around a gateway, orchestrator, workspace memory, founder workflow engines, connector surfaces, Scrapling-backed ingestion, and a HELM governance boundary for consequential actions. ## Audience Use this page if you are a developer, operator, or evaluator who needs to understand the public architecture before self-hosting or extending Pilot. It avoids internal planning language and focuses on the current service boundaries. ## Outcome After this page you should be able to: - identify the gateway, web UI, orchestrator, database, job queue, connectors, ingestion runtime, and HELM sidecar; - explain how founder workflows move through tasks, operators, approvals, and audit records; - understand where Scrapling and browser session capture fit; - know which actions should produce receipts or denials; - choose the right reference page for APIs, environment variables, security, or integrations. ## System Map ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Founder["Founder or operator"] Web["Web UI and Telegram surfaces"] Auth["Auth and workspace context"] Orchestrator["Orchestrator and agent loop"] Jobs["pg-boss jobs"] Engines["Discover, Decide, Build, Launch, Apply engines"] Helm["HELM governance boundary"] Connectors["Connectors and OAuth/session grants"] IngestionRuntime["Scrapling Python runtime"] Postgres["PostgreSQL + pgvector"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Gateway["Gateway API"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Receipts["evidence_packs and audit events"] end %% Operational Flow Edges Founder --> Web Web --> Gateway Gateway --> Auth Gateway --> Orchestrator Orchestrator --> Jobs Orchestrator --> Engines Orchestrator --> Helm Orchestrator --> Connectors Orchestrator --> IngestionRuntime Gateway --> Postgres Jobs --> Postgres Engines --> Postgres Helm --> Receipts Receipts --> Postgres %% Premium Styling Rules style Gateway fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Receipts fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Source Truth Architecture claims are backed by: - `services/gateway/` - `services/orchestrator/` - `services/launch-engine/` - `packages/db/src/schema/` - `packages/helm-client/` - `packages/connectors/` - `pipelines/requirements.txt` - `scripts/install-python-runtime.sh` - `docs/helm-integration.md` - `docs/security.md` If code, schema, or deployment files disagree with this page, the source artifact wins. ## Gateway The gateway is the public HTTP boundary for auth, workspaces, tasks, connectors, governance status, launch surfaces, YC ingestion routes, audit, and health. It owns request authentication, workspace context, rate limiting, secure headers, and route-level validation. ## Orchestrator The orchestrator runs founder tasks and agent loops. It receives workspace context, model configuration, policy configuration, and connector grants. It is where Pilot turns a founder goal into an executable plan, tool calls, approvals, artifacts, and audit entries. ## Memory And Truth PostgreSQL with pgvector is the durable system of record. It stores workspaces, members, tasks, operators, knowledge, applications, connector grants, audit events, approvals, task runs, evidence packs, launch records, and ingestion metadata. pgvector supports knowledge and opportunity search; keyword search remains available when embedding providers are absent. ## Engines Pilot's founder-facing modes are product journeys, not separate products: | Mode | Output Shape | | --- | --- | | Discover | opportunities, research notes, knowledge pages, scored signals | | Decide | decision artifacts, comparison tables, evidence-backed recommendations | | Build | tasks, specs, implementation plans, generated artifacts | | Launch | deploy targets, launch checklists, support-bot workflows | | Apply | application drafts, program tracking, review artifacts | ## Scrapling Ingestion Scrapling-backed ingestion runs through a local Python runtime. It supports public YC ingestion, session-backed founder-authorized syncs, replay of stored captures, and adaptive selector storage. Browser session material is encrypted and treated as sensitive operator data; public docs should explain the mechanism without exposing private session content. ## HELM Governance Boundary HELM is the policy and receipt boundary for LLM inference and non-trivial external actions in production. Pilot's local trust boundary can pre-check kill switches, budgets, blocklists, connector scope, and approval requirements, but production allows should be backed by HELM decisions where configured. ## Troubleshooting | Symptom | Likely Cause | Fix | | --- | --- | --- | | architecture page disagrees with routes | service changed without docs update | check `services/gateway/src/routes` and update docs | | task runs but no audit event appears | orchestrator path bypassed persistence | inspect `task_runs`, `audit_events`, and route wiring | | ingestion works locally but not in Docker | Python/browser runtime mismatch | verify `PYTHON_BIN`, Playwright, and Patchright paths | | HELM receipts are absent | sidecar or `HelmClient` is not configured | check `HELM_GOVERNANCE_URL` and startup logs | --- title: "Pilot Backend and Protocols" canonical: "https://pilot.docs.mindburn.org/backend" source: "pilot/docs/backend.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/backend.md" section: "backend" access: "public" sensitivity: "public" last_reviewed: "2026-05-18" checksum_sha256: "sha256:cc58d87a6547a93110c623efeccd75709ebc9c5172b13b05f6636d99ed448236" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot Backend and Protocols This page is the public backend map for Pilot. It keeps implementation-facing readers oriented without duplicating the API reference or internal source-owner READMEs. ## Source truth - HTTP API behavior: `docs/api.md` and `services/gateway/src/routes/`. - Orchestration behavior: `services/orchestrator/README.md` and `services/orchestrator/src/`. - Shared contracts and sanitizers: `packages/shared/README.md`. - Database domains and migrations: `packages/db/README.md` and `packages/db/migrations/README.md`. - HELM integration: `docs/helm-integration.md` and `packages/helm-client/README.md`. - Connector behavior: `docs/integrations.md` and `packages/connectors/README.md`. ## Backend boundary The gateway owns HTTP ingress, authentication, workspace context, rate limiting, and route-level validation. The orchestrator owns task execution, agent loop coordination, tool mediation, and governance handoff. Database schema docs are source maps, not product promises; capability promotion remains governed by the shared capability registry and production-readiness evidence. ## Request path ```mermaid sequenceDiagram participant User participant Gateway participant DB as PostgreSQL participant Orchestrator participant HELM User->>Gateway: Authenticated API request Gateway->>DB: Resolve workspace and policy context Gateway->>Orchestrator: Dispatch governed task when needed Orchestrator->>HELM: Evaluate consequential action when configured HELM-->>Orchestrator: Decision and receipt Orchestrator-->>DB: Persist task, audit, and evidence state Gateway-->>User: Response or status reference ``` ## Validation Run `npm run docs:coverage && npm run docs:truth` before changing backend docs. Use the route tests in `services/gateway` and orchestrator tests for behavioral claims. --- title: "Pilot Changelog" canonical: "https://pilot.docs.mindburn.org/changelog" source: "pilot/CHANGELOG.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/CHANGELOG.md" section: "changelog" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:a6eb012ec456480f8059ce51bc2dac7427ee889926afa88a6b777325419960fa" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Changelog All notable public changes to Pilot are documented here. Entries focus on self-hosting, APIs, governance, founder workflows, connectors, security, and documentation impact. ## [1.3.0] - 2026-04-24 ### Added - Long-running autonomous execution checkpoint support for task runs, including checkpoint state and stalled-run detection. - Cost attribution dashboard material for spend, cache savings, monthly burn projection, cache hit rate, and per-workspace or per-operator views. - Skills marketplace client for installing verified skill archives into a local Pilot skill directory. - Ollama provider support for local/self-hosted inference when no cloud provider key is configured. ### Changed - Pilot version updated to `1.3.0`. - LLM configuration supports self-hosted Ollama endpoint and model settings. - Environment examples include self-hosted inference and skills registry configuration. ## [1.2.1] - 2026-04-24 ### Fixed - A2A task dispatch now routes into the governed orchestrator instead of returning canned responses. - Evidence-pack writes run L1 conformance validation. - Tool output sanitization now runs before untrusted tool results return to the model context. ### Added - Subagent certification CLI for validating evidence packs. - A2A gateway tests, connector unit tests, and sanitizer tests. - Operator-ready environment examples for A2A and optional PDF parsing. ## [1.2.0] - 2026-04-20 ### Added - Connector breadth across Slack, Notion, Linear, Stripe read-only, Google Calendar, and HubSpot. - Vision and PDF ingestion helpers with `parse_pdf` and `analyze_image` tools. - Agent2Agent discovery and JSON-RPC endpoint with bearer authentication. - L1/L2 conformance validators for receipt and evidence-pack checking. ### Changed - Built-in orchestrator tools expanded from 27 to 47. - Default registered connectors expanded from 8 to 13. ## [1.1.0] - 2026-04-19 ### Added - MCP consumer and provider support, including a Pilot MCP server for DB-backed tools. - Compliance framework overlays, attestations, and governance UI hooks. - Sandbox abstraction for generated-code execution checks. - Optional Langfuse and Braintrust observability hooks. - Skill loader for `SKILL.md` based operator behavior. - HELM endpoint integration, budget/cost surfaces, and proof graph support. - Threat scanning and sanitizer tests for untrusted scraping output. - Anthropic prompt-caching telemetry. - Live conduct streaming through server-sent events. ## [1.0.0] - 2026-04-19 ### Added - Connector token refresh infrastructure. - CI/CD hardening with SBOM, provenance, vulnerability scanning, secret scanning, and restore drill workflow. - Observability alerts, dashboards, Sentry/OpenTelemetry documentation, and cost/governance views. - DigitalOcean production deployment path with Pilot, HELM sidecar, PostgreSQL, Caddy, and backup scheduling. - Web dashboard mode pages, governance DAG, Telegram command routing, E2E tests, and load-test baseline. ## [0.1.0] - 2026-04-16 ### Added - HELM governance sidecar integration. - Multi-tenant workspace isolation with encrypted secret storage. - Scoring, semantic deduplication, clustering, decision court, build, launch, and apply workflows. - Magic-link and Telegram authentication. - DigitalOcean deployment configuration and Docker build. --- title: "Pilot Frontend and Console" canonical: "https://pilot.docs.mindburn.org/frontend" source: "pilot/docs/frontend.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/frontend.md" section: "frontend" access: "public" sensitivity: "public" last_reviewed: "2026-05-18" checksum_sha256: "sha256:63194d3ed7b4ae44fc901ae098f04f71b2af497af1e32f977c093d5a914543b7" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot Frontend and Console This page is the public frontend map for Pilot. It covers user-facing surfaces and points readers to the implementation paths that own behavior. ## Source truth - Web application shell and founder workflows: `apps/web/README.md` and `apps/web/src/app/`. - Telegram Mini App behavior: `docs/telegram-miniapp-8.md` and `apps/telegram-miniapp/README.md`. - Telegram bot behavior: `apps/telegram-bot/README.md`. - Public workflow routes and capability labels: `docs/capabilities.md`, `docs/pilot-capability-matrix.md`, and the shared capability registry. - API payloads used by UI surfaces: `docs/api.md`. ## Frontend boundary Pilot frontend docs may describe available screens, route families, and the state each screen reads. They must not promote prototype or implemented capabilities to production-ready unless the shared registry and evidence-backed evals have already promoted the capability. ## Surface map ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Founder["Founder or operator"] Web["apps/web"] Telegram["Telegram bot and mini app"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Gateway["Gateway API"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] State["Workspace, task, evidence, and capability state"] end %% Operational Flow Edges Founder --> Web Founder --> Telegram Web --> Gateway Telegram --> Gateway Gateway --> State State --> Web %% Premium Styling Rules style Gateway fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style State fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Validation Run `npm run docs:coverage && npm run docs:truth` before changing frontend docs. Run the relevant web or Telegram package checks when a UI claim depends on actual route behavior. --- title: "Scrapling Ingestion" canonical: "https://pilot.docs.mindburn.org/ingestion/scrapling" source: "pilot/docs/ingestion/scrapling-v045.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/ingestion/scrapling-v045.md" section: "architecture" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:edcd682236fbd5efcc74f0db7afbdab313f5621f2c21f44fe972ad1ee4f31d32" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Scrapling 0.4.5 Ingestion Upgrade Status: implemented behind the existing Pilot ingestion boundary. Linear: MIN-254, MIN-245 ## Source Check - Scrapling 0.4.x adds the MCP server and anti-bot/Turnstile-oriented fetch surface used by the Pilot bridge. - Scrapling 0.4.5 changes redirect handling so safe redirects reject loopback, private, and link-local targets by default. - 0.4.5 also adds spider development mode. Pilot exposes it only through explicit development flags and disables the env override in production. Primary sources: - https://github.com/D4Vinci/Scrapling/releases/tag/v0.4.0 - https://github.com/D4Vinci/Scrapling/releases/tag/v0.4.5 - https://scrapling.readthedocs.io/en/latest/ai/mcp-server.html ## Repository Changes - `pipelines/requirements.txt` pins `scrapling[ai]==0.4.5`. - `pipelines/scraper/lib/scrapling_adapter.py` centralizes `follow_redirects="safe"` for fetcher, dynamic, and stealthy paths. - `pipelines/scraper/run_fetch.py` reports safe redirect mode and accepts `--development-mode`. - `pipelines/yc-scraper/scrape_startup_school.py` can use development mode for spider iteration outside production. - `services/orchestrator/src/tools.ts` validates `scrapling_fetch` with a Zod schema before the Python bridge runs. ## MCP Boundary Scrapling MCP exposure must use the existing MCP registry path: 1. Configure Scrapling in `packs/mcp/servers.json` or `MCP_SERVERS_CONFIG_PATH`. 2. Let `McpServerRegistry` instantiate the server. 3. Let `ToolRegistry.registerMcpTools("scrapling", client)` create namespaced tools such as `mcp.scrapling.fetch`. 4. Let `AgentLoop.evaluateToolGovernance()` evaluate every `mcp.scrapling.*` call through `packages/helm-client` before execution. Do not call the Scrapling MCP server directly from services or Telegram handlers. ## Validation Dry-run examples: ```bash PYTHONPATH=pipelines python pipelines/scraper/run_fetch.py \ --url https://www.ycombinator.com/companies \ --strategy fetcher \ --selector title \ --limit 1 PYTHONPATH=pipelines python pipelines/yc-scraper/scrape_startup_school.py \ --limit 2 \ --dry-run \ --development-mode ``` Live YC validation should be run with a short `--limit` first, then a scheduled crawl after the Python runtime is rebuilt with `scripts/install-python-runtime.sh`. ## Public Operator Checklist A public Scrapling ingestion claim is complete only when it names the source type, capture boundary, replay behavior, redaction rule, and validation command. Keep private session-backed captures out of anonymous exports; public docs should describe deterministic parsing, operator-triggered replay, error capture, and evidence metadata without exposing cookies, private YC session state, or raw browser storage. When the parser changes, update the fixture or replay example first, then update this page and the public manifest. ## Expected Output For a successful public ingestion run, the operator should see a queued or completed ingestion record, a source label, count metadata, replay counters when the replay migration is present, and a redacted error field when parsing fails. For a failed run, collect the source, replay reference, parser version, sanitized payload shape, and worker logs. Public examples should include a local command, a queued job or ingestion row, a replay reference, and a sanitized parser result. The docs must also say how to recover from partial capture, parser drift, duplicate replay, rate limiting, and unavailable upstream pages. Operators should validate that failed captures remain inspectable without exposing protected session state. ## Boundary Do not include raw session cookies, connector tokens, private page bodies, or founder-specific application material in public examples. If upstream layout drift changes extraction, update the parser test and explain the operator action: rerun capture, replay stored input, or mark the record stale. ## Troubleshooting If ingestion is queued but never completes, check worker availability, rate limits, source reachability, and database migrations before changing docs. If replay produces different output, preserve the original capture metadata, compare parser version, and document the drift as a deterministic replay finding rather than a new public claim. --- title: "Integrations" canonical: "https://pilot.docs.mindburn.org/integrations" source: "pilot/docs/integrations.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/integrations.md" section: "integrations" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:efcea73c820dfceead539a741861723c635de8bad1ea177b503283877a964700" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Integrations Pilot integrations connect founder workflows to Telegram, Telegram Mini App, OAuth providers, session-auth connectors, Scrapling ingestion, and HELM governance. This landing page explains the public integration surface and links to deeper pages. ## Audience Use this page if you are enabling external systems, reviewing connector boundaries, or deciding which integration to configure first. It is written for founder/developers and self-hosting operators. ## Outcome After this page you should know: - which integrations are required for a minimal local install; - how Telegram bot and Mini App surfaces relate to the gateway; - how OAuth and session-auth connector grants are stored; - where YC session capture is safe to discuss publicly; - where HELM fits as a governance boundary rather than a product name. ## Integration Map ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Founder["Founder"] Telegram["Telegram bot"] MiniApp["Telegram Mini App"] Web["Web UI"] OAuth["OAuth connectors"] Session["Session-auth connectors"] Scrapling["Scrapling ingestion"] Secrets["Encrypted token/session storage"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Gateway["Gateway API"] end subgraph Execution["3. Execution & Verdict Plane"] Helm["HELM governed actions"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Receipts["Receipts and audit"] end %% Operational Flow Edges Founder --> Telegram Founder --> MiniApp Founder --> Web Telegram --> Gateway MiniApp --> Gateway Web --> Gateway Gateway --> OAuth Gateway --> Session Gateway --> Scrapling Gateway --> Helm OAuth --> Secrets Session --> Secrets Helm --> Receipts %% Premium Styling Rules style Gateway fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Helm fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff style Receipts fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Source Truth Integration docs are backed by: - `docs/telegram-miniapp-8.md` - `docs/helm-integration.md` - `docs/env-reference.md` - `docs/security.md` - `services/gateway/src/routes/connectors*` - `services/gateway/src/services/managed-telegram-bots.ts` - `packages/connectors/` - `packages/helm-client/` - `docs/ingestion/scrapling-v045.md` If code and docs disagree, update this page and the deeper integration page together. ## Minimal Local Install For the first local run, the only required integration is the database. You can add direct LLM provider keys for local development or configure HELM if you want the production governance shape. Telegram, OAuth connectors, YC session capture, S3 storage, Sentry, and external email are optional until the workflow needs them. ## Telegram Bot Telegram bot support is enabled with `TELEGRAM_BOT_TOKEN` and, in production webhook mode, `TELEGRAM_WEBHOOK_SECRET`. The bot can route commands into Pilot workflows and approval notifications. Managed launch/support bots are created through Telegram Managed Bots and persisted workspace-by-workspace. ## Telegram Mini App The Mini App is served from the gateway and should point BotFather to `https://your-domain.com/app/`. It is useful when founders want a lightweight mobile surface for modes, approvals, launch support, and governed task status. ## OAuth Connectors OAuth connectors such as GitHub, Gmail, and Google Drive use provider-specific client IDs, secrets, callback URLs, and encrypted token storage. Connector grants are workspace-scoped. Production deployments should set explicit redirect URIs and rotate secrets with the security guide. ## Session-Auth Connectors The YC connector uses founder-authorized browser session capture rather than OAuth. Public docs may explain the shape: the founder grants a connector, saves a browser storage-state snapshot, validates it, and Pilot stores it encrypted at rest. Public docs should not expose private session data, matching results, or personal material. ## Scrapling Ingestion Scrapling runs in a pinned Python runtime with Playwright and Patchright browser caches. Use it for public ingestion, replay of stored captures, and session-backed syncs when the connector is valid. ## HELM Governance HELM governs non-trivial external actions and LLM inference in production. Use HELM Integration (protected staff doc) for sidecar, receipt, fail-closed, and environment details. ## Troubleshooting | Symptom | Likely Cause | Fix | | --- | --- | --- | | Telegram webhook returns unauthorized | secret token mismatch | set Telegram webhook secret and `TELEGRAM_WEBHOOK_SECRET` to the same value | | OAuth callback fails | redirect URI mismatch | register the exact callback URL from env reference | | connector says granted but cannot act | token or session validation failed | revalidate grant and inspect encrypted storage health | | Scrapling fetch fails | browser runtime missing | rerun `scripts/install-python-runtime.sh` | | governed action is denied | HELM policy or approval gate blocked it | inspect receipt, reason code, and policy bundle | --- title: "Degradation Matrix" canonical: "https://pilot.docs.mindburn.org/operations/degradation-matrix" source: "pilot/docs/degradation-matrix.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/degradation-matrix.md" section: "self-hosting" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:909069eea873cdea6cb85a4df3e9d725e305c3396c9cecdf45e7bcf1d697fe7d" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Graceful Degradation Matrix Pilot is designed so that missing optional services cause specific, bounded feature loss rather than cascading failure. This document lists every optional env var and what happens when it is absent. | Env Var | Unset behavior | Features affected | | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | `HELM_GOVERNANCE_URL` / `HELM_FAIL_CLOSED` | Production requires HELM. If HELM is unreachable with fail-closed enabled, `/health` degrades and governed LLM calls are denied. | Agent runs and founder-intel paths that need governed inference. | | `OPENROUTER_API_KEY` / `ANTHROPIC_API_KEY` / `OPENAI_API_KEY` / `OLLAMA_BASE_URL` | Direct-provider fallback only when HELM is not configured. If none, agent loop returns `completed` immediately with `No LLM configured`. | Local/dev agent runs, founder-intel extraction, knowledge truth compilation, opportunity scoring. | | `OPENAI_API_KEY` / `VOYAGE_API_KEY` | Embedding provider falls back to deterministic hash pseudo-embedding (not semantically meaningful). | Vector search returns low-quality results. Keyword search still works. | | `TELEGRAM_BOT_TOKEN` | Telegram bot is not started. | `/api/telegram/webhook` not mounted. Mini-app still served. Approval notifications disabled. | | `TELEGRAM_WEBHOOK_SECRET` | Webhook secret validation skipped (dev) / rejected (prod — set it). | Bot webhook security. | | `TELEGRAM_MANAGER_BOT_USERNAME` | Manager username is resolved from Telegram `getMe` when `TELEGRAM_BOT_TOKEN` is set. | Web-created launch/support bot setup links fail if neither value is available. | | `APP_URL` | Child managed-bot webhook setup fails. | Founder-owned launch/support bots cannot receive customer updates. | | `REDIS_URL` | Optional legacy limiter backend. Main API rate limits use Postgres `ratelimit_buckets`. | Legacy in-memory limiter paths only. | | `SENTRY_DSN` | Error reporting disabled, errors only hit local logs. | No remote error alerting. | | `EMAIL_PROVIDER` (`noop`) | Magic-link emails are not sent; dev-only auth responses return codes while provider logs keep only redacted delivery metadata. **Unsafe for production.** | Users cannot log in unless they are using the local development auth response. | | `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET` | GitHub connector is registered but OAuth initiation throws. Manual-token mode still works if tokens are supplied directly. | OAuth self-serve for GitHub. | | `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` | Same as above for Gmail + Drive. | OAuth self-serve for Google. | | `MICROSOFT_CLIENT_ID` / `MICROSOFT_CLIENT_SECRET` | Same as above for Outlook. | OAuth self-serve for Outlook. | | `STORAGE_PROVIDER` | Defaults to `local` — files written to `STORAGE_PATH` (default `./data/storage`). | S3-backed storage disabled. | | `COHERE_API_KEY` | Search reranking disabled. | Slightly less relevant opportunity and knowledge ranking. | | `ENCRYPTION_KEY` | **Required in production** (process exits 1). Dev fallback is insecure. | Connector token encryption at rest. | | `SESSION_SECRET` | Defaults to dev value. **Unsafe for production.** | Session token + OAuth state HMAC. | | `ENABLED_CONNECTORS` | All connectors register with warnings if credentials missing. | Strict startup validation disabled. | | `RUN_MIGRATIONS_ON_STARTUP` | Defaults to `true`. DigitalOcean production sets `false` and runs migrations explicitly during deploy. | Schema drift risk if set false and migrations forgotten. | | `DAILY_BUDGET_MAX` / `PER_TASK_BUDGET_MAX` | Defaults to 500 / 100 (EUR). | Cost enforcement tightness. | | `DB_POOL_MAX` / `DB_IDLE_TIMEOUT` / etc. | Sensible defaults applied. | Pool tuning only matters under high concurrency. | ## Startup refusal conditions (fatal) The process exits non-zero if any of these fail: 1. `DATABASE_URL` is missing. 2. `SESSION_SECRET`, `ENCRYPTION_KEY`, or configured `EVIDENCE_SIGNING_KEY` is missing or still a default development value when `NODE_ENV=production`. 3. `TELEGRAM_BOT_TOKEN` is set in production without `TELEGRAM_WEBHOOK_SECRET`. 4. `drizzle-kit migrate` fails during startup (when `RUN_MIGRATIONS_ON_STARTUP=true`). 5. Any connector listed in `ENABLED_CONNECTORS` has missing credentials (prod only). 6. `EMAIL_PROVIDER=resend` without `RESEND_API_KEY`, or `EMAIL_PROVIDER=smtp` without `SMTP_HOST`. ## Degraded but running The process starts successfully and runs, but specific features are degraded: - **HELM down in production:** `/health` reports degraded and governed inference fails closed. Restore the sidecar before resuming autonomous work. - **No direct LLM keys in dev:** All direct-provider agent operations return immediately. Surface a clear "AI provider required" error in the UI. - **No Telegram:** Telegram surface disappears. Web UI + REST API still fully operational. - **No Redis:** Main API rate limits still use Postgres token buckets. Legacy in-memory limiter paths remain per-process. - **No Sentry:** You'll only see errors in logs. Set up `SENTRY_DSN` before you hit real traffic. - **No email provider:** Dev mode only. Magic link codes appear in logs and HTTP responses. ## Verifying degradation Start the gateway with no optional env vars set and hit `/health`: ```bash curl http://localhost:3100/health ``` Response includes a `checks` object listing what's connected. Use that to confirm which optional services are active. --- title: "Runbook" canonical: "https://pilot.docs.mindburn.org/operations/runbook" source: "pilot/docs/runbook.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/runbook.md" section: "self-hosting" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:d2d0298c2acfe85b0d540780b96ac3ff0c9ab402e39ef12c0023f368a69f970f" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot Runbook On-call response procedures, diagnostic commands, and rollback playbooks for operators running Pilot in production. --- ## 0. Start Here (Every Incident) 1. **Check `/health`** — `curl https:///health`. Anything non-200 is a hard incident. 2. **Check Sentry** — are there recent unhandled errors spiking? 3. **Check Grafana dashboards** — `api.json`, `orchestrator.json`, `infrastructure.json`. 4. **Check DigitalOcean Droplet and Compose status** — `bash infra/digitalocean/deploy.sh status`. 5. Declare severity and communicate on your internal channel. --- ## 1. Common Incidents ### 1A. Auth broken — users cannot log in Symptoms: users report not receiving email magic link, or verify returns 401 despite correct code. - Check `/health` → `checks.db` true? - Check `EMAIL_PROVIDER` config — if it reverted to `noop`, emails are not sent; only local development auth responses expose the code. ```bash ssh root@$DO_DROPLET_IP 'cd /opt/pilot/current && docker compose -f infra/digitalocean/docker-compose.yml exec pilot env | grep EMAIL' ``` - Check Resend/SMTP dashboard for bounces or rate limiting. - Check the `sessions` table — are rows being written? ```sql SELECT COUNT(*), MAX(created_at) FROM sessions WHERE channel = 'email_pending'; ``` - If Resend API is down: temporarily switch to a backup SMTP provider in `.env.production.pilot`, redeploy with `bash infra/digitalocean/deploy.sh deploy`, and verify login. ### 1B. Database down Symptoms: `/health` returns 503 with `checks.db: false`. - DigitalOcean Compose Postgres: `docker compose --env-file .env.production.shared -f infra/digitalocean/docker-compose.yml exec postgres psql -U helm -d pilot -c '\l'`. - Check DB logs: `docker compose --env-file .env.production.shared -f infra/digitalocean/docker-compose.yml logs --tail=200 postgres`. - Connection pool exhausted? Query `SELECT count(*) FROM pg_stat_activity`. If >80 increase `DB_POOL_MAX` or investigate slow queries. - If the DB is truly down, restore the latest verified backup on a replacement Droplet, then point DNS at the new Droplet IP. ### 1C. HELM sidecar or LLM provider outage Symptoms: `/health` returns degraded with `checks.helm: "unreachable"`, or agent tasks fail with HELM unreachability / upstream provider errors. - Check HELM first: `docker compose --env-file .env.production.shared -f infra/digitalocean/docker-compose.yml logs --tail=200 helm` and `docker compose --env-file .env.production.shared -f infra/digitalocean/docker-compose.yml exec helm wget -qO- http://localhost:8081/healthz`. - Check provider status pages for the upstream configured on HELM (`HELM_UPSTREAM_URL`). In production, provider keys belong on the HELM sidecar, not on the Pilot app. - If HELM is healthy but the upstream is down, rotate the sidecar upstream/key in `.env.production.helm` and redeploy. - Enable kill switch temporarily: set `policy.killSwitch=true` via a workspace-settings update. Tasks will be blocked rather than retry-storming. ### 1D. Rate-limiting users unexpectedly Symptoms: users report 429 on normal usage. - Check Postgres token buckets in `ratelimit_buckets` for the reported subject and route class. - If a single bad IP is hammering, block it with Cloudflare, Caddy, or the DigitalOcean Cloud Firewall before changing app limits. - Bump limits temporarily via code — update `services/gateway/src/index.ts` rate-limit configs and ship. ### 1E. Disk full / storage exhausted - Check Droplet and Docker volume usage with `df -h` and `docker system df`. - Rotate old Pino logs (if writing to disk): `find /app/logs -mtime +7 -delete`. - Clean old S3 backups via `scripts/backup.sh` retention (default 30 days). - If `STORAGE_PROVIDER=local`, migrate to S3. ### 1F. Task stuck in 'running' status Symptoms: a task shows status='running' for hours. - Expected: the reaper job should move it to 'failed' after 10 minutes (`tasks.reap_stuck` scheduled every 5 min). - If not happening, check pg-boss health: `SELECT * FROM pgboss.schedule WHERE name = 'tasks.reap_stuck'`. - Manual reap: `UPDATE tasks SET status='failed' WHERE id='' AND status='running'`. ### 1G. YC private sync / Scrapling runtime broken Symptoms: YC connector validates in UI but private sync jobs fail, or operator fetches return browser/runtime errors. - Check the configured Python runtime: ```bash PYTHON_BIN=${PYTHON_BIN:-./.venv-pipelines/bin/python} $PYTHON_BIN scripts/verify-python-runtime.py ``` - If browser binaries are missing, re-run: ```bash bash scripts/install-python-runtime.sh ``` - Verify `ENCRYPTION_KEY` did not rotate without a connector/session migration plan. - Confirm the `yc` connector still shows a validated session in the workspace settings UI. - If validation fails after a YC auth change, capture and save a fresh session snapshot. --- ## 2. Diagnostic Commands ### Health ```bash curl https:///health | jq # full health JSON curl https:///metrics | head -50 # Prometheus metrics sample ``` ### Logs (DigitalOcean Compose) ```bash ssh root@$DO_DROPLET_IP cd /opt/pilot/current docker compose -f infra/digitalocean/docker-compose.yml logs --tail=200 pilot docker compose -f infra/digitalocean/docker-compose.yml logs pilot | grep ERROR docker compose -f infra/digitalocean/docker-compose.yml logs pilot | grep requestId=XXX ``` ### Database (DigitalOcean Compose Postgres) ```bash docker compose -f infra/digitalocean/docker-compose.yml exec postgres psql -U helm -d pilot # Most active tables SELECT schemaname, relname, n_live_tup FROM pg_stat_user_tables ORDER BY n_live_tup DESC LIMIT 10; # Pool saturation SELECT count(*), state FROM pg_stat_activity GROUP BY state; # Long-running queries SELECT pid, now() - pg_stat_activity.query_start AS duration, query FROM pg_stat_activity WHERE state = 'active' AND now() - pg_stat_activity.query_start > interval '30 seconds'; ``` ### pg-boss queue ```sql SELECT name, state, COUNT(*) FROM pgboss.job GROUP BY name, state; SELECT * FROM pgboss.archive ORDER BY completed_on DESC LIMIT 20; -- recently completed ``` ### Python / Scrapling ```bash PYTHON_BIN=${PYTHON_BIN:-./.venv-pipelines/bin/python} $PYTHON_BIN scripts/verify-python-runtime.py ls -la ${PLAYWRIGHT_BROWSERS_PATH:-./.cache/ms-playwright} ls -la ${PATCHRIGHT_BROWSERS_PATH:-./.cache/ms-patchright} ``` --- ## 3. Rollback Procedure ### 3A. Application rollback 1. Find the last known-good release: ```bash git log --oneline -20 ``` 2. Roll back to the previous release: ```bash DO_DROPLET_IP= bash infra/digitalocean/deploy.sh rollback ``` 3. Watch health: ```bash while true; do curl -s https:///health | jq -c; sleep 2; done ``` ### 3B. Database rollback (destructive — last resort) 1. Stop the gateway: `docker compose -f infra/digitalocean/docker-compose.yml stop pilot`. 2. Take a snapshot of current DB state. 3. Restore from last known-good backup: ```bash bash scripts/backup.sh restore ``` 4. Start gateway: `docker compose -f infra/digitalocean/docker-compose.yml up -d pilot`. 5. **Warning:** connector tokens encrypted with a since-rotated ENCRYPTION_KEY will be unreadable. Plan carefully. ### 3C. Migration rollback Drizzle migrations are forward-only by default. To roll a migration back: 1. Write a reverse migration SQL file manually (e.g., `0005_revert_xyz.sql` that drops the column added in `0004`). 2. Apply it normally via `npm run db:push`. 3. Redeploy the app pinned to the pre-`0004` schema version. --- ## 4. Escalation - **SEV-1 (data loss, security breach):** Wake the on-call immediately. Preserve state (snapshots, logs) before attempting fixes. - **SEV-2 (service down for all users):** Respond within 15 min. Start incident channel. - **SEV-3 (degraded, subset of users):** Respond within 1h. File ticket, fix during business hours. --- ## 5. Post-Incident Review Template ``` # Incident YYYY-MM-DD — ## Timeline - HH:MM UTC — Detected (how?) - HH:MM UTC — First responder on call - HH:MM UTC — Mitigated (what was done?) - HH:MM UTC — Fully resolved ## Impact - Users affected: / - Features affected: - Data loss: ## Root Cause ## Contributing Factors ## Action Items - [ ] - [ ] - [ ] - [ ] ## Lessons Learned ``` --- title: "Pilot Platform and DevOps" canonical: "https://pilot.docs.mindburn.org/platform" source: "pilot/docs/platform.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/platform.md" section: "platform" access: "public" sensitivity: "public" last_reviewed: "2026-05-18" checksum_sha256: "sha256:3032342368a2458c247a2dba5bac0edcaf4c1686bfbfdde1bbd77306b13f5b7e" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot Platform and DevOps This page is the public platform map for Pilot. It is intentionally narrow: deployment, operations, infrastructure prerequisites, and recovery paths. Product workflow behavior belongs in `docs/architecture.md`, API details belong in `docs/api.md`, and environment variables belong in `docs/env-reference.md`. ## Source truth - Local and production deployment: `docs/self-hosting.md`, `infra/README.md`, `infra/digitalocean/README.md`, and `infra/docker/README.md`. - Runtime operation and incident response: `docs/runbook.md`, `docs/degradation-matrix.md`, and `infra/monitoring/README.md`. - Configuration: `docs/env-reference.md` and `.env.example`. - Migration safety: `packages/db/migrations/README.md` and `scripts/check-migration-journal.ts`. ## Platform boundary Pilot runs as a self-hosted stack: web UI, gateway, orchestrator, PostgreSQL, pgvector, background jobs, optional Telegram surfaces, and optional HELM governance. The platform docs should explain how to run, observe, back up, and upgrade that stack. They should not claim managed hosting, production availability, or customer deployment status unless a release artifact and smoke evidence prove it. ## Public operator path ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] SelfHosting["Self-hosting guide"] Env["Environment reference"] Deploy["Infra and Docker docs"] Degrade["Degradation matrix"] end subgraph Execution["3. Execution & Verdict Plane"] Runbook["Runbook"] Recovery["Recovery or rollback action"] end %% Operational Flow Edges SelfHosting --> Env Env --> Deploy Deploy --> Runbook Runbook --> Degrade Degrade --> Recovery %% Premium Styling Rules style Runbook fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff style Recovery fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff ``` ## Validation Run `npm run docs:coverage && npm run docs:truth` before publishing platform claims. Run `npm run compose:config` when changing Docker or DigitalOcean deployment instructions. --- title: "Pilot Positioning" canonical: "https://pilot.docs.mindburn.org/product" source: "pilot/docs/positioning/pilot-vs-general-purpose-autonomy.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/positioning/pilot-vs-general-purpose-autonomy.md" section: "product" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:1fac09c328c181b6ec43a10d9ab6341ce0b4a153d1de5beb877fc437c8d0fc42" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Product Pilot is a self-hostable founder operating system for discovery, decision, build, launch, and application workflows. It is not positioned as a generic coding agent or broad customer-support agent. The product shape is a governed operating loop for founders who need useful work, evidence, and approval boundaries in the same system. ## Audience Use this page if you are evaluating Pilot as a product, building a founder workflow, writing public copy, or deciding whether a feature belongs in user guidance or operator guidance. ## Outcome After this page you should be able to: - explain the five founder journeys: Discover, Decide, Build, Launch, and Apply; - identify what artifact each journey should produce; - separate founder user guidance from self-hosting operator guidance; - explain why Pilot uses HELM for governance without making HELM part of the product name; - compare Pilot to general-purpose autonomy without drifting into unsupported claims. ## Founder Workflow Map ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Discover["Discover opportunities"] Build["Build tasks and artifacts"] Launch["Launch and support"] Apply["Apply to programs"] Memory["Workspace memory"] end subgraph Execution["3. Execution & Verdict Plane"] Governance["HELM-governed actions"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Decide["Decide with evidence"] end %% Operational Flow Edges Discover --> Decide Decide --> Build Build --> Launch Launch --> Apply Apply --> Memory Memory --> Discover Build --> Governance Launch --> Governance Apply --> Governance %% Premium Styling Rules style Decide fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff style Governance fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff ``` ## Source Truth Product guidance is backed by: - `README.md` - `docs/roadmap.md` - `docs/api.md` - `services/orchestrator/` - `services/launch-engine/` - `services/gateway/src/routes/launch.ts` - `packages/db/src/schema/` - `packs/founder_ops.v1.json` - `docs/helm-integration.md` Product language should stay grounded in working routes, schema, tasks, artifacts, approvals, and receipts. ## Founder Journeys | Journey | Founder Job | Expected Output | | --- | --- | --- | | Discover | find and organize opportunities | scored opportunities, knowledge pages, research notes | | Decide | make a high-stakes choice with evidence | decision artifact, comparison table, recommendation, audit trail | | Build | turn a goal into implementation work | tasks, specs, code or artifact plan, governed commits where configured | | Launch | prepare deployment and go-to-market work | deploy target, launch checklist, support-bot setup, health records | | Apply | draft and manage application material | application record, draft sections, status updates, review history | ## Founder Guidance Vs Operator Guidance Founder guidance should focus on workflows, decisions, artifacts, and approvals. Operator guidance should focus on Docker, Postgres, env vars, backups, security, integrations, and uptime. When a page mixes both, split it or move the operational material into [Self-Hosting](/self-hosting). ## Governance Positioning Pilot uses HELM as the governance boundary for consequential actions. Public product copy should say "Pilot" for the product and "HELM" only when describing policy evaluation, receipts, sidecar deployment, or trust-boundary behavior. ## Contrast With Generic Autonomy | Surface | Pilot | General-purpose autonomy | | --- | --- | --- | | Primary user | founder/operator | broad user or enterprise admin | | Workflow shape | Discover, Decide, Build, Launch, Apply | open-ended task execution | | Deployment | self-hostable stack | often vendor-hosted | | Governance | HELM decisions, approvals, evidence packs | product-specific logs | | Memory | workspace-scoped founder data | runtime-specific memory model | ## Troubleshooting | Symptom | Likely Cause | Fix | | --- | --- | --- | | product copy says too much | claim is not backed by routes or schema | link it to source truth or remove it | | founder docs mention deployment internals | operator content leaked into user guidance | move it to self-hosting | | operator docs describe workflows vaguely | product content is missing expected outputs | add artifacts and success criteria | | HELM appears as product branding | rename work is incomplete | use Pilot for product name and HELM only for governance | --- title: "API Reference" canonical: "https://pilot.docs.mindburn.org/reference" source: "pilot/docs/api.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/api.md" section: "reference" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:497af58abaf099a649e866ff226f4b01d84391764cdb064158d954b094f2b052" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot API Reference Base URL: `http://localhost:3100` (or your deployed gateway URL) ## Audience Use this page if you are writing a client, testing a self-hosted deployment, wiring a connector, or auditing API behavior. It is a public reference hub for routes, auth, workspace context, payload shapes, job surfaces, receipts, and diagnostics. ## Outcome After this page you should be able to: - authenticate with email, Telegram, or API keys; - pass workspace context correctly; - use the public-safe auth, workspace, task, capability, and audit endpoints; - understand where HELM governance receipts appear; - collect useful diagnostics when an API call fails. ## API Surface Map ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Client["Client"] Auth["Auth"] Workspace["Workspace context"] Tasks["Tasks and operators"] Knowledge["Knowledge and applications"] Connectors["Connectors"] Launch["Launch"] Incidents["Incidents"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Audit["Audit and approvals"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Governance["Governance receipts"] end %% Operational Flow Edges Client --> Auth Auth --> Workspace Workspace --> Tasks Workspace --> Knowledge Workspace --> Connectors Workspace --> Launch Workspace --> Incidents Workspace --> Audit Tasks --> Governance Connectors --> Governance Launch --> Governance %% Premium Styling Rules style Audit fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Governance fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Source Truth Route behavior is backed by `services/gateway/src/routes/`, Zod payloads and shared types in `packages/shared/`, database domains in `packages/db/src/schema/`, and HELM receipt wiring in `packages/helm-client/`. If a route description differs from code or tests, update this file. ## Authentication All protected endpoints require one of: - **Bearer token:** `Authorization: Bearer ` - **API key:** `X-API-Key: hp_` - **Workspace context:** authenticated non-auth routes should also include `X-Workspace-Id: ` when the workspace is not already encoded in the path. Session tokens are obtained via `/api/auth/email/request` + `/api/auth/email/verify` or `/api/auth/telegram`. Session tokens rotate automatically after 24 hours — check the `X-New-Token` response header. --- ## Auth ### POST /api/auth/email/request Request a magic link code. ```json { "email": "you@example.com" } ``` Response: `{ "sent": true, "email": "...", "code": "123456" }` (code only in dev mode) ### POST /api/auth/email/verify Verify magic link code and get a session. ```json { "email": "you@example.com", "code": "123456" } ``` Response: `{ "token": "...", "user": { "id", "name", "email" }, "workspace": { "id", "name" } }` ### POST /api/auth/telegram Authenticate via Telegram Web App initData. ```json { "initData": "" } ``` ### POST /api/auth/apikey Create an API key (requires auth). ```json { "name": "my-key" } ``` Response: `{ "key": "hp_...", "name": "...", "expiresAt": "..." }` ### DELETE /api/auth/session Logout / invalidate current session. ### POST /api/auth/invite/:token Accept a workspace invite. ```json { "email": "you@example.com" } ``` --- ## Workspace ### GET /api/workspace/:id Get workspace details with members. ### GET /api/workspace/:id/settings Get workspace settings (policy, budget, model config). ### PUT /api/workspace/:id/settings Update workspace settings. ```json { "policyConfig": { "maxIterationBudget": 50, "blockedTools": [] }, "budgetConfig": { "monthlyLlmBudget": 100, "currency": "USD" }, "modelConfig": { "provider": "openrouter", "model": "anthropic/claude-sonnet-4-20250514", "temperature": 0.7 } } ``` ### PUT /api/workspace/:id/mode Switch workspace mode. ```json { "mode": "discover" } ``` Valid modes: `discover`, `decide`, `build`, `launch`, `apply` ### POST /api/workspace/:id/invite Generate invite link. ```json { "role": "member", "email": "partner@example.com" } ``` Response: `{ "inviteUrl": "...", "inviteToken": "...", "role": "member", "expiresIn": "7 days" }` --- ## Tasks ### GET /api/tasks List tasks for the workspace. Query: `?workspaceId=...` or pass `X-Workspace-Id`. ### POST /api/tasks Create a task and write audit/evidence metadata. ```json { "title": "Draft launch checklist", "description": "Use existing workspace evidence.", "workspaceId": "..." } ``` ### PUT /api/tasks/:id/status Update a task status. ```json { "status": "queued" } ``` ### POST /api/tasks/:id/run Run a governed task through the orchestrator. ### GET /api/tasks/:id/runs List task run history for the workspace-scoped task. ## Audit ### GET /api/audit List audit log entries. Query: `?workspaceId=...&limit=50`. ### GET /api/audit/approvals List approvals. Query: `?workspaceId=...&status=pending`. ### PUT /api/audit/approvals/:id Resolve an approval. ```json { "status": "approved" } ``` ### GET /api/audit/violations List policy violations for the workspace. ## Capabilities ### GET /api/capabilities Return the public capability registry and readiness summary. ### GET /api/capabilities/:key Return one capability state, blocker list, owner, and required eval gate. ## Protected Route Families Pilot also has protected operator, connector, launch, YC, governance, compliance, command, browser, eval, lifecycle, secret, managed Telegram, and tenant administration route families. Anonymous docs intentionally do not expose those route paths. Authenticated customer or staff docs connect those surfaces when the viewer has access, and secret-bearing payloads remain excluded from sitemap, search, Markdown, LLM, and MCP exports. ## Receipts And Governance When HELM is configured, governed LLM calls and consequential tool actions should attach decision metadata to task runs and evidence rows. Use: - `GET /api/governance/status` - `GET /api/governance/receipts` - `GET /api/governance/receipts/:decisionId` - `GET /api/audit?workspaceId=...` ## Error Codes And Diagnostics For a failed API call, collect method, path, workspace ID, request ID, response status, sanitized payload shape, auth mechanism, task ID if relevant, and HELM decision ID when present. Do not include secrets, raw connector tokens, or private session snapshots. ## Troubleshooting | Symptom | Likely Cause | Fix | | ------------------------------------ | ------------------------------------------------- | ------------------------------------------------------------ | | request returns 401 | missing bearer token, API key, or expired session | authenticate again and check `X-New-Token` | | route returns 404 for workspace data | workspace context is missing or wrong | pass `X-Workspace-Id` or use workspace-scoped path | | connector action fails | grant is missing, expired, or not validated | inspect connector grant status and revalidate | | task creates but never runs | pg-boss or orchestrator is unavailable | check `/health` and job queue status | | governed action has no receipt | HELM is not configured or route is not governed | check `HELM_GOVERNANCE_URL`, startup logs, and audit entries | --- title: "Environment Reference" canonical: "https://pilot.docs.mindburn.org/reference/env" source: "pilot/docs/env-reference.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/env-reference.md" section: "reference" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:8f656128d4b27b24cb43d5e2f4497f0a9db4d3fa013356073fb784c06c107d89" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Environment Variables Reference Complete reference for all Pilot configuration variables. ## Required Variables | Variable | Description | Example | | ---------------- | ----------------------------------------------------------------------------------- | --------------------------------------------- | | `DATABASE_URL` | PostgreSQL connection string with pgvector | `postgresql://helm:helm@localhost:5432/pilot` | | `SESSION_SECRET` | 64-char hex secret for session token signing. Generate: `openssl rand -hex 32` | `a1b2c3...` | | `ENCRYPTION_KEY` | 64-char hex secret for connector token encryption. Generate: `openssl rand -hex 32` | `a1b2c3...` | | `APP_URL` | Public gateway base URL used for OAuth, email login links, and callbacks | `https://pilot.example.com` | ## Telegram | Variable | Required | Default | Description | | ------------------------------- | -------- | ------- | -------------------------------------------------------------------------------------- | | `TELEGRAM_BOT_TOKEN` | No | — | Bot token from [@BotFather](https://t.me/BotFather). Enables the Telegram bot. | | `TELEGRAM_WEBHOOK_SECRET` | Prod | — | HMAC secret for webhook validation. Generate: `openssl rand -hex 32` | | `TELEGRAM_OWNER_CHAT_ID` | No | — | Telegram chat ID of the bot owner. Enables admin commands and proactive notifications. | | `TELEGRAM_MANAGER_BOT_USERNAME` | No | `getMe` | Optional manager bot username override for Telegram Managed Bots setup links. | > **Finding your chat ID:** Send `/start` to [@userinfobot](https://t.me/userinfobot) on Telegram. Managed launch/support bots require enabling **Bot Management Mode** for the main bot in BotFather's Mini App. `APP_URL` must be publicly reachable because child bot webhooks are configured under the protected managed-Telegram webhook namespace documented in authenticated operator docs. ## LLM Providers Production deployments should route Pilot through the HELM sidecar. In that shape, direct provider keys live on the HELM app, not on the Pilot app. Direct providers are for local development, fallback labs, or non-HELM deployments. | Variable | Required | Description | | --------------------- | ---------- | ----------------------------------------------------------------------------------------- | | `HELM_GOVERNANCE_URL` | Prod | HELM sidecar governed API URL, e.g. `http://helm:8080` on the Docker network. | | `HELM_HEALTH_URL` | Prod | HELM sidecar health URL, e.g. `http://helm:8081`. | | `HELM_FAIL_CLOSED` | Prod | Must be `1` in production. Unreachable HELM blocks governed LLM calls. | | `HELM_UPSTREAM_URL` | Prod | OpenAI-compatible upstream endpoint used by the HELM sidecar after policy approval. | | `HELM_PORT` | Local | Host port exposed by the local HELM governance API. Defaults to `8420`. | | `HELM_HEALTH_PORT` | Local | Host port exposed by the local HELM health API. Defaults to `8421`. | | `HELM_REGION` | No | Region label included in local/deployed HELM sidecar metadata and evidence context. | | `PILOT_LLM_MODEL` | No | Model name passed through the HELM proxy. | | `OPENROUTER_API_KEY` | Direct dev | OpenRouter key when Pilot runs without HELM. | | `ANTHROPIC_API_KEY` | Direct dev | Direct Anthropic key when Pilot runs without HELM. | | `OPENAI_API_KEY` | Direct dev | Direct OpenAI key when Pilot runs without HELM. Also used for embeddings when configured. | | `OLLAMA_BASE_URL` | Direct dev | Local/self-hosted Ollama endpoint used only when no cloud key is set. | | `OLLAMA_MODEL` | If Ollama | Ollama model id, e.g. `llama3.1:8b`. | For DigitalOcean production, set upstream provider keys on the `helm` sidecar service; do not set direct provider keys on `pilot`. ## Connectors (OAuth) These enable real OAuth flows for external services. Without them, connectors operate in manual-token mode. | Variable | Required | Description | | ------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `GITHUB_CLIENT_ID` | No | GitHub OAuth App client ID | | `GITHUB_CLIENT_SECRET` | No | GitHub OAuth App client secret | | `GOOGLE_CLIENT_ID` | No | Google OAuth client ID (for Gmail + Drive) | | `GOOGLE_CLIENT_SECRET` | No | Google OAuth client secret | | `GOOGLE_REDIRECT_URI` | No | Google OAuth redirect URI. Default: `{APP_URL}/api/connectors/gmail/oauth/callback` | | `MICROSOFT_CLIENT_ID` | No | Microsoft identity platform OAuth app client ID for Outlook | | `MICROSOFT_CLIENT_SECRET` | No | Microsoft identity platform OAuth app client secret for Outlook | | `MICROSOFT_REDIRECT_URI` | No | Microsoft OAuth redirect URI. Default: `{APP_URL}/api/connectors/outlook/oauth/callback` | | `ENABLED_CONNECTORS` | No | Comma-separated list of connectors to strictly validate at startup (e.g., `github,gmail,gdrive,outlook`). Missing credentials for enabled connectors cause `fatal error → exit 1` in production. | ### Setting up GitHub OAuth 1. Go to [GitHub Developer Settings → OAuth Apps](https://github.com/settings/developers) 2. Create a new OAuth App 3. Set Authorization callback URL to `https://your-domain.com/api/connectors/github/oauth/callback` 4. Copy Client ID and Client Secret to `.env` ### Setting up Google OAuth (Gmail + Drive) 1. Go to [Google Cloud Console → APIs & Services → Credentials](https://console.cloud.google.com/apis/credentials) 2. Create an OAuth 2.0 Client ID (Web application) 3. Add authorized redirect URIs: - `https://your-domain.com/api/connectors/gmail/oauth/callback` - `https://your-domain.com/api/connectors/gdrive/oauth/callback` 4. Enable the Gmail API and Google Drive API in the API Library 5. Copy Client ID and Client Secret to `.env` ### Setting up Microsoft OAuth (Outlook) 1. Go to [Microsoft Entra admin center → App registrations](https://entra.microsoft.com/) 2. Create an app registration for a web application 3. Add redirect URI `https://your-domain.com/api/connectors/outlook/oauth/callback` 4. Add delegated Microsoft Graph permissions for `User.Read`, `Mail.Read`, `Mail.Send`, and `offline_access` 5. Copy Client ID and Client Secret to `.env` ## Connectors (Session Auth) The YC connector uses founder-authorized browser session capture instead of OAuth. | Variable | Required | Description | | ---------------- | -------- | ------------------------------------------------------------------------ | | `APP_URL` | Yes | Base URL used to return from the guided YC session-capture flow. | | `ENCRYPTION_KEY` | Yes | Encrypts stored browser storage-state snapshots in `connector_sessions`. | YC session state is stored separately from OAuth tokens. It powers authenticated reads and syncs for YC cofounder matching and other private YC workflows. ## Security | Variable | Required | Default | Description | | ---------------------- | -------- | ------------------------- | ----------------------------------------------------------------------------------------- | | `SESSION_SECRET` | Yes | `change-me-in-production` | Session token HMAC signing secret. **Must change in production.** | | `ENCRYPTION_KEY` | Prod | dev fallback | AES-256-GCM key for encrypting connector tokens at rest. Generate: `openssl rand -hex 32` | | `EVIDENCE_SIGNING_KEY` | Prod | dev fallback | Ed25519 evidence-pack signing key used by governed receipt and audit flows. | | `DAILY_BUDGET_MAX` | No | `500` | Maximum daily spend (EUR) across all tasks before kill switch | | `PER_TASK_BUDGET_MAX` | No | `100` | Maximum spend per individual task | > ⚠️ **Production requirement:** Both `SESSION_SECRET` and `ENCRYPTION_KEY` must be set to unique, random values. ## Server | Variable | Required | Default | Description | | --------------------------- | -------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `PORT` | No | `3100` | HTTP server port | | `NODE_ENV` | No | `development` | Environment. Set to `production` for production. | | `LOG_LEVEL` | No | `info` | Pino log level (`debug`, `info`, `warn`, `error`, `fatal`) | | `ALLOWED_ORIGINS` | No | `*` (dev) | Comma-separated CORS allowed origins. Set to your domain in production. | | `APP_URL` | No | `http://localhost:3100` | Public-facing URL of the app (used for OAuth redirect URIs) | | `RUN_MIGRATIONS_ON_STARTUP` | No | `true` | When `true` (default), gateway runs pending Drizzle migrations on boot. DigitalOcean production sets `false` and runs migrations explicitly during deploy. | | `PYTHON_BIN` | No | `python3` | Python executable used by the orchestrator for Scrapling-backed pipelines. For local installs, prefer `./.venv-pipelines/bin/python`. | | `PLAYWRIGHT_BROWSERS_PATH` | No | repo-local cache or `/ms-playwright` in Docker | Browser binary cache used by dynamic Scrapling fetchers. | | `PATCHRIGHT_BROWSERS_PATH` | No | repo-local cache or `/ms-patchright` in Docker | Browser binary cache used by stealth Scrapling sessions. | ## Email (Transactional) Required in production to send magic-link login codes. In development, the `noop` provider does not send email; the local auth route returns the code in the HTTP response and provider logs include only redacted delivery metadata. | Variable | Required | Default | Description | | ---------------- | ----------- | ------------------------------ | ------------------------------------------------------------- | | `EMAIL_PROVIDER` | No | `noop` | `resend` \| `smtp` \| `noop`. Use `noop` only in development. | | `EMAIL_FROM` | No | `Pilot ` | Sender address | | `RESEND_API_KEY` | If `resend` | — | API key from [resend.com](https://resend.com) | | `SMTP_HOST` | If `smtp` | — | SMTP server hostname | | `SMTP_PORT` | If `smtp` | `587` | SMTP port (587 STARTTLS, 465 TLS) | | `SMTP_USER` | No | — | SMTP auth username | | `SMTP_PASS` | No | — | SMTP auth password | | `SMTP_SECURE` | No | auto | `true` for port 465, else STARTTLS | > ⚠️ **Production requirement:** `EMAIL_PROVIDER` must be `resend` or `smtp`. The `noop` provider is dev-only; users cannot log in. ## Object Storage For storing artifacts, launch assets, and raw ingestion captures. Falls back to local filesystem if not configured. | Variable | Required | Default | Description | | ------------------ | -------- | ---------------- | ----------------------------------------------------- | | `STORAGE_PROVIDER` | No | `local` | `local` or `s3` | | `STORAGE_PATH` | No | `./data/storage` | Local storage directory (when using `local` provider) | | `S3_ENDPOINT` | Prod | — | S3-compatible endpoint URL, e.g. DO Spaces | | `S3_BUCKET` | Prod | — | S3 bucket name | | `S3_ACCESS_KEY` | Prod | — | S3 access key | | `S3_SECRET_KEY` | Prod | — | S3 secret key | | `S3_REGION` | Prod | `fra1` | S3 signing region | When using local storage, Pilot also persists: - Scrapling adaptive selector databases under `STORAGE_PATH/adaptive` - raw crawl captures under `STORAGE_PATH/raw` - crawl checkpoints under `STORAGE_PATH/crawls` ## Error Reporting (Optional) | Variable | Required | Default | Description | | ----------------- | -------- | ------- | -------------------------------------------------------------------------- | | `SENTRY_DSN` | No | — | Sentry DSN for error reporting. When unset, errors only hit local logs. | | `RELEASE_VERSION` | No | — | Release tag for Sentry (e.g., git SHA). Helps correlate errors to deploys. | ## Search & Ranking | Variable | Required | Default | Description | | ---------------- | -------- | ------- | ---------------------------------------------------------------------------------------- | | `COHERE_API_KEY` | No | — | Cohere API key for reranking search results. Improves opportunity and knowledge ranking. | ## DigitalOcean (Production) | Variable | Required | Default | Description | | ------------------- | -------- | -------------------------------------------- | ----------------------------------------------------- | | `DOMAIN` | Prod | `localhost` | Public hostname served by Caddy on the Droplet | | `TLS_EMAIL` | Prod | `admin@pilot.dev` | ACME registration email for Caddy certificates | | `POSTGRES_PASSWORD` | Prod | `helm` | PostgreSQL password used by the Droplet compose stack | | `POSTGRES_IMAGE` | Prod | `pgvector/pgvector:0.8.0-pg17` | Pinned pgvector PostgreSQL image | | `CADDY_IMAGE` | Prod | `caddy:2.10.2-alpine` | Pinned Caddy TLS proxy image | | `OFELIA_IMAGE` | Prod | `mcuadros/ofelia:0.3.22` | Pinned backup scheduler image | | `HELM_IMAGE` | Prod | `ghcr.io/mindburn-labs/helm-ai-kernel:0.5.0` | Published HELM sidecar image pulled by the Droplet | | `PILOT_IMAGE` | Prod | `ghcr.io/mindburn-labs/pilot:` | Published Pilot gateway image pulled by the Droplet | | `WEB_IMAGE` | Prod | `ghcr.io/mindburn-labs/pilot-web:` | Published Next.js web image pulled by the Droplet | Deployment-time `doctl` variables such as `DO_REGION`, `DO_SIZE`, `DO_SSH_KEYS`, and `DO_DROPLET_IP` are consumed by `infra/digitalocean/deploy.sh`; they are not runtime app variables. ## Vercel And Cloudflare Pages (Production Proof) These variables are consumed by the launch-engine provider adapters when a deployment target is configured for Vercel or Cloudflare Pages. Real deployments also require target config to include the provider-specific project/deployment payload that the admin UI stores for that workspace; the providers fail closed in production when credentials or payload metadata are missing. | Variable | Required | Default | Description | | ----------------------- | ------------- | ------- | ----------------------------------------------------------------- | | `VERCEL_TOKEN` | If Vercel | — | Bearer token for Vercel project and deployment API calls | | `VERCEL_TEAM_ID` | If team scope | — | Optional Vercel team id query scope | | `VERCEL_TEAM_SLUG` | If team scope | — | Optional Vercel team slug query scope | | `CLOUDFLARE_API_TOKEN` | If Cloudflare | — | Cloudflare API token with Pages Write permission | | `CF_API_TOKEN` | If Cloudflare | — | Backward-compatible alias for `CLOUDFLARE_API_TOKEN` | | `CLOUDFLARE_ACCOUNT_ID` | If Cloudflare | — | Cloudflare account id used by Pages project and deployment routes | ## Backup | Variable | Required | Default | Description | | ------------------------------ | -------- | ----------- | ------------------------------------------------------ | | `BACKUP_DIR` | No | `./backups` | Local backup directory for `scripts/backup.sh` | | `BACKUP_ENCRYPTION_PASSPHRASE` | Prod | — | GPG symmetric passphrase for encrypted remote backups | | `BACKUP_CRON_SCHEDULE` | No | `0 3 * * *` | Ofelia cron expression for the DigitalOcean backup job | Production backups are uploaded as encrypted `.sql.gz.gpg` files using the `S3_*` variables above. Plaintext upload is blocked unless `BACKUP_ALLOW_PLAINTEXT_UPLOAD=1` is set for a non-production drill. --- title: "Roadmap" canonical: "https://pilot.docs.mindburn.org/roadmap" source: "pilot/docs/roadmap.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/roadmap.md" section: "product" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:737ea8b1c5fb6dc302a27021b7d9ba4a6325d3c16555abd21bb1e1c9b501a602" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Pilot — Public roadmap ## Diagram This scheme maps the main sections of Roadmap in reading order. ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Page["Roadmap"] A["v1.0.0 — shipped surfaces"] B["Phase 14 — v1.1 (next)"] C["v1.2+ research"] D["Upstream dependencies"] E["How to propose changes"] end %% Operational Flow Edges Page --> A A --> B B --> C C --> D D --> E %% Premium Styling Rules ``` Snapshot as of the v1.0.0-rc cut. Items marked (🟢) ship in v1.0.0; (🟡) defer to v1.1 / Phase 14; (🔵) longer-range research. Production-readiness source of truth: this roadmap is not a production autonomy claim. Use `docs/capabilities.md` (protected staff doc), `GET /api/capabilities`, and `packages/shared/src/capabilities/index.ts` for current capability state. A shipped surface may still be `prototype`, `scaffolded`, `stub`, or `blocked` until it passes the required eval and the registry marks it `production_ready`. ## v1.0.0 — shipped surfaces - 🟢 HELM governance sidecar integration with signed evidence packs - 🟢 Multi-tenant isolation + per-tenant secrets + rate-limit partitioning + per-tenant backup - 🟢 Discover mode: YC + HN + ProductHunt + IndieHackers + GitHub trending + Reddit r/SaaS + Crunchbase RSS + YC blog intelligence pipeline - 🟢 Decide mode: adversarial decision court with bull/bear/referee/scenarios and cofounder matching - 🟢 Build mode: spec generation + scaffolding templates + HELM-governed GitHub commits - 🟢 Launch mode: DigitalOcean deploy target with guarded deploy receipts - 🟢 Apply mode: YC/Techstars/Antler templates with HELM-escalated submissions - 🟢 Governance surface: receipts list + proof-graph DAG viewer + `dump-proof-graph.ts` CLI - 🟢 Governed subagents: Conductor + three built-ins (opportunity_scout, decision_facilitator, founder_diagnostician) - 🟢 Observability: OTel GenAI semantic conventions, 5 Alertmanager-routed alerts, Grafana GenAI dashboard - 🟢 Supply-chain hardening: CycloneDX SBOM, SLSA L3 provenance, Trivy + gitleaks + license gates ## Phase 14 — v1.1 (next) - 🟡 Vercel launch provider parity and deeper DigitalOcean automation - 🟡 `decision_court_run` tool wrapper enabling `decision_facilitator` to invoke the full court - 🟡 Four more built-in subagents: `build_engineer`, `launch_captain`, `application_drafter`, plus governed wrappers around the SMTM-Phase-11 ops (`content_strategist`, `seo_analyst`, `ads_operator`, `social_operator`, `finance_ops_lead`) - 🟡 Interactive DAG replay — re-execute an evidence-pack node with modified inputs (diff-render the new subtree) - 🟡 Mini App full per-mode tabs (decide / launch / apply / governance) + operator chat wired to SSE - 🟡 Playwright docker-stack shared fixture + 10 more E2E cases (build-spec, launch-deploy, apply-yc, helm-down, reauth, discover-flow) - 🟡 `dump-proof-graph` artifact generation wired into the landing-page demo ## v1.2+ research - 🔵 Dynamic subagent creation from chat (registry currently loads from disk only) - 🔵 MCP server attachment on subagents (`mcp_servers:` frontmatter currently ignored) - 🔵 Streaming partial subagent results back to the parent - 🔵 Subagent marketplace + remote hosting - 🔵 Policy-pack marketplace (v1 ships with a single `founder_ops` pack) - 🔵 Advanced compliance overlays: GDPR, HIPAA, SOX - 🔵 Multi-region deployment (v1 is single-region) - 🔵 Enterprise SSO (SAML/OIDC) - 🔵 Cross-workspace subagent / cofounder-candidate search ## Upstream dependencies - `helm-ai-kernel` governance parity — keep `POST /api/v1/evaluate` and the `/api/v1/guardian/evaluate` compatibility alias covered by contract tests; migration `0012_reverify_spawn_receipts.sql` remains available for re-signing older Path-A rows. ## How to propose changes Open an issue via `.github/ISSUE_TEMPLATE/feature.md`. Roadmap changes of significance (reorderings, removals) require a PR against `docs/roadmap.md` with a Why / How-to-apply paragraph so the rationale survives the next snapshot. ## Public Roadmap Discipline The roadmap is public only where it describes capability state, self-hosting expectations, and source-backed delivery boundaries. Do not use it to promise private customer timelines, investor material, YC session details, or production autonomy that has not passed the relevant eval gate. Every public roadmap item should name the owning source family, the validation command that will graduate it, and the docs page that will change when it ships. If an item depends on protected operator work, describe the public outcome and keep the implementation route or runbook in authenticated docs. The public roadmap should help a self-hosted operator decide what is usable today, what is blocked by missing configuration, and what requires customer or staff guidance. --- title: "Security" canonical: "https://pilot.docs.mindburn.org/security" source: "pilot/docs/security.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/security.md" section: "security" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:37dc3d940bcccf5d65edc3c5db896f7d083bc79fbe46f59bc81e82f7c97866e3" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Security Hardening Guide Production-grade security configuration for Pilot. ## Audience Use this page if you are operating Pilot in production, reviewing autonomous-action constraints, hardening connectors, or checking the self-hosting threat model. It is for operators, platform engineers, and security reviewers. ## Outcome After this page you should be able to: - generate and rotate required secrets; - explain how sessions, API keys, connector tokens, and session snapshots are protected; - understand autonomous-action constraints and fail-closed behavior; - configure OAuth, webhooks, CORS, TLS, rate limits, backups, and audit logs; - distinguish Pilot application security from HELM governance responsibilities. ## Security Boundary ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] User["User or Telegram"] Workspace["Workspace tenancy"] Helm["HELM governance"] Secrets["Encrypted token/session storage"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Gateway["Gateway auth and rate limits"] Policy["Trust boundary policy"] Approval["Approval gates"] Audit["Audit events"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Receipts["Receipts and evidence packs"] end %% Operational Flow Edges User --> Gateway Gateway --> Workspace Workspace --> Policy Policy --> Approval Policy --> Helm Gateway --> Secrets Gateway --> Audit Helm --> Receipts %% Premium Styling Rules style Gateway fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Policy fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Approval fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Audit fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Receipts fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Source Truth Security guidance is backed by: - `services/gateway/src/middleware/` - `services/gateway/src/routes/auth.ts` - `services/gateway/src/routes/connectors*` - `services/orchestrator/src/trust.ts` - `packages/db/src/schema/` - `scripts/rotate-encryption-key.ts` - `scripts/rotate-master-key.ts` - `docs/helm-integration.md` - `docs/degradation-matrix.md` If code and this guide disagree, update the guide before publishing. ## Secrets Management ### Required Secrets Every production deployment **must** generate unique values for: ```bash # Generate all secrets at once export SESSION_SECRET=$(openssl rand -hex 32) export ENCRYPTION_KEY=$(openssl rand -hex 32) export TELEGRAM_WEBHOOK_SECRET=$(openssl rand -hex 32) ``` | Secret | Purpose | Risk if compromised | | ------------------------- | ------------------------------------------------ | ---------------------------------- | | `SESSION_SECRET` | HMAC signing for session tokens and OAuth state | Session forgery, CSRF bypass | | `ENCRYPTION_KEY` | AES-256-GCM encryption of connector OAuth tokens | Token theft for GitHub/Gmail/Drive | | `TELEGRAM_WEBHOOK_SECRET` | HMAC validation of incoming Telegram webhooks | Spoofed bot commands | > ⚠️ **Never** use the default dev values in production. The `launch-gate.sh` script will flag this. ### Secret Rotation Rotating secrets requires careful coordination: 1. **SESSION_SECRET rotation:** All active sessions become invalid. Users must re-authenticate. 2. **ENCRYPTION_KEY rotation:** Use the rotation tool (below) to re-encrypt tokens without losing them. 3. **TELEGRAM_WEBHOOK_SECRET rotation:** Update the webhook via Telegram API after changing. #### ENCRYPTION_KEY Rotation Procedure ```bash # 1. Generate the new key NEW_KEY=$(openssl rand -hex 32) # 2. Dry run to see how many rows will rotate ENCRYPTION_KEY_OLD=$CURRENT_ENCRYPTION_KEY \ ENCRYPTION_KEY_NEW=$NEW_KEY \ DATABASE_URL=$PROD_DATABASE_URL \ tsx scripts/rotate-encryption-key.ts --dry-run # 3. Run the rotation (writes to DB) ENCRYPTION_KEY_OLD=$CURRENT_ENCRYPTION_KEY \ ENCRYPTION_KEY_NEW=$NEW_KEY \ DATABASE_URL=$PROD_DATABASE_URL \ tsx scripts/rotate-encryption-key.ts # 4. Update .env.production.pilot, then redeploy through the DO script DO_DROPLET_IP= bash infra/digitalocean/deploy.sh deploy # 5. Verify — a subsequent agent run that uses a connector token should succeed ``` The rotation is idempotent per row; failed rows are logged and skipped so the rest continue. ## Prompt Injection Defense The agent loop treats all user-controlled and tool-output content as **untrusted data**, not instructions. Strategy: 1. **Tagged context blocks.** User input (task context, operator goal, role, tool outputs) is JSON-encoded and wrapped in `...` tags. The LLM sees explicit framing, not raw prose. 2. **System-level instruction.** The plan prompt begins with a `SECURITY NOTICE` that tells the model content inside `` blocks is untrusted. 3. **Tool allowlist.** The tool registry presents only the tools available for the current mode; requests for any other tool are rejected by the trust boundary. 4. **Trust boundary checks.** Before executing any tool call, the `TrustBoundary` evaluates kill switches, blocklists, budget, connector scope, and approval requirements. Fail-closed. **Known gaps:** - LLMs can still be convinced to misuse _allowed_ tools in unexpected ways. Defense-in-depth: approval-gated sensitive tools (email send, financial actions, external posts). - The model may leak short strings from context into its reply. Do not place credentials, other users' data, or raw secrets into agent-visible context. **Testing:** See `services/orchestrator/src/__tests__/agent-loop.test.ts` for injection-resistance assertions. ## Data Deletion (GDPR Right to Erasure) Authenticated users can delete their account via: ```http DELETE /api/users/me Authorization: Bearer ``` Behaviour: - The user row is deleted. FK cascades clean up `sessions`, `api_keys`, and `workspace_members`. - Founder profile rows are `set_null`'d (FK policy). - Any workspace where the user was the **sole member** is also deleted, cascading to its `tasks`, `operators`, `audit_log`, etc. - Workspaces with other members are left intact; the user is just unlinked. Admins may execute the same deletion on behalf of a user via a direct DB query; follow the same sequence. ## Authentication ### Session Tokens - 30-day expiry, stored in database - Transmitted via `Authorization: Bearer ` header - Session can be revoked via `DELETE /api/auth/session` ### Email Magic Codes - 15-minute expiry, stored as `SESSION_SECRET`-keyed HMAC digests rather than plaintext codes - Timing-safe comparison during verification - One-time use: successful verification deletes the pending code before creating the session - Failed verification increments the pending-code attempt counter and deletes the code after 5 attempts - Request and verification outcomes write best-effort audit entries ### API Keys - 365-day expiry, stored as SHA-256 hash (never plaintext) - Transmitted via `X-API-Key: ` header - One-time display on creation (hash is not reversible) ### Rate Limiting Built-in rate limiting by endpoint category: | Endpoint | Limit | Window | | ------------------------- | ------- | ------ | | `/api/auth/*` | 5 req | 1 min | | `/api/connectors/*/grant` | 10 req | 1 min | | `/api/connectors/*/token` | 10 req | 1 min | | `/api/tasks` | 30 req | 1 min | | `/api/*` (general) | 100 req | 1 min | For production, consider adding an external rate limiter (Cloudflare, nginx) for DDoS protection. ## OAuth Security ### CSRF Protection OAuth flows use HMAC-signed state parameters: - State = `{connectorId}:{workspaceId}:{nonce}:{hmac}` - HMAC computed with `SESSION_SECRET` - States expire after 10 minutes - Each state is single-use (deleted after callback) ### Redirect URI Validation - Redirect URIs are registered per-provider at startup - Only exact-match redirect URIs are accepted - Production URIs must use HTTPS ### Token Storage - OAuth access and refresh tokens are encrypted at rest using AES-256-GCM - Key derivation: `scrypt(ENCRYPTION_KEY, 'pilot-salt', 32)` - IV is randomly generated per encryption operation - Auth tag is stored alongside ciphertext for integrity verification ## Network Security ### HTTPS / TLS Pilot does not terminate TLS directly. Use a reverse proxy: **Caddy (recommended — automatic HTTPS):** ``` your-domain.com { reverse_proxy localhost:3100 } ``` **nginx:** ```nginx server { listen 443 ssl; server_name your-domain.com; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; location / { proxy_pass http://127.0.0.1:3100; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } ``` **Cloudflare Tunnel (zero-config):** ```bash cloudflared tunnel --url http://localhost:3100 ``` ### CORS In production, explicitly set `ALLOWED_ORIGINS`: ``` ALLOWED_ORIGINS=https://your-domain.com,https://app.your-domain.com ``` The wildcard (`*`) is only allowed in development mode. ### Security Headers Pilot applies these headers automatically via Hono's `secureHeaders()`: - `X-Content-Type-Options: nosniff` - `X-Frame-Options: SAMEORIGIN` - `X-XSS-Protection: 0` (modern CSP preferred) - `Referrer-Policy: no-referrer` - `Strict-Transport-Security` (when behind HTTPS proxy) ## Trust Boundary The orchestrator enforces a multi-layer trust boundary on every agent action: ``` Kill Switch → Policy Validation → Tool Blocklist → Budget → Connector Allowlist → Content Bans → Approval Gate ``` ### Configuration Set via workspace settings (`/api/workspace/settings`): ```json { "policyConfig": { "killSwitch": false, "failClosed": true, "toolBlocklist": ["github_create_repo"], "contentBans": ["password", "secret"], "connectorAllowlist": ["github"], "requireApprovalFor": ["gmail_send", "github_create_repo"] }, "budgetConfig": { "dailyTotalMax": 500, "perTaskMax": 100, "perOperatorMax": 200, "emergencyKill": 1000, "currency": "EUR" } } ``` ### Fail-Closed Behavior If `failClosed` is `true` (default), any policy validation failure blocks all actions. This means: - Missing or malformed budget config → all actions denied - Budget values out of range → all actions denied - Invalid `perTaskMax > dailyTotalMax` → all actions denied ## Database Security ### Connection Security For production PostgreSQL: ``` DATABASE_URL=postgresql://helm:STRONG_PASSWORD@db-host:5432/pilot?sslmode=require ``` ### Backup Encryption Production backup uploads are encrypted before leaving the Droplet. Set `BACKUP_ENCRYPTION_PASSPHRASE` and DO Spaces `S3_*` settings, then use: ```bash # Create, GPG-encrypt, checksum, and upload bash scripts/backup.sh create-and-upload # Verify or restore an encrypted backup bash scripts/backup.sh verify backups/pilot_YYYY...sql.gz.gpg bash scripts/backup.sh restore backups/pilot_YYYY...sql.gz.gpg ``` Remote plaintext upload is blocked unless `BACKUP_ALLOW_PLAINTEXT_UPLOAD=1` is explicitly set for a non-production drill. ## Audit Trail All mutating API requests are logged to the `audit_events` table: - User ID, workspace ID, action, resource, timestamp - Request body (sanitized — tokens/secrets redacted) - Response status code Query audit logs via `GET /api/audit?workspaceId=...`. ## Checklist Before going to production, verify: - [ ] `SESSION_SECRET` is a unique random value (not `change-me-in-production`) - [ ] `ENCRYPTION_KEY` is set (not using dev fallback) - [ ] `TELEGRAM_WEBHOOK_SECRET` is set for webhook mode - [ ] `ALLOWED_ORIGINS` is set to specific domains (not `*`) - [ ] `NODE_ENV=production` is set - [ ] PostgreSQL uses SSL (`?sslmode=require`) - [ ] HTTPS is terminated via reverse proxy - [ ] Database password is strong and unique - [ ] Backups are configured and tested - [ ] `launch-gate.sh` passes all checks - [ ] `npm run production:readiness` passes; DigitalOcean deploy blocks while any capability remains below `production_ready` ## Troubleshooting | Symptom | Likely Cause | Fix | | ------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------- | | users cannot log in | email provider, session secret, or database is misconfigured | check auth routes, email provider, and session rows | | connector token cannot decrypt | encryption key rotated without migration | use the rotation procedure and verify rows before deploy | | webhook accepts unexpected requests | webhook secret is missing or mismatched | set provider webhook secret and verify request validation | | autonomous action bypasses approval | trust boundary policy is too permissive | update workspace policy and require approval for sensitive tools | | production keeps running without HELM | fail-closed config is missing | set `HELM_FAIL_CLOSED=1` and verify `/health` reports HELM state | --- title: "Self-Hosting Pilot" canonical: "https://pilot.docs.mindburn.org/self-hosting" source: "pilot/docs/self-hosting.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/self-hosting.md" section: "self-hosting" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:3ee8b308cc1f16f3b08a8fd12f26c24514d3eb47573a1cfaa995f410cac6016f" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Self-Hosting Pilot Pilot is designed to run on your own infrastructure. This operator guide covers Docker, PostgreSQL, pgvector, environment variables, upgrades, backups, restore, DigitalOcean deployment, local development, and production readiness. ## Audience Use this page if you are operating Pilot rather than just trying the first founder workflow. It is for self-hosting operators, platform engineers, and developers who own runtime health, secrets, migrations, backups, upgrades, and rollback. ## Outcome After this page you should be able to: - run Pilot locally or on a single-node Docker Compose deployment; - install the Python/Scrapling runtime; - configure PostgreSQL with pgvector and required environment variables; - configure HELM as the production governance sidecar; - back up and restore database plus local storage; - perform upgrades and diagnose common startup failures. ## Deployment Topology ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Operator["Operator"] Proxy["Caddy, nginx, or tunnel"] Web["Pilot web UI :3000"] Postgres["PostgreSQL 17 + pgvector"] Jobs["pg-boss jobs"] Helm["HELM sidecar"] Storage["local or S3-compatible storage"] Upstream["OpenAI-compatible upstream"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Gateway["Pilot gateway :3100"] end subgraph Execution["3. Execution & Verdict Plane"] Python["Python/Scrapling runtime"] end %% Operational Flow Edges Operator --> Proxy Proxy --> Gateway Proxy --> Web Gateway --> Postgres Gateway --> Jobs Gateway --> Python Gateway --> Helm Gateway --> Storage Helm --> Upstream %% Premium Styling Rules style Gateway fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Python fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff ``` ## Source Truth This page is backed by: - `scripts/setup.sh` - `scripts/install-python-runtime.sh` - `scripts/verify-python-runtime.py` - `infra/docker/docker-compose.yml` - `infra/digitalocean/README.md` - `infra/digitalocean/deploy.sh` - `docs/env-reference.md` - `docs/security.md` - `docs/degradation-matrix.md` If commands here differ from scripts or deployment files, update the docs and source together. ## Prerequisites - **Docker** 24+ and **Docker Compose** v2 - **Python** 3.10+ with `venv` and `pip` available for local pipeline execution - A domain name (optional, for production) - A Telegram bot token (for bot/mini-app features) - For production: a HELM sidecar with an upstream LLM key configured on the sidecar - For local direct-provider development: OpenRouter, Anthropic, OpenAI, or Ollama ## Quick Start (Clean Install) We provide an interactive setup script for first-time deployments. ```bash git clone https://github.com/mindburn-labs/pilot.git cd pilot # Run the interactive setup script bash scripts/setup.sh ``` The script will check prerequisites, generate necessary secrets, start the database, run migrations, and test the API. The gateway runs on port **3100**, the web UI on **3000**. ## Python / Scrapling Runtime Pilot uses a local Python runtime for Scrapling-backed ingestion, YC sync, and operator-triggered fetch/extract work. For local installs: ```bash bash scripts/install-python-runtime.sh ./.venv-pipelines/bin/python scripts/verify-python-runtime.py ``` This creates `./.venv-pipelines`, installs the pinned pipeline dependencies from `pipelines/requirements.txt`, and installs Chromium for both Playwright and Patchright. Set `PYTHON_BIN=./.venv-pipelines/bin/python` in `.env` so the orchestrator uses that runtime. ## Environment Variables For a complete list of required and optional environment variables, see the [Environment Reference](/reference/env). The setup script will generate the critical security tokens for you. Production deployments should set `HELM_GOVERNANCE_URL` and `HELM_FAIL_CLOSED=1`; direct provider keys are only needed when running Pilot without HELM. ## Development Setup (without Docker) ```bash # Install dependencies npm install # Install the pinned Python pipeline runtime bash scripts/install-python-runtime.sh # Start PostgreSQL (or use Docker for just the DB) docker compose -f infra/docker/docker-compose.yml up -d postgres # Set DATABASE_URL export DATABASE_URL=postgresql://helm:helm@localhost:5432/pilot # Run migrations npm run db:push # Start all services in development npm run dev ``` Verify the full runtime before using private YC flows: ```bash PYTHON_BIN=./.venv-pipelines/bin/python ./scripts/launch-gate.sh ``` ## Telegram Bot Setup 1. Create a bot via [@BotFather](https://t.me/botfather) 2. Set the bot token in `.env` as `TELEGRAM_BOT_TOKEN` 3. Set up the webhook: ```bash curl -X POST "https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/setWebhook" \ -H "Content-Type: application/json" \ -d '{"url": "https://your-domain.com/api/telegram/webhook", "secret_token": "your-secret", "allowed_updates": ["message", "callback_query", "managed_bot"]}' ``` 4. Set `TELEGRAM_WEBHOOK_SECRET` to match the `secret_token` above ### Founder-Owned Launch/Support Bot Pilot can provision one founder-owned child bot through Telegram Managed Bots. 1. Open the main bot in BotFather's Mini App and enable **Bot Management Mode**. 2. Ensure `APP_URL` is the public HTTPS gateway URL. 3. Optionally set `TELEGRAM_MANAGER_BOT_USERNAME` to the main bot username without `@`. 4. From the main bot, run `/launchbot`, or use the Launch page / Mini App settings to create the native Telegram setup link. Child bot tokens are encrypted in `tenant_secrets`. Child webhook secrets are stored only as hashes, and outbound support replies cross the HELM governance path before being sent. ## Telegram Mini App 1. Via BotFather, create a Web App for your bot 2. Set the URL to `https://your-domain.com/app/` 3. The mini app is served as static files from the gateway ## YC Session Capture The `yc` connector uses founder-authorized session capture instead of OAuth. 1. Open **Settings** in the web app. 2. Grant the `yc` connector to your workspace. 3. Paste an exported browser storage-state JSON for your authorized YC session into the connector session box. 4. Click **Save Session**, then **Validate Session**. 5. Once validation succeeds, private YC matching syncs can run from the Discover surface or through background jobs. All YC session snapshots are encrypted at rest using `ENCRYPTION_KEY`. ## Custom Domain (Non-DigitalOcean) For non-DigitalOcean deployments, put a reverse proxy (Caddy, nginx, or Cloudflare Tunnel) in front: ``` # Caddyfile example your-domain.com { reverse_proxy localhost:3100 } app.your-domain.com { reverse_proxy localhost:3000 } ``` ## DigitalOcean Deployment Pilot ships on DigitalOcean as a Docker Compose stack on one Droplet. The HELM sidecar stays private on the Docker network, Pilot talks to `http://helm:8080`, and production remains fail-closed with `HELM_FAIL_CLOSED=1`. ```bash cp infra/digitalocean/env.production.shared.example .env.production.shared cp infra/digitalocean/env.production.helm.example .env.production.helm cp infra/digitalocean/env.production.pilot.example .env.production.pilot # Fill domain, database, pinned images, Pilot secrets, email, # DO Spaces backup settings, evidence signing, and sidecar provider key. export DO_SSH_KEYS= export DO_REGION=fra1 export DO_SIZE=s-2vcpu-4gb bash infra/digitalocean/deploy.sh doctor bash infra/digitalocean/deploy.sh create export DO_DROPLET_IP= bash infra/digitalocean/deploy.sh preload-helm bash infra/digitalocean/deploy.sh deploy ``` `deploy.sh deploy` runs `npm run production:readiness` before copying a release. It fails closed until the shared capability registry reports every capability as `production_ready` with eval metadata; use `npm run production:readiness:expect-blocked` only to verify the current pre-production blocked state. See infra/digitalocean/README.md (protected staff doc) for the full runbook. ## Database Pilot uses PostgreSQL 17 with pgvector for knowledge base search. The schema is managed by Drizzle ORM. ```bash # Push schema to database npm run db:push # Generate a migration npm run db:generate # Open Drizzle Studio (DB browser) npm run db:studio ``` ## pgAdmin (Optional) For local database inspection, start the dev-only Docker stack's debug profile: ```bash docker compose -f infra/docker/docker-compose.yml --profile debug up -d ``` pgAdmin will be available at http://localhost:5050 (admin@pilot.local / admin). ## Backup & Restore Use the included backup tooling to manage your PostgreSQL data. ```bash # Create, encrypt, and upload a backup when S3_* and BACKUP_ENCRYPTION_PASSPHRASE are set bash scripts/backup.sh create-and-upload # List available backups bash scripts/backup.sh list # Verify a backup bash scripts/backup.sh verify backups/pilot_YYYY...sql.gz.gpg # Restore from a backup bash scripts/backup.sh restore backups/pilot_YYYY...sql.gz.gpg ``` Production uploads must be encrypted. The deploy doctor requires DO Spaces settings and `BACKUP_ENCRYPTION_PASSPHRASE`. In addition to the database backup, preserve local storage when using the default filesystem backend: ```bash tar -czf pilot_storage_$(date +%Y%m%d_%H%M%S).tar.gz data/storage ``` That archive contains: - raw crawl captures - Scrapling adaptive selector databases - crawl checkpoint directories YC session snapshots live in the database (`connector_sessions`) and are therefore covered by PostgreSQL backups. ## Updating ```bash git pull docker compose -f infra/docker/docker-compose.yml build docker compose -f infra/docker/docker-compose.yml up -d # Run any new migrations docker compose -f infra/docker/docker-compose.yml exec pilot npx drizzle-kit push ``` ## Troubleshooting **Health check fails:** Check `DATABASE_URL` is correct and PostgreSQL is reachable. **Agent loop doesn't execute:** In production, check `/health` for `checks.helm: ok` and verify the HELM sidecar has its upstream provider key. In direct-provider development, set `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or `OLLAMA_BASE_URL` plus `OLLAMA_MODEL`. **YC private sync fails:** Re-run `scripts/verify-python-runtime.py`, confirm the `yc` connector shows `Validated`, and save a fresh YC session snapshot if needed. **Scrapling browser fetches fail locally:** Re-run `bash scripts/install-python-runtime.sh` and check that `PLAYWRIGHT_BROWSERS_PATH` and `PATCHRIGHT_BROWSERS_PATH` point to existing browser caches. **Telegram commands don't work:** Verify the webhook is set and `TELEGRAM_BOT_TOKEN` matches. **CORS errors in web UI:** Set `ALLOWED_ORIGINS` to include your web app URL. --- title: "Start with Pilot" canonical: "https://pilot.docs.mindburn.org/start" source: "pilot/docs/start.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/start.md" section: "start" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:c700a993b200433a34899a8cda6f9f3e10ae8de1aed6475b5347ddc12349a124" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Start With Pilot Pilot is a self-hostable founder operating system for discovery, decision, build, launch, and application workflows. This start path gets a founder/developer from a clean checkout to one useful governed workflow without forcing them to understand every operator concern first. ## Audience Use this page if you are a founder, solo developer, or technical evaluator who wants to run Pilot locally, create a workspace, connect the minimum required services, execute one useful workflow, and see what crosses the HELM governance boundary. ## Outcome At the end you should have: - a local Pilot gateway and web UI; - PostgreSQL with pgvector running; - a workspace you can authenticate into; - one task or workflow queued through Pilot; - a clear distinction between founder guidance and operator/self-hosting guidance; - one governed action that produces an allow, deny, escalation, or health signal from the HELM path when configured. ## First Workflow Path ```mermaid flowchart TD subgraph Ingestion["1. Ingestion & Context Plane"] Clone["Clone Pilot"] Workspace["Create workspace"] Mode["Choose Discover, Decide, Build, Launch, or Apply"] end subgraph Evaluation["2. Evaluation & Policy Plane"] Trust["HELM-governed action check"] end subgraph Execution["3. Execution & Verdict Plane"] Setup["Run setup script"] Task["Run one useful workflow"] end subgraph Ledger["4. Tamper-Evident Ledger Plane"] Output["Artifact, receipt, or denial reason"] end %% Operational Flow Edges Clone --> Setup Setup --> Workspace Workspace --> Mode Mode --> Task Task --> Trust Trust --> Output %% Premium Styling Rules style Setup fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff style Task fill:#3182ce,stroke:#2b6cb0,stroke-width:2px,color:#fff style Trust fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#fff style Output fill:#2f855a,stroke:#276749,stroke-width:2px,color:#fff ``` ## Source Truth This onboarding page is backed by: - `docs/self-hosting.md` - `docs/env-reference.md` - `docs/api.md` - `docs/helm-integration.md` - `docs/security.md` - `services/gateway/` - `services/orchestrator/` - `packages/db/` - `packs/founder_ops.v1.json` If a command here differs from setup scripts or service code, update this page. ## 1. Install The Minimal Stack ```bash git clone https://github.com/Mindburn-Labs/pilot.git cd pilot bash scripts/setup.sh ``` The setup script checks prerequisites, generates local secrets, starts the database, runs migrations, and tests the API. The gateway runs on port `3100`; the web UI runs on port `3000`. For local development without Docker, use the operator-focused [Self-Hosting](/self-hosting) page. ## 2. Verify Runtime Health ```bash curl http://localhost:3100/health | jq ``` Expected output includes overall status and checks for the database and job queue. If HELM is configured, health should also show whether the governance sidecar is reachable. ## 3. Create Or Enter A Workspace Use the web UI or auth API to create a workspace. Workspace context is the unit for tasks, operators, connector grants, approvals, audit events, and budget policy. For API-driven setup, request and verify an email code: ```bash curl -sS -X POST http://localhost:3100/api/auth/email/request \ -H 'Content-Type: application/json' \ -d '{"email":"founder@example.com"}' | jq ``` Development mode may return a code directly. Production should send it through the configured email provider. ## 4. Run One Useful Workflow Start with a low-risk task: - Discover: score an opportunity or search knowledge. - Decide: compare options and produce a decision artifact. - Build: create a structured task or implementation plan. - Launch: create a deploy target or launch checklist artifact. - Apply: draft an application section for review. For a simple API smoke: ```bash curl -sS http://localhost:3100/api/product/modes | jq ``` Then create a task from the UI or `POST /api/tasks` with a workspace ID. ## 5. Understand The Governed Action Pilot can run with direct providers for local development, but production should route through HELM. When `HELM_GOVERNANCE_URL` and `HELM_FAIL_CLOSED=1` are set, LLM inference and consequential tool execution are expected to pass through the HELM boundary. You should see one of four outcomes: | Outcome | Meaning | | --- | --- | | allow | action passed policy and may execute | | deny | action was blocked with a reason | | escalate | approval is required before execution | | unreachable/degraded | HELM is required but unavailable, so Pilot fails closed | ## Founder Vs Operator Docs | Need | Page | | --- | --- | | run the first workflow | this page | | operate Docker, Postgres, backups, and upgrades | [Self-Hosting](/self-hosting) | | inspect API groups and payloads | [API Reference](/reference) | | understand governance boundary | HELM Integration (protected staff doc) | | harden secrets, sessions, and connectors | [Security](/security) | ## Troubleshooting | Symptom | Likely Cause | Fix | | --- | --- | --- | | setup script fails on prerequisites | Docker, Node, or Python runtime is missing | install prerequisites and rerun the setup script | | `/health` reports database failure | PostgreSQL is stopped or `DATABASE_URL` is wrong | start the compose database and check env reference | | task completes without useful output | no LLM provider or HELM upstream is configured | configure HELM for production or direct provider keys for local development | | governed action fails closed | HELM is required but unreachable | check `HELM_GOVERNANCE_URL`, `HELM_HEALTH_URL`, and sidecar logs | | Telegram workflow does not respond | bot token or webhook is missing | use the integrations page and Telegram setup steps | --- title: "Telegram Mini App" canonical: "https://pilot.docs.mindburn.org/telegram-miniapp" source: "pilot/docs/telegram-miniapp-8.md" edit: "https://github.com/Mindburn-Labs/pilot/edit/main/docs/telegram-miniapp-8.md" section: "integrations" access: "public" sensitivity: "public" last_reviewed: "unknown" checksum_sha256: "sha256:11e7ad115bfbd12017a75482319fe8cc50025e327311c4fe45a259cf378cf54c" build_timestamp: "2026-05-24T13:42:52.320Z" --- # Telegram Mini App 8.0 Surface Status: implemented as optional capability wrappers. Linear: MIN-272, MIN-258 ## Source Check Telegram Bot API 8.0 added Mini App support for fullscreen mode, persistent device storage, secure local storage, and home-screen shortcuts. Primary source: - https://core.telegram.org/bots/api-changelog ## Repository Changes - `apps/telegram-miniapp/src/telegram-capabilities.ts` wraps fullscreen, home-screen, DeviceStorage, and SecureStorage with feature detection. - The Mini App stores only non-secret UI hints client-side: - last active tab in DeviceStorage; - workspace/user/session metadata in SecureStorage. - The JWT remains in memory via `setAuthToken`; OAuth refresh tokens still belong behind server-side connectors and HELM-governed flows. - The home dashboard exposes fullscreen and home-screen actions when Telegram reports support. ## Boundary Notes Mini App storage is not a replacement for PostgreSQL, Drizzle, connector token storage, or HELM approvals. External actions still enter through gateway/orchestrator and are governed before execution. ## Public Mini App Checklist The public Mini App docs should cover only the safe setup and user-facing integration path: Telegram Web App configuration, callback origin, login handoff, workspace selection, task creation, approval visibility, and error handling. Managed bot provisioning, webhook secrets, child bot token rotation, and production support flows belong in authenticated operator documentation. Keep examples focused on what a self-hosted developer can reproduce locally: configure the bot, open the Mini App, authenticate, create a task, confirm the task appears in the gateway, and verify that governed actions record audit/evidence metadata. ## Expected Output After setup, a developer should be able to open the Mini App from Telegram, receive a valid Telegram initData login, resolve a workspace, create or inspect a task, and see predictable errors for missing bot token, invalid origin, expired session, or unavailable gateway. Capture browser console output, gateway request ID, sanitized initData shape, and workspace ID for diagnostics. Never paste bot tokens, webhook secrets, raw Telegram user payloads, or managed support-bot routes into anonymous docs or LLM exports. ## Browser And Mobile Checks Include a simple smoke path for desktop Telegram, mobile Telegram, and browser preview. The expected page should fit small screens, preserve safe-area padding, recover from offline gateway errors, and show the same workspace/task state after reload. Public docs should name the diagnostic IDs to collect, while protected docs own bot provisioning, token rotation, support escalation, and child bot lifecycle details. ## Audience Split Founder guidance covers opening the Mini App, choosing a workspace, creating a task, reviewing approvals, and understanding why an action was blocked. Operator guidance covers environment variables, bot configuration, origin checks, gateway reachability, logging, and rollback to a browser-only workflow when Telegram is unavailable. Each section should name the visible success signal and the first safe diagnostic to collect. ## Troubleshooting If login fails, check bot token configuration, Telegram auth age, allowed origin, gateway logs, and workspace membership. If the Mini App renders but actions fail, capture the request ID, current workspace ID, task ID, response status, and sanitized payload shape. Do not include private Telegram user payloads or managed support-bot webhook details in public support artifacts.