# /goal — The Factory Floor (the swarm runs itself)

> **A system /goal, not a Lab app.** The north star for turning the ~28
> mutually-unaware launchd cron jobs into a **Factorio-style production line** —
> machines with declared recipes, belts moving typed artifacts, a power grid
> metering shared resources, one git lane, and a live factory map. Read alongside
> [`docs/NORTH-STAR.md`](../NORTH-STAR.md) (the flywheel keystone — the factory is
> what *runs* it cheaply), [`docs/goals/router-model.md`](router-model.md), and the
> subsystem docs in [`factory/README.md`](../../factory/README.md).

A destination to *reach for*, not a spec to implement once.

---

## The reality this goal exists to fix

The swarm's dependency graph is **100% implicit in launchd launch times**. There
is no DAG, no orchestrator, no master runner — just 28 jobs that each *hope* their
upstream already ran. The damage this causes is documented scar tissue:

- **9 generators each run their own `git pull`+`commit`+`push origin master`** on
  overlapping schedules → `.git/index.lock` races, rebase-abort-on-unstaged,
  push rejections, and a **12-day deploy freeze** from branch stranding.
- **Silent fallbacks are the worst class** — claude off launchd's PATH / `$USER`
  unset / a missing Gemini key → **21 garbage-template comics** and un-illustrated
  editorials shipped undetected; the watchdog noticed *hours* later.
- **Stale-by-construction** — `blog-gen` @9AM consumed `trends/` not produced until
  1:30PM, so the editorial read *yesterday's* trends, every day.
- **No owner for shared artifacts** — `feed.xml` was regenerated daily but committed
  by no machine, so RSS froze for days at a time.
- **The repo can't rebuild itself** — 13 installed launchd jobs had **no plist in
  the repo** (launchd was the only source of truth).
- Plus: GPU thermal pile-up (renders 20min→5h), disk-full batch aborts, a 7-day
  NaN-in-JSON outage, a 30MB tracked cache that stalled `git pull`, a dual checkout,
  Qdrant-not-auto-started, and ~$2.4K/mo of API spend against a $200 pool.

Everything below turns "28 jobs that hope" into **one factory that knows** — making
implicit explicit, silent loud, and stale impossible.

> **Status (2026-06-29):** the read-only/additive core is **built and verified** —
> the DAG, the fail-loud preflight, the single committer (shadow), the launchd
> source-of-truth, and a CI guard. Nothing is wired into launchd yet; the cutover
> (M4–M6) is deliberately operator-gated. See the roadmap.

---

## North star (the destination)

1. **One orchestrator runs the machines as a data-readiness DAG.** A consumer fires
   only when *all* its inputs are fresh-for-today (`@today`), in dependency order —
   so `trend-gen` precedes `blog-gen` by construction and a late/failed producer
   *blocks* its consumers instead of letting them publish stale data. Sources
   (arxiv/funding/market/comics) stay time-triggered — the raw-resource patches.
2. **One git lane.** All commits go through a single serialized committer that asserts
   on-master + not-behind-origin and stages only each machine's *declared* outputs —
   killing the 9-way race, branch stranding, the worktree-sweep, and the orphaned
   `feed.xml` in one move.
3. **Nothing degrades silently.** A shared preflight fails *loud* (machine → `blocked`
   + alarm) when a hard dependency is missing; `require_llm` machines abort rather than
   ship template prose. The editorial **Voice Gate stays inviolate.**
4. **A metered power grid.** A single GPU lease (with thermal cooldown + SIGKILL
   escalation) replaces the manual comic stagger; an LLM-$ budget meter downshifts
   Opus→Sonnet→Haiku→the local flywheel model under pressure; a disk governor and the
   git lane round out the grid. Saturation → **backpressure**, never pile-up.
5. **The repo is the source of truth.** `factory/schedule.toml` generates every launchd
   plist; CI fails on drift. Every scheduled machine has a version-controlled recipe.
6. **A live factory map.** The gateway + `status.html` render the DAG as a schematic —
   machine states, belt freshness, power draw, the critical-path bottleneck — with the
   watchdog folded in as machine states and one-click idempotent replay/backfill.

**Reaching it looks like:** a single launchd heartbeat ticks `orchestrator.py`; the
day's content flows source→`signals`→`digest`/editorial with each machine firing the
moment its inputs are ready; one committer pushes once per machine; a claude outage
lights a machine red instead of shipping a template; the operator watches the factory
map fill in — and never touches the editorial.

---

## The roadmap (ordered by leverage)

| M | Milestone | Status | Why / where |
|---|-----------|--------|-------------|
| **M0** | **Make the DAG explicit** — recipe manifests for all 27 machines; `factory.py` (load → topo-sort → readiness) + `orchestrator.py` (`validate`/`graph`/`plan`/`tick --dry-run`). | ✅ **shipped** | Implicit→explicit. The whole thing rests on this. |
| **M1** | **Kill silent fallbacks** — `factory/lib.sh` (one safety floor, replaces the 15× copy-paste) + `preflight.py` (claude/USER/Gemini/Qdrant/disk/on-master, fail-loud). | ✅ core shipped; **rollout open** | `blog-gen` wired + live-validated; **18 entrypoints** still inline. |
| **M2** | **One git lane** — `committer.py` (flock, on-master + not-behind assertions, stages only changed declared outputs, push-retry). | ✅ shipped (shadow) | Cutover (generators stop pushing) is M5/M6. |
| **M3** | **Source-of-truth** — `schedule.toml` ← installed jobs; `gen-plists.py` emit/check; hermetic `check-factory.py` in `verify.sh`. | ✅ **shipped** | Repo can rebuild launchd; CI catches drift (0 drift today). |
| **M4** | **Shadow the orchestrator** beside launchd for ~1 week — log the order + would-commit set it *would* run; launchd still executes. Require **0 divergence** before any cutover. | ⬜ open | Proves DAG == reality at zero risk. **Do next.** |
| **M5** | **Cut over the comics subtree** — the 4 comic jobs → orchestrator + the GPU lease (arbitration replaces the 00/02/04/13:30 manual stagger). GO: comics publish on time 3 days; rollback: re-`launchctl load` the 4 plists. | ⬜ open | Isolated subtree → safest first cutover. |
| **M6** | **Cut over the main bus** — sources keep cron; everything downstream becomes data-triggered; digest renders explicit "section absent today" instead of silently masking. | ⬜ open | Kills stale-by-construction + the masking. |
| **M7** | **Power grid + factory map** — LLM-$ meter + model downshift, disk governor; the live DAG schematic in gateway/`status.html`; fold in the watchdog. | ⬜ open | Observability + the $200-pool fix (Phase 6). |
| **M8** | **Event-driven daemon + backfill** — one KeepAlive process; producer-done emits a belt event that wakes downstream immediately; idempotent replay keyed by `(recipe, date, input-hash)`. | ⬜ open | The latency end-state (Phase 7). |

**Do first:** finish **M1 rollout** (the other 18 entrypoints) and **M4 shadow**
(prove equivalence). M5–M6 are the operator-gated cutover; M7–M8 follow.

---

## The /loop (one self-paced iteration)

This goal is pursued by a recurring **`/loop`** — each iteration is bounded, reversible,
and **operator-gated at every launchd change**. Per iteration:

1. **Read state** — `bash factory/selfcheck.sh` (recipes+DAG, launchd drift, coverage,
   the staged single-committer / shared-lib readiness counts), `orchestrator.py plan`,
   and which entrypoints already `source factory/lib.sh`.
2. **Pick the lowest OPEN milestone.** M1 not done → wire **one** more entrypoint to
   `lib.sh`+`factory_preflight` (behavior-preserving; `bash -n` + run its preamble). M1
   done, M4 not → run `orchestrator.py tick --dry-run` and diff the planned order +
   would-commit set against what launchd actually produced that day; append the diff.
   M4 proven (0 divergence ≥5 days) → propose **one** isolated cutover (comics first)
   **and stop for a human GO/NO-GO** before any `launchctl` change.
3. **Produce** — the wired entrypoint **or** a shadow-equivalence diff **or** a cutover
   proposal (the exact `launchctl load/unload` + rollback), plus a one-line
   `python3 unified-gen/daily_log.py append` note.
4. **Measure** — `factory/selfcheck.sh` still green; shared-lib/`git push` counts moving
   toward their targets (19/19 wired; 0 direct pushers); shadow divergence count.
5. **Stop / checkpoint** — halt when either (a) a **launchctl cutover** is the next step
   → checkpoint for a **human GO** (never `launchctl` autonomously), or (b) shadow shows
   **any divergence** → stop and surface the mismatch instead of cutting over.

**No launchd/git behavior change happens without an operator GO.** The loop proposes
and prepares (with rollback); a human flips the scheduler.

### Running it

```
/loop Advance docs/goals/factory.md by ONE bounded, reversible iteration, then checkpoint:
1) READ: `bash factory/selfcheck.sh`, `python3 factory/orchestrator.py plan`, and which *-gen/generate_*.sh
   already source factory/lib.sh.
2) PICK the lowest OPEN milestone: M1 (not all wired) -> wire ONE more entrypoint to factory/lib.sh +
   factory_preflight, behavior-preserving (bash -n + run its preamble, do NOT run the full generator).
   Else M4 -> `orchestrator.py tick --dry-run` and diff planned order + would-commit set vs what launchd
   produced today; record it. Else (M4 proven, 0 divergence >=5d) -> PREPARE one isolated cutover (comics
   first): the exact launchctl unload/load + rollback, and STOP for human GO.
3) PRODUCE: the wired entrypoint OR a shadow-equivalence diff OR a cutover proposal + a one-line
   `python3 unified-gen/daily_log.py append` note.
4) MEASURE: factory/selfcheck.sh green; shared-lib count toward 19/19; direct-`git push` count toward 0;
   shadow divergence count.
5) STOP/CHECKPOINT: if the next step is a launchctl change -> STOP for human GO (never run launchctl
   autonomously). If shadow shows ANY divergence -> STOP and surface it. Never touch the editorial voice;
   commit only factory/ + entrypoint + CI artifacts. Report what changed + next step, then end.
```

Omit an interval so the loop self-paces. `/loop stop` to halt.

---

## Anti-goals

- **Don't cut over launchd autonomously.** Every `launchctl load/unload` is operator-gated
  with a one-command rollback. The loop *prepares*; a human flips the scheduler.
- **Don't reinvent with a heavy engine.** No Airflow/Dagster/Prefect/Temporal — no daemon,
  DB, server, or build step. Stdlib single-file modules that *reuse* the blackboard,
  watchdog, and gateway. (Make/Snakemake's file-target-DAG idea is borrowed, not adopted.)
- **Don't `git add -A`, ever.** The committer stages only a machine's *declared* outputs —
  the worktree-sweep that ate operator edits and the 30MB-cache-breaks-pull class must stay
  impossible. Coordination state (`swarm-state/`, `*-state/`) is never a committer output.
- **Don't let a consumer run on stale data.** `@today` gating is the point; a late producer
  blocks its consumers and lights up — it does not silently publish yesterday.
- **Don't touch the editorial voice.** The factory *runs* the writer; it never writes. The
  Voice Gate (`check-editorial-model.js`) stays inviolate; `require_llm` machines fail loud
  rather than ship template prose.
- **Don't grow the swarm to fill the DAG.** Smallest factory that keeps the publication alive
  (the North-Star anti-goal). A new machine earns a recipe only when it produces something a
  reader or another machine consumes.

---

## Where it lives

`factory/` — `factory.py` (DAG core), `orchestrator.py` (CLI), `committer.py` (git lane),
`preflight.py` + `lib.sh` (fail-loud floor), `gen-plists.py` + `schedule.toml` (launchd
source-of-truth), `recipes/*.toml` (27 blueprints), `selfcheck.sh` (operator dashboard),
`README.md`. CI guard: `scripts/ci/check-factory.py` (in `verify.sh`). Full design + phased
rollout: the approved plan `snoopy-crunching-fox` (and this /goal supersedes it as the
living north star).