kb://learnings/2026-05-23-streaming-product-modularization-wavestable2026-05-23

Streaming product modularization wave — overnight blitz 2026-05-23

modularizationrefactorauditpersonavoiceauthchat-bridgestream-hostapprtoopstate-of-the-art

Streaming product modularization wave — overnight blitz 2026-05-23

TL;DR

13 refactor waves + 1 ecosystem fix landed in a single overnight session (~6h wall clock). Roughly 22,000 LOC of monolithic Rust/TypeScript reshaped into ~90 focused submodules across 13 repos. Tests green throughout (~1190 passes across the surface). Zero behavioural changes. -58 net clippy warnings.

NAO's directive: «делаем все идеально OOP state of the art», «граним алмаз — удаляем старое и ненужное, минимальный универсальный код без регрессий».

What landed (by wave)

waverepoLOC deltatestsclippy
7Aopenalice-chat-bridge — main.rs1739 → 76 (-96%)87 → 870 new
7Bopenalice-live-control — recovery.rs672 → 3-module folder17 → 170 new
8openalice-{chat-bridge,live-control} clippy polish-57 warnings104 → 104-57
10openalice-auth — routes.rs3141 → 51 (-98%)29 → 290 new
12Aopenalice-voice — signal/mod.rs530 → 21 (-96%)66 → 660 new
12Bopenalice-tenants — export.rs + auth_proxy.rs + me.rs2028 → 18 submodules24 → 24-1
13Aopenalice-stream-host — main.rs2347 → 82 (-96%)81 → 81-1
13Bopenalice-persona/server — lounge.rs2088 → 7-module folder558 → 5580 new
13Copenalice-app — 3 oversized pages3745 → 774 (-79%)tsc greenn/a
EXTturbopack symlink fix — 4 Next.js reposn/atsc ✅n/a
14Aopenalice-persona/server — sounds.rs + audio.rs + state.rs3765 → 15 submodules558 → 5580 new
14Bopenalice-app — streamThemes.ts1130 → 23 leaf filestsc greenn/a
15Aopenalice-rt — lib.rs + store.rs1857 → 11 submodules78 → 780 new
15Bopenalice-voice-client — client.ts739 → 607 + 3 helpers12 → 120 new

How it was done

Pattern (proven over the night)

  1. Audit first (inspector + sub-agent) — confirm god modules + complexity hotspots.
  2. Dispatch per repo — one parallel sub-agent per crate, never two on the same crate.
  3. Cap parallelism at 3 cargo agents per feedback-cargo-j4-server-safety-2026-05-16. Use -j 4 strictly.
  4. Hard rules per agent: - Zero behavioural changes — only relocate - Tests green at every commit, not just at the end - @feature annotations preserved verbatim - Conventional commits per logical group - pub use re-exports preserve call-sites - Push to mvp (only branch)
  5. Soul-block check on persona — long string literals must show as paired +/- diffs only (move, not edit).
  6. Smart deferral allowed — if a fn body resists clean extraction (closures capturing 6+ private fields), relocate the whole fn into its own file rather than force a per-arm split. Per-arm extraction queued for a later wave that introduces a proper *Session struct.

Per-repo divergences worth noting

  • auth: agent's natural grouping diverged from NAO's brief (added profile.rs, mfa.rs, config.rs, streams.rs, chat_sources.rs, health.rs). Better than my initial layout.
  • persona/lounge: no roles.rs/presence.rs/invites.rs files — those were helpers or value-types, not endpoints. Agent added audit.rs + reports.rs (Phase 5/6) as their own domains.
  • voice/signal: alternative shape chosen (messages + transport_io + negotiation + session) since there was no PeerState enum to extract and ICE handling was 4 lines (not enough for its own file).
  • voice-client/client.ts: 130-LOC onWsMessage switch (CC=41) kept whole — splitting would force exposing 8 private fields across the boundary = worse abstraction. Per-arm extraction blocked on a WsSession struct refactor.
  • stream-host/run_daemon: CC=124 fn relocated whole into daemon/event_loop.rs. Same blocker — needs WsSession-style refactor for per-arm split.

Ecosystem fix: turbopack workspace symlinks

Wave 13-C surfaced that next build --turbopack failed across 4 Next.js apps with «Symlink … points out of the filesystem root». Root cause: node_modules/@openalicelabs/* are symlinks to sibling repos (../../../openalice-sdk) — turbopack security refuses symlinks escaping the project root.

Canonical fix applied to app / live / dashboard / social:

import path from "node:path";
outputFileTracingRoot: path.resolve(__dirname, ".."),

Tells Next.js the parent dir is part of the same trace scope. Webpack continued to work; turbopack now matches. Future-default builder unblocked.

What stayed monolithic (intentional)

These were flagged by audits but deferred — split would be a worse abstraction or out-of-scope:

  • openalice-media (11k LOC DSP / music theory) — 102 star imports indicate re-export anti-pattern, not god-modules. Different problem class.
  • openalice-ui (10k LOC design system) — 26 components, none individually >500 LOC. Issue is breadth, not depth. Splitting would just create more files for the same surface.
  • openalice-broadcasterzero Rust files by design. Dockerfile + entrypoint.sh + Chromium is the product. Not empty — intentional architecture.
  • CC=124 `run_daemon`, CC=61 `handle_ws`, CC=41 `onWsMessage` — relocated whole; per-arm split needs a session-struct refactor first.

Tooling notes

  • openalice-inspector (Python AST tool) — used for audits via sub-agents. Direct main-session use sometimes hits a "No source files found" stale-cache quirk; clearing .openalice-cache/ and re-running fixes it.
  • Server safety: 3 cargo-heavy agents max concurrent per feedback-cargo-j4-server-safety-2026-05-16. Kept the rule strictly tonight — peak was ~9 rustc procs across 2 cargo agents, well within the safety band that historically locked up at 15+ rustc.
  • Persona test gating: cargo test -p persona-server --lib -j 4 -- --test-threads=4 is the standard. 558 tests, all green after every commit in W13B + W14A.

What this unlocks

  • Easier per-feature edits — adding a new chat platform = touch platforms/<x>.rs, not weave into 1739-LOC main.
  • Cleaner per-agent dispatch — future LLM agents working on persona can focus on one submodule without loading 2K-LOC files into context.
  • Atlas-deps drift fix — every @feature annotation preserved verbatim, so atlas-ingestor cron will re-discover and re-link them after next scan.
  • `turbopack` ready for the entire frontend surface.

What this is NOT

This is infrastructure work, not feature work. None of the 13 Mission Control milestones flipped status — the modularization advances them indirectly (cleaner code = faster feature shipping) but doesn't complete any of them.

Per the «state of the art axioms» — minimal, universal, working, no regressions — the splits succeed because they relocate without changing behaviour. The metric isn't "milestones moved" but "future-velocity multiplier earned".

Cross-links