Search

BGM panner + spatial audio singleton for multi-source scene playback inside persona-web.

Spatial WebRTC voice chat via @openalicelabs/voice-client + spatialAudio panner — distance attenuation tuned, occlusion behind portals, mobile audio unlock flow polished.

Wave M/N of the streaming product — splitscreen layouts, WASAPI capture, BGM/voice volume mixer, Helix EventSub follows, pty/browse compose surfaces. Stream scene modes 8/8 covered.

Rust + WASM AI-agent-native audio creation platform — audio graph, euclidean rhythms, Schroeder reverb, 9 genre KBs, MIDI export, 1342 WAV samples.

End-to-end **BGM (background-music) channel selection** for a streamer's agent. The streamer picks a radio channel in Hub → persona-server spawns a per-agent WebSocket bridge to `radio.blal.pro` → `no

> **One sentence:** This page is about teaching a computer to *hear* speech (ASR), to *speak* (TTS), and — at the 2024–2026 frontier — to do both inside a single transformer that listens and talks at

Replace silent / solid-colour placeholders with royalty-free real BGM, SFX, scene, and default-VRM assets so every preset URL serves usable content out of the box.

Rust SFU media server for real-time voice rooms (LiveKit-inspired): WebRTC/ICE/DTLS/SRTP signalling, pluggable TTS/STT providers, and transcript dispatch to live-control and social-api.

**Scope:** The self-serve / SMB chatbot-widget builder category — our direct competitors as an EU-sovereign, self-hostable AI-agent + **embodied-avatar** website widget. Deep-dives on Chatbase, Tidio

13 refactor waves + 1 ecosystem fix landed in a single overnight session (~6h wall clock). Roughly **22,000 LOC** of monolithic Rust/TypeScript reshaped into **~90 focused submodules** across **13 rep

Webcam → VMC/VRM puppet for human co-hosts streaming alongside Alice (Mediapipe Face Landmarker + Pose).

Server-authoritative weather state — clear/rain/snow/storm presets with smooth blends, particle-driven by WorldParticles, audio cues via soundManager.

Replace per-utterance multipart POST with streaming Whisper (or Deepgram) for sub-500ms transcript latency.

Streaming radio service: shared + per-tenant channels with scheduler, now-playing WS fanout, and tracks CRUD. BGM source for the live page picker and Hub /account/radio BFF.

Derive real lipsync visemes from TTS audio (Opus PCM → phoneme estimator) instead of the current placeholder array.

Spawn agent on A2A 'join voice room' task instead of relying on VOICE_AGENT_AUTO_JOIN_ROOM bootstrap env.

The lab uses a deliberate runtime vocabulary that recurs in design docs, code comments, and agent instructions. This note is the canonical short reference. Charter section 4 holds the long form.

Mirror /v1/embed as /v1/embeddings with OpenAI-compatible request/response shape so off-the-shelf SDKs can target the gateway without custom adapters.

Rust AI voice participant — STT→A2A LLM→TTS pipeline that joins openalice-voice SFU rooms as an agent.

Vision perception PRODUCER — turns a camera/screen/scene frame into a consent-gated visual_perception event (agent-protocol v0.2) via an EU-sovereign VLM (mistral-small-3.2 over OpenRouter). The symmetric analog of openalice-stt (audio→text). VlmClient trait + OpenRouterVlm (multimodal) + build_visual_perception (camera-off-until-consent, enforced at the producer) + structured JSON reading + PerceptionCadence (when to read). Live-proven against the real model. The consumer/bridge is in openalice-agent-basic.

First commit on openalice-voice — the Rust SFU that powers real-time voice between Alice, the persona, and viewers.

Deployable vision PRODUCER — Axum HTTP service wrapping openalice-vision. POST /v1/vision/read (frame → consent-gated visual_perception event from the EU VLM) + GET /health. The TS product frontends (Persona camera / Live render / World scene) call this instead of embedding the Rust crate, like the tts/stt services for audio. Camera-without-consent → 403, the frame is never sent to the model (privacy gate before the VLM call). Live-proven over HTTP. Deploy (Dockerfile + compose + Traefik vision.blal.pro) is the next step.

> **One-line summary.** A Mixture-of-Experts layer replaces one big feed-forward network with *many* smaller ones ("experts") plus a tiny **router** that, for each token, picks just a handful to run.

NAO 2026-05-23 directive: unify VRM + movement + speech + lipsync across live / persona / world / social so they snap together like LEGO. The first concrete cleanup is the parallel-impl problem betwee

> **One-line summary.** Not every token needs the whole model. **Mixture-of-Depths** puts a tiny router in front of each transformer block that, under a fixed budget, lets only the top-`k` most "deser

> **For NAO + anyone building multi-model systems.** This is the deep companion > to [[fusion-and-llm-councils]]. Where that article explains *why* a panel of > models beats one, MoA is the specific,

**Scope.** Deep-dive on the voice-AI *agent* platforms that compete with OpenAlice's 3-layer voice stack (`openalice-voice` / `-agent` / `-client`, STT/TTS/voice-clone): **ElevenLabs Conversational AI

Three-state (Closed → Open → HalfOpen) per-target circuit breaker. Process-local DashMap registry — same target name from any caller returns the same shared state. Configurable trip threshold and cool-down via OA_CB_<TARGET>_THRESHOLD / OA_CB_<TARGET>_COOLDOWN_MS env vars. Prevents cascading failure when downstream services (voice, anim, scheduler) go hard-down; callers fast-fail instead of queuing full-timeout round-trips.

Step 2 deliverable (a) of the Phase-2 research wave. Cross-cutting market-data substrate underneath the three vertical pitch decks (steps 3-5). All numbers are direction-true, not investment-bank-prec

BGM channel selection closed loop: streamer picks a radio channel in Hub → persona-server spawns a per-agent WebSocket bridge to radio.blal.pro → now_playing frames land on the PersonaEvent bus → the live page renders «currently playing …» without a second WS to radio. Per-(tenant, agent) bridge, Drop-aborts-task BridgeHandle, exponential backoff to 30s, startup-restore so deploys don't drop subscriptions. KB note docs the integration.