Search

**Scope.** Deep-dive on the voice-AI *agent* platforms that compete with OpenAlice's 3-layer voice stack (`openalice-voice` / `-agent` / `-client`, STT/TTS/voice-clone): **ElevenLabs Conversational AI

> **One sentence:** This page is about teaching a computer to *hear* speech (ASR), to *speak* (TTS), and — at the 2024–2026 frontier — to do both inside a single transformer that listens and talks at

Replace per-utterance multipart POST with streaming Whisper (or Deepgram) for sub-500ms transcript latency.

Spatial WebRTC voice chat via @openalicelabs/voice-client + spatialAudio panner — distance attenuation tuned, occlusion behind portals, mobile audio unlock flow polished.

> **One sentence:** A vision-language-action model is a multimodal LLM that, instead of (or in addition to) emitting words, emits **robot actions** — it looks at a camera image, reads a command like "

Derive real lipsync visemes from TTS audio (Opus PCM → phoneme estimator) instead of the current placeholder array.

Rust + WASM AI-agent-native audio creation platform — audio graph, euclidean rhythms, Schroeder reverb, 9 genre KBs, MIDI export, 1342 WAV samples.

Rust SFU media server for real-time voice rooms (LiveKit-inspired): WebRTC/ICE/DTLS/SRTP signalling, pluggable TTS/STT providers, and transcript dispatch to live-control and social-api.

**Status:** DRAFT — pending NAO review **Author:** Norbert Wiener (alice-core) **Date:** 2026-05-31 **Scope:** `openalice` core — connector inbound gating, coalesce/lane FSM, agentic loop, config.

First commit on openalice-voice — the Rust SFU that powers real-time voice between Alice, the persona, and viewers.

A single LLM call is a black box you can almost reason about: one prompt in, one completion out, a token count on the receipt. An **agent** is a black box wrapped in a loop wrapped in a tree. It calls

Thin, embed-agnostic agent core — the reusable perceive→reason→act loop over openalice-agent-protocol, shared by Persona/World/Live/Social. Extraction TARGET from openalice-agent-lite (embed-specifics — brand RAG, visitor memory, quota, billing, tenants config — stay in agent-lite). M1 scaffold = the AgentCore trait surface + the net-new perception→observation bridge (visual_/spatial_perception → one LLM line, the way voice_utterance becomes a user line). Real ReAct-loop extraction is lab-gated M2/M3.

Rust AI voice participant — STT→A2A LLM→TTS pipeline that joins openalice-voice SFU rooms as an agent.

13 refactor waves + 1 ecosystem fix landed in a single overnight session (~6h wall clock). Roughly **22,000 LOC** of monolithic Rust/TypeScript reshaped into **~90 focused submodules** across **13 rep

Strategic research output. NAO directed «изучи тему всю и предложи лучше варианты и узнаем сходится ли наш взор» — full-context study + proposed options to align with his vision.

End-to-end **BGM (background-music) channel selection** for a streamer's agent. The streamer picks a radio channel in Hub → persona-server spawns a per-agent WebSocket bridge to `radio.blal.pro` → `no

> **One-line summary.** Inside a running language model there are *directions in activation space* that mean human things — "this is honest," "I'm about to refuse," "the sentiment is positive." **Repr

openalice-landing audit-and-fix wave: 15 viewport fixes, added delights, and an infinite hero-chat surface — part of the brand wave that moved the landing visuals to living surfaces.

Export pty_up, pty_active_sessions, pty_spawned_total, and pty_errors_total on /metrics so the monitoring stack can alarm on stuck or crashing PTY children.

OpenAliceLabs operates as a **house of brands**: two distinct identities governed independently. Decisions in one do not silently propagate to the other.

Detect health up→down transitions, post to /incidents feed + Telegram channel for live ops awareness.

Mel-spectrogram → VRM pose stream: ZeroEGGS ONNX gesture synthesis + PFNN procedural locomotion fallback, served over WebSocket to openalice-persona.

Vision perception PRODUCER — turns a camera/screen/scene frame into a consent-gated visual_perception event (agent-protocol v0.2) via an EU-sovereign VLM (mistral-small-3.2 over OpenRouter). The symmetric analog of openalice-stt (audio→text). VlmClient trait + OpenRouterVlm (multimodal) + build_visual_perception (camera-off-until-consent, enforced at the producer) + structured JSON reading + PerceptionCadence (when to read). Live-proven against the real model. The consumer/bridge is in openalice-agent-basic.

Per-repo + org-wide activity feed at /activity — every commit, manifest revision, lifecycle flip across the org in the last fortnight.

Spawn agent on A2A 'join voice room' task instead of relying on VOICE_AGENT_AUTO_JOIN_ROOM bootstrap env.

**Window**: ISO week 2026-W25 (2026-06-16 Mon → 2026-06-22 Sun), UTC. Security hardening, embed launch-readiness, and the modularity campaign dominated the week.

CI/CD harness for testing stateful AI agents — Docker incubator + Inspect-style Task/Solver/Scorer + ed25519 bench-integrity certs + Prompt Evolution Studio (block CRUD, A/B tests, reflexion review).

Wire ElevenLabs streaming WebSocket TTS path so first audio chunk plays before the full sentence is synthesized.

Deny-by-default SSRF guard for the `perform_action` tool's outbound HTTP. The `perform_action` tool dispatches HTTP requests to URLs that originate from a tenant's client-action registry. Even though those URLs are https-validated at registration time in openalice-tenants, agent-lite re-validates at the CALL boundary and additionally resolves the host so a URL can never be used to reach the operator's own internal network (metadata service, docker hostnames, RFC-1918 ranges, loopback, …). SECURITY MODEL (deny-by-default): 1. scheme MUST be exactly `https`. 2. the host MUST be a fully-qualified domain name — NOT `localhost`, NOT a bare single-label hostname (e.g. an internal docker service name like `openalice-tenants-dev`), and NOT a bare IP literal. 3. EVERY address the host resolves to MUST be a public, routable unicast address. If ANY resolved address falls in a private / loopback / link-local / reserved / unspecified / metadata range, the whole URL is rejected (so a "public" name that resolves to an internal IP — DNS rebinding — is also blocked). Blocked IPv4 ranges: 0.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10 (CGNAT), 127.0.0.0/8, 169.254.0.0/16 (link-local, incl. 169.254.169.254 metadata), 172.16.0.0/12, 192.0.0.0/24, 192.0.2.0/24, 192.168.0.0/16, 198.18.0.0/15, 198.51.100.0/24, 203.0.113.0/24, 224.0.0.0/4 (multicast), 240.0.0.0/4 (reserved), 255.255.255.255 (broadcast). Blocked IPv6: ::, ::1 (loopback), fc00::/7 (ULA), fe80::/10 (link-local), multicast, and IPv4-mapped/compatible addresses (re-checked as IPv4). This module is pure + unit-tested: the range checks take an `IpAddr` so they can be exercised exhaustively without any DNS, and the host-shape checks are tested directly. The one impure step (DNS resolution) is isolated in `check_url_safe`, which is deny-by-default on resolve failure.