Telegram Rich-Message Formatting v1 — Design

Status: ✅ APPROVED by NAO 2026-06-14 — implement on its own branch immediately after Lane v2 (#450). · Author: Tycho Brahe · Date: 2026-06-14 Trigger: NAO request — "мы можем улучшить наш телеграм модуль для форматировок?" + tg format.txt (Bot API 10.1 rich-message spec, shipped 2026-06-11). Task: #517 · Sequencing: separate branch off mvp, implemented AFTER Lane v2 (#450) heavy builds settle.

1. Motivation

Alice's LLM emits rich Markdown — headings, tables, bullet/ordered lists, task lists, fenced code, block quotes, and (for Katya's diploma) LaTeX math. Telegram's legacy parse_mode=HTML supports only a small tag set, so today all of that structure is lost or garbled. Bot API 10.1 added sendRichMessage / sendRichMessageDraft, which render Markdown natively. Goal: universal + SOTA — Alice's structure shows up properly, while every client (old or new) still gets a clean message.

2. Current state (verified by code trace via Atlas + Explore, 2026-06-14)

Outbound path:

reply.rs:162  markdown_to_telegram_html(body)            ← lossy transpiler
  → host.rs:1539  markdown_to_telegram_html(body)         ← REDUNDANT 2nd call
    → rest_client.rs:1568  split_message(text, 4096)       ← splits mid-<tag>
      → rest_client.rs:1578  send_message(parse_mode="HTML")

Gaps found:

• Lossy transpiler (crates/modules-connectors/src/connectors/telegram/formatting.rs:25 markdown_to_telegram_html): headings #, tables |…|, task lists - [ ], footnotes, --- rules are stripped; ordered/unordered lists pass through as raw text.
• Chunk-break bug: split_message (crates/core/src/utils/mod.rs:86) at 4096 chars is not tag-aware — an opening <b> near the boundary becomes orphaned → Telegram returns 400 → the bridge falls back to stripping ALL formatting for the whole message (host.rs:1551-1558).
• Double conversion: markdown_to_telegram_html is called at both reply.rs:162 and host.rs:1539.
• No reference anywhere to sendRichMessage / sendRichMessageDraft (greenfield).

3. Constraints (verified against the Bot API 10.1 doc NAO sent)

Old-client behaviour is undocumented (it's client-side, not in the API). We therefore do not assume the server degrades gracefully — the legacy floor is mandatory, not optional.

4. Architecture — ONE unified outbound formatter (not two tracks)

NAO's directive: "universal + SOTA, one ideal formatter." So A (fix legacy) is not a separate skippable task — its essential fixes become the fallback floor of B.

Alice raw markdown (+ [BUTTONS] block)
        │
        ▼
  TelegramOutbound::render(chat_id, markdown)        ← new decision layer
        │
        ├─ extract [BUTTONS] → inline keyboard (unchanged)
        │
        ├─ resolve mode: per-chat `rich_messages` flag + global kill-switch
        │
        ├─ RICH path  ─ build InputRichMessage{ markdown: normalize(md) }
        │              ─ POST sendRichMessage (32768-char chunks if needed)
        │              ─ on error (unsupported / parse / 4xx) → FALL THROUGH ↓
        │
        └─ LEGACY floor (improved "A") ─ universal, always valid:
             ─ tag-aware 4096 chunker: never split inside a tag; re-open
               open tags across chunk boundaries
             ─ headings → <b>bold</b>; lists → • / 1. bullets;
               tables → <pre> monospace; --- → ────────; task lists → ☐/☑
             ─ existing tags (bold/italic/code/pre/quote/spoiler/link) preserved
             ─ POST sendMessage(parse_mode="HTML")

Key cleanups folded in:

• Remove the double markdown_to_telegram_html call (single conversion site).
• Centralise mode resolution + fallback in one module so reply.rs / host.rs / edit paths share one code path.

Module layout

• connectors/telegram/rich.rs — InputRichMessage builder + markdown normalisation + the sendRichMessage / (later) sendRichMessageDraft REST calls.
• connectors/telegram/formatting.rs — upgrade markdown_to_telegram_html (headings/lists/ tables/hr/checkbox) = the legacy floor.
• connectors/telegram/outbound.rs — the decision layer (mode resolve → rich → fallback).
• core/src/utils — tag-aware split_message (or a new split_html_safe).
• rest_client.rs — add send_rich_message; wire outbound into send_response.

5. Scope tiers

v1 (this branch):

• Rich text passthrough: headings, ordered/unordered/task lists, tables, fenced code with language, block quotes, spoilers, links, LaTeX formulas ($…$, $$…$$), collapsible <details> for long replies.
• Improved legacy floor + tag-aware chunking + double-convert dedup.
• Per-chat rich_messages flag + global kill-switch.

v2 (later, separate):

• sendRichMessageDraft + <tg-thinking> native streaming in DMs (replaces 👀 there).
• Rich media (collage/slideshow) once generated images have public HTTPS URLs (C2).
• <tg-map> for location replies.

6. The one genuine decision: default rich on or off?

• Option 1 — rich OFF by default, opt-in per-chat. Safest-universal; zero risk to any client; we flip it on per known-modern chat.
• Option 2 — rich ON by default + attempt-and-fallback. Most-SOTA; assumes modern clients (true for NAO/Katya); legacy floor still catches any failure.

Recommendation: Option 2 — ON by default with a global kill-switch + per-chat override. Audience is modern clients, the legacy floor is always present as the safety net, and it delivers the "сразу идеально" experience NAO asked for. Downgrade to Option 1 trivially if a client problem appears.

DECISION (NAO, 2026-06-14): Option 2 — rich ON by default + global kill-switch + per-chat override + attempt-and-fallback. Locked.

7. Testing

• Golden snapshots (byte-stable): markdown → InputRichMessage.markdown and markdown → improved-legacy-HTML.
• Tag-aware chunker unit tests: tag never split; tags reopened across boundary; UTF-8 safe.
• Live smoke (lab + dev DM + group): heading, table, LaTeX, ordered list, fenced code, a >4096-char reply, and a forced rich-error → legacy-fallback path.
• No regression on the existing formatting.rs tests (preserve current tag behaviour).

8. Sequencing & risk

• Separate branch off `mvp`, own `CARGO_TARGET_DIR` to avoid contention with Lane v2's shared target. Implement after Lane v2 heavy builds settle (disk at 98% / 12G free — two heavy builds at once risk ENOSPC).
• Touches reply.rs + host.rs, which Lane v2 M4b also edits → rebase onto mvp after Lane v2 lands to avoid merge conflicts.
• Lives in prod (openalice-dev) → branch + lab-validate before any deploy; no push/deploy without explicit NAO word.