| | |
| | | |
| | | `lmrobot-app` remains the business authority for call state, role permission, greeting prepare/consume, ASR, prompt/history, LLM, TTS, message persistence, activity, diagnostics and reasonCode. |
| | | |
| | | ## Turn stream fixture fast path |
| | | |
| | | For TTS streaming contract changes, run the fixture validator before Docker or iPhone smoke: |
| | | |
| | | ```bash |
| | | node tools/validate-turn-stream-fixture.mjs fixtures/turn-stream-happy.ndjson |
| | | node tools/validate-turn-stream-fixture.mjs fixtures/turn-stream-mp3-chunks.ndjson |
| | | node tools/validate-asr-realtime-fixture.mjs fixtures/asr-realtime-happy.ndjson |
| | | ``` |
| | | |
| | | The fixture validates only the protocol shape and fast-path invariants: |
| | | |
| | | - `reply_playback_mode_selected` appears before audio chunks. |
| | | - `reply_playback_started` appears before the first audio chunk. |
| | | - `reply_audio_chunk.audioChunk.format` is `pcm_s16le`, `mp3`, `mpeg` or `wav`. |
| | | - `pcm_s16le` chunks are `16000Hz` or `48000Hz` mono and 16-bit aligned. |
| | | - Encoded chunks such as `mp3` must have non-empty payload bytes. This validates contract shape only; decoder and LiveKit write behavior still require runtime smoke. |
| | | - A terminal event exists. |
| | | |
| | | This is not a media smoke. Docker build, LiveKit room join and iPhone `first_reply_remote_audio` still remain the runtime acceptance path. |
| | | |
| | | ## LiveKit Data Message contract |
| | | |
| | | The helper publishes reply playback state through LiveKit reliable Data Message: |
| | | |
| | | ```text |
| | | topic = combrabo_voice.reply_state |
| | | ``` |
| | | |
| | | The topic is the LiveKit channel. The payload `type` is the business message type and must not reuse the topic value: |
| | | |
| | | ```json |
| | | { |
| | | "type": "reply_state", |
| | | "schemaVersion": "1.0", |
| | | "callId": "cv_xxx", |
| | | "traceId": "trace_xxx", |
| | | "turnId": "turn_xxx", |
| | | "replyPlaybackMode": "streaming_tts", |
| | | "state": "reply_playback_started", |
| | | "seq": 3, |
| | | "tsMs": 1234567890 |
| | | } |
| | | ``` |
| | | |
| | | ## Runtime modes |
| | | |
| | | The helper has two runtime modes: |
| | | |
| | | - `worker` mode: the original one-call process. It reads `CV_*` variables and joins one LiveKit room. |
| | | - `service` mode: a long-running control-plane wrapper. It exposes internal HTTP endpoints and spawns one worker child process per session. |
| | | |
| | | Service mode is enabled by either: |
| | | |
| | | ```bash |
| | | CV_HELPER_SERVICE_ENABLED=true ./combrabo-voice-runtime-helper |
| | | ./combrabo-voice-runtime-helper service |
| | | ``` |
| | | |
| | | Service mode does not change the media worker boundary. It only replaces `lmrobot-app -> ProcessBuilder` with `lmrobot-app -> helper HTTP control plane`. |
| | | |
| | | ## Bot audio output profile |
| | | |
| | | The bot output profile is a helper deployment setting, not a per-call business field. Worker mode reads it directly from `CV_*` variables. Service mode workers inherit it from the helper service process. |
| | | |
| | | For the current Lmtest / Combrabo Voice dynamic reply target, the business chain is fixed to `16000Hz / mono / pcm_s16le` from Java stream bridge to helper bot-track output. The helper must not explicitly convert dynamic reply output to `48000Hz` in this path. `livekit-48k` is retained only for historical paths or emergency rollback, not as this round's target profile. |
| | | |
| | | | Profile | NativeAudioSource | Purpose | |
| | | | --- | --- | --- | |
| | | | `pcm-16k` | `16000Hz / mono` | Default and current dynamic reply target profile. | |
| | | | `livekit-48k` | `48000Hz / mono` | Historical / emergency rollback profile only. | |
| | | | `custom` | `CV_BOT_SAMPLE_RATE_HZ / CV_BOT_NUM_CHANNELS` | Local debug only unless promoted by a separate decision. | |
| | | |
| | | ```bash |
| | | CV_BOT_AUDIO_PROFILE=pcm-16k |
| | | CV_BOT_AUDIO_PROFILE=livekit-48k |
| | | CV_BOT_AUDIO_PROFILE=custom CV_BOT_SAMPLE_RATE_HZ=16000 CV_BOT_NUM_CHANNELS=1 |
| | | ``` |
| | | |
| | | `bot_track_ready` is the authoritative runtime signal for the selected profile: |
| | | |
| | | ```json |
| | | { |
| | | "eventType": "bot_track_ready", |
| | | "eventPayload": { |
| | | "trackName": "bot-main-audio", |
| | | "audioProfile": "pcm-16k", |
| | | "sampleRate": 16000, |
| | | "numChannels": 1 |
| | | } |
| | | } |
| | | ``` |
| | | |
| | | Dynamic streaming reply completion must expose both source and target audio metadata: |
| | | |
| | | ```json |
| | | { |
| | | "eventType": "bot_reply_audio_write_finished", |
| | | "eventPayload": { |
| | | "format": "pcm_s16le", |
| | | "sourceSampleRate": 16000, |
| | | "sourceChannels": 1, |
| | | "targetAudioProfile": "pcm-16k", |
| | | "targetSampleRate": 16000, |
| | | "targetChannels": 1, |
| | | "networkChunkCount": 68, |
| | | "debugSourcePath": "/tmp/.../stream-reply-turn-0001-source.wav", |
| | | "debugPcmWavPath": "/tmp/.../stream-reply-turn-0001-target.wav" |
| | | } |
| | | } |
| | | ``` |
| | | |
| | | This profile affects bot output only. User audio observation, VAD and turn artifacts remain `48000Hz / mono` in this phase. |
| | | |
| | | The current acceptance requires both `stream-reply-{turnId}-source.wav` and `stream-reply-{turnId}-target.wav` to be `16000Hz / mono / pcm_s16le`. If the target dump is `48000Hz`, the dynamic reply 16k chain is not complete. |
| | | |
| | | ## Service mode endpoints |
| | | |
| | | All session endpoints require: |
| | | |
| | | ```http |
| | | Authorization: Bearer {helperAuthToken} |
| | | X-Voice-Trace-Id: {traceId} |
| | | ``` |
| | | |
| | | ### Health |
| | | |
| | | ```http |
| | | GET /internal/combrabo-voice/health |
| | | ``` |
| | | |
| | | For Jenkins and ops probes, `/health` is also supported and returns the same body. |
| | | |
| | | Response: |
| | | |
| | | ```json |
| | | { |
| | | "code": 0, |
| | | "msg": "", |
| | | "data": { |
| | | "status": "UP", |
| | | "version": "0.1.0", |
| | | "mode": "service" |
| | | } |
| | | } |
| | | ``` |
| | | |
| | | ### Start session |
| | | |
| | | ```http |
| | | POST /internal/combrabo-voice/sessions/start |
| | | Idempotency-Key: {callId} |
| | | Content-Type: application/json |
| | | ``` |
| | | |
| | | Request body follows the `lmrobot-app` service-mode contract: |
| | | |
| | | ```json |
| | | { |
| | | "callId": "cv_xxx", |
| | | "traceId": "trace_xxx", |
| | | "runtimeSessionNonce": "nonce_xxx", |
| | | "authProfile": "local-dev", |
| | | "livekit": { |
| | | "url": "ws://127.0.0.1:7880", |
| | | "roomId": "room_xxx", |
| | | "botToken": "dynamic_bot_token", |
| | | "botParticipantIdentity": "bot_xxx", |
| | | "userParticipantIdentity": "user_xxx" |
| | | }, |
| | | "turnBridge": { |
| | | "url": "http://127.0.0.1:19102/internal/sdk/combrabo-voice/runtime/turns/stream" |
| | | }, |
| | | "audio": { |
| | | "firstAudioSource": "fixed_greeting_tts", |
| | | "greetingAudio": { |
| | | "type": "local_file", |
| | | "pathRef": "greeting/cv_xxx.mp3", |
| | | "format": "mp3" |
| | | } |
| | | }, |
| | | "runtime": { |
| | | "turnArtifactDir": "/tmp/combrabo-voice/artifacts", |
| | | "audioDebugDumpEnabled": false |
| | | } |
| | | } |
| | | ``` |
| | | |
| | | `botToken` is dynamic LiveKit connection material. It is accepted only through this internal control plane and must not be logged. |
| | | |
| | | Success response: |
| | | |
| | | ```json |
| | | { |
| | | "code": 0, |
| | | "msg": "", |
| | | "data": { |
| | | "callId": "cv_xxx", |
| | | "status": "STARTED", |
| | | "runtimeSessionId": "rt_cv_xxx", |
| | | "botParticipantJoined": true, |
| | | "botTrackReady": true, |
| | | "firstAudioSource": "fixed_greeting_tts" |
| | | } |
| | | } |
| | | ``` |
| | | |
| | | `sessions/start` must not treat a spawned process as ready. In service mode the worker emits |
| | | `cv_activity` lines on stdout. The service updates its session registry from these events and |
| | | waits for: |
| | | |
| | | - `bot_participant_joined` |
| | | - `bot_track_ready` |
| | | |
| | | Only after both are observed can `botParticipantJoined=true` and `botTrackReady=true` be returned. |
| | | If the worker exits first, return `RUNTIME_START_FAILED`; if the ready window expires, return |
| | | `RUNTIME_START_TIMEOUT`. |
| | | |
| | | ### Query session |
| | | |
| | | ```http |
| | | GET /internal/combrabo-voice/sessions/{callId} |
| | | ``` |
| | | |
| | | The response contains only state aliases, never token, roomId or participantIdentity. |
| | | |
| | | ### Stop session |
| | | |
| | | ```http |
| | | POST /internal/combrabo-voice/sessions/{callId}/stop |
| | | Content-Type: application/json |
| | | ``` |
| | | |
| | | ```json |
| | | { |
| | | "reason": "client_end", |
| | | "runtimeSessionNonce": "nonce_xxx" |
| | | } |
| | | ``` |
| | | |
| | | Repeated stop returns `code=0` with `alreadyStopped=true`. If nonce mismatches, helper returns `RUNTIME_SESSION_MISMATCH`. |
| | | |
| | | ## Service mode auth profiles |
| | | |
| | | `authProfile` is an alias, not a token. The first version allows `default`, `local-dev` and `dev`. |
| | | |
| | | For local smoke, helper accepts these environment variables: |
| | | |
| | | ```bash |
| | | CV_HELPER_AUTH_TOKEN=local-helper-token |
| | | CV_RUNTIME_TURN_BRIDGE_TOKEN=local-turn-bridge-token |
| | | ``` |
| | | |
| | | Profile-specific override is supported by suffix: |
| | | |
| | | ```bash |
| | | CV_HELPER_AUTH_TOKEN_LOCAL_DEV=local-helper-token |
| | | CV_TURN_BRIDGE_TOKEN_LOCAL_DEV=local-turn-bridge-token |
| | | ``` |
| | | |
| | | The helper never receives `turnBridgeToken` in the `sessions/start` body. It resolves the token from its own deployment config, using `authProfile`. |
| | | |
| | | ## Required environment variables |
| | | |
| | | | Name | Source | Notes | |