edit | blame | history | raw

Runtime Contract

This repository contains only the Combrabo Voice LiveKit runtime helper. It is a media worker, not a business service.

Responsibilities

  • Connect to the LiveKit room using bot credentials generated by lmrobot-app.
  • Publish one bot-main-audio track.
  • Play prepared greeting audio from CV_GREETING_AUDIO_FILE or CV_GREETING_AUDIO_URL.
  • Receive user audio in S7 and pass one completed voice turn to the Java turn bridge.
  • Publish reliable LiveKit Data Message for device_output smoke and later voice command output.

Non-goals

  • No SDK token authentication.
  • No database access.
  • No order, entitlement or billing logic.
  • No prompt, ASR, LLM, TTS or message persistence authority.
  • No LiveKit server modification.

lmrobot-app remains the business authority for call state, role permission, greeting prepare/consume, ASR, prompt/history, LLM, TTS, message persistence, activity, diagnostics and reasonCode.

Turn stream fixture fast path

For TTS streaming contract changes, run the fixture validator before Docker or iPhone smoke:

node tools/validate-turn-stream-fixture.mjs fixtures/turn-stream-happy.ndjson
node tools/validate-turn-stream-fixture.mjs fixtures/turn-stream-mp3-chunks.ndjson

The fixture validates only the protocol shape and fast-path invariants:

  • reply_playback_mode_selected appears before audio chunks.
  • reply_playback_started appears before the first audio chunk.
  • reply_audio_chunk.audioChunk.format is pcm_s16le, mp3, mpeg or wav.
  • pcm_s16le chunks are 16000Hz or 48000Hz mono and 16-bit aligned.
  • Encoded chunks such as mp3 must have non-empty payload bytes. This validates contract shape only; decoder and LiveKit write behavior still require runtime smoke.
  • A terminal event exists.

This is not a media smoke. Docker build, LiveKit room join and iPhone first_reply_remote_audio still remain the runtime acceptance path.

LiveKit Data Message contract

The helper publishes reply playback state through LiveKit reliable Data Message:

topic = combrabo_voice.reply_state

The topic is the LiveKit channel. The payload type is the business message type and must not reuse the topic value:

{
  "type": "reply_state",
  "schemaVersion": "1.0",
  "callId": "cv_xxx",
  "traceId": "trace_xxx",
  "turnId": "turn_xxx",
  "replyPlaybackMode": "streaming_tts",
  "state": "reply_playback_started",
  "seq": 3,
  "tsMs": 1234567890
}

Runtime modes

The helper has two runtime modes:

  • worker mode: the original one-call process. It reads CV_* variables and joins one LiveKit room.
  • service mode: a long-running control-plane wrapper. It exposes internal HTTP endpoints and spawns one worker child process per session.

Service mode is enabled by either:

CV_HELPER_SERVICE_ENABLED=true ./combrabo-voice-runtime-helper
./combrabo-voice-runtime-helper service

Service mode does not change the media worker boundary. It only replaces lmrobot-app -> ProcessBuilder with lmrobot-app -> helper HTTP control plane.

Bot audio output profile

The bot output profile is a helper deployment setting, not a per-call business field. Worker mode reads it directly from CV_* variables. Service mode workers inherit it from the helper service process.

For the current Lmtest / Combrabo Voice dynamic reply target, the business chain is fixed to 16000Hz / mono / pcm_s16le from Java stream bridge to helper bot-track output. The helper must not explicitly convert dynamic reply output to 48000Hz in this path. livekit-48k is retained only for historical paths or emergency rollback, not as this round's target profile.

Profile NativeAudioSource Purpose
pcm-16k 16000Hz / mono Default and current dynamic reply target profile.
livekit-48k 48000Hz / mono Historical / emergency rollback profile only.
custom CV_BOT_SAMPLE_RATE_HZ / CV_BOT_NUM_CHANNELS Local debug only unless promoted by a separate decision. 
CV_BOT_AUDIO_PROFILE=pcm-16k
CV_BOT_AUDIO_PROFILE=livekit-48k
CV_BOT_AUDIO_PROFILE=custom CV_BOT_SAMPLE_RATE_HZ=16000 CV_BOT_NUM_CHANNELS=1

bot_track_ready is the authoritative runtime signal for the selected profile:

{
  "eventType": "bot_track_ready",
  "eventPayload": {
    "trackName": "bot-main-audio",
    "audioProfile": "pcm-16k",
    "sampleRate": 16000,
    "numChannels": 1
  }
}

Dynamic streaming reply completion must expose both source and target audio metadata:

{
  "eventType": "bot_reply_audio_write_finished",
  "eventPayload": {
    "format": "pcm_s16le",
    "sourceSampleRate": 16000,
    "sourceChannels": 1,
    "targetAudioProfile": "pcm-16k",
    "targetSampleRate": 16000,
    "targetChannels": 1,
    "networkChunkCount": 68,
    "debugSourcePath": "/tmp/.../stream-reply-turn-0001-source.wav",
    "debugPcmWavPath": "/tmp/.../stream-reply-turn-0001-target.wav"
  }
}

This profile affects bot output only. User audio observation, VAD and turn artifacts remain 48000Hz / mono in this phase.

The current acceptance requires both stream-reply-{turnId}-source.wav and stream-reply-{turnId}-target.wav to be 16000Hz / mono / pcm_s16le. If the target dump is 48000Hz, the dynamic reply 16k chain is not complete.

Service mode endpoints

All session endpoints require:

Authorization: Bearer {helperAuthToken}
X-Voice-Trace-Id: {traceId}

Health

GET /internal/combrabo-voice/health

For Jenkins and ops probes, /health is also supported and returns the same body.

Response:

{
  "code": 0,
  "msg": "",
  "data": {
    "status": "UP",
    "version": "0.1.0",
    "mode": "service"
  }
}

Start session

POST /internal/combrabo-voice/sessions/start
Idempotency-Key: {callId}
Content-Type: application/json

Request body follows the lmrobot-app service-mode contract:

{
  "callId": "cv_xxx",
  "traceId": "trace_xxx",
  "runtimeSessionNonce": "nonce_xxx",
  "authProfile": "local-dev",
  "livekit": {
    "url": "ws://127.0.0.1:7880",
    "roomId": "room_xxx",
    "botToken": "dynamic_bot_token",
    "botParticipantIdentity": "bot_xxx",
    "userParticipantIdentity": "user_xxx"
  },
  "turnBridge": {
    "url": "http://127.0.0.1:19102/internal/sdk/combrabo-voice/runtime/turns/stream"
  },
  "audio": {
    "firstAudioSource": "fixed_greeting_tts",
    "greetingAudio": {
      "type": "local_file",
      "pathRef": "greeting/cv_xxx.mp3",
      "format": "mp3"
    }
  },
  "runtime": {
    "turnArtifactDir": "/tmp/combrabo-voice/artifacts",
    "audioDebugDumpEnabled": false
  }
}

botToken is dynamic LiveKit connection material. It is accepted only through this internal control plane and must not be logged.

Success response:

{
  "code": 0,
  "msg": "",
  "data": {
    "callId": "cv_xxx",
    "status": "STARTED",
    "runtimeSessionId": "rt_cv_xxx",
    "botParticipantJoined": true,
    "botTrackReady": true,
    "firstAudioSource": "fixed_greeting_tts"
  }
}

sessions/start must not treat a spawned process as ready. In service mode the worker emits
cv_activity lines on stdout. The service updates its session registry from these events and
waits for:

  • bot_participant_joined
  • bot_track_ready

Only after both are observed can botParticipantJoined=true and botTrackReady=true be returned.
If the worker exits first, return RUNTIME_START_FAILED; if the ready window expires, return
RUNTIME_START_TIMEOUT.

Query session

GET /internal/combrabo-voice/sessions/{callId}

The response contains only state aliases, never token, roomId or participantIdentity.

Stop session

POST /internal/combrabo-voice/sessions/{callId}/stop
Content-Type: application/json

{
  "reason": "client_end",
  "runtimeSessionNonce": "nonce_xxx"
}

Repeated stop returns code=0 with alreadyStopped=true. If nonce mismatches, helper returns RUNTIME_SESSION_MISMATCH.

Service mode auth profiles

authProfile is an alias, not a token. The first version allows default, local-dev and dev.

For local smoke, helper accepts these environment variables:

CV_HELPER_AUTH_TOKEN=local-helper-token
CV_RUNTIME_TURN_BRIDGE_TOKEN=local-turn-bridge-token

Profile-specific override is supported by suffix:

CV_HELPER_AUTH_TOKEN_LOCAL_DEV=local-helper-token
CV_TURN_BRIDGE_TOKEN_LOCAL_DEV=local-turn-bridge-token

The helper never receives turnBridgeToken in the sessions/start body. It resolves the token from its own deployment config, using authProfile.

Required environment variables

Name Source Notes
CV_CALL_ID lmrobot-app Public call alias, not DB id.
CV_TRACE_ID lmrobot-app Used to correlate logs and activity.
CV_LIVEKIT_URL lmrobot-app Helper reachable LiveKit URL.
CV_LIVEKIT_ROOM_ID lmrobot-app Sensitive connection material; do not log raw value in shared docs.
CV_LIVEKIT_BOT_TOKEN lmrobot-app Sensitive connection token.
CV_LIVEKIT_BOT_PARTICIPANT_IDENTITY lmrobot-app Sensitive connection material.

Optional first-audio variables:

Name Notes
CV_FIRST_AUDIO_SOURCE fixed_greeting_tts, cached_tts or diagnostic value.
CV_GREETING_AUDIO_FILE Local WAV / MP3 file path.
CV_GREETING_AUDIO_URL WAV / MP3 URL fetched by helper.
CV_GREETING_TEXT Diagnostic metadata only; do not log user/private content.

Optional device-output smoke variables:

Name Notes
CV_DEVICE_OUTPUT_SMOKE_ENABLED true enables deterministic Data Message smoke. Default false.
CV_DEVICE_OUTPUT_DESTINATION_IDENTITIES Comma-separated target identities. Empty falls back to user identity or room broadcast.

LiveKit Data Message payload

For device output, helper sends reliable Data Message:

{
  "type": "device_output",
  "schemaVersion": "1.0",
  "commandId": "cmd_xxx",
  "callId": "cv_call_xxx",
  "traceId": "trace_xxx",
  "turnId": "turn_xxx",
  "commandCode": "vibration.start",
  "params": {
    "step": 1,
    "durationSec": 3,
    "extension": {}
  },
  "source": {
    "kind": "voice_command"
  }
}

The payload must not include token, room credential, user raw speech, ASR text, LLM prompt or full reply text.