cverse-voice-llm-inference-adapter v0.1.0
adapter
capsule://quake0day/[email protected]
Provides adapters for voice-enabled Large Language Models, including specific integrations for Doubao and Qwen Omni, and a Persona Agent framework for complex conversational flows. This allows the core inference service to handle real-time voice interactions.
Owns
- Base VoiceLLM plugin interface
- Doubao VoiceLLM integration (protocol, config, real-time client)
- Qwen Omni real-time integration
- Persona Agent framework (supervisor, subagents, tools, schemas)
Does not own
- The core inference service
- Other AI model types (ASR, LLM, TTS, Avatar, etc.)
AI orientation
An AI agent working on this capsule would focus on integrating new real-time conversational AI models, enhancing the Persona Agent's capabilities (e.g., adding new tools, improving agent reasoning), or optimizing real-time interaction flows. It requires knowledge of conversational AI, agent frameworks, and Python plugin development.
Avoid
- Modifying the core inference service or other AI model types.
Provides
library:voice-llm-plugin— VoiceLLM implementations conforming to the inference core's plugin interface.
Requires
library:llm-pluginfromcverse-llm-inference-adapter— Uses LLM plugins for text-based reasoning within the Persona Agent.
Dependencies
Capsules
cverse-inference-core>=0.1.0cverse-llm-inference-adapter>=0.1.0
Invariants (must always hold)
- VoiceLLM plugins must process real-time audio/text inputs and generate appropriate responses.
- Persona Agent must maintain conversational context and execute tasks reliably.
- Plugins must adhere to the defined VoiceLLM plugin interface.
Source files (17)
Click any file to view its content; the path on the right shows where the file lands when this capsule is installed.
src/inference/plugins/voice_llm/__init__.py→plugins/voice_llm/__init__.pysrc/inference/plugins/voice_llm/base.py→plugins/voice_llm/base.pysrc/inference/plugins/voice_llm/doubao_config.py→plugins/voice_llm/doubao_config.pysrc/inference/plugins/voice_llm/doubao_protocol.py→plugins/voice_llm/doubao_protocol.pysrc/inference/plugins/voice_llm/doubao_realtime.py→plugins/voice_llm/doubao_realtime.pysrc/inference/plugins/voice_llm/persona/__init__.py→plugins/voice_llm/persona/__init__.pysrc/inference/plugins/voice_llm/persona/i18n.py→plugins/voice_llm/persona/i18n.pysrc/inference/plugins/voice_llm/persona/llm.py→plugins/voice_llm/persona/llm.pysrc/inference/plugins/voice_llm/persona/runtime.py→plugins/voice_llm/persona/runtime.pysrc/inference/plugins/voice_llm/persona/schemas.py→plugins/voice_llm/persona/schemas.pysrc/inference/plugins/voice_llm/persona/subagents/__init__.py→plugins/voice_llm/persona/subagents/__init__.pysrc/inference/plugins/voice_llm/persona/subagents/agent.py→plugins/voice_llm/persona/subagents/agent.pysrc/inference/plugins/voice_llm/persona/subagents/default_tools.py→plugins/voice_llm/persona/subagents/default_tools.pysrc/inference/plugins/voice_llm/persona/supervisor.py→plugins/voice_llm/persona/supervisor.pysrc/inference/plugins/voice_llm/persona/tools.py→plugins/voice_llm/persona/tools.pysrc/inference/plugins/voice_llm/persona_agent.py→plugins/voice_llm/persona_agent.pysrc/inference/plugins/voice_llm/qwen_omni_realtime.py→plugins/voice_llm/qwen_omni_realtime.py
Plus capsule.yaml and
install.json.