cverse-llm-inference-adapter v0.1.0
adapter
capsule://quake0day/[email protected]
Provides adapters for various Large Language Model (LLM) providers, enabling the core inference service to interact with different LLMs (e.g., OpenAI, Qwen) through a consistent interface.
Owns
- Base LLM plugin interface
- Specific LLM provider integrations (e.g., OpenAI, Qwen)
- Qwen-specific endpoint utilities
Does not own
- The core inference service
- Other AI model types (ASR, TTS, Avatar, etc.)
AI orientation
An AI agent working on this capsule would focus on integrating new LLM providers, optimizing API calls to existing LLMs, or enhancing prompt engineering within the plugin. It requires knowledge of LLM APIs and Python plugin development.
Avoid
- Modifying the core inference service or other AI model types.
Provides
library:llm-plugin— LLM implementations conforming to the inference core's plugin interface.
Dependencies
Capsules
cverse-inference-core>=0.1.0
Invariants (must always hold)
- LLM plugins must correctly generate text responses.
- Plugins must adhere to the defined LLM plugin interface.
Source files (5)
Click any file to view its content; the path on the right shows where the file lands when this capsule is installed.
src/inference/plugins/llm/__init__.py→plugins/llm/__init__.pysrc/inference/plugins/llm/base.py→plugins/llm/base.pysrc/inference/plugins/llm/openai_plugin.py→plugins/llm/openai_plugin.pysrc/inference/plugins/llm/qwen_plugin.py→plugins/llm/qwen_plugin.pysrc/inference/plugins/qwen_endpoint.py→plugins/qwen_endpoint.py
Plus capsule.yaml and
install.json.