feat: Moss Integration for LiveKit Agents

[CoderOMaster]

PR Description: Moss Integration for LiveKit Agents

Description

This PR introduces livekit-plugins-moss, integrating Moss with LiveKit Agents.

The problem: Voice agents have a ~800ms total latency budget to feel natural. Traditional RAG pipelines add 100–500ms of network round-trips per retrieval call, making real-time knowledge retrieval impractical during live conversations.

The solution: Moss is an edge-native semantic search runtime that runs in the same process as your LiveKit agent. Once assets are downloaded, search executes locally with sub-10ms latency and zero network hops - making it practical to retrieve knowledge mid-conversation without breaking conversational flow.

What's New

• New Plugin (livekit-plugins-moss): A dedicated plugin for Moss integration.
• MossClient: An asynchronous wrapper around the inferedge-moss SDK, designed for LiveKit's event-driven architecture.
• Type Definitions: Comprehensive type hints for all operations (IndexInfo, DocumentInfo, SearchResult, etc.).

Key Capabilities

Real-Time Semantic Search

• Sub-10ms retrieval latency — search runs locally alongside the agent, not over the network
• Hybrid search with tunable alpha parameter: blend semantic similarity (α=1.0) with keyword matching (α=0.0) to balance precision and recall for your use case

Index Lifecycle Management

• Create, delete, and list indexes
• Get detailed index metadata
• Explicitly load/unload indexes from memory for resource control

Document Management

• Add, delete, and retrieve documents within indexes
• Moss handles indexing, packaging, and distribution of your knowledge base

Async-First API

• All I/O operations are non-blocking async/await, ensuring agent responsiveness during live sessions

Privacy & Offline Support

• Data stays local once indexed — no queries leave the device
• Works offline after initial asset download

Usage

Here is a simple example of how to use the MossClient in your agent:

import asyncio
import os
from livekit.plugins.moss import MossClient, DocumentInfo

async def main():
    # Ensure env vars are set: MOSS_PROJECT_ID, MOSS_PROJECT_KEY
    # Client will automatically use these environment variables if not passed explicitly
    client = MossClient()
    index_name = "agent-knowledge-base"

    # 1. Create an index with initial documents
    initial_docs = [
        DocumentInfo(id="2",text="LiveKit is an open source project for real-time communication."),
        DocumentInfo(id="3",text="Moss provides semantic search capabilities for AI agents."),
    ]
    print(f"Creating index '{index_name}'...")
    await client.create_index(index_name, initial_docs)

    # 2. Add more documents later
    new_docs = [
        DocumentInfo(id="1",text="Agents can use plugins to extend their functionality.")
    ]
    await client.add_documents(index_name, new_docs, options=None)

    # 3. Query the index (auto-loads if needed)
    query_text = "What is Moss offering?"
    print(f"Querying: '{query_text}'")
    results = await client.query(index_name, query_text, top_k=2)

    for doc in results.docs:
        print(f"{doc.id}: {doc.text} (score: {doc.score:.3f})")

    # 4. Cleanup
    await client.delete_index(index_name)
    print("Index deleted.")

if __name__ == "__main__":
    asyncio.run(main())

Environment Variables

The client relies on the following environment variables for authentication:

Variable	Description
MOSS_PROJECT_ID	Your Moss Project ID. Found in the Moss dashboard.
MOSS_PROJECT_KEY	Your Moss Secret Key. Used for API authentication.

[CLAassistant]

All committers have signed the CLA.

[r4ghu]

nit: remove this change

[r4ghu]

We have an updated Python version b14 - https://pypi.org/project/inferedge-moss/

[r4ghu]

3.10 and above

[r4ghu]

remove this line.

[r4ghu]

https://moss.dev

[r4ghu]

If this is removed, then please update this documentation.