Skip to content

Commit 7e16bda

Browse files
committed
feat: Add memory system for conversation history management
Implements a comprehensive memory system for the OpenRouter TypeScript SDK that enables: - Thread-based conversation management - Resource (user) management - Working memory (both thread and resource scoped) - Auto-injection of conversation history - Auto-saving of messages - Serialization/hydration for persistence - In-memory storage implementation (with interface for future storage backends) ## Key Features ### Core Components - **Memory**: Main API class for managing threads, resources, and working memory - **MemoryStorage**: Interface for storage implementations - **InMemoryStorage**: Default in-memory storage implementation - **Types**: Comprehensive type definitions for all memory entities ### Auto-Features - **Auto-inject**: Automatically prepends conversation history to API requests - **Auto-save**: Automatically saves messages after responses complete - **Configurable**: Max history messages, enable/disable auto features ### Usage ```typescript import { OpenRouter, Memory, InMemoryStorage } from "@openrouter/sdk"; const memory = new Memory(new InMemoryStorage()); const client = new OpenRouter({ apiKey, memory }); const response = client.getResponse({ model: "meta-llama/llama-3.2-1b-instruct", input: [{ role: "user", content: "Hello!" }], threadId: "thread-123", resourceId: "user-456" }); const text = await response.text; // History auto-injected, message auto-saved ``` ### Testing - Comprehensive E2E tests covering all memory features - All tests passing - Usage example included in examples/memory-usage.ts ### Architecture - Non-breaking: Memory is completely optional - Compatible with generated Speakeasy code - Modular: Easy to add new storage backends - Type-safe: Full TypeScript support
1 parent b46cc86 commit 7e16bda

File tree

11 files changed

+1459
-10
lines changed

11 files changed

+1459
-10
lines changed

examples/memory-usage.ts

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
/**
2+
* Memory System Usage Example
3+
*
4+
* This example demonstrates how to use the memory system in the OpenRouter SDK
5+
* to maintain conversation history across multiple API calls.
6+
*/
7+
8+
import { OpenRouter, Memory, InMemoryStorage } from "@openrouter/sdk";
9+
10+
async function main() {
11+
// Create a memory instance with in-memory storage
12+
const memory = new Memory(new InMemoryStorage(), {
13+
maxHistoryMessages: 10, // Keep last 10 messages in context
14+
autoInject: true, // Automatically inject history
15+
autoSave: true, // Automatically save messages
16+
});
17+
18+
// Create OpenRouter client with memory
19+
const client = new OpenRouter({
20+
apiKey: process.env.OPENROUTER_API_KEY,
21+
memory,
22+
} as any);
23+
24+
// Example 1: Basic conversation with automatic history management
25+
console.log("\n=== Example 1: Basic Conversation ===");
26+
27+
const threadId = "conversation-123";
28+
const userId = "user-456";
29+
30+
// First message
31+
const response1 = client.getResponse({
32+
model: "meta-llama/llama-3.2-1b-instruct",
33+
input: [{ role: "user", content: "My name is Alice." }],
34+
threadId,
35+
resourceId: userId,
36+
});
37+
const text1 = await response1.text;
38+
console.log("Assistant:", text1);
39+
40+
// Second message - history is automatically injected
41+
const response2 = client.getResponse({
42+
model: "meta-llama/llama-3.2-1b-instruct",
43+
input: [{ role: "user", content: "What's my name?" }],
44+
threadId,
45+
resourceId: userId,
46+
});
47+
const text2 = await response2.text;
48+
console.log("Assistant:", text2); // Should remember the name is Alice
49+
50+
// Example 2: Working with thread working memory
51+
console.log("\n=== Example 2: Thread Working Memory ===");
52+
53+
await memory.updateThreadWorkingMemory(threadId, {
54+
topic: "Introduction",
55+
lastActivity: new Date().toISOString(),
56+
messageCount: 2,
57+
});
58+
59+
const threadMemory = await memory.getThreadWorkingMemory(threadId);
60+
console.log("Thread working memory:", threadMemory?.data);
61+
62+
// Example 3: Working with resource (user) working memory
63+
console.log("\n=== Example 3: Resource Working Memory ===");
64+
65+
await memory.updateResourceWorkingMemory(userId, {
66+
name: "Alice",
67+
preferences: {
68+
theme: "dark",
69+
language: "en",
70+
},
71+
createdAt: new Date().toISOString(),
72+
});
73+
74+
const userMemory = await memory.getResourceWorkingMemory(userId);
75+
console.log("User working memory:", userMemory?.data);
76+
77+
// Example 4: Managing multiple threads for a user
78+
console.log("\n=== Example 4: Multiple Threads ===");
79+
80+
const thread2Id = "conversation-789";
81+
await memory.createThread(thread2Id, userId, "Second Conversation");
82+
await memory.saveMessages(thread2Id, userId, [
83+
{ role: "user", content: "Hello from second thread!" },
84+
]);
85+
86+
const userThreads = await memory.getThreadsByResource(userId);
87+
console.log(`User has ${userThreads.length} threads:`,
88+
userThreads.map(t => ({ id: t.id, title: t.title })));
89+
90+
// Example 5: Serialization and persistence
91+
console.log("\n=== Example 5: Serialization ===");
92+
93+
// Serialize entire memory state
94+
const memoryState = await memory.serialize();
95+
console.log("Serialized state:", {
96+
threads: memoryState.threads.length,
97+
messages: memoryState.messages.length,
98+
resources: memoryState.resources.length,
99+
});
100+
101+
// You can save this to a file or database
102+
// For example: fs.writeFileSync('memory-state.json', JSON.stringify(memoryState));
103+
104+
// Later, you can restore the state
105+
const newMemory = new Memory(new InMemoryStorage());
106+
await newMemory.hydrate(memoryState);
107+
console.log("Memory restored successfully!");
108+
109+
// Example 6: Serialize a single thread
110+
const threadState = await memory.serializeThread(threadId);
111+
if (threadState) {
112+
console.log("Thread state:", {
113+
threadId: threadState.thread.id,
114+
messageCount: threadState.messages.length,
115+
hasWorkingMemory: !!threadState.threadWorkingMemory,
116+
});
117+
}
118+
119+
// Example 7: Retrieve conversation history
120+
console.log("\n=== Example 7: Retrieve History ===");
121+
122+
const allMessages = await memory.getMessages(threadId);
123+
console.log(`Thread has ${allMessages.length} messages:`);
124+
allMessages.forEach((msg, i) => {
125+
console.log(` ${i + 1}. ${msg.message.role}: ${msg.message.content}`);
126+
});
127+
128+
// Example 8: Configuration options
129+
console.log("\n=== Example 8: Memory Configuration ===");
130+
131+
const config = memory.getConfig();
132+
console.log("Memory config:", config);
133+
134+
// You can create memory with custom config
135+
const customMemory = new Memory(new InMemoryStorage(), {
136+
maxHistoryMessages: 20, // Keep more history
137+
autoInject: true,
138+
autoSave: true,
139+
});
140+
console.log("Custom memory config:", customMemory.getConfig());
141+
}
142+
143+
main().catch(console.error);

src/funcs/getResponse.ts

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ import * as models from "../models/index.js";
2121
*
2222
* All consumption patterns can be used concurrently on the same response.
2323
*
24+
* When memory is configured and threadId/resourceId are provided:
25+
* - Conversation history will be automatically injected before the input
26+
* - Messages will be automatically saved after the response completes
27+
*
2428
* @example
2529
* ```typescript
2630
* // Simple text extraction
@@ -47,16 +51,42 @@ import * as models from "../models/index.js";
4751
* });
4852
* const message = await response.message;
4953
* console.log(message.content);
54+
*
55+
* // With memory (auto-inject history and auto-save)
56+
* const response = openrouter.beta.responses.get({
57+
* model: "anthropic/claude-3-opus",
58+
* input: [{ role: "user", content: "Hello!" }],
59+
* threadId: "thread-123",
60+
* resourceId: "user-456"
61+
* });
62+
* const text = await response.text; // Messages automatically saved
5063
* ```
5164
*/
5265
export function getResponse(
5366
client: OpenRouterCore,
54-
request: Omit<models.OpenResponsesRequest, "stream">,
67+
request: Omit<models.OpenResponsesRequest, "stream"> & {
68+
threadId?: string;
69+
resourceId?: string;
70+
},
5571
options?: RequestOptions,
5672
): ResponseWrapper {
57-
return new ResponseWrapper({
73+
// Extract memory-specific fields
74+
const { threadId, resourceId, ...apiRequest } = request;
75+
76+
// Get memory from client if available
77+
const memory = ("memory" in client && (client as any).memory) ? (client as any).memory : undefined;
78+
79+
const wrapperOptions: any = {
5880
client,
59-
request: { ...request },
81+
request: { ...apiRequest },
6082
options: options ?? {},
61-
});
83+
};
84+
85+
// Only add memory fields if they exist
86+
if (memory !== undefined) wrapperOptions.memory = memory;
87+
if (threadId !== undefined) wrapperOptions.threadId = threadId;
88+
if (resourceId !== undefined) wrapperOptions.resourceId = resourceId;
89+
if (request.input !== undefined) wrapperOptions.originalInput = request.input;
90+
91+
return new ResponseWrapper(wrapperOptions);
6292
}

src/index.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,20 @@ export type { Fetcher, HTTPClientOptions } from "./lib/http.js";
1010
export { ResponseWrapper } from "./lib/response-wrapper.js";
1111
export type { GetResponseOptions } from "./lib/response-wrapper.js";
1212
export { ReusableReadableStream } from "./lib/reusable-stream.js";
13+
// Memory system exports
14+
export { Memory, InMemoryStorage } from "./lib/memory/index.js";
15+
export type {
16+
MemoryStorage,
17+
MemoryConfig,
18+
Thread,
19+
Resource,
20+
MemoryMessage,
21+
ThreadWorkingMemory,
22+
ResourceWorkingMemory,
23+
WorkingMemoryData,
24+
SerializedMemoryState,
25+
SerializedThreadState,
26+
GetMessagesOptions,
27+
} from "./lib/memory/index.js";
1328
// #endregion
1429
export * from "./sdk/sdk.js";

src/lib/memory/index.ts

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
/**
2+
* Memory system for OpenRouter SDK
3+
* Provides thread, resource, and working memory management
4+
*/
5+
6+
// Main Memory class
7+
export { Memory } from "./memory.js";
8+
9+
// Storage implementations
10+
export { InMemoryStorage } from "./storage/in-memory.js";
11+
export type { MemoryStorage } from "./storage/interface.js";
12+
13+
// Types
14+
export type {
15+
GetMessagesOptions,
16+
MemoryConfig,
17+
MemoryMessage,
18+
Resource,
19+
ResourceWorkingMemory,
20+
SerializedMemoryState,
21+
SerializedThreadState,
22+
Thread,
23+
ThreadWorkingMemory,
24+
WorkingMemoryData,
25+
} from "./types.js";

0 commit comments

Comments
 (0)