Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/kernel-language-model-service/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,16 @@
"default": "./dist/ollama/nodejs.cjs"
}
},
"./test-utils": {
"import": {
"types": "./dist/test-utils/index.d.mts",
"default": "./dist/test-utils/index.mjs"
},
"require": {
"types": "./dist/test-utils/index.d.cts",
"default": "./dist/test-utils/index.cjs"
}
},
"./package.json": "./package.json"
},
"files": [
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export { makeQueueService } from './queue/service.ts';
export { makeQueueModel } from './queue/model.ts';
export type { QueueLanguageModel } from './queue/model.ts';
export type { QueueLanguageModelService } from './queue/service.ts';
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Queue-based Language Model Service (Testing Utility)

[`makeQueueService`](./service.ts) is a testing utility that creates a `LanguageModelService` implementation for use in tests. It provides a queue-based language model where responses are manually queued using the `push()` method and consumed by `sample()` calls.

## Usage

1. Create a service using `makeQueueService()`
2. Create a model instance using `makeInstance()`
3. Queue responses using `push()` on the model instance
4. Consume responses by calling `sample()`

Note that `makeInstance` and `sample` ignore their arguments, but expect them nonetheless.

## Examples

### Basic Example

```typescript
import { makeQueueService } from '@ocap/kernel-language-model-service/test-utils';

const service = makeQueueService();
const model = await service.makeInstance({ model: 'test' });

// Queue a response
model.push('Hello, world!');

// Consume the response
const result = await model.sample({ prompt: 'Say hello' });
for await (const chunk of result.stream) {
console.log(chunk.response); // 'Hello, world!'
}
```

### Multiple Queued Responses

```typescript
const service = makeQueueService();
const model = await service.makeInstance({ model: 'test' });

// Queue multiple responses
model.push('First response');
model.push('Second response');

// Each sample() call consumes the next queued response
const first = await model.sample({ prompt: 'test' });
const second = await model.sample({ prompt: 'test' });

// Process streams...
```
Loading
Loading