Using JavaScript to Interface with C++ Dictionaries

[yokofly]

version support: already in every 2.7/2.8/3.0

Dictionary Access Guide: Using JavaScript to Interface with C++ Dictionaries

Overview

This guide explains how to use the JavaScript API to interact with dictionaries backed by C++ in the system. The Dictionary Access feature provides high-performance access to dictionary data through a set of JavaScript functions, supporting both simple and complex key scenarios.

Setup Requirements

Before using the Dictionary Access API, you need to:

1. Create backend storage (mutable streams) to store the actual data
2. Create dictionary objects that reference the storage
3. Ensure your dictionary is properly loaded before first use

1. Create Mutable Streams for Backend Storage

-- Create a mutable stream for complex key dictionary
CREATE MUTABLE STREAM my_database.complex_key_storage (
    -- Key columns
    key1 string,
    key2 string,
    
    -- Value columns
    str_val string,
    int32_val int32,
    int64_val int64,
    float64_val float64 ,
    
    -- Timestamp for when the record was created/updated
    created_at datetime64(3)
) PRIMARY KEY (key1, key2);

-- Create a mutable stream for simple key dictionary
CREATE MUTABLE STREAM my_database.simple_key_storage (
    -- Simple key column
    id uint64,
    
    -- Value columns
    str_val string,
    int32_val int32,
    int64_val int64,
    float64_val float64,
    
    -- Timestamp
    created_at datetime64(3)
) PRIMARY KEY (id);

2. Create Dictionaries

-- Create a complex key dictionary
CREATE DICTIONARY my_database.complex_key_dict (
    -- Key columns
    key1 string,
    key2 string,
    
    -- Value columns
    str_val string,
    int32_val int32,
    int64_val int64,
    float64_val float64,
    
    -- Timestamp
    created_at datetime64(3)
)
PRIMARY KEY (key1, key2)
SOURCE(TIMEPLUS(USER 'username' PASSWORD 'password' DB 'my_database' STREAM 'complex_key_storage'))
LAYOUT(complex_key_direct);

-- Create a simple key dictionary
CREATE DICTIONARY my_database.simple_key_dict (
    -- Key column
    id uint64,
    
    -- Value columns
    str_val string,
    int32_val int32,
    int64_val int64,
    float64_val float64,
    
    -- Timestamp
    created_at datetime64(3)
)
PRIMARY KEY id
SOURCE(TIMEPLUS(USER 'username' PASSWORD 'password' DB 'my_database' STREAM 'simple_key_storage'))
LAYOUT(direct);

Key Functions

The API provides the following core functions for dictionary operations:

Function	Description
getCardinality(dictName)	Returns the number of items in the dictionary
getValue(dictName, key, [columns])	Retrieves a single value from the dictionary
setValue(dictName, key, value)	Stores a single value in the dictionary
batchGetValues(dictName, keys, [columns])	Retrieves multiple values in a batch operation
batchSetValues(dictName, keys, values)	Stores multiple values in a batch operation

Dictionary Types

The system supports two types of dictionaries:

1. Simple Key Dictionary: Uses a single numeric identifier as the key
2. Complex Key Dictionary: Uses multiple fields to form a composite key

Reading and Updating Records

A common pattern is to retrieve a record, modify it, and write it back:

// 1. Get the current record
const record = getValue('my_simple_dict', 1001);

// 2. Modify one or more fields
record.str_val = "Updated string value";
record.int32_val = 500;

// 3. Write the updated record back
setValue('my_simple_dict', 1001, record);

For complex key dictionaries:

// 1. Get the current record
const complexRecord = getValue('my_complex_dict', {key1: 'user123', key2: 'profile'});

// 2. Modify fields
complexRecord.value1 = "Updated user information";
complexRecord.active = false;

// 3. Write back with the same key
setValue('my_complex_dict', {key1: 'user123', key2: 'profile'}, complexRecord);

Reading Data with getValue

// Simple key dictionary lookup
const record = getValue('my_simple_dict', 1001);
console.log(record.str_val, record.int32_val);

// Complex key dictionary lookup
const complexRecord = getValue('my_complex_dict', {key1: 'user123', key2: 'profile'});
console.log(complexRecord.value1, complexRecord.value2);

// Optional: specify only certain columns to retrieve
const partialRecord = getValue('my_dict', 1001, ['str_val', 'int32_val']);

Writing Data with setValue

// Simple key dictionary update
setValue('my_simple_dict', 1001, {
    str_val: 'New string value',
    int32_val: 42,
    created_at: '2023-01-01 12:00:00.000'
});

// Complex key dictionary update
setValue('my_complex_dict', {key1: 'user123', key2: 'profile'}, {
    value1: 'User information',
    value2: 100,
    active: true
});

Recommended Approach: Batch Operations

For best performance, especially with multiple records, we strongly recommend using batch operations:

// Batch get multiple values
const keys = [1001, 1002, 1003, 1004]; // For simple key dictionary
const results = batchGetValues('my_simple_dict', keys);

// Batch set multiple values
const complexKeys = [
    {key1: 'user1', key2: 'profile'},
    {key1: 'user2', key2: 'profile'},
    {key1: 'user3', key2: 'profile'}
];
const values = [
    {value1: 'User 1 data', value2: 101},
    {value1: 'User 2 data', value2: 102},
    {value1: 'User 3 data', value2: 103}
];
batchSetValues('my_complex_dict', complexKeys, values);

Performance Recommendations

1. Use Simple Key Dictionaries when possible for maximum performance
2. Prefer Batch Operations over individual operations for bulk data
3. Specify Only Required Columns in getValue and batchGetValues to reduce data transfer
4. Reuse Dictionary References rather than repeatedly looking up the same dictionary

Working with Dates

Timestamps are automatically converted between datetime64 in the database and JavaScript Date objects:

// Setting a date value
setValue('my_dict', {key1: 'test', key2: 'a'}, {
    created_at: '2023-01-01 12:00:00.000'  // String format
});

// Reading a date value
const result = getValue('my_dict', {key1: 'test', key2: 'a'});
console.log(result.created_at);  // Returns a Date object

Complete Example

Here's a complete example demonstrating the recommended usage:

// Dictionary names with full path
const COMPLEX_DICT = 'my_database.complex_key_dict';
const SIMPLE_DICT = 'my_database.simple_key_dict';

// Get and update pattern for simple key dictionary
const record = getValue(SIMPLE_DICT, 1001);
record.str_val = "Modified string value";
record.int32_val += 10;  // Increment the current value
setValue(SIMPLE_DICT, 1001, record);

// Batch operations for better performance
const batchKeys = [1001, 1002, 1003, 1004];
const results = batchGetValues(SIMPLE_DICT, batchKeys);

// Update multiple records from batch results
for (let i = 0; i < results.length; i++) {
    if (results[i]) {  // Check if record exists
        results[i].int32_val += 5;  // Increment all values
    }
}

// Write back the modified batch
batchSetValues(SIMPLE_DICT, batchKeys, results);

[yokofly]

report(internal): https://timeplus-workspace.slack.com/files/U0418G6VD1V/F09G87UNBAM/javascript_udf_mutable_stream_operations.pdf

test script:

fintech_dict_write_only.sql

fintech_dict_read_only.sql

run_dict_perf_comprehensive.sh

[yokofly]

Leaving this open deliberately during offboarding triage.

PR #658 already writes this page -- it adds docs/js-udf-dictionary.md plus sidebar wiring and declares Closes #347, so this issue closes automatically when that PR merges. Closing it by hand now would hide a real gap if the PR does not land.

One correction worth recording, since the issue body predates the implementation: the page as written reflects current engine behaviour, where missing keys return default-filled objects rather than null unless javascript_udf_getvalue_null_on_missing_key is set. That setting and by-name column projection landed in 3.3.1. Do not use the draft content in this issue as the source of truth.

Action needed: PR #658 needs a reviewer. Its author is leaving, so it will not move on its own.

---

Context added during @yokofly offboarding (account deactivated 2026-08-14). Deliberately left open -- see above.