Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 30 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

# Light Protocol - AI Assistant Reference Guide

## Repository Overview
Expand Down Expand Up @@ -32,11 +36,13 @@ light-protocol/
│ ├── verifier/ # ZKP verification logic in Solana programs
│ ├── zero-copy/ # Zero-copy serialization for efficient account access
│ └── zero-copy-derive/ # Derive macros for zero-copy serialization
├── programs/ # Light Protocol Solana programs
├── programs/ # Light Protocol Solana programs (pinocchio-based)
│ ├── account-compression/ # Core compression program (owns Merkle tree accounts)
│ ├── system/ # Light system program (compressed account validation)
│ ├── compressed-token/ # Compressed token implementation (similar to SPL Token)
│ └── registry/ # Protocol configuration and forester access control
├── anchor-programs/ # Anchor-based program variants
│ └── system/ # Anchor variant of the system program
├── sdk-libs/ # Rust libraries used in custom programs and clients
│ ├── client/ # RPC client for querying compressed accounts
│ ├── sdk/ # Core SDK for Rust/Anchor programs
Expand Down Expand Up @@ -73,10 +79,27 @@ light-protocol/

## Development Workflow

### Setup
```bash
./scripts/install.sh # Install dependencies into .local/
./scripts/devenv.sh # Activate development environment (uses .local/ toolchain)
solana-keygen new -o ~/.config/solana/id.json # Generate keypair (required before testing)
```

### Build Commands
```bash
# Build entire monorepo (uses Nx)
./scripts/build.sh
# Build with just (preferred)
just build # Build programs + JS + CLI
just programs::build # Build only Solana programs

# Or with scripts
./scripts/build.sh # Build entire monorepo (uses Nx)
```

### Format and Lint
```bash
just format # cargo +nightly fmt --all + JS formatting
just lint # Rust fmt check + clippy + README checks + JS lint
```

### Testing Patterns
Expand Down Expand Up @@ -121,6 +144,7 @@ cargo test-sbf -p compressed-token-test

#### 3. SDK Tests (sdk-tests/)
SDK integration tests for various SDK implementations (native, Anchor, Pinocchio, token).
V1 = concurrent Merkle trees (height 26, separate queue/tree accounts). V2 = batched Merkle trees (state height 32, address height 40, combined queue/tree accounts).

```bash
# Run with: cargo test-sbf -p <package-name>
Expand Down Expand Up @@ -186,13 +210,11 @@ TEST_MODE=local cargo test --package forester e2e_test -- --nocapture
- `REDIS_URL=redis://localhost:6379`

#### 7. Linting
Format and clippy checks across the entire codebase.

```bash
./scripts/lint.sh
just lint # Preferred: fmt check + clippy + README checks + JS lint
./scripts/lint.sh # Alternative: shell script
```

**Note:** This requires nightly Rust toolchain and clippy components.
Requires nightly Rust toolchain and clippy components.

### Test Organization Principles

Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,10 +183,9 @@ solana-keygen new -o ~/.config/solana/id.json

Program tests are located in program-tests.
Many tests start a local prover server.
To avoid conflicts between local prover servers run program tests with `--test-threads=1` so that tests are executed in sequence.

```bash
cargo test-sbf -p account-compression-test -- --test-threads=1
cargo test-sbf -p account-compression-test
```

### SDK tests
Expand Down
16 changes: 15 additions & 1 deletion program-libs/batched-merkle-tree/src/batch.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use light_bloom_filter::BloomFilter;
use light_bloom_filter::{BloomFilter, BloomFilterRef};
use light_hasher::{Hasher, Poseidon};
use light_zero_copy::vec::ZeroCopyVecU64;
use zerocopy::{FromBytes, Immutable, IntoBytes, KnownLayout};
Expand Down Expand Up @@ -396,6 +396,20 @@ impl Batch {
Ok(())
}

/// Immutable version of `check_non_inclusion` using `BloomFilterRef`.
pub fn check_non_inclusion_ref(
num_iters: usize,
bloom_filter_capacity: u64,
value: &[u8; 32],
store: &[u8],
) -> Result<(), BatchedMerkleTreeError> {
let bloom_filter = BloomFilterRef::new(num_iters, bloom_filter_capacity, store)?;
if bloom_filter.contains(value) {
return Err(BatchedMerkleTreeError::NonInclusionCheckFailed);
}
Ok(())
}

/// Marks the batch as inserted in the merkle tree.
/// 1. Checks that the batch is ready.
/// 2. increments the number of inserted zkps.
Expand Down
2 changes: 2 additions & 0 deletions program-libs/batched-merkle-tree/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,10 @@ pub mod initialize_address_tree;
pub mod initialize_state_tree;
pub mod merkle_tree;
pub mod merkle_tree_metadata;
pub mod merkle_tree_ref;
pub mod queue;
pub mod queue_batch_metadata;
pub mod queue_ref;
pub mod rollover_address_tree;
pub mod rollover_state_tree;

Expand Down
1 change: 1 addition & 0 deletions program-libs/batched-merkle-tree/src/merkle_tree.rs
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ impl<'a> BatchedMerkleTreeAccount<'a> {
account_data: &'a mut [u8],
pubkey: &Pubkey,
) -> Result<BatchedMerkleTreeAccount<'a>, BatchedMerkleTreeError> {
light_account_checks::checks::check_discriminator::<Self>(account_data)?;
Self::from_bytes::<ADDRESS_MERKLE_TREE_TYPE_V2>(account_data, pubkey)
}

Expand Down
170 changes: 170 additions & 0 deletions program-libs/batched-merkle-tree/src/merkle_tree_ref.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
use std::ops::Deref;

use light_account_checks::{
checks::check_account_info,
discriminator::{Discriminator, DISCRIMINATOR_LEN},
AccountInfoTrait,
};
use light_compressed_account::{
pubkey::Pubkey, ADDRESS_MERKLE_TREE_TYPE_V2, STATE_MERKLE_TREE_TYPE_V2,
};
use light_merkle_tree_metadata::errors::MerkleTreeMetadataError;
use light_zero_copy::{cyclic_vec::ZeroCopyCyclicVecRefU64, errors::ZeroCopyError};
use zerocopy::Ref;

use crate::{
batch::Batch, constants::ACCOUNT_COMPRESSION_PROGRAM_ID, errors::BatchedMerkleTreeError,
merkle_tree::BatchedMerkleTreeAccount, merkle_tree_metadata::BatchedMerkleTreeMetadata,
};

/// Immutable batched Merkle tree reference.
///
/// Uses `try_borrow_data()` + `&'a [u8]` instead of
/// `try_borrow_mut_data()` + `&'a mut [u8]`, avoiding UB from
/// dropping a `RefMut` guard while a raw-pointer-based mutable
/// reference continues to live.
///
/// Only contains the fields that external consumers actually read:
/// metadata, root history, and bloom filter stores.
/// Hash chain stores are not parsed (only needed inside account-compression).
#[derive(Debug)]
pub struct BatchedMerkleTreeRef<'a> {
pubkey: Pubkey,
metadata: Ref<&'a [u8], BatchedMerkleTreeMetadata>,
root_history: ZeroCopyCyclicVecRefU64<'a, [u8; 32]>,
pub bloom_filter_stores: [&'a [u8]; 2],
}

impl Discriminator for BatchedMerkleTreeRef<'_> {
const LIGHT_DISCRIMINATOR: [u8; 8] = BatchedMerkleTreeAccount::LIGHT_DISCRIMINATOR;
const LIGHT_DISCRIMINATOR_SLICE: &'static [u8] =
BatchedMerkleTreeAccount::LIGHT_DISCRIMINATOR_SLICE;
}

impl<'a> BatchedMerkleTreeRef<'a> {
/// Deserialize a batched state Merkle tree (immutable) from account info.
pub fn state_from_account_info<A: AccountInfoTrait>(
account_info: &A,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
Self::from_account_info::<STATE_MERKLE_TREE_TYPE_V2, A>(
&ACCOUNT_COMPRESSION_PROGRAM_ID,
account_info,
)
}

/// Deserialize an address tree (immutable) from account info.
pub fn address_from_account_info<A: AccountInfoTrait>(
account_info: &A,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
Self::from_account_info::<ADDRESS_MERKLE_TREE_TYPE_V2, A>(
&ACCOUNT_COMPRESSION_PROGRAM_ID,
account_info,
)
}

pub(crate) fn from_account_info<const TREE_TYPE: u64, A: AccountInfoTrait>(
program_id: &[u8; 32],
account_info: &A,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
check_account_info::<BatchedMerkleTreeAccount, A>(program_id, account_info)?;
let data = account_info.try_borrow_data()?;
// SAFETY: We extend the lifetime of the borrowed data to 'a.
// The borrow is shared (immutable), so dropping the Ref guard
// restores pinocchio's borrow state correctly for shared borrows.
let data_slice: &'a [u8] = unsafe { std::slice::from_raw_parts(data.as_ptr(), data.len()) };
Self::from_bytes::<TREE_TYPE>(data_slice, &account_info.key().into())
}

/// Deserialize a state tree (immutable) from bytes.
#[cfg(not(target_os = "solana"))]
pub fn state_from_bytes(
account_data: &'a [u8],
pubkey: &Pubkey,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
light_account_checks::checks::check_discriminator::<BatchedMerkleTreeAccount>(
account_data,
)?;
Self::from_bytes::<STATE_MERKLE_TREE_TYPE_V2>(account_data, pubkey)
}

/// Deserialize an address tree (immutable) from bytes.
#[cfg(not(target_os = "solana"))]
pub fn address_from_bytes(
account_data: &'a [u8],
pubkey: &Pubkey,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
light_account_checks::checks::check_discriminator::<BatchedMerkleTreeAccount>(
account_data,
)?;
Self::from_bytes::<ADDRESS_MERKLE_TREE_TYPE_V2>(account_data, pubkey)
}

pub(crate) fn from_bytes<const TREE_TYPE: u64>(
account_data: &'a [u8],
pubkey: &Pubkey,
) -> Result<BatchedMerkleTreeRef<'a>, BatchedMerkleTreeError> {
// 1. Skip discriminator.
let (_discriminator, account_data) = account_data.split_at(DISCRIMINATOR_LEN);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check discriminator here


// 2. Parse metadata.
let (metadata, account_data) =
Ref::<&'a [u8], BatchedMerkleTreeMetadata>::from_prefix(account_data)
.map_err(ZeroCopyError::from)?;
if metadata.tree_type != TREE_TYPE {
return Err(MerkleTreeMetadataError::InvalidTreeType.into());
}

// 3. Parse root history (cyclic vec).
let (root_history, account_data) =
ZeroCopyCyclicVecRefU64::<[u8; 32]>::from_bytes_at(account_data)?;

// 4. Parse bloom filter stores (immutable).
let bloom_filter_size = metadata.queue_batches.get_bloomfilter_size_bytes();
let (bf_store_0, account_data) = account_data.split_at(bloom_filter_size);
let (bf_store_1, _account_data) = account_data.split_at(bloom_filter_size);

Comment on lines +117 to +125
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Guard bloom filter parsing against short buffers.
Line 125 uses split_at without a length check; malformed or truncated input can panic instead of returning a structured error. Prefer an explicit size check and return a ZeroCopyError.

🛠️ Suggested fix
-        let bloom_filter_size = metadata.queue_batches.get_bloomfilter_size_bytes();
-        let (bf_store_0, account_data) = account_data.split_at(bloom_filter_size);
-        let (bf_store_1, _account_data) = account_data.split_at(bloom_filter_size);
+        let bloom_filter_size = metadata.queue_batches.get_bloomfilter_size_bytes();
+        let total_bf_bytes = bloom_filter_size
+            .checked_mul(2)
+            .ok_or(ZeroCopyError::InvalidConversion)?;
+        if account_data.len() < total_bf_bytes {
+            return Err(
+                ZeroCopyError::InsufficientMemoryAllocated(account_data.len(), total_bf_bytes)
+                    .into(),
+            );
+        }
+        let (bf_store_0, account_data) = account_data.split_at(bloom_filter_size);
+        let (bf_store_1, _account_data) = account_data.split_at(bloom_filter_size);
🤖 Prompt for AI Agents
In `@program-libs/batched-merkle-tree/src/merkle_tree_ref.rs` around lines 119 -
127, The bloom filter parsing currently calls
account_data.split_at(bloom_filter_size) without validating buffer length, which
can panic on truncated input; update the code around
ZeroCopyCyclicVecRefU64::from_bytes_at and the subsequent bf_store_0/bf_store_1
parsing to first check that account_data.len() >=
bloom_filter_size.checked_mul(2).ok_or(ZeroCopyError::ShortBuffer)? (or
equivalent) and if not return Err(ZeroCopyError::ShortBuffer), then safely
slice/split the two bloom filter stores (bf_store_0, bf_store_1); reference the
ZeroCopyCyclicVecRefU64::from_bytes_at call and the bf_store_0/bf_store_1
variables to locate the fix.

// 5. Stop here -- hash_chain_stores are not needed for read-only access.

Ok(BatchedMerkleTreeRef {
pubkey: *pubkey,
metadata,
root_history,
bloom_filter_stores: [bf_store_0, bf_store_1],
})
}

/// Check non-inclusion in all bloom filters which are not zeroed.
pub fn check_input_queue_non_inclusion(
&self,
value: &[u8; 32],
) -> Result<(), BatchedMerkleTreeError> {
for i in 0..self.queue_batches.num_batches as usize {
Batch::check_non_inclusion_ref(
self.queue_batches.batches[i].num_iters as usize,
self.queue_batches.batches[i].bloom_filter_capacity,
value,
self.bloom_filter_stores[i],
)?;
}
Ok(())
}
Comment on lines +136 to +150
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Doc comment is slightly misleading — the code checks all bloom filters, not just "not zeroed" ones.

The comment says "Check non-inclusion in all bloom filters which are not zeroed," but the implementation unconditionally iterates all num_batches bloom filters without checking bloom_filter_is_zeroed(). This is functionally correct — a zeroed bloom filter has no bits set, so check_non_inclusion_ref will always pass on it. But the comment creates a false expectation that there's a conditional skip happening.

📝 Suggested doc fix
-    /// Check non-inclusion in all bloom filters which are not zeroed.
+    /// Check non-inclusion across all bloom filters.
+    /// Zeroed bloom filters pass trivially (no bits set).
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Check non-inclusion in all bloom filters which are not zeroed.
pub fn check_input_queue_non_inclusion(
&self,
value: &[u8; 32],
) -> Result<(), BatchedMerkleTreeError> {
for i in 0..self.queue_batches.num_batches as usize {
Batch::check_non_inclusion_ref(
self.queue_batches.batches[i].num_iters as usize,
self.queue_batches.batches[i].bloom_filter_capacity,
value,
self.bloom_filter_stores[i],
)?;
}
Ok(())
}
/// Check non-inclusion across all bloom filters.
/// Zeroed bloom filters pass trivially (no bits set).
pub fn check_input_queue_non_inclusion(
&self,
value: &[u8; 32],
) -> Result<(), BatchedMerkleTreeError> {
for i in 0..self.queue_batches.num_batches as usize {
Batch::check_non_inclusion_ref(
self.queue_batches.batches[i].num_iters as usize,
self.queue_batches.batches[i].bloom_filter_capacity,
value,
self.bloom_filter_stores[i],
)?;
}
Ok(())
}
🤖 Prompt for AI Agents
In `@program-libs/batched-merkle-tree/src/merkle_tree_ref.rs` around lines 145 -
159, The docstring for check_input_queue_non_inclusion is misleading: the
function iterates all batches unconditionally and calls
Batch::check_non_inclusion_ref for each bloom filter, it does not skip zeroed
filters; update the comment on check_input_queue_non_inclusion to state it
checks every bloom filter (including zeroed ones) or remove the "which are not
zeroed" phrase and optionally reference that zeroed filters trivially pass
check_non_inclusion_ref rather than implying a conditional skip via
bloom_filter_is_zeroed().


pub fn pubkey(&self) -> &Pubkey {
&self.pubkey
}
}

impl Deref for BatchedMerkleTreeRef<'_> {
type Target = BatchedMerkleTreeMetadata;

fn deref(&self) -> &Self::Target {
&self.metadata
}
}

impl<'a> BatchedMerkleTreeRef<'a> {
/// Return root from the root history by index.
pub fn get_root_by_index(&self, index: usize) -> Option<&[u8; 32]> {
self.root_history.get(index)
}
}
Loading
Loading