Skip to content

Fix sol test adapter#22359

Open
agusaldasoro wants to merge 11 commits into
developfrom
fix/sol-test-adapter
Open

Fix sol test adapter#22359
agusaldasoro wants to merge 11 commits into
developfrom
fix/sol-test-adapter

Conversation

@agusaldasoro
Copy link
Copy Markdown
Contributor

@agusaldasoro agusaldasoro commented May 8, 2026

Fix flaky Solana↔EVM integration tests — startBlock capture and SolEventEmitter hardening

Problem

Test_CCIPMessaging_EVM2Solana and Test_CCIPMessaging_Solana2EVM frequently timeout at 30 minutes on first CI run but pass quickly on retry. The root cause is SolEventEmitter scanning all historical transactions from slot 0, which can include setup noise that delays or blocks event discovery.

Root Cause

When startBlock is nil (the default for Solana dest chains), SolEventEmitter scans from slot 0, processing every transaction ever sent to the OffRamp address. On first run with a fresh environment, this includes deployment and configuration transactions that slow down event discovery. On retry, timing differences or cached state allow it to pass quickly.

Changes

startBlock capture for Solana dest chains (deployment/ccip/changeset/testhelpers/messagingtest/helpers.go)

  • Capture the current Solana slot before sending messages using a currentBlocker interface assertion on the dest adapter. This tells SolEventEmitter to skip all transactions before the message was sent.
  • Reduced SleepAndReplay duration from 30s to 10s for Solana lanes since Solana finality is faster than EVM.

SolEventEmitter hardening (deployment/ccip/changeset/testhelpers/test_adapter_svm.go)

  • Added CurrentBlock method to SVMAdapter that returns the current finalized slot.
  • Added isTransientSolanaRPCError function to gracefully handle transient RPC errors like "Transaction not found" (code -32020) instead of crashing the test.
  • Applied transient error handling to both GetSignaturesForAddressWithOpts and GetTransaction calls.
  • Added batchCompleted tracking: when a transient GetTransaction error occurs mid-batch, the cursor is not advanced, ensuring the batch is retried on the next tick instead of permanently skipping events.
  • Added case <-ctx.Done() to the main select loop for prompt goroutine exit on context cancellation.
  • Wrapped all errorCh <- err sends in select { case errorCh <- err: case <-done: } to prevent goroutine leaks.
  • Replaced require.Equal for source chain selector with an if-check + log + continue, so commit reports from other source chains are gracefully skipped instead of failing the test.
  • Reduced polling interval from 2s to 1s.
  • Added elapsed time and remaining seqNr count to timeout error messages.
  • Added startup log messages with timeout duration.

Related PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

✅ No conflicts with other open PRs targeting develop

@trunk-io
Copy link
Copy Markdown

trunk-io Bot commented May 8, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

Failed Test Failure Summary Logs
TestConfigureForwarder/set_config_with_mcms The test failed without a specific error message, indicating an unspecified failure during the test execution. Logs ↗︎
TestConfigureForwarder/set_config_with_mcms The test failed without a specific error message indicating what went wrong. Logs ↗︎
TestConfigureForwarder The test failed during the configuration of the forwarder, but the specific cause of failure is not provided in the log. Logs ↗︎
TestConfigureForwarder The test failed during the configuration of the forwarder, likely due to an error occurring after copying the Solana programs. Logs ↗︎

... and 8 more

View Full Report ↗︎Docs

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Risk Rating: MEDIUM

This PR refines Solana-related CCIP test helpers by making event polling more responsive and improving timeout/log diagnostics, and also adjusts messaging test helper behavior around replay timing and (intended) Solana start-slot selection.

Changes:

  • Reduced Solana event polling interval from 2s to 1s and added more informative “waiting/elapsed” logs and timeout errors in test_adapter_svm.go.
  • Added logic in messaging tests intended to set a Solana destination startBlock to avoid scanning historical transactions.
  • Shortened SleepAndReplay delay when either source or destination chain is Solana.

Areas requiring scrupulous human review:

  • messagingtest/helpers.go start-block capture: currently relies on an interface method that is not implemented anywhere, so the optimization does not take effect.
  • Replay timing change (30s → 10s for Solana-involved tests): verify it doesn’t introduce flakiness in environments where logpoller/indexing latency is higher.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
deployment/ccip/changeset/testhelpers/test_adapter_svm.go More frequent Solana event polling and improved logging/timeout diagnostics for commit/exec event waits.
deployment/ccip/changeset/testhelpers/messagingtest/helpers.go Attempts to set Solana start slot for destination validation and tweaks replay sleep duration for Solana paths.

Comment thread deployment/ccip/changeset/testhelpers/messagingtest/helpers.go
Comment thread deployment/ccip/changeset/testhelpers/messagingtest/helpers.go Outdated
Comment thread deployment/ccip/changeset/testhelpers/messagingtest/helpers.go Outdated
Comment thread deployment/ccip/changeset/testhelpers/messagingtest/helpers.go Outdated
@agusaldasoro agusaldasoro requested a review from Copilot May 11, 2026 14:00
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread deployment/ccip/changeset/testhelpers/test_adapter_svm.go Outdated
Comment thread deployment/ccip/changeset/testhelpers/messagingtest/helpers.go Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread deployment/ccip/changeset/testhelpers/test_adapter_svm.go
@agusaldasoro agusaldasoro requested a review from Copilot May 11, 2026 15:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@agusaldasoro agusaldasoro requested a review from a team as a code owner May 11, 2026 17:55
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 11, 2026

CORA - Pending Reviewers

Codeowners Entry Overall Num Files Owners
/deployment/ 1 @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/operations-platform, @smartcontractkit/core
/deployment/ccip/ 3 @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/operations-platform, @smartcontractkit/core

Legend: ✅ Approved | ❌ Changes Requested | 💬 Commented | 🚫 Dismissed | ⏳ Pending | ❓ Unknown

For more details, see the full review summary.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread deployment/utils/nodetestutils/node.go Outdated
Comment thread deployment/ccip/changeset/testhelpers/test_adapter_svm.go
@cl-sonarqube-production
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants