eth/consensus : implement eccpow consensus engine#10
Open
mmingyeomm wants to merge 3188 commits intocryptoecc:worldlandfrom
Open
eth/consensus : implement eccpow consensus engine#10mmingyeomm wants to merge 3188 commits intocryptoecc:worldlandfrom
mmingyeomm wants to merge 3188 commits intocryptoecc:worldlandfrom
Conversation
…ng (#33180) head.AuthData is assigned later in the function, so the earlier assignment can safely be removed.
Adds a flag to specify how many blobs a node is willing to include in their locally build block as specified in https://eips.ethereum.org/EIPS/eip-7872 I deviated from the EIP in one case, I allowed for specifying 0 as the minimum blobs/block
EIP-8024: Backward compatible SWAPN, DUPN, EXCHANGE Introduces additional instructions for manipulating the stack which allow accessing the stack at higher depths. This is an initial implementation of the EIP, which is still in Review stage.
YAML supports leaving out the value, so we should handle this condition in our limited parser.
Looks like (in some very EVM specific tests) we spent a lot of time
resizing memory. If the underlying array is big enough, we can speed it
up a bit by simply slicing the memory.
goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/core/vm
cpu: Intel(R) Core(TM) Ultra 7 155U
│ /tmp/old.txt │ /tmp/new.txt │
│ sec/op │ sec/op vs base │
Resize-14 6.145n ± 9% 1.854n ± 14% -69.83% (p=0.000 n=10)
│ /tmp/old.txt │ /tmp/new.txt │
│ B/op │ B/op vs base │
Resize-14 5.000 ± 0% 5.000 ± 0% ~ (p=1.000 n=10)
│ /tmp/old.txt │ /tmp/new.txt │
│ allocs/op │ allocs/op vs base │
Resize-14 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
From the blocktest benchmark:
620ms 10.93s (flat, cum) 9.92% of Total
. . 80:func (m *Memory) Resize(size uint64) {
30ms 60ms 81: if uint64(m.Len()) < size {
590ms 10.87s 82: m.store = append(m.store, make([]byte, size-uint64(m.Len()))...)
. . 83: }
. . 84:}
---------
Co-authored-by: Felix Lange <fjl@twurst.com>
A new pointless fad appeared recently where people just create a fairly low information tag at the beginning of their github PR titles. Something like `feat` or other keywords. This seems to originate from the angular community and to be used for automation scripts over there. We do not use any of those scripts and if we did we would be using the github labels, which offer strictly equivalent functionalities without wasting useful PR title space. In order for these keywords to fail the validation, I am adding a check that these directories listed indeed exist in the repository.
Fixes error messages to print the actual blob gas value instead of the pointer address by dereferencing `ExcessBlobGas`, `BlobGasUsed` and `ParentBeaconRoot`.
…on (#33301) This change fixes a stall in the legacy blob sidecar conversion pipeline where tasks that arrived during an active batch could remain unprocessed indefinitely after that batch completed, unless a new external event arrived. The root cause was that the loop did not restart processing in the case <-done: branch even when txTasks had accumulated work, relying instead on a future event to retrigger the scheduler. This behavior is inconsistent with the billy task pipeline, which immediately chains to the next task via runNextBillyTask() without requiring an external trigger. The fix adds a symmetric restart path in `case <-done`: that checks `len(txTasks) > 0`, clones the accumulated tasks, clears the queue, and launches a new run with a fresh done and interrupt. This preserves batching semantics, prevents indefinite blocking of callers of convert(), and remains safe during shutdown since the quit path still interrupts and awaits the active batch. No public interfaces or logging were changed.
… to 0.18.1 in /cmd/keeper (#33256) Bumps [github.com/consensys/gnark-crypto](https://github.com/consensys/gnark-crypto) from 0.18.0 to 0.18.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/consensys/gnark-crypto/releases">github.com/consensys/gnark-crypto's releases</a>.</em></p> <blockquote> <h2>v0.18.1</h2> <p><strong>Full Changelog</strong>: <a href="https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1">https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/Consensys/gnark-crypto/blob/master/CHANGELOG.md">github.com/consensys/gnark-crypto's changelog</a>.</em></p> <blockquote> <h2>[v0.18.1] - 2025-10-28</h2> <h3>Docs</h3> <ul> <li>add CHANGELOG for 0.18.1</li> </ul> <h3>Perf</h3> <ul> <li>limit memory allocation during Vector deserialization (<a href="https://redirect.github.com/Consensys/gnark-crypto/issues/759">#759</a>)</li> </ul> <p><!-- raw HTML omitted --><!-- raw HTML omitted --></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Consensys/gnark-crypto/commit/fb04e95c3b2fca34f2cc773d3e8c1f930ede271a"><code>fb04e95</code></a> docs: add CHANGELOG for 0.18.1</li> <li><a href="https://github.com/Consensys/gnark-crypto/commit/0a4d04ae621bc19b6b40c6c7b8fa91f08b434533"><code>0a4d04a</code></a> perf: limit memory allocation during Vector deserialization (<a href="https://redirect.github.com/consensys/gnark-crypto/issues/759">#759</a>)</li> <li>See full diff in <a href="https://github.com/consensys/gnark-crypto/compare/v0.18.0...v0.18.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/ethereum/go-ethereum/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fix for #33060 which I can't directly fix in the branch.
Fixes error messages to print the actual blob gas value instead of the pointer address by dereferencing `ExcessBlobGas`, `BlobGasUsed` and `ParentBeaconRoot`.
No matter what value of P2P.DiscoveryV4 or DiscoveryV5 is set in config file, it will be overwritten by the CLI flag, even if the flag is not set. This fixes it to apply the flag only if set.
Enable blocktest to read filenames from stdin when no path argument is provided, matching the existing statetest behavior. This allows efficient batch processing of blockchain tests. Usage: - Single file: evm blocktest <path> - Batch mode: find tests/ -name "*.json" | evm blocktest --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de> Co-authored-by: Felix Lange <fjl@twurst.com>
…3282) This was found because other clients are failing RPC tests generated by Geth. Nethermind and Besu return the correct error code, -32602, in this situation.
This adds checks into getPayload to ensure the correct version is called for the fork which applies to the payload. --------- Co-authored-by: jsvisa <delweng@gmail.com>
Fixes a test/lint regression introduced by #32754
Recently in #31630 we removed support for overriding the network id in preset networks. While this feature is niche, it is useful for shadow forks. This PR proposes we add the functionality back, but in a simpler way. Instead of checking whether the flag is set in each branch of the network switch statement, simply apply the network flag after the switch statement is complete. This retains the following behavior: 1. Auto network id based on chain id still works, because `IsSet` only returns true if the flag is _actually_ set. Not if it just has a default set. 2. The preset networks will set their network id directly and only if the network id flag is set is it overridden. This, combined with the override genesis flag is what allows the shadow forks. 3. Setting the network id to the same network id that the preset _would have_ set causes no issues and simply emits the `WARN` that the flag is being set explicitly. I don't think people explicitly set the network id flag often. ``` WARN [10-22|09:36:15.052] Setting network id with flag id=10 ``` --------- Co-authored-by: Felix Lange <fjl@twurst.com>
We still default to legacy txes for methods like eth_sendTransaction, eth_signTransaction. We can default to 0x2 and if someone would like to stay on legacy they can do so by setting the `gasPrice` field. cc @deffrian
This PR introduces a new debug feature, logging the slow blocks with detailed performance statistics, such as state read, EVM execution and so on. Notably, the detailed performance statistics of slow blocks won't be logged during the sync to not overwhelm users. Specifically, the statistics are only logged if there is a single block processed. Example output ``` ########## SLOW BLOCK ######### Block: 23537063 (0xa7f878611c2dd27f245fc41107d12ebcf06b4e289f1d6acf44d49a169554ee09) txs: 248, mgasps: 202.99 EVM execution: 63.295ms Validation: 1.130ms Account read: 6.634ms(648) Storage read: 17.391ms(1434) State hash: 6.722ms DB commit: 3.260ms Block write: 1.954ms Total: 99.094ms State read cache: account (hit: 622, miss: 26), storage (hit: 1325, miss: 109) ############################## ```
…33338) geth --nodiscover=false may result in ctx.IsSet(NoDiscoverFlag.Name) is true, but cfg. NoDiscovery should be false, not true.
This introduces two main changes to Pebble's configuration: (a) Remove the Bloom filter at Level 6 The Bloom filter is never used at the bottom-most level, so keeping it serves no purpose. Removing it saves storage without affecting read performance. (b) Re-enable read-sampling compaction Read-sampling compaction was previously disabled in the hash-based scheme because all data was identified by hashes and basically no data overwrite. Read sampling compaction makes no sense. After switching to the path-based scheme, data overwrites are much more common, making read-sampling compaction beneficial and reasonable to re-enable.
Fixes priceheap comparison in some edge cases. --------- Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
This PR introduces a threshold (relative to current market base fees), below which we suppress the diffusion of low fee transactions. Once base fees go down, and if the transactions were not evicted in the meantime, we release these transactions. The PR also updates the bucketing logic to be more sensitive, removing the extra logarithm. Blobpool description is also updated to reflect the new behavior. EIP-7918 changed the maximim blob fee decrease that can happen in a slot. The PR also updates fee jump calculation to reflect this. --------- Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
…#33908) The payload rebuild loop resets the timer with the full Recommit duration after generateWork returns, making the actual interval generateWork_elapsed + Recommit instead of Recommit alone. Since fillTransactions uses Recommit (2s) as its timeout ceiling, the effective rebuild interval can reach ~4s under heavy blob workloads — only 1–2 rebuilds in a 6s half-slot window instead of the intended 3. Fix by subtracting elapsed time from the timer reset. ### Before this fix ``` t=0s timer fires, generateWork starts t=2s fillTransactions times out, timer.Reset(2s) t=4s second rebuild starts t=6s CL calls getPayload — gets the t=2s result (1 effective rebuild) ``` ### After ``` t=0s timer fires, generateWork starts t=2s fillTransactions times out, timer.Reset(2s - 2s = 0) t=2s second rebuild starts immediately t=4s timer.Reset(0), third rebuild starts t=6s CL calls getPayload — gets the t=4s result (3 effective rebuilds) ```
We got a report that after v1.17.0 a geth-teku node starts to time out on engine_getBlobsV2 after around 3h of operation. The culprit seems to be our optional http2 service which Teku attempts first. The exact cause of the timeout is still unclear. This PR is more of a workaround than proper fix until we figure out the underlying issue. But I don't expect http2 to particularly benefit engine API throughput and latency. Hence it should be fine to disable it for now.
#33916 + cmd/keeper go mod tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
In `buildPayload()`, the background goroutine uses a `select` to wait on the recommit timer, the stop channel, and the end timer. When both `timer.C` and `payload.stop` are ready simultaneously, Go's `select` picks a case non-deterministically. This means the loop can enter the `timer.C` case and perform an unnecessary `generateWork` call even after the payload has been resolved. Add a non-blocking check of `payload.stop` at the top of the `timer.C` case to exit immediately when the payload has already been delivered.
Return the Amsterdam instruction set from `LookupInstructionSet` when `IsAmsterdam` is true, so Amsterdam rules no longer fall through to the Osaka jump table. --------- Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
For bal-devnet-3 we need to update the EIP-8024 implementation to the latest spec changes: ethereum/EIPs#11306 > Note: I deleted tests not specified in the EIP bc maintaining them through EIP changes is too error prone.
Pebble maintains a batch pool to recycle the batch object. Unfortunately batch object must be explicitly returned via `batch.Close` function. This PR extends the batch interface by adding the close function and also invoke batch.Close in some critical code paths. Memory allocation must be measured before merging this change. What's more, it's an open question that whether we should apply batch.Close as much as possible in every invocation.
Implements https://eips.ethereum.org/EIPS/eip-7778 --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Mainnet was already overriding --cache to 4096. This PR just makes this the default.
…33927) The BatchSpanProcessor queue size was incorrectly set to DefaultMaxExportBatchSize (512) instead of DefaultMaxQueueSize (2048). I noticed the issue on bloatnet when analyzing the block building traces. During a particular run, the miner was including 1000 transactions in a single block. When telemetry is enabled, the miner creates a span for each transaction added to the block. With the queue capped at 512, spans were silently dropped when production outpaced the span export, resulting in incomplete traces with orphaned spans. While this doesn't eliminate the possibility of drops under extreme load, using the correct default restores the 4x buffer between queue capacity and export batch size that the SDK was designed around.
Fixes a regression in #33593 where a block gas limit > gasCap resulted in more execution than the gas cap.
Eth currently has a flaky test, related to the tx fetcher. The issue seems to happen when Unsubscribe is called while sub is nil. It seems that chain.Stop() may be invoked before the loop starts in some tests, but the exact cause is still under investigation through repeated runs. I think this change will at least prevent the error.
The computation of `MAIN_STORAGE_OFFSET` was incorrect, causing the last byte of the stem to be dropped. This means that there would be a collision in the hash computation (at the preimage level, not a hash collision of course) if two keys were only differing at byte 31.
…le tree (#33961) This is an optimization that existed for verkle and the MPT, but that got dropped during the rebase. Mark the nodes that were modified as needing recomputation, and skip the hash computation if this is not needed. Otherwise, the whole tree is hashed, which kills performance.
`GenerateChain` commits trie nodes asynchronously, and it can happen that some nodes aren't making it to the db in time for `GenerateChain` to open it and find the data it is looking for.
Add nil checks to prevent potential panics when keystore backend is unavailable in the Clef signer API.
Reduce allocations in calculation of tx cost. --------- Co-authored-by: weixie.cui <weixie.cui@okg.com> Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Updates go-eth-kzg to https://github.com/crate-crypto/go-eth-kzg/releases/tag/v1.5.0 Significantly reduces the allocations in VerifyCellProofBatch which is around ~5% of all allocations on my node --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
I observed failing tests in Hive `engine-withdrawals`: - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=146 - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=147 ```shell DEBUG (Withdrawals Fork on Block 2): NextPayloadID before getPayloadV2: id=0x01487547e54e8abe version=1 >> engine_getPayloadV2("0x01487547e54e8abe") << error: {"code":-38005,"message":"Unsupported fork"} FAIL: Expected no error on EngineGetPayloadV2: error=Unsupported fork ``` The same failure pattern occurred for Block 3. Per Shanghai engine_getPayloadV2 spec, pre-Shanghai payloads should be accepted via V2 and returned as ExecutionPayloadV1: - executionPayload: ExecutionPayloadV1 | ExecutionPayloadV2 - ExecutionPayloadV1 MUST be returned if payload timestamp < Shanghai timestamp - ExecutionPayloadV2 MUST be returned if payload timestamp >= Shanghai timestamp Reference: - https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#engine_getpayloadv2 Current implementation only allows GetPayloadV2 on the Shanghai fork window (`[]forks.Fork{forks.Shanghai}`), so pre-Shanghai payloads are rejected with Unsupported fork. If my interpretation of the spec is incorrect, please let me know and I can adjust accordingly. --------- Co-authored-by: muzry.li <muzry.li1@ambergroup.io>
This PR fixes a regression introduced in https://github.com/ethereum/go-ethereum/pull/33836/changes Before PR 33836, running mainnet would automatically bump the cache size to 4GB and trigger a cache re-calculation, specifically setting the key-value database cache to 2GB. After PR 33836, this logic was removed, and the cache value is no longer recomputed if no command line flags are specified. The default key-value database cache is 512MB. This PR bumps the default key-value database cache size alongside the default cache size for other components (such as snapshot) accordingly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network