Repository navigation
benchmark: weekly engine benchmark report #2141
Replies: 2 comments
Engine benchmark on ubuntu-26.04-armMeasured natively on Linux ARM64, image ubuntu26-arm64 20261023.150.3, Test Environment
----------------
.. list-table::
* - Image
- ``GitHub-hosted ubuntu-26.04-arm, image ubuntu26-arm64 20261023.150.3``
* - ``pcapkit`` revision
- ``c264485``
* - Python
- CPython 3.10.22, 3.11.17, 3.12.15, 3.13.16, 3.14.8, 3.15.1
* - Architecture
- ``aarch64``
* - Capture
- ``in.pcap`` -- 605 bytes, SHA-256 ``7aabadca49b329c6...``
* - Packets per extraction
- 6
* - Iterations
- 1000 timed extractions per engine per pass, the first discarded as a warm-up
* - Passes
- 3 over the whole engine set, in 8 environments: ``3.10-pcap_ct``, ``3.10-pypcap``, ``3.11-pcap_ct``, ``3.11-pypcap``, ``3.12-pcap_ct``, ``3.13-pcap_ct``, ``3.14-pcap_ct``, ``3.15-pcap_ct``
* - tshark
- ``TShark (Wireshark) 4.6.4.``
* - libpcap
- ``libpcap version 1.10.6 (64-bit time_t, with TPACKET_V3)``
Resolved package versions:
============== ============ =========== ============ =========== ============ ============ ============ ============
Package 3.10-pcap_ct 3.10-pypcap 3.11-pcap_ct 3.11-pypcap 3.12-pcap_ct 3.13-pcap_ct 3.14-pcap_ct 3.15-pcap_ct
============== ============ =========== ============ =========== ============ ============ ============ ============
``dpkt`` 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8
``libpcap`` 1.11.0b29 *absent* 1.11.0b29 *absent* 1.11.0b29 1.11.0b29 1.11.0b29 1.11.0b29
``lxml`` 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3
``pcap-ct`` 1.3.0b3 *absent* 1.3.0b3 *absent* 1.3.0b3 1.3.0b3 1.3.0b3 1.3.0b3
``pypcap`` *absent* 1.3.0 *absent* 1.3.0 *absent* *absent* *absent* *absent*
``pypcapfile`` 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0
``pypcapkit`` 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14
``pyshark`` 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6
``scapy`` 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0
============== ============ =========== ============ =========== ============ ============ ============ ============
Test Results (Relative)
-----------------------
Times relative to ``pcapkit``, which is 1 by definition; lower is
faster. Each ratio is taken against the baseline measured in the same
environment on the same pass, so the machine cancels out. "Observed range" is
the smallest and largest ratio actually seen across the passes.
Ratios are pooled across every environment in the run, which means across
every Python version measured as well as both ``pcap`` providers. The
observed range therefore includes any disagreement between interpreters
about an engine; the per-version table is where that is broken out.
============== ============= ============== =======
Engine Relative time Observed range Samples
============== ============= ============== =======
``pypcapfile`` 0.0496 0.0456-0.0542 12
``pypcap`` 0.0650 0.0600-0.0702 6
``pcap_ct`` 0.0967 0.0852-0.103 18
``dpkt`` 0.107 0.103-0.141 24
``scapy`` 0.353 0.340-0.369 24
``pcapkit`` 1 *baseline* 24
``pyshark`` ¶ 29.7 23.4-31.5 18
============== ============= ============== =======
The single absolute figure, for converting the ratios back: ``pcapkit``
itself ran at **0.5917 ms per packet** on this run
(0.566-0.715 across 24 passes). Absolute times are a property of the
machine as much as of the engine, which is why every other figure here is a
ratio.
Run-to-run spread: the median row varied by 17% of its ratio across
the passes and the widest by 35%. A gap between two rows that is
smaller than that is noise, not a result.
``¶`` measured, but on less evidence than the rows beside it.
What was collected is real; what was lost is named here rather than left to
be inferred from the pass count:
* ``pyshark`` -- 3 individual extraction(s) failed and were discardedPosted from Engine Benchmark run at commit |
0 replies
Engine benchmark on ubuntu-26.04-armMeasured natively on Linux ARM64, image ubuntu26-arm64 20261023.150.3, Test Environment
----------------
.. list-table::
* - Image
- ``GitHub-hosted ubuntu-26.04-arm, image ubuntu26-arm64 20261023.150.3``
* - ``pcapkit`` revision
- ``7951ee4``
* - Python
- CPython 3.10.22, 3.11.17, 3.12.15, 3.13.16, 3.14.8, 3.15.2
* - Architecture
- ``aarch64``
* - Capture
- ``in.pcap`` -- 605 bytes, SHA-256 ``7aabadca49b329c6...``
* - Packets per extraction
- 6
* - Iterations
- 1000 timed extractions per engine per pass, the first discarded as a warm-up
* - Passes
- 3 over the whole engine set, in 8 environments: ``3.10-pcap_ct``, ``3.10-pypcap``, ``3.11-pcap_ct``, ``3.11-pypcap``, ``3.12-pcap_ct``, ``3.13-pcap_ct``, ``3.14-pcap_ct``, ``3.15-pcap_ct``
* - tshark
- ``TShark (Wireshark) 4.6.4.``
* - libpcap
- ``libpcap version 1.10.6 (64-bit time_t, with TPACKET_V3)``
Resolved package versions:
============== ============ =========== ============ =========== ============ ============ ============ ============
Package 3.10-pcap_ct 3.10-pypcap 3.11-pcap_ct 3.11-pypcap 3.12-pcap_ct 3.13-pcap_ct 3.14-pcap_ct 3.15-pcap_ct
============== ============ =========== ============ =========== ============ ============ ============ ============
``dpkt`` 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8 1.9.8
``libpcap`` 1.11.0b29 *absent* 1.11.0b29 *absent* 1.11.0b29 1.11.0b29 1.11.0b29 1.11.0b29
``lxml`` 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3 6.1.3
``pcap-ct`` 1.3.0b3 *absent* 1.3.0b3 *absent* 1.3.0b3 1.3.0b3 1.3.0b3 1.3.0b3
``pypcap`` *absent* 1.3.0 *absent* 1.3.0 *absent* *absent* *absent* *absent*
``pypcapfile`` 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0 0.12.0
``pypcapkit`` 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14 1.5.0b14
``pyshark`` 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6
``scapy`` 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0 2.8.0
============== ============ =========== ============ =========== ============ ============ ============ ============
Test Results (Relative)
-----------------------
Times relative to ``pcapkit``, which is 1 by definition; lower is
faster. Each ratio is taken against the baseline measured in the same
environment on the same pass, so the machine cancels out. "Observed range" is
the smallest and largest ratio actually seen across the passes.
Ratios are pooled across every environment in the run, which means across
every Python version measured as well as both ``pcap`` providers. The
observed range therefore includes any disagreement between interpreters
about an engine; the per-version table is where that is broken out.
================ ============= ============== =======
Engine Relative time Observed range Samples
================ ============= ============== =======
``pypcapfile`` 0.0510 0.0464-0.0558 12
``pypcap`` 0.0644 0.0583-0.0708 6
``pcap_ct`` 0.0958 0.0837-0.101 18
``dpkt`` 0.106 0.102-0.143 24
``scapy`` 0.356 0.346-0.374 24
``pcapkit`` 1 *baseline* 24
``pyshark`` [*]_ 28.0 23.1-29.8 18
================ ============= ============== =======
.. [*] Measured, but on less evidence than the rows beside it. What was
collected is real; what was lost is named here rather than left to be
inferred from the pass count: 3 individual extraction(s) failed and were
discarded.
The single absolute figure, for converting the ratios back: ``pcapkit``
itself ran at **0.5733 ms per packet** on this run
(0.546-0.685 across 24 passes). Absolute times are a property of the
machine as much as of the engine, which is why every other figure here is a
ratio.
Run-to-run spread: the median row varied by 18% of its ratio across
the passes and the widest by 39%. A gap between two rows that is
smaller than that is noise, not a result.Posted from Engine Benchmark run at commit |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Reports from the job this thread is named after, one comment per report. Posted by
util/discussion_report.py(#2030).All reactions