Skip to content

About

Secure-coding lab for CVE-2026-114938 (Java4Shell): local reproduction of the HotSpot AVX-512 Base64 decoder heap overflow, with an AddressSanitizer proof, QEMU negative control, detection hunts, and a CTI alert.

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

CVE-2026-114938_Java4Shell_lab — a secure-coding laboratory

Background

CVE-2026-114938 — "Java4Shell" — was disclosed on 2026-11-17 (Oracle's November Critical Security Patch Update and the matching OpenJDK advisory, following a write-up by the finder "Calif"). It is a heap buffer overflow in the Java runtime itself, and the name is a nod to Log4Shell: like that bug, it sits in ubiquitous, trusted code and is reachable through data that arrives before a user authenticates.

The surprising part is "memory corruption in Java." Java is memory-safe, so the flaw is not in Java code — it is in the native machine code the JDK ships and runs on your behalf. For speed, HotSpot replaces java.util.Base64.Decoder.decode() with a hand-written AVX-512 assembly routine when the CPU supports it. That routine is not bounds-checked the way the Java method is, and it mishandles a short, padded Base64 tail — writing past the end of the output array. Because Base64 is decoded all over the stack before authentication (HTTP Basic headers, JWTs, SAML assertions), the finder frames it as a pre-auth issue.

Two things are worth stating up front, and this lab keeps them distinct throughout:

  • Confirmed: the out-of-bounds write is real and has been reproduced independently (and here). It fires only on a narrow set of preconditions — see Scope reminder.
  • A claim, not confirmed: the finder reports unauthenticated remote code execution. That has not been independently demonstrated, and this lab does not achieve it. How common a genuinely exploitable app is in the real world is an open question.

The rest of this README is the hands-on lab: it reproduces the overflow, shows why it only fires on specific hardware, and proves the shipped two-line fix rejects it — all on hosts you control.


The bug, concretely

On an affected JVM, a short malformed Base64 tail drives the intrinsic's output-length arithmetic negative; the negative value is turned into an all-ones store mask, and a full 64-byte vector store lands 62 bytes past the output array.

The cause is a single missing floor: the 1–63-byte tail handler computes output_len = (len/4)*3 and decrements it once per trailing =, with no guard against going below zero. That negative length reaches an x86 BZHI instruction, which reads only the low 8 bits of its index — so -1 becomes 0xFF = 255, 255 ≥ 64, and the mask is left all-ones instead of exposing the handful of real tail bytes.

This repo proves the overflow, shows why it only fires on specific hardware, and proves the shipped two-line guard rejects it — all on hosts you control.

The bug in four steps

Keep this picture in mind; every demo below is one of these four lines made concrete.

# What happens Value
1 Tail handler computes output_len = (tail_len/4)*3 for a 1–3-byte fragment 0
2 It subtracts 1 for the trailing = — with no floor -1 ← the bug
3 -1 feeds BZHI, which reads only the low 8 bits of the index: 0xFF = 255 ≥ 64 → mask left untouched 0xFFFF…FF (all-ones)
4 The all-ones mask lets a full 64-byte vector store write every lane — starting at offset 48 — into a 50-byte array, so it runs to offset 111 62 bytes out of bounds (offsets 50–111)

The entire fix is to refuse step 1 for tails shorter than one 4-char block — a two-line guard (cmpl length,4 / jcc less,L_bruteForce) that sends them to the safe scalar decoder before any of this arithmetic runs.

How to read this lab

Pick the path that fits you — each builds on the one before:

  1. Just show me the bug. make show — an animated memory map of the overflow (a few seconds; add --fast to skip the delays). Then make crash for the AddressSanitizer proof.
  2. Teach me the mechanism. Read WRITEUP.md — start with §2 ("why memory corruption in Java") for the native/intrinsic-boundary framing, then §3 for the defect — and docs/DEMO-ASAN-MODEL.md (what the proof demo establishes and why).
  3. Why is my fleet (not) exposed? DEFAULT-VS-VULNERABLE.md — the four preconditions and the hardware gate, with make qemu-control as empirical proof.
  4. How would I catch it in production? DETECTION.md — hunt the decoder crashing, not the payload, with make detect-demo to watch the hunts fire.

Every demo's exact captured output is in docs/DEMO-OUTPUTS.md if you want to read the results without running anything.

On the canonical 64×'A'+'=' case: java.util.Base64 ultimately rejects that literal 65-char string as malformed (wrong 4-byte ending unit). Whether the AVX-512 bulk store runs before that rejection (making the canonical string a working trigger) or the validator rejects first (so a real trigger must be a well-formed padded string whose final 1–3-byte fragment hits the tail handler, repro/Trigger2.java) is an ordering this ARM lab cannot decide — the intrinsic never compiles here. Either way the lab uses the geometry to prove the tail-handler arithmetic. See DEFAULT-VS-VULNERABLE.md §3 and TECHNICAL_AUDIT.md (AUD-003) for the two hypotheses and what would settle them.

The honest constraint (read first)

The real crash requires the AVX-512 VBMI Base64 decode intrinsic (generate_base64_decodeBlock() in stubGenerator_x86_64.cpp) to execute. HotSpot enables it only when UseAVX > 2 and the CPU reports AVX-512 VL/BW/VBMI — real Ice Lake / Tiger Lake / Zen 4-class x86-64 silicon. ARM is not affected, and QEMU on ARM exposes no AVX-512, so a faithful intrinsic-level crash cannot run on an Apple Silicon host.

This lab therefore proves the defect two ways:

  1. a logic-level reproduction of the exact vulnerable arithmetic in C, caught by AddressSanitizer — runs natively on ARM; and
  2. a negative control: an affected x86-64 JDK under Docker QEMU that does not crash, demonstrating empirically that the hardware precondition is required.

Safety and intended use

This is defensive security education. The C model is a standalone arithmetic reproduction — it corrupts only its own heap allocation and is caught by ASan; it contains no exploit, no payload, and touches no external system. The Docker control runs a stock, affected JDK against input you generate locally. Nothing here scans or attacks anything you do not own. See SECURITY.md.

Prerequisites

  • A C compiler with AddressSanitizer (cc/clang; Apple clang is fine) and make.
  • Optional, for the negative control only: Docker with x86-64 emulation (--platform linux/amd64). Pulls the pinned affected build eclipse-temurin:21.0.13_9-jdk (~450 MB); override with make qemu-control QEMU_IMAGE=… (a digest for full immutability).

Quick start

make show         # ** watch it ** — animated color memory-map of the 62-byte overflow
make crash        # the lab: fixed control (SURVIVES) then vuln (ASan heap-buffer-overflow)
make fixed        # shipped guard applied -> no overflow, exit 0
make vuln         # the defect -> AddressSanitizer aborts on the OOB write
make detect-demo  # run the crash/exception hunts against synthetic fixtures (see docs/DETECTION.md)
make proof        # regenerate repro/CRASH_PROOF.txt (the committed crash transcript)
make smoke        # non-TTY CI check: asserts fixed=exit0, vuln=OOB, hunts fire
make qemu-control # affected x86-64 JDK under QEMU -> scalar path, no crash (needs Docker)
make clean        # remove build artifacts

make show is the one to run first if you want to see the bug: it draws the output array filling green, then the masked store spilling red bytes past offset 50, lane by lane, while narrating the output_len = -1 → all-ones mask arithmetic. Add --fixed (or make show-fixed) to watch the guarded path refuse to overflow.

If make show is for a non-color terminal (a piped log, a bare SSH session, NO_COLOR): it detects that and prints a pointer instead of garbled escapes. Read the captured final frame at repro/VISUAL_VULN_FRAME.txt (or docs/DEMO-OUTPUTS.md), or pass --force to emit the raw stream anyway.

Expect make crash and make vuln to exit non-zero — that is success, not breakage. The vuln run deliberately triggers an AddressSanitizer abort (SIGABRT), so make reports a failed target. The abort, with heap-buffer-overflow … 0 bytes after 50-byte region, is the proof. make fixed is the one that exits 0. (For garbled make show output on a non-color terminal, see the note just above — the demo detects that case and points you at the captured frame; --force emits the raw stream.)

The committed crash transcript is at repro/CRASH_PROOF.txt if you want the evidence without a toolchain; every target's captured output is in docs/DEMO-OUTPUTS.md.

What each target shows

make target What it demonstrates Result
show animated color memory-map of the masked store spilling 62 bytes OOB visual
show-fixed the same demo with the guard applied no overflow
vuln the negative-length → all-ones-mask → 64-byte store (base64_tail_model.c) heap-buffer-overflow (ASan)
fixed the shipped guard (tail < 4 → scalar path) applied to the same input SURVIVES (exit 0)
crash both of the above in sequence — the before/after pair fixed then crash
detect-demo the crash-dump and exception-burst hunts firing on synthetic fixtures HIGH + BURST
proof capture the full run to repro/CRASH_PROOF.txt transcript
smoke non-TTY CI assertion that the deterministic demos still behave PASS / non-zero
verify-proof assert CRASH_PROOF.txt still matches a fresh run (ASLR-insensitive) PASS / drift
qemu-control affected Temurin 21.0.13 (x86-64) under QEMU, trigger sweep no crash (scalar path)

Detection is documented in DETECTION.md: there are no network IOCs, so the lab hunts for the decoder crashing — JVM hs_err_pid dumps faulting in StubRoutines::_base64_decodeBlock and bursts of Base64$Decoder decode exceptions — with runnable scripts (repro/hunt_hserr.sh, repro/hunt_logs.sh) and a Sigma rule. For the perimeter and the SOC there is also a lower-confidence pack under repro/detection/ — Suricata/Snort (regression-tested via make detect-test), a Sigma multi-rule, ModSecurity, YARA memory-forensics rules, and a passive Nuclei version-triage template — plus repro/verify_corruption.sh, a bounded self-check that reproduces the overflow and the -XX:-UseBASE64Intrinsics mitigation on a host you control.

The fixed/vuln pair is the instructive contrast: the only difference is the two-line guard from the upstream fix. Remove it and the overflow returns — that is the mutation check that proves the guard, not luck, is what makes the input safe.

Layout

README.md                              this file — start here
CLAUDE.md                              repo guidance for Claude Code
Makefile                               lab driver (wraps repro/)
docs/
  index.html                           redirect to the current alert (GitHub Pages landing page)
  Ghost in the Runtime_ ... v1.4.html  the CTI alert this lab accompanies (current)
  Ghost in the Runtime_ ... v1.4.pdf   v1.4 alert as print-ready A4 PDF (9 pages)
  Ghost in the Runtime_ ... v1.3.html  the prior alert version (superseded by v1.4)
  Ghost in the Runtime_ ... v1.3.pdf   v1.3 alert as PDF (superseded)
  Ghost in the Runtime_ ... v1.2.html  older alert version (superseded)
  Ghost in the Runtime_ ... v1.2.pdf   v1.2 alert as PDF (superseded)
  Ghost in the Runtime_ ... v1.1.html  the first disclosed version (superseded)
  Ghost in the Runtime_ ... v1.1.pdf   v1.1 alert as PDF (superseded)
  WRITEUP.md                           full technical write-up
  DEFAULT-VS-VULNERABLE.md             why most JVMs are safe; what makes one vulnerable
  DETECTION.md                         hunt-for-the-crash detection guidance
  SECURITY.md                          responsible-use policy
  DEMO-ASAN-MODEL.md                   demo 1 (the crash proof) + learnings
  DEMO-VISUAL.md                       demo 2 (the animated memory map) + learnings
  DEMO-OUTPUTS.md                      captured output of every demo + what each proves
repro/
  base64_tail_model.c           faithful C model of the defect (ASan proof)
  visual_demo.c                 animated color memory-map demo
  Trigger.java / Trigger2.java  trigger inputs for the QEMU control
  Base64Trigger.java            bounded real-JVM probe (verifies 62-byte OOB on affected silicon)
  verify_corruption.sh          local self-check: trigger + -XX:-UseBASE64Intrinsics mitigation
  hunt_hserr.sh                 crash-dump hunt (primary detection signal)
  hunt_logs.sh                  decode-exception-burst hunt (secondary signal)
  detect_demo.sh                synthetic-fixture detection demo
  fixtures/                     committed synthetic hs_err dumps (Java 17 + 21 shapes)
  java4shell_decoder_crash.sigma  Sigma rule for the crash signal
  detection/                    network/WAF/host/forensics/triage pack
    java4shell.rules            Suricata/Snort rules (regression-tested)
    java4shell.yaml             Sigma multi-rule pack (web/host/app)
    modsecurity.conf            ModSecurity WAF rules
    java4shell.yar              YARA rules for crash/heap-dump memory forensics
    java4shell_version.yaml     Nuclei passive remote version-triage template
    test/                       synthetic-pcap regression harness for the Suricata rules
      make_pcap.py              deterministic labelled-pcap generator (stdlib only)
      score.py                  joins Suricata alerts to cases, scores TP/FP
      run.sh                    generate -> replay -> score (SKIPs without suricata)
      DETECT_TEST.txt           committed PASS transcript
  make_proof.sh                 regenerates CRASH_PROOF.txt (invoked by `make proof`)
  Makefile                      native ASan/visual build + QEMU control + hunts
  CRASH_PROOF.txt               committed ASan crash transcript
  QEMU_CONTROL.txt              negative-control transcript (no AVX-512 -> no crash)
  DETECT_DEMO.txt               detection-hunt transcript (HIGH + BURST)
  VISUAL_VULN_FRAME.txt         final frame of `make show` (62 bytes OOB)
  VISUAL_FIXED_FRAME.txt        final frame of `make show-fixed` (no store)
  README.md                     repro-specific notes

Scope reminder

Affected only: HotSpot Java 17.0.6+ / 21 / 25 / 27 on x86-64 with AVX-512 VBMI and the intrinsic active. Stock Java 8, 11, 17.0.0–17.0.5 and ARM are not affected — but note Oracle's Java SE 8 Enterprise Performance Pack backported the intrinsic, so 8u511b33-perf is affected (fixed in 8u513-perf). The overflow is confirmed and independently reproduced; the unauthenticated-RCE claim is the finder's and was not independently confirmed at the alert's cut-off.

Analyst assessment

This is a judgement, not a confirmed fact — held to the same confirmed-vs-claimed discipline as the rest of the lab.

Java4Shell's exploitable scope is narrow and specific: HotSpot Java 17.0.6+ / 21 / 25 / 27 on x86-64 with AVX-512 VBMI and the Base64 intrinsic JIT-compiled, where attacker data reaches a Base64 decoder. Stock Java 8 / 11, 17.0.0–17.0.5, and ARM are not affected (the Java SE 8 Enterprise Performance Pack 8u511b33-perf is the one Java 8 exception — see above). (The hardware is not exotic — AVX-512 VBMI is standard on Intel Ice Lake/Sapphire/Emerald/Granite Rapids and AMD Zen 4+ — so the real limiter is whether attacker-controlled Base64 reaches a hot decoder path before auth, not the silicon.)

I assess widespread compromise is unlikely, and no in-the-wild exploitation has been observed to date — only lab PoCs like the one in this repo. A full unauthenticated-RCE chain is hard to build and remains the finder's claim, not an independently demonstrated result.

The finder reports chaining the out-of-bounds write into memory read/write primitives and eventually RCE — but says there is no universal exploit: each target needs specific conditions, and the weaponised chain is bespoke per product and sensitive to network timing (Calif's own words: "not a universal point-and-shoot"). Treat the read/write-primitive and RCE steps as reported, not confirmed.

What independent reproduction has shown, on real AVX-512 VBMI silicon (our lab cannot — it is ARM, see the constraint above), is the step below RCE. A third-party kit (r3dvei1/java4shell, Ryzen 7 PRO 8845HS, openjdk 21.0.13.0.3) reproduced the overflow as a real JVM crash (SIGSEGV downstream of the store, 3/4 runs, layout-dependent — consistent with the CVE's AC:H) and, with sun.misc.Unsafe, observed the store overwriting an adjacent object's length and klass words (4/4 deterministic). That corroborates the overflow landing on live object metadata — a write primitive — but still short of demonstrated code execution: the same kit's attempt to crash a stock nginx→Tomcat target produced no RCE, because the write landed in free TLAB space with nothing live behind it (the kit's own conclusion: adjacency is "the hard part," per-application and per-collector). So the ladder is: overflow — confirmed and independently reproduced on silicon; adjacent-metadata corruption / write primitive — independently reproduced; a working chain against an arbitrary host — not demonstrated in public material; RCE against named products (Keycloak, Jira, Confluence, Jenkins, Artifactory, PingFederate) — the finder's claim. Upgrade each rung only with evidence for that rung.

The underlying primitive is an out-of-bounds write (a 64-byte vector store past the output array), not a direct over-read. But that write can be groomed into a heap-memory leak relatively easily — overwrite an adjacent length or pointer field, and a later read returns neighbouring memory. That brings Heartbleed to mind on impact — secrets sitting in adjacent memory flowing back to an attacker — while being meaningfully harder and narrower on mechanism: Heartbleed was a trivially reliable pure over-read, whereas this is an over-write you must convert into a leak via heap grooming on specific hardware.

The real danger of such a leak is that sensitive data — AWS access keys, JWT signing keys, session tokens, TLS private keys — may sit in neighbouring heap memory next to the request-handling buffers.

One operational caveat on the interim mitigation: -XX:-UseBASE64Intrinsics is a JVM startup flag, not a live hot-fix. HotSpot caches the per-intrinsic compile decision, so you cannot disable the intrinsic on an already-running process — applying it means restarting each affected JVM with the flag in its startup args. The durable fix is patching to a fixed build.

Acknowledgements

This lab stands on the public work of others. We credit each for the specific contribution we drew on, and we keep this repo's confirmed-vs-claimed discipline when we do: a citation is not an endorsement of a source's exploitation claims.

Disclosure and primary analysis

  • "Calif" — the finder, for the original disclosure and write-up. The out-of-bounds write is reproduced here; the unauthenticated-RCE chain remains Calif's claim, not an achieved result in this lab (see Analyst assessment).
  • Qi Deng — for a second, independent write-up. Two single-source details are drawn from it and labelled as such throughout (docs/WRITEUP.md, docs/DETECTION.md): the reported 0x9a 0x69 0xa6 residue pattern and the ~200k-request warm-up rate. Independent report, not reproduced here except where real-silicon evidence later corroborated it.
  • OpenJDK / Oracle — for the upstream fix, JDK-8393212 / 5177a70b (the two-line tail-length guard this lab's fixed/vuln mutation check is built around).

Independent reproduction

  • r3dvei1/java4shell — for reproducing the overflow on real AVX-512 VBMI silicon (Ryzen 7 PRO 8845HS, openjdk 21.0.13.0.3), which this ARM lab cannot: a real JVM crash plus a deterministic Unsafe read of an overwritten adjacent object's metadata. Cited in the Analyst assessment as the write-primitive rung of the ladder, and it is the silicon evidence that let us promote the YARA residue rule from modelled to corroborated. Their own retracted negative-RCE experiment is noted as retracted.

Detection and tooling adapted (defensive portions only)

  • ikow/java4shell-poc (MIT) — the detection pack (repro/detection/) and the bounded verifier (repro/Base64Trigger.java, repro/verify_corruption.sh) were adapted from it, with severities rescoped to this lab's discipline. We deliberately do not adopt that repo's downstream exploit chains or active network scanner.
  • jsik22/java4shell-detection — for the synthetic-pcap regression-harness approach behind repro/detection/test/ (generate → replay → score true/false positives).
  • darkfibr/yara-java4shell (MIT) — for informing the YARA memory-forensics rules. We note it for honesty about provenance: this lab declined to hard-code its residue triplet, because that capture came from a VBMI2 host via an LLM-authored artifact. The triplet was adopted only once independent real-silicon evidence corroborated it (see repro/detection/java4shell.yar).

Formal license notices for the adapted third-party portions are in THIRD_PARTY_NOTICES.md.

About

Secure-coding lab for CVE-2026-114938 (Java4Shell): local reproduction of the HotSpot AVX-512 Base64 decoder heap overflow, with an AddressSanitizer proof, QEMU negative control, detection hunts, and a CTI alert.

Topics

Resources

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages