Skip to content

Conditional κ = 0.001730980255989036959962 (+1.140e-9 over #4088): one-third fourth-entry fill attaining #4091's pooled ceiling - #4181

Draft
rohanarun wants to merge 1 commit into
CrocSwap:mainfrom
rohanarun:staged544-pooled-ceiling-fill
Draft

rohanarun wants to merge 1 commit into
CrocSwap:mainfrom
rohanarun:staged544-pooled-ceiling-fill

Conversation

@rohanarun

Copy link
Copy Markdown
Contributor

Author-checked conditional candidate; not maintainer-reviewed. This keeps #4088's four words, native receipts, clock (ORDER.json), template and plan unchanged. It changes two things:

  • Selector. Exactly the calls at ticks t < 1080 use the four-entry word of their orientation.
  • Fill. A new literal chronological owner fill for that selector.
#4088 this PR
fourth-entry calls 1,090 (109·t mod 324 < 109) 1,080 (t < 1080)
idle histogram {3:22783, 5:1625, 48:138492} {3:22788, 5:1620, 48:138492}
complete profile {1:8060060, 2:2310120, 3:751813, 4:327210, 5:66425, 8:25920, 48:138492} {1:8060040, 2:2310120, 3:751788, 4:327240, 5:66420, 8:25920, 48:138492}
root lower bound 0.001730979115670792472760 0.001730980255989036959962

κ = 0.001730980255989036959962 − 10⁻⁴⁶ − 10⁻⁷⁰, which is +1.1403·10⁻⁹ over #4088 (+0.0000659%).

The root bracket […962, …963] equals the sharp pooled ceiling that @RossRichardson proves in #4091 for this fixed two-provider invoice; #4170 measured the gap as 1.1403e-9. This fill attains that ceiling, so no fill of these four words can do better.

Why #4088 stopped at 1,090 calls.

  • At exactly one third of the calls, the window-free LP optimum is integral: 1,620 idle-5 bins 32:(9,9,9), 22,788 idle-3 bins 48:(9×5) and 138,492 empty 48-bins.
  • The time windows are what block it. I adapted the staged544-clock-ceiling per-tick Hall LP to per-call supply. It rates Improve conditional κ with a mixed fourth staged entry (+8.696e-9) #4088's 109-mod rule at 1,080 calls at only 0.0017309790862, below Improve conditional κ with a mixed fourth staged entry (+8.696e-9) #4088.
  • The idle-5 bins close by about tick 208, and the arithmetic selector spreads the early sevens and eights too thinly.
  • With the fourth entry in the first 1,080 ticks, the windowed LP reaches the ceiling, and the reverse-time fill builder (--distinct 40) gives an integral fill that attains it.

Verification. python3.12 -B research/staged544-pooled-ceiling-fill/verify.py --controls runs in about 8 s and needs only the standard library.

Scope. No word, native or operator change: those claims are #4088's, and this inherits all of #4088's conditional hypotheses. As with #4088, the full ordinary/outer release has not been rerun for a mixed dispatch (see #4165).

Open next questions.

Credit.

Prepared by Rohan Arun with Anthropic Claude assistance.

🤖 Generated with Claude Code

https://claude.ai/code/session_011S35E2acyQDaVRjPRw4TDy

…): one-third fourth-entry fill at PR4091's pooled ceiling

Select the first 1,080 ticks for PR4088's four-entry words and build a new
literal T360 owner fill. PR4088's unchanged owner walk accepts all 1,321,920
roles; idle {3:22788, 5:1620, 48:138492}; exact root bracket
[0.001730980255989036959962, ...963] equals PR4091's pooled ceiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011S35E2acyQDaVRjPRw4TDy
@rcfc-jreed

Copy link
Copy Markdown
Contributor

Independent audit of the owner fill, profile and moment at 4ed92bed: no defect found in the scope below

This was a code-disjoint check, standard library only, posted under this CLAIM. It reads only committed data through git show and imports none of this package's code, including verify.py, owner_verify.py and exact_root.py. Frame widths use an independent F₂ RREF.

Inputs (blob ids):

  • FILL.json.gz 44187864
  • MIXED-DISPATCH.json d4d64c5d
  • RESULT.json 5a0ba7ae
  • Predecessor (BASE 12fa8c5c) ORDER 58d4b4c3, TEMPLATE 114808c3, PLAN 7a1db55a, DISPATCH 9e93daea
  • All SOURCES.json usage: input blobs match BASE/HEAD.

What passes (26 of 26 checks):

  1. Dispatch. MIXED-DISPATCH equals the predecessor DISPATCH with -four exactly on ticks t < 1080. There are exactly 1,080 fourth-entry calls, all on extra-four (the early ticks of this clock never hit gauge44).
  2. Providers. All four words expose 408 helpers. THREE local bill {1:2487,2:713,3:224,4:102,5:20,8:8} and FOUR {1:2489,2:713,3:227,4:99,5:20,8:8} match the declared PR4088 profiles. Width sums are 3301 (THREE) and 3300 (FOUR).
  3. Windows, re-derived. For each of 360 replicas, walking the template schedule with that replica's clock yields 6,480 owner bins (cap, lo, hi) with the chronology rule lo < t ≤ hi. Child ranks are {8:6,16:4,32:4,48:4} (by child id), all ≤ 51. Same-family windows on one replica are pairwise disjoint.
  4. Literal fill (1,321,920 = 3240 × 408 roles).
    • Helper identity (t, h) = (j//408, j%408).
    • Same-call families are distinct; every role lies in its window.
    • Offsets are compact prefixes; cells stay within capacity.
  5. Profile. Idle recount {3:22788, 5:1620, 48:138492} plus 3240 × paid local profiles reproduces RESULT exactly. Rank mass is 65088 × 360 = 23,431,680.
  6. Moment. Signed moment Σ n_r r^(1−a) with unpaid term 73: −896·360, evaluated at 90-digit Decimal:
    • At lower, M ≈ −1.7e−17; at upper, M ≈ +4.9e−17.
    • Bisection root ≈ 0.001730980255989036959962… lies in the committed 1e−24 bracket.
    • conditional_kappa = lower − 10⁻⁴⁶ − 10⁻⁷⁰ and is strictly above PR4088's root_upper (gain 1.140318e−9), matching RESULT.
    • This is a high-precision numerical confirmation, not a second interval proof.
  7. Negative controls, each rejected:
    • PR4088's 109·t mod 324 < 109 selector on this fill;
    • one fourth-entry call reverted to three;
    • same-call family collision;
    • compact-offset hole;
    • host at the closed lower window boundary t = lo;
    • PR4088's profile leaves the new root bracket.

Not covered: native word admission, #4091's pooled-ceiling LP proof, the Hall construction of the fill, outer ledger, and all-size / analytic assumptions. The chronology rule lo < t ≤ hi is re-derived from the template and not re-justified.

This is a finite replay of committed data and not a formal proof.

check_4181.py (run: python3 -B check_4181.py --repo <clone with 4ed92bed>, ≈14 s)
#!/usr/bin/env python3
"""Independent check of PR4181 (staged544 pooled-ceiling fill).

Reads committed data via git show. Imports no package code (not owner_verify,
not exact_root, not the PR's verify.py). Frame widths use an independent F2
RREF. The moment is checked with 90-digit Decimal ln/exp on the signed
profile {**H, 73: -896*360}; that is a numerical confirmation, not an
interval proof.

James Reed (@rcfc-jreed), with substantial AI assistance (Claude, in Cursor).
Apache-2.0.
"""
import argparse, collections, copy, gzip, hashlib, json, subprocess, sys
from decimal import Decimal as D, getcontext
from fractions import Fraction as Q

HEAD = '4ed92bed95dd483f5662e98ef246b258c3cfe64f'
BASE = '12fa8c5c844be21bfefb38a19503fbd867a11244'
PKG = 'research/staged544-pooled-ceiling-fill/'
PRED = 'research/staged544-mixed-fourth-entry/'
FOURTH = 1080
CALLS = 3240
REPLICAS = 360
HELPERS = 408
MASS = 65088 * REPLICAS
LIVE_SIGN = 896 * REPLICAS  # unpaid 73-term in the signed moment
results = []


def check(name, ok, detail=''):
    results.append((name, bool(ok), detail))
    print(('PASS ' if ok else 'FAIL ') + name + ((': ' + str(detail)) if detail else ''), flush=True)
    return bool(ok)


def show(repo, path, rev=HEAD):
    return subprocess.run(['git', '-C', repo, 'show', f'{rev}:{path}'],
                          capture_output=True, check=True).stdout


def blob(repo, path, rev=HEAD):
    return subprocess.run(['git', '-C', repo, 'rev-parse', f'{rev}:{path}'],
                          capture_output=True, text=True, check=True).stdout.strip()


def js(b, gz=False):
    return json.loads(gzip.decompress(b) if gz else b)


# ---------------------------------------------------------------- F2 frames
def rref(vectors):
    """Independent F2 Gaussian elimination; returns the pivot basis as a frozenset of ints."""
    pivots = {}
    for x in vectors:
        assert type(x) is int and 0 <= x < 512
        while x:
            p = x.bit_length() - 1
            if p in pivots:
                x ^= pivots[p]
            else:
                pivots[p] = x
                break
    for p in sorted(pivots):
        for q in list(pivots):
            if q > p and (pivots[q] >> p) & 1:
                pivots[q] ^= pivots[p]
    return frozenset(pivots.values())


FULL = rref([1 << j for j in range(9)])


def width(word, r):
    end = rref(word['helper_final'].get(str(r), list(FULL)))
    start = rref(word['start_extra'].get(str(r), []))
    return len(end) - len(start)


# ---------------------------------------------------------------- load
def load(repo):
    g = lambda p, rev=HEAD: show(repo, p, rev)
    gb = lambda p, rev=HEAD: blob(repo, p, rev)
    sources = js(g(PKG + 'SOURCES.json'))
    d = dict(
        result=js(g(PKG + 'RESULT.json')),
        fill=js(g(PKG + 'owner/FILL.json.gz'), True),
        mixed=js(g(PKG + 'owner/MIXED-DISPATCH.json')),
        sources=sources,
        pred_result=js(g(PRED + 'RESULT.json', BASE)),
        order=js(g(PRED + 'owner/ORDER.json', BASE)),
        template=js(g(PRED + 'owner/TEMPLATE.json.gz', BASE), True),
        plan=js(g(PRED + 'owner/PLAN.json.gz', BASE), True),
        old_dispatch=js(g(PRED + 'owner/DISPATCH.json', BASE)),
    )
    # pin-check every SOURCES input that we rely on
    pins = {}
    for entry in sources['files']:
        if entry['usage'] != 'input':
            continue
        p = entry['path']
        # PR packages live on HEAD for the new files; predecessor inputs on BASE
        rev = HEAD if p.startswith(PKG) else BASE
        # predecessor paths are under PRED and must match main/BASE blob
        try:
            pins[p] = gb(p, rev)
        except subprocess.CalledProcessError:
            pins[p] = None
    d['pins'] = pins
    d['blobs'] = {
        PKG + 'owner/FILL.json.gz': gb(PKG + 'owner/FILL.json.gz'),
        PKG + 'owner/MIXED-DISPATCH.json': gb(PKG + 'owner/MIXED-DISPATCH.json'),
        PKG + 'RESULT.json': gb(PKG + 'RESULT.json'),
        PRED + 'owner/ORDER.json': gb(PRED + 'owner/ORDER.json', BASE),
        PRED + 'owner/TEMPLATE.json.gz': gb(PRED + 'owner/TEMPLATE.json.gz', BASE),
        PRED + 'owner/PLAN.json.gz': gb(PRED + 'owner/PLAN.json.gz', BASE),
        PRED + 'owner/DISPATCH.json': gb(PRED + 'owner/DISPATCH.json', BASE),
    }
    # words + native origins
    words = {}
    origins = {}
    for tag, pid in [('base', 'gauge44'), ('extra', 'extra')]:
        origins[pid] = js(g(PRED + f'owner/old-NATIVE-{tag}.json.gz', BASE), True)['helper_origin']
        for kind, suffix in [('THREE', ''), ('FOUR', '-four')]:
            w = js(g(PRED + f'inputs/{kind}-{tag}.json.gz', BASE), True)
            words[pid + suffix] = w
    d['words'], d['origins'] = words, origins
    return d


# ---------------------------------------------------------------- windows (lo < t <= hi)
def derive_windows(template, plan, order):
    clock = {tuple(v): t for t, v in enumerate(order)}
    children = {x['child']: x['rank'] for x in template['idle_children']}
    pair = {x['stage']: x['pair'] for x in template['recipe'] if x['kind'] == 'local'}
    family = {c: 2 * plan['pairs'][str(c)] + plan['sides'][str(c)] for c in children}
    windows = {}
    for o in range(REPLICAS):
        touched, tick = {}, None
        for e in template['execution_schedule']:
            k = e['kind']
            if k == 'local_cover_sweep':
                tick = clock[e['stage'], o]
                touched[pair[e['stage']]] = tick
            elif k == 'bridge':
                touched[e['src']] = tick
                touched[e['dst']] = tick
            elif k == 'idle_cover_sweep':
                for c in e['child_indices']:
                    if c in children:
                        windows[o, c] = (children[c], touched.get(e['pair'], -1), tick)
    return windows, family, children, clock


def walk(rows, order, provider, windows, family, widths, local):
    bad = collections.Counter()
    used = {}
    families = set()
    if len(rows) != CALLS * HELPERS:
        bad['role-count'] += 1
        return bad, {}, {}
    for j, row in enumerate(rows):
        if len(row) != 6 or any(type(v) is not int for v in row):
            bad['row'] += 1
            continue
        t, h, o, c, p, z = row
        if t != j // HELPERS or h != j % HELPERS:
            bad['helper-identity'] += 1
        if h == 0:
            families = set()
        if (o, c) not in windows or not (0 <= p < 112):
            bad['bin'] += 1
            continue
        cap, lo, hi = windows[o, c]
        if not (lo < t <= hi):
            bad['window'] += 1
        f = (o, family[c], p)
        if f in families:
            bad['family'] += 1
        families.add(f)
        prefix = used.get((o, c, p), 0)
        if z != prefix:
            bad['offset'] += 1
        w = widths[provider[tuple(order[t])]][h]
        if prefix + w > cap:
            bad['capacity'] += 1
        used[o, c, p] = prefix + w
    idle = collections.Counter()
    for (o, c), (cap, lo, hi) in windows.items():
        for p in range(112):
            left = cap - used.get((o, c, p), 0)
            if left:
                idle[left] += 1
    profile = collections.Counter(idle)
    for call in order:
        profile.update(local[provider[tuple(call)]])
    return bad, dict(sorted(profile.items())), dict(sorted(idle.items()))


def signed_moment(profile, a):
    """Σ n_r r^(1-a) including the unpaid 73-term, via Decimal."""
    one = D(1)
    s = D(0)
    for r, n in {**profile, 73: -LIVE_SIGN}.items():
        s += D(n) * (D(r) ** (one - a))
    return s


def run(d):
    getcontext().prec = 90
    r = d['result']
    sources = d['sources']
    check('I1 SOURCES base_commit equals canonical main', sources['base_commit'] == BASE)
    pin_ok = True
    pin_detail = []
    for entry in sources['files']:
        if entry['usage'] != 'input':
            continue
        p = entry['path']
        got = d['pins'].get(p)
        if got != entry['git_blob']:
            pin_ok = False
            pin_detail.append((p, entry['git_blob'][:12], None if got is None else got[:12]))
    check('I2 every SOURCES input blob matches BASE/HEAD', pin_ok, pin_detail[:5] if pin_detail else '')

    order = [tuple(x) for x in d['order']]
    clock = {tuple(v): t for t, v in enumerate(order)}
    check('O1 ORDER has 3240 = 9 x 360 calls covering every (stage, replica)',
          len(order) == CALLS and set(clock) == {(s, r) for s in range(9) for r in range(REPLICAS)})
    check('O2 every replica runs stages 0..8 in increasing clock order',
          all(clock[s, r] < clock[s + 1, r] for r in range(REPLICAS) for s in range(8)))

    expected = [[s, r, pid + ('-four' if clock[s, r] < FOURTH else '')]
                for s, r, pid in d['old_dispatch']]
    check('D1 MIXED-DISPATCH equals old DISPATCH with -four exactly on t < 1080',
          d['mixed'] == expected)
    provider = {(s, r): pid for s, r, pid in d['mixed']}
    n_four = sum(pid.endswith('-four') for pid in provider.values())
    check('D2 exactly 1080 fourth-entry calls', n_four == FOURTH, dict(collections.Counter(provider.values())))

    # widths and local profiles from the four words
    widths, local = {}, {}
    for pid, origin_key in [('gauge44', 'gauge44'), ('extra', 'extra')]:
        origin = d['origins'][origin_key]
        for suffix in ['', '-four']:
            key = pid + suffix
            w = d['words'][key]
            need_origin = sorted(set(range(224, 704)) - set(w['removed']))
            if origin != need_origin:
                check(f'V origin mismatch {key}', False, (len(origin), len(need_origin)))
            widths[key] = [width(w, r) for r in origin]
            local[key] = {int(k): v for k, v in w['report']['local_profile'].items()}
    check('V1 four providers: 408 helpers each; width sums and local mass',
          all(len(w) == HELPERS for w in widths.values()),
          {k: (sum(w), sum(r * n for r, n in local[k].items())) for k, w in widths.items()})
    check('V2 THREE and FOUR local profiles match PR4088 declared bills',
          local['gauge44'] == local['extra'] == {1: 2487, 2: 713, 3: 224, 4: 102, 5: 20, 8: 8}
          and local['gauge44-four'] == local['extra-four'] == {1: 2489, 2: 713, 3: 227, 4: 99, 5: 20, 8: 8})

    windows, family, children, _ = derive_windows(d['template'], d['plan'], order)
    check('W1 360 replicas x idle children yield owner bins; ranks <= 51',
          len(windows) == len({(o, c) for (o, c) in windows}) and max(cap for cap, _, _ in windows.values()) <= 51,
          f'bins={len(windows)} child-ranks={dict(collections.Counter(children.values()))}')
    # same-family windows on one replica must be disjoint (inherited PR3945 rule)
    shared = collections.defaultdict(list)
    for (o, c), (_, lo, hi) in windows.items():
        shared[o, family[c]].append((lo, hi))
    disjoint = all(all(max(a, c) >= min(b, d) for i, (a, b) in enumerate(xs) for c, d in xs[i + 1:])
                   for xs in shared.values())
    check('W2 same-family windows on one replica are pairwise disjoint', disjoint)

    rows = d['fill']['rows']
    check('F0 fill has 3240 x 408 = 1,321,920 roles and no dispatch_overrides',
          len(rows) == CALLS * HELPERS and d['fill'].get('dispatch_overrides') == []
          and d['fill']['columns'] == ['time', 'helper', 'owner_replica', 'child', 'port', 'offset'])
    bad, profile, idle = walk(rows, order, provider, windows, family, widths, local)
    check('F1 literal fill: windows (lo<t<=hi), helpers, families, compact offsets, capacity',
          not bad, dict(bad))
    check('F2 idle histogram matches RESULT', idle == {int(k): v for k, v in r['idle'].items()}, idle)
    check('P1 idle + per-call paid profiles == RESULT profile',
          profile == {int(k): v for k, v in r['profile'].items()}, profile)
    mass = sum(rk * n for rk, n in profile.items())
    check('P2 rank mass == 65088 * 360', mass == MASS, mass)

    lo, hi = Q(r['root_lower']), Q(r['root_upper'])
    check('M1 root bracket width is exactly 1e-24', hi - lo == Q(1, 10 ** 24))
    Dq = lambda q: D(q.numerator) / D(q.denominator)
    mlo, mhi = signed_moment(profile, Dq(lo)), signed_moment(profile, Dq(hi))
    err = D('1e-40')
    check('M2 signed moment(lower) < 0 < signed moment(upper)',
          mlo + err < 0 < mhi - err, f'M(lo)={mlo:.3e}, M(hi)={mhi:.3e}')
    a, b = Dq(lo), Dq(hi)
    for _ in range(80):
        c = (a + b) / 2
        a, b = (c, b) if signed_moment(profile, c) < 0 else (a, c)
    check('M3 bisected root lies in [lower, upper]', Dq(lo) <= a <= Dq(hi), f'root ~ {a:.40f}')
    kappa = lo - Q(1, 10 ** 46) - Q(1, 10 ** 70)
    pred_hi = Q(d['pred_result']['root_upper'])
    check('M4 conditional_kappa == lower - 1e-46 - 1e-70 and strictly above PR4088 upper',
          Q(r['conditional_kappa']) == kappa and kappa > pred_hi,
          f'gain {float(kappa - pred_hi):.6e}')
    check('M5 RESULT gain_over_pr4088_upper matches kappa - pred_upper',
          Q(r['gain_over_pr4088_upper']) == kappa - pred_hi)

    # negative controls
    def expect(label, tag, mrows=None, mprov=None):
        prov = mprov if mprov is not None else provider
        b2, _, _ = walk(mrows if mrows is not None else rows, order, prov, windows, family, widths, local)
        check(f'C {label} rejected as {tag}', b2.get(tag, 0) > 0, dict(b2))

    m = copy.deepcopy(rows)
    # use PR4088's 109-mod selector on this fill
    prov109 = {(s, r): pid + ('-four' if (clock[s, r] * 109 % 324 < 109) else '')
               for s, r, pid in d['old_dispatch']}
    b109, _, _ = walk(rows, order, prov109, windows, family, widths, local)
    check('C PR4088 109-mod selector with this fill is rejected',
          sum(b109.values()) > 0, dict(b109))
    # one fourth call reverted: t=0 must be four
    call0 = tuple(order[0])
    assert provider[call0].endswith('-four')
    prov1079 = dict(provider)
    prov1079[call0] = provider[call0].removesuffix('-four')
    expect('one fourth-entry call reverted to three', 'capacity', mprov=prov1079)
    m = copy.deepcopy(rows)
    m[1][2:5] = m[0][2:5]
    expect('same-call family collision', 'family', m)
    m = copy.deepcopy(rows)
    m[0][5] += 1
    expect('compact-offset hole', 'offset', m)
    # host at the closed lower boundary t == lo (when lo >= 0)
    jb = next(j for j, x in enumerate(rows) if windows[x[2], x[3]][1] >= 0)
    m = copy.deepcopy(rows)
    m[jb][0] = windows[rows[jb][2], rows[jb][3]][1]  # t = lo, violates lo < t
    # also breaks helper-identity; still expect window
    expect('host at closed lower window boundary', 'window', m)
    # moment control: PR4088 profile must not sit in the new bracket
    pred_prof = {int(k): v for k, v in d['pred_result']['profile'].items()}
    check('C PR4088 profile leaves the new root bracket',
          not (signed_moment(pred_prof, Dq(lo)) < 0 < signed_moment(pred_prof, Dq(hi))))


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument('--repo', required=True)
    ap.add_argument('--json')
    a = ap.parse_args()
    d = load(a.repo)
    for p, b in d['blobs'].items():
        print('blob', b, p)
    run(d)
    ok = all(r[1] for r in results)
    print(('PASS' if ok else 'FAIL'), f'{sum(r[1] for r in results)}/{len(results)} checks', flush=True)
    if a.json:
        open(a.json, 'w').write(json.dumps(
            dict(head=HEAD, blobs=d['blobs'],
                 checks=[dict(name=n, ok=o, detail=str(x)) for n, o, x in results]), indent=1))
    sys.exit(0 if ok else 1)


if __name__ == '__main__':
    main()

James Reed (@rcfc-jreed), with substantial AI assistance (Claude, in Cursor).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants