Skip to content

perf(portfolio): share executable revision across one evidence read - #6990

Open
hhyykk wants to merge 1 commit into
loopx-project:mainfrom
hhyykk:codex/next-architecture-payoff-20261021
Open

hhyykk wants to merge 1 commit into
loopx-project:mainfrom
hhyykk:codex/next-architecture-payoff-20261021

Conversation

@hhyykk

@hhyykk hhyykk commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

A bounded Goal portfolio read currently ends the status request scope before reading each registered Agent's quota. The remaining native decisions repeatedly rescan the same TS executable tree. This makes one evidence read pay for code-identity resolution many times.

Reuse the existing effect_runtime_request_scope at build_goal_portfolio, the shared interface consumed by CLI, Chat context and manager inspection. Nested status scopes join it. The next portfolio read resolves current code again; registry/provider/history data remain live reads.

Basis: TypeScript control-plane migration RFC Stage 3 caller adoption at d98804591271380db9737443ed5345e3f4e5b890. Base: main. No semantic migration or R1–R3 completion is claimed.

Author Declaration

  • Written by: model_agent, OpenAI GPT-6 Codex, for the requesting operator.

Implemented against

Specification: docs/architecture/rfcs/typescript-control-plane-migration-v0.md, revision d98804591271380db9737443ed5345e3f4e5b890.

Criterion Disposition Symbol / evidence
Actual caller adoption of the existing typed transport implemented bounded build_goal_portfolio request; real File/SQLite and normal CLI
Complete-command payoff, not RPC-count inference implemented matched full CLI and direct-reader benchmark below; native methods/counts unchanged
Preserve source freshness and authority scope implemented next read observes changed registration; existing limits, stopped/external scope, conflict and unavailable-source tests
Replacement-first economics implemented within this scope existing revision owner reused; +2 production lines, no new bridge/cache/semantic owner, no semantic deletion claimed
Whole migration/default-provider/golden-query qualification out_of_scope no provider routing or authority changes

Self-check: clean-code-guard, test-guard and docs-guard; related entrypoint/caller tests, real providers, baseline-sensitive regression and installed artifacts. Real models and live external delivery were not invoked.

Scope And Continuation

Validation

  • Tested revision: 79c8cf80e290fc3bf5fd8614b5e6566dfaf00efc.
  • Run state: finished. Input classes: synthetic, public_fixture.
Check kind Result Evidence / limitation
integration passed 169 tests: portfolio, executable request scope, Chat manager context, manager inspection, evidence scope and CLI entrypoint
regression_parity passed final new File/SQLite regression uses actual code fingerprint IO and canonical providers, observes changed registration on the next read, preserves state bytes
regression_parity passed the same final regression test loaded against baseline fails on repeated code resolution; passes on HEAD; actual import roots verified
real_entrypoint passed 120 fresh source-verified CLI processes, plus 120 direct-reader samples across both arms; real managed native IPC, no transport mocks
real_backend passed real disposable File and SQLite canonical stores; fresh recorded read observations, one Goal with 1/8 registered Agents; result quality asserted verified
integration passed normal wheel and sdist builds; separately installed environments, actual console from an empty project cwd without source PYTHONPATH; File/SQLite repeat reads, unknown Goal coverage and unchanged registry
static passed scoped Ruff, control-plane typecheck, paired RFC staged/head checks and diff whitespace

Matched complete-read performance

Pinned main d98804591271380db9737443ed5345e3f4e5b890 versus this HEAD. Linux, Python 3.11, Node 24.21.0 / SQLite 3.53.4. Both arms use the same disposable fixtures, sequential execution, 15 samples/arm/profile and warm managed native runtimes. Full CLI starts a fresh Python process every sample, includes imports/parser/registry/typed reads/output, and verifies actual Python and TS source roots. Bytecode writing is disabled equally in both arms. Warm direct-reader times include the complete portfolio function.

Provider Agents Direct-read p95 ms, base → head Complete CLI p95 ms, base → head Code scans per portfolio read
file 1 280.9 → 61.5 (-78.1%) 2948.6 → 2697.2 (-8.5%) 16 → 1
file 8 1778.9 → 190.1 (-89.3%) 4245.2 → 2725.0 (-35.8%) 121 → 1
sqlite 1 258.3 → 59.3 (-77.0%) 2811.9 → 2675.6 (-4.8%) 16 → 1
sqlite 8 1779.1 → 164.7 (-90.7%) 4391.1 → 2964.8 (-32.5%) 121 → 1

All retained semantic fields match across arms and samples. Wall-clock observation fields and their derived snapshot/source hashes are excluded from cross-run equality; each raw portfolio snapshot hash is independently recomputed. Registry revision, persisted source metadata revisions, scope, coverage, Agents, warnings, progress and limitations remain in the comparison. Every raw local observation is retained privately. Native method sequences/counts are unchanged: this removes redundant executable-tree scans, not RPCs or source reads.

The early exploratory probe used an incomplete registry and a cold first arm; it is excluded from these numbers. These results measure local evidence-read overhead with empty canonical work queues, not model quality, task completion, large Todo/history workloads, cold native startup or all of LoopX.

Portable reproduction harness

Save the script below outside both checkouts. Install Python dependencies and run npm ci in both checkouts. Select the same Python interpreter and a qualified Node version (24.21.0 / SQLite 3.53.4 above). Use empty output directories and run sequentially on an idle host:

python3 qualify_portfolio.py HEAD_CHECKOUT SETUP_OUTPUT --fixtures FIXTURE_DIRECTORY --setup
python3 qualify_portfolio.py BASE_CHECKOUT BASE_OUTPUT --fixtures FIXTURE_DIRECTORY --samples 15
python3 qualify_portfolio.py HEAD_CHECKOUT HEAD_OUTPUT --fixtures FIXTURE_DIRECTORY --samples 15

The fixture directory is deliberately shared between arms, so registry bytes and persisted file-metadata revisions can be compared. The script asserts verified source quality, checks snapshot hashes, records all timings/native methods and emits per-subprocess import proofs. Source fixtures are synthetic; usage pings are disabled. The process stops its own managed runtime.

Harness SHA256: 08cf6157c173dc73fb280665cb65194b3247cdb0822238a0f8cf69b14a574076.

import argparse,contextlib,hashlib,io,json,os,statistics,subprocess,sys,tempfile,time
from pathlib import Path
p=argparse.ArgumentParser();p.add_argument('repo');p.add_argument('out');p.add_argument('--fixtures');p.add_argument('--setup',action='store_true');p.add_argument('--samples',type=int,default=15);a=p.parse_args()
repo=Path(a.repo).resolve();out=Path(a.out).resolve();out.mkdir(parents=True,exist_ok=True);fixtures=Path(a.fixtures).resolve() if a.fixtures else out/'fixtures';tmp=out/'tmp';tmp.mkdir(exist_ok=True)
os.environ.update(TMPDIR=str(tmp),TEMP=str(tmp),TMP=str(tmp),LOOPX_USAGE_PING='0',PYTHONDONTWRITEBYTECODE='1');tempfile.tempdir=str(tmp);sys.path[:0]=[str(repo),str(repo/'tests'),str(repo/'tests/control_plane')]
import loopx
from loopx.control_plane import effect_runtime as er
from loopx.goal_portfolio import build_goal_portfolio,_digest
assert Path(loopx.__file__).resolve()==repo/'loopx/__init__.py';assert er._control_plane_root().resolve()==repo/'loopx/control_plane'
if a.setup:
 from canonical_authority_fixture import promoted_create_fixture
 from datetime import datetime,timezone,timedelta
 for provider in ('file','sqlite'):
  for agents in (1,8):
   folder=fixtures/f'{provider}-{agents}';registry,root,state=promoted_create_fixture(folder,provider=provider,registered_agents=[f'agent-{i}' for i in range(agents)])
   data=json.loads(registry.read_text());data['goals'][0].update(domain='portfolio',adapter={'kind':'generic_project_goal_v0'},status='active');registry.write_text(json.dumps(data))
   index=root/'goals/goal-a/runs/index.jsonl';index.parent.mkdir(parents=True,exist_ok=True)
   # Fresh recorded observations, no fabricated delivery/acceptance evidence.
   index.write_text(''.join(json.dumps({'run_id':f'fixture-{i}','goal_id':'goal-a','agent_id':f'agent-{i}','generated_at':(datetime.now(timezone.utc)-timedelta(minutes=1)).isoformat(),'classification':'read_only_refresh','recommended_action':'Inspect the empty canonical Goal.'})+'\n' for i in range(agents)))
 print('fixtures created');er.restart_effect_runtime();sys.exit(0)
scans=[];calls=[];fingerprint=er._runtime_fingerprint;request=er._request_with_info

def scan():
 t=time.perf_counter();r=fingerprint();scans.append((time.perf_counter()-t)*1000);return r

def rpc(*args,**kw):calls.append(kw['method']);return request(*args,**kw)
er._runtime_fingerprint=scan;er._request_with_info=rpc

def check(value):
 payload=dict(value);identity=payload.pop('snapshot_id');assert identity==_digest(payload)
 assert payload['ok'] is True and len(payload['goals'])==1
 assert payload['goals'][0]['quality']=='verified',payload['goals'][0]
 return payload

def norm(value):
 if isinstance(value,dict):
  # Only wall-clock observations and their derived identities are excluded.
  # Raw snapshots and every sample remain in the receipt; inventory and file revisions stay exact.
  return {k:norm(v) for k,v in value.items() if k not in {'collected_at','collection_completed_at','snapshot_id'} }
 if isinstance(value,list):return [norm(v) for v in value]
 return value

def semantic(value):
 value=norm(value)
 for row in value['goals']:
  row['source'].pop('revision',None);row['source'].pop('observed_at',None)
 return value

def percentile(v):return sorted(v)[min(len(v)-1,int((len(v)-1)*.95+.5))]
records=[]
try:
 for provider in ('file','sqlite'):
  for agents in (1,8):
   registry=fixtures/f'{provider}-{agents}'/'registry.json';argv=['--registry',str(registry),'--format','json','goal-portfolio'];build_goal_portfolio(registry_path=registry)
   warm=[];fresh=[];scancounts=[];methods=[];observations=[]
   for n in range(a.samples):
    scans.clear();calls.clear();t=time.perf_counter();value=build_goal_portfolio(registry_path=registry);warm.append((time.perf_counter()-t)*1000);check(value);scancounts.append(len(scans));methods.append(list(calls));observations.append(value)
   script='''import pathlib,sys,json,time,loopx
from loopx.control_plane import effect_runtime as er
from loopx.entrypoint import main
expected=pathlib.Path(sys.argv[1]);sys.argv=sys.argv[:1]+sys.argv[2:]
assert pathlib.Path(loopx.__file__).resolve()==expected/'loopx/__init__.py'
assert er._control_plane_root().resolve()==expected/'loopx/control_plane'
scans=[];original=er._runtime_fingerprint
def scan():
 result=original();scans.append(result);return result
er._runtime_fingerprint=scan
code=main()
sys.stderr.write('PROOF='+json.dumps({'module':str(pathlib.Path(loopx.__file__).resolve()),'code_scans':len(scans)})+'\\n')
raise SystemExit(code)
'''
   proofs=[];outputs=[]
   for n in range(a.samples):
    t=time.perf_counter();pr=subprocess.run([sys.executable,'-c',script,str(repo),*argv],cwd=repo,env={**os.environ,'PYTHONPATH':str(repo)},capture_output=True,text=True,timeout=60);fresh.append((time.perf_counter()-t)*1000);assert pr.returncode==0,(pr.stdout,pr.stderr);value=json.loads(pr.stdout);check(value);outputs.append(value);proof=json.loads(next(x.removeprefix('PROOF=') for x in pr.stderr.splitlines() if x.startswith('PROOF=')));assert proof['module']==str(repo/'loopx/__init__.py');proofs.append(proof)
   normalized=semantic(observations[0]);assert all(semantic(v)==normalized for v in observations+outputs)
   row={'provider':provider,'agents':agents,'warm_samples_ms':warm,'fresh_cli_samples_ms':fresh,'warm_p50_ms':statistics.median(warm),'warm_p95_ms':percentile(warm),'cli_p50_ms':statistics.median(fresh),'cli_p95_ms':percentile(fresh),'scan_counts':scancounts,'methods':methods,'source_proofs':proofs,'semantic':normalized,'raw_observations':observations,'raw_cli':outputs};records.append(row);print(json.dumps({k:row[k] for k in ('provider','agents','warm_p95_ms','cli_p95_ms','scan_counts')}),flush=True)
 (out/'receipt.json').write_text(json.dumps({'revision':subprocess.check_output(['git','-C',str(repo),'rev-parse','HEAD'],text=True).strip(),'module':str(Path(loopx.__file__).resolve()),'product_sha256':hashlib.sha256((repo/'loopx/goal_portfolio.py').read_bytes()).hexdigest(),'harness_sha256':hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),'records':records},indent=2)+'\n')
finally:er.restart_effect_runtime()

Coverage/gaps: no TS product diff; related Python/native real-path tests cover the affected read and lifetime. Full repository suite, Windows, PostgreSQL, browser/model roundtrips and live providers were not run. Hosted CI is separate from these local checks. Already-scoped callers retain their prior performance expectations.

Frontend / Visual Evidence

UI impact: none. Existing evidence interfaces and schemas stay unchanged; no screen or new user capability is introduced.

Change Area

  • Control plane: bounded evidence-reader caller adoption.

Technical Direction

Existing TypeScript control-plane migration, Stage 3; shared code-revision owner remains effect_runtime.py. Provider and domain semantic owners are unchanged.

Shared-authority RFC fixture impact

  • Schema: one disposable canonical Goal, fresh read-observation run index, 1/8 registered Agents, existing portfolio output; no fabricated task/evidence acceptance.
  • Semantic dimensions: scope, limits, stopped Goals, registration freshness, conflicting inventory, lifecycle gaps and unavailable sources retain existing behavior. The new real-provider regression covers separate request lifetimes and live registrations.
  • Provider arms: real File/SQLite; installed artifacts repeat both.
  • Legacy/file/PostgreSQL promotion rehearsal: not run. This change does not promote a provider, select authority/runtime backend routing or modify a compatibility projection. Full D1–D3/R1–R3 qualification remains separate.

Boundary Checklist

  • Public-safe diff/body; no private state, raw traces, credentials, internal URLs or local machine paths.
  • Existing PRs reviewed; no duplicate maintainer benchmark work.
  • Scoped to the requested measurable improvement.
  • English/Chinese RFC pair changed; bilingual staged/head checks pass.
  • UI impact none.
  • DCO sign-off included.

Signed-off-by: aihop <4408344+hhyykk@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant