Skip to content

Review: Add IKEA WebHarbor mirror (#49) - #77

Open
TabsPhasers wants to merge 5 commits into
aiming-lab:mainfrom
TabsPhasers:review/pr-49-ikea
Open

Review: Add IKEA WebHarbor mirror (#49)#77
TabsPhasers wants to merge 5 commits into
aiming-lab:mainfrom
TabsPhasers:review/pr-49-ikea

Conversation

@TabsPhasers

@TabsPhasers TabsPhasers commented Aug 17, 2026

Copy link
Copy Markdown

Summary

Reviews and carries forward #49 by @Lxr-max on top of the current main branch.

This PR preserves the original IKEA mirror contribution, resolves its current integration conflicts, implements the code-side catalog-image fix, tracks the required asset delivery in HF Asset PR #49, and completes the deterministic grading contract for all 18 IKEA tasks.

Original review resolution

The original review requested two asset fixes:

  1. Real product images — implemented. The reviewed asset package contains 156 JPEG product images, replacing the previous SVG product placeholders.
  2. Publish ikea.tar.gz and pin the merged asset revision — the asset package has been submitted in HF Asset PR #49. Updating .assets-revision remains blocked until that PR is merged.

Review changes

  • integrate IKEA as the 17th WebHarbor site on port 40016
  • integrate the catalog with real JPEG product images, with delivery tracked in HF Asset PR Add IKEA WebHarbor mirror #49
  • add deterministic verifiers and focused tests for Task 0–17
  • provide verifier_path and non-empty judge_rubric for all 18 tasks
  • require relevant IKEA page evidence for information tasks while accepting different valid navigation paths
  • use task-appropriate seeded accounts, with multiple seeded credentials stated in the relevant task prompts
  • remove the shared account/password hint from the login page
  • discover initial.db and after.db directly from the standard run directory, keeping IKEA grading self-contained

Asset delivery status — blocking

The archive was audited before upload. Local username/group metadata and macOS provenance attributes were removed. No local paths, credentials, API keys, private keys, EXIF/GPS/XMP metadata, or other sensitive data were found.

This GitHub PR must not merge until:

  • HF Asset PR Add IKEA WebHarbor mirror #49 is merged
  • .assets-revision is updated to the HF merge commit
  • a clean checkout successfully runs fetch_assets.sh ikea
  • clean-build, image, reset byte-identity, and page smoke checks pass

Scope

  • The original contribution and attribution remain traceable to Add IKEA WebHarbor mirror #49 and @Lxr-max.
  • Reviewer-only trajectories, screenshots, CUA runners, local environment overlays, and generated run artifacts are excluded.
  • Generic evaluator success/pass changes are intentionally excluded and preserved on a separate branch for independent review.

Validation

  • 18/18 tasks have unique IDs, verifier_path, and judge_rubric
  • 100/100 committed IKEA verifier tests pass
  • no-op, positive, wrong-account, incomplete-state, navigation-evidence, checkout-review, and order-number mismatch cases are covered
  • git diff --check origin/main..review/pr-49-ikea passes

Reviews #49.

Original asset review: #49 (comment)

Review lock: #49 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant