From 2f683772810df72dbef3f0e389b3a5cadf68076f Mon Sep 17 00:00:00 2001 From: Mateusz Dalke <150788324+dalkedev@users.noreply.github.com> Date: Sat, 7 Nov 2026 07:52:51 +0000 Subject: [PATCH 01/18] docs: specify Scope-owned Android interception boundary --- .../plans/2026-11-07-owned-android-engine.md | 52 +++++++++++++++++++ .../2026-11-07-owned-android-engine-design.md | 37 +++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 docs/superpowers/plans/2026-11-07-owned-android-engine.md create mode 100644 docs/superpowers/specs/2026-11-07-owned-android-engine-design.md diff --git a/docs/superpowers/plans/2026-11-07-owned-android-engine.md b/docs/superpowers/plans/2026-11-07-owned-android-engine.md new file mode 100644 index 0000000..24558ef --- /dev/null +++ b/docs/superpowers/plans/2026-11-07-owned-android-engine.md @@ -0,0 +1,52 @@ +# Scope-owned Android engine Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace HTTP Toolkit runtime hooks with an original CA-scoped Android interception engine in one independently reviewed PR. +**Architecture:** Original JavaScript hook/policy code runs in the selected app through the existing Python/Frida worker. Rust retains proxy/recovery management and materializes only the new assets. Imported system trust injection and unsupported root profiles are removed explicitly. +**Tech Stack:** JavaScript, Python, Rust, existing Frida Java bridge; pinned existing dependencies. +**Spec:** docs/superpowers/specs/2026-11-07-owned-android-engine-design.md + +## Global Constraints + +- Preserve existing Rust proxy/TLS/archive behavior and source history; no license change. +- New runtime implementations must be original, not copied or mechanically translated upstream hooks. +- Supported profiles are `proxy`, `certificate`, `unpinning`; an empty selection means all three. `unpinning` requires `certificate`. +- Unsupported `root`, `system-ca` or unknown profiles fail before any device probes or state mutation. +- No trust-all manager, hostname-verifier bypass, certificate-expiry bypass or global system trust-store mutation is allowed. +- Only validated Scope-CA chains may bypass supported OkHttp pin checks; all other chains keep original checks/exceptions. +- Readiness requires installed requested capabilities. Stop terminates only the exact worker-spawned process; failures remain visible. +- Keep legacy CA removal recovery; document proxy/root/native coverage reductions. +- Public artifacts contain no private context, paths, captures, credentials or machine identifiers. +- Never claim mock tests are Android results. Real unavailable gates are NOT RUN. No merge/release/tags/admin/engine changes. + +## Review Focus + +- Unrelated/expired/malformed chains must not gain trust or bypass pins. +- Partial hook installation and missing overloads must roll back, never report ready. +- Explicit profiles including stale root/system-ca values must fail before device writes. +- Worker cancellation/startup error must not leave an owned instrumented process active or kill unrelated PIDs. +- Old recovery records must still clean only Scope-owned CA/mappings after injection is removed. + +### Task 1: Original hook engine and worker contract + +**Files:** Create `resources/frida/scope-engine.js`, `resources/frida/tests/scope-engine.test.cjs`, `resources/device/tests/test_frida_worker.py`; modify `resources/device/frida_worker.py`. +**Interfaces:** Worker `build_script(root, ca, port, profiles) -> str` remains available. `normalize_profiles(profiles) -> list[str]` is shared by the CLI/builder. New JS exports a testable engine factory/policy boundary and runs through documented Frida `Java` APIs. JSON runtime config uses public CA/loopback proxy/normalized profiles. Readiness payload identifies engine `scope-java-v1`, selected profiles and installed capabilities; Python validates it. Existing Rust JSONL `ready/error/stopped/detached` kinds remain compatible. + +- [ ] Write executable JS and Python failing tests for CA-scoped pin decisions, original rejection propagation, original trust delegation, explicit/default/unsupported profiles, missing classes and transactional rollback, hostile config serialization, readiness validation and owned-PID cleanup. +- [ ] Observe RED with expected missing implementation or wrong existing behavior; record exact commands/output in report. +- [ ] Implement original engine (small focused helpers in one runtime file), worker JSON config construction, capability-aware ready gate and finite owned-process cleanup; preserve CLI listing/version and bounded diagnostics. Use the actual Frida API, not a production test stub. Any unimplemented requested capability is an error. +- [ ] Run focused tests plus complete Python device suite; self-review. Do not claim fixture-based adapter tests are device verification. +- [ ] Commit only Task1 files and report exact RED/GREEN and remaining limitations. + +### Task 2: Application integration, migration and release verification + +**Files:** Modify `crates/scope-device/src/lib.rs`, `src/App.tsx`, `README.md`, `docs/ANDROID.md`, `THIRD_PARTY_NOTICES.md`, `.github/workflows/checks.yml`, affected existing tests; remove six imported HTK runtime JS/config files and `resources/device/install-system-ca.template.sh`; retain Frida bridge/licenses. Add `docs/ANDROID_ENGINE.md` with explicit coverage/evidence. +**Interfaces:** Consume Task1 `scope-engine.js`, `normalize_profiles` and JSONL engine/capability payload. Rust rejects unsupported profiles before ADB/root probes; ASSETS materializes only `frida/scope-engine.js`, existing Frida bridge and worker. Remove automatic system CA installation, preserve cleanup of old recovery ca_hash. UI removes unsupported root option and explains Stop ends the instrumented process. + +- [ ] Add regression tests for early unsupported profile rejection, asset construction and legacy owned-CA recovery behavior where affected; observe RED. +- [ ] Wire original engine and remove imported runtime source/injection path. Preserve unrelated Rust proxy and recovery semantics. +- [ ] Update docs/notices accurately including retained bridge distribution provenance, current original engine, explicit reduced coverage and device NOT RUN gates; no licensing cleanup beyond truthful component inventory. +- [ ] Add the executable JS engine test command to the existing three-platform checks using existing pinned actions. Keep existing checks/security unchanged. +- [ ] Run focused Rust device tests, all JS/Python suites, UI tests/build when available, formatting and source/privacy/provenance checks; record unavailable desktop/device gates explicitly. +- [ ] Commit, report evidence. Root obtains independent spec/quality review, fixes findings with regression tests, creates exactly one PR only after local review, reconciles hosted checks on exact head. Keep draft if real device gates are unavailable. diff --git a/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md b/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md new file mode 100644 index 0000000..8504754 --- /dev/null +++ b/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md @@ -0,0 +1,37 @@ +# Scope-owned Android interception engine + +## Intent and scope + +Replace HTTP Toolkit-origin interception/unpinning scripts with original Scope code using documented Android, Java and Frida APIs. Retain Frida and its independently attributed Java bridge as third-party instrumentation infrastructure. This is a source migration, not a new TLS implementation or a license change. Retain public source history and accurate third-party notices. + +The existing Scope Rust HTTP(S) proxy, CA, archive and upstream certificate verification stay unchanged. New runtime code must not be copied or mechanically translated from the removed upstream hooks. No claim of legally certified clean-room development is made. + +## Approach + +A small per-process Java engine is the default and only built-in hook backend. Keeping the imported hook backend indefinitely would leave the ownership problem unresolved. Reimplementing Frida or every native networking stack would expand this work beyond a reviewable change. Scope-owned Java hooks on Frida provide a bounded, testable boundary. + +Supported profiles are `proxy`, `certificate`, `unpinning`; an empty selection means all three. `unpinning` requires `certificate`. Unsupported `root`, `system-ca` or unknown profiles fail before any device probes or state mutation. No native TLS, QUIC, Flutter, Cronet, custom obfuscated pinners, child process or root-hiding coverage is claimed. + +`proxy` directs supported Java HTTP clients to the ADB-reversed loopback Scope listener. `certificate` adds trust only for chains validated against the supplied Scope CA while preserving original validation for other chains. No trust-all manager, hostname-verifier bypass, certificate-expiry bypass or global system trust-store mutation is allowed. `unpinning` may skip supported OkHttp pin checks only after the peer chain validates against the configured Scope CA; non-Scope chains must execute original pin checks and preserve their result/exception. + +## Runtime and lifecycle contract + +The Python worker validates finite profile, PEM and port inputs before device work. Source construction uses JSON serialization for runtime data, not source-text replacement. The engine reports per-capability installation results. Readiness requires successful installation of every requested profile, not merely entry into Java.perform. Unsupported classes/overloads or hook errors produce actionable bounded diagnostics and no success claim. Installation failure rolls back installed method implementations. + +Process-created proxy/trust objects can outlive method unhooking. Therefore Stop terminates the exact process spawned by this worker, then unloads/detaches; it does not promise that unloading alone restores the running app. It must not kill an unrelated process. Cleanup errors remain visible, and no clean-stop receipt is emitted after failed cleanup. Existing Rust recovery of owned ADB/server state and legacy CA removal remains available after restart. + +## Compatibility changes + +Remove the imported system-CA injection template and automatic rooted proxy-mode trust-store injection. System proxy mode remains device-wide; HTTPS requires manually configured application trust. Do not remove legacy CA cleanup for recovery records created by earlier versions. Remove root-hiding selection from the UI; stale API/CLI selections are rejected before mutations. Explain these reductions prominently instead of claiming full parity with the imported scripts. + +## Validation and delivery + +Use executable JavaScript policy/adapter tests, worker builder/lifecycle tests and existing Rust/UI/device/release suites. Include controls proving unrelated or invalid chains still fail, original pin exceptions survive, unsupported capabilities never become ready, partial startup rolls back, and Stop ends only the owned PID. Simulated Frida/Java adapter tests are explicitly not Android execution evidence. + +Real Android/Frida/OkHttp TLS, app-stop restoration and API/ABI compatibility are separate acceptance gates. Record exact PASS / FAIL / NOT RUN without inventing coverage. An independently reviewed PR may remain draft if these device gates cannot be executed. No merge, release, tags, engine/service changes or live external account actions. + +## Primary API references + +- https://frida.re/docs/javascript-api/#java +- https://developer.android.com/privacy-and-security/security-config +- https://square.github.io/okhttp/features/https/ From 8754c03368b7b105cfba5c83cad83328e242a8f9 Mon Sep 17 00:00:00 2001 From: Mateusz Dalke <150788324+dalkedev@users.noreply.github.com> Date: Sat, 7 Nov 2026 08:04:00 +0000 Subject: [PATCH 02/18] feat: add Scope-owned Java interception engine and worker lifecycle --- resources/device/frida_worker.py | 302 ++++++++++++++------ resources/device/tests/test_frida_worker.py | 179 ++++++++++++ resources/frida/scope-engine.js | 299 +++++++++++++++++++ resources/frida/tests/scope-engine.test.cjs | 254 ++++++++++++++++ 4 files changed, 950 insertions(+), 84 deletions(-) create mode 100644 resources/device/tests/test_frida_worker.py create mode 100644 resources/frida/scope-engine.js create mode 100644 resources/frida/tests/scope-engine.test.cjs diff --git a/resources/device/frida_worker.py b/resources/device/frida_worker.py index 9e3d5fe..2dbf8c5 100644 --- a/resources/device/frida_worker.py +++ b/resources/device/frida_worker.py @@ -1,53 +1,234 @@ #!/usr/bin/env python3 -"""Scope's isolated Frida worker. Stdout is JSON Lines; stdin EOF requests cleanup. +"""Scope's isolated Frida worker. Stdout is JSON Lines; stdin EOF requests Stop. -The worker controls exactly one selected application. Hook source stays JavaScript. -Python/Frida is deliberately outside the desktop process so an ART crash cannot -take the session archive or UI with it. No HTTP Toolkit installation is used. +The worker owns one spawned application process. Stop ends that process because +unloading methods cannot revoke already-created proxy or TLS objects. """ import argparse +from collections import deque +from contextlib import nullcontext import json from pathlib import Path import re import signal import sys import threading +import time -PROFILES = { - "proxy": ["android/android-proxy-override.js"], - "certificate": ["android/android-system-certificate-injection.js"], - "unpinning": ["android/android-certificate-unpinning.js", "android/android-certificate-unpinning-fallback.js"], - "root": ["android/android-disable-root-detection.js"], -} +PROFILES = ("proxy", "certificate", "unpinning") +CAPABILITIES = {"proxy": "java-url", "certificate": "sslcontext-scope-ca", "unpinning": "okhttp3-certificate-pinner"} def emit(kind, **fields): - # Keep Frida diagnostics local and avoid logging tokens inside query strings. if "message" in fields: fields["message"] = re.sub(r"(https?://[^\s?]+)\?[^\s]+", r"\1?[query omitted]", str(fields["message"]))[:4000] print(json.dumps({"kind": kind, **fields}), flush=True) +def normalize_profiles(profiles): + if not isinstance(profiles, (list, tuple)) or any(not isinstance(p, str) or p not in PROFILES for p in profiles): + raise ValueError("Unsupported hook profile; supported: proxy, certificate, unpinning") + result = [p for p in PROFILES if p in profiles] if profiles else list(PROFILES) + if "unpinning" in result and "certificate" not in result: + raise ValueError("unpinning profile requires certificate") + return result + + def build_script(root, ca, port, profiles): - cert = ca.read_text().strip() - if not re.fullmatch(r"-----BEGIN CERTIFICATE-----\s+[A-Za-z0-9+/=\s]+-----END CERTIFICATE-----", cert): + profiles = normalize_profiles(profiles) + if type(port) is not int or not 1 <= port <= 65535: + raise ValueError("A valid proxy port is required") + if ca.stat().st_size > 65536: + raise ValueError("Scope CA PEM is too large") + cert = ca.read_text(encoding="utf-8").strip() + if len(cert) > 65536 or not re.fullmatch(r"-----BEGIN CERTIFICATE-----\s+[A-Za-z0-9+/=\s]+-----END CERTIFICATE-----", cert, flags=re.ASCII): raise ValueError("Scope CA must be a public PEM certificate") - unknown = set(profiles) - set(PROFILES) - if unknown: - raise ValueError("Unknown hook profile: " + ", ".join(sorted(unknown))) - config = (root / "config-template.js").read_text() - config, count = re.subn(r"const CERT_PEM = `.*?`;", lambda _: "const CERT_PEM = `" + cert + "`;", config, count=1, flags=re.S) - if count != 1: - raise ValueError("Unexpected Frida configuration template") - config, count = re.subn(r"const PROXY_PORT = \d+;", "const PROXY_PORT = %d;" % port, config, count=1) - if count != 1: - raise ValueError("Missing proxy port configuration") - scripts = [(root / "frida-java-bridge.js").read_text(), config] - for profile in profiles: - for name in PROFILES[profile]: - scripts.append((root / name).read_text()) - scripts.append("Java.perform(() => send({kind: 'scope-hooks-ready'}));\n") - return "\n".join(scripts) + config = {"caPem": cert, "proxyHost": "127.0.0.1", "proxyPort": port, "profiles": profiles} + return "\n".join([ + (root / "frida-java-bridge.js").read_text(encoding="utf-8"), + (root / "scope-engine.js").read_text(encoding="utf-8"), + "globalThis.ScopeJavaEngine.start(" + json.dumps(config, ensure_ascii=True) + ");\n", + ]) + + +def validate_ready(payload, profiles): + selected = normalize_profiles(profiles) + if not isinstance(payload, dict) or payload.get("kind") != "scope-hooks-ready" or payload.get("engine") != "scope-java-v1" or payload.get("profiles") != selected: + raise ValueError("Invalid Scope engine readiness identity or profiles") + capabilities = payload.get("capabilities") + if not isinstance(capabilities, dict) or set(capabilities) != set(selected): + raise ValueError("Missing requested profile capabilities") + for profile in selected: + installed = capabilities[profile] + if not isinstance(installed, list) or any(not isinstance(c, str) for c in installed) or CAPABILITIES[profile] not in installed: + raise ValueError("Missing installed capability for profile " + profile) + return payload + + +def bounded_call(label, operation, timeout): + done = threading.Event() + outcome = [] + + def run(): + try: + outcome.append((True, operation())) + except BaseException as error: + outcome.append((False, error)) + finally: + done.set() + + threading.Thread(target=run, daemon=True).start() + if not done.wait(timeout): + raise RuntimeError(label + " timed out; completion is unknown") + success, value = outcome[0] + if not success: + raise RuntimeError(label + ": " + str(value)) from value + return value + + +def cleanup_owned(device, pid, script, session, timeout=3, process_not_found=()): + errors = [] + + def kill(): + try: + device.kill(pid) + except process_not_found: + pass # Frida confirms this process already ended. + + def unload(): + if script.is_destroyed is not True: + try: + script.unload() + except Exception: + if script.is_destroyed is not True: + raise + + def detach(): + if session.is_detached is not True: + try: + session.detach() + except Exception: + if session.is_detached is not True: + raise + + operations = [] + if pid is not None: + operations.append(("kill owned application", kill)) + if script is not None: + operations.append(("unload hooks", unload)) + if session is not None: + operations.append(("detach session", detach)) + for label, operation in operations: + try: + bounded_call(label, operation, timeout) + except Exception as error: + errors.append(str(error)) + if errors: + raise RuntimeError("Cleanup failed: " + "; ".join(errors)) + + +def run_session(device, package, source, profiles, version, stopped, ready_timeout=30, cleanup_timeout=3, process_not_found=(), cancellable_factory=None): + profiles = normalize_profiles(profiles) + ready = threading.Event() + exited = threading.Event() + failure = deque(maxlen=2) + receipt = [] + replaced = threading.Event() + session = script = pid = None + + def receive(message, _data): + payload = message.get("payload") + if message.get("type") == "send" and isinstance(payload, dict): + if payload.get("kind") == "scope-hooks-ready": + if failure or stopped.is_set(): + return + try: + receipt[:] = [validate_ready(payload, profiles)] + ready.set() + except ValueError as error: + failure.append(str(error)) + stopped.set() + return + if payload.get("kind") == "scope-hooks-error": + failure.append(str(payload.get("message", "Hook installation failed"))[:2000]) + stopped.set() + return + if message.get("type") == "error": + failure.append(str(message.get("stack") or message.get("description") or "Hook error")[:2000]) + stopped.set() + emit("log", message=message.get("stack") or payload or message.get("description") or message) + + def detached(reason, crash=None): + if reason == "process-terminated": + exited.set() + elif reason == "process-replaced": + replaced.set() + failure.append("Owned process image was replaced; termination cannot be confirmed safely") + emit("detached", message=reason, crash=bool(crash)) + stopped.set() + + primary_error = None + startup_complete = threading.Event() + deadline = time.monotonic() + ready_timeout + cancellable = cancellable_factory() if cancellable_factory else None + + def cancel_startup(): + while not startup_complete.wait(0.025): + if stopped.is_set(): + cancellable.cancel() + return + if time.monotonic() >= deadline: + failure.append("Startup deadline expired; pending operation completion may be unknown") + stopped.set() + cancellable.cancel() + return + + if cancellable is not None: + threading.Thread(target=cancel_startup, daemon=True).start() + attempted_spawn = False + try: + if stopped.is_set(): + return + with cancellable if cancellable is not None else nullcontext(): + attempted_spawn = True + pid = device.spawn([package]) + emit("spawned", engine="scope-java-v1", pid=pid) + session = device.attach(pid) + session.on("detached", detached) + script = session.create_script(source) + script.on("message", receive) + script.load() + if not stopped.is_set(): + device.resume(pid) + startup_complete.set() + while not ready.is_set() and not stopped.is_set() and time.monotonic() < deadline: + ready.wait(min(0.05, max(0, deadline - time.monotonic()))) + if failure: + raise RuntimeError("Hook startup failed: " + "; ".join(failure)) + if stopped.is_set(): + if failure: + raise RuntimeError("Hook startup failed: " + "; ".join(failure)) + return + if not ready.is_set(): + raise RuntimeError("Java hooks did not become ready within the startup deadline") + emit("ready", pid=pid, version=version, engine=receipt[0]["engine"], profiles=profiles, capabilities=receipt[0]["capabilities"]) + stopped.wait() + if failure: + raise RuntimeError("Hook runtime failed: " + "; ".join(failure)) + except BaseException as error: + primary_error = RuntimeError("Spawn did not return an owned PID; completion is unknown: " + str(error)) if attempted_spawn and pid is None else error + finally: + startup_complete.set() + try: + cleanup_owned(device, None if exited.is_set() or replaced.is_set() else pid, script, session, cleanup_timeout, process_not_found) + except Exception as error: + if primary_error: + raise RuntimeError(str(primary_error) + "; " + str(error)) from primary_error + raise + if not replaced.is_set() and pid is not None: + emit("stopped", engine="scope-java-v1", pid=pid, processTerminated=True, message="Owned application process ended; hooks unloaded and session detached") + if primary_error: + raise primary_error def main(): @@ -62,6 +243,12 @@ def main(): parser.add_argument("--profile", action="append", default=[]) parser.add_argument("--list-apps", action="store_true") args = parser.parse_args() + profiles = normalize_profiles(args.profile) # Reject stale profiles before any device operation. + source = None + if not args.version and not args.list_apps: + if not args.package or not args.ca or not args.scripts: + raise ValueError("Package, public CA and scripts are required") + source = build_script(args.scripts, args.ca, args.port, profiles) try: import frida except ImportError as error: @@ -76,29 +263,7 @@ def main(): if args.list_apps: emit("apps", apps=[{"package": app.identifier, "name": app.name, "running": bool(app.pid)} for app in device.enumerate_applications()]) return - if not args.package or not args.ca or not args.scripts or not 1 <= args.port <= 65535: - raise ValueError("Package, public CA, scripts and a valid port are required") - source = build_script(args.scripts, args.ca, args.port, args.profile or ["proxy", "certificate", "unpinning"]) stopped = threading.Event() - ready = threading.Event() - failure = [] - session = None - script = None - pid = None - resumed = False - - def receive(message, _data): - payload = message.get("payload") - if message.get("type") == "send" and isinstance(payload, dict) and payload.get("kind") == "scope-hooks-ready": - ready.set() - return - if message.get("type") == "error": - failure.append(message.get("stack") or message.get("description") or "Hook error") - emit("log", message=message.get("stack") or payload or message.get("description") or message) - - def detached(reason, crash=None): - emit("detached", message=reason, crash=bool(crash)) - stopped.set() def wait_for_stop(): for line in sys.stdin: @@ -109,39 +274,8 @@ def wait_for_stop(): signal.signal(signal.SIGINT, lambda *_: stopped.set()) signal.signal(signal.SIGTERM, lambda *_: stopped.set()) threading.Thread(target=wait_for_stop, daemon=True).start() - emit("log", message="Experimental Java hooks: Android 16 ART/GC crashes were observed upstream. Native TLS/QUIC stacks are not covered.") - try: - pid = device.spawn([args.package]) - session = device.attach(pid) - session.on("detached", detached) - script = session.create_script(source) - script.on("message", receive) - script.load() - device.resume(pid) - resumed = True - if not ready.wait(30): - raise RuntimeError("Java hooks did not become ready within 30 seconds. " + "; ".join(failure[-2:])) - if failure: - raise RuntimeError("A hook failed during startup: " + "; ".join(failure[-2:])) - emit("ready", pid=pid, version=frida.__version__, profiles=args.profile or ["proxy", "certificate", "unpinning"]) - stopped.wait() - finally: - if script: - try: - script.unload() - except Exception: - pass - if session: - try: - session.detach() - except Exception: - pass - if pid and not resumed: - try: - device.kill(pid) - except Exception: - pass - emit("stopped", message="Frida hooks detached; the selected application was left running") + emit("log", message="Experimental Scope Java hooks; Stop ends the spawned app. Native TLS/QUIC and custom pinners are not covered.") + run_session(device, args.package, source, profiles, frida.__version__, stopped, process_not_found=(frida.ProcessNotFoundError,), cancellable_factory=frida.Cancellable) if __name__ == "__main__": diff --git a/resources/device/tests/test_frida_worker.py b/resources/device/tests/test_frida_worker.py new file mode 100644 index 0000000..840e776 --- /dev/null +++ b/resources/device/tests/test_frida_worker.py @@ -0,0 +1,179 @@ +"""Worker contract tests; fake Frida calls are not Android coverage.""" +import importlib.util +import json +from pathlib import Path +import subprocess +import tempfile +import time +import unittest +from unittest import mock + +SOURCE = Path(__file__).resolve().parents[1] / "frida_worker.py" +SPEC = importlib.util.spec_from_file_location("scope_frida_worker", SOURCE) +WORKER = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(WORKER) +PEM = "-----BEGIN CERTIFICATE-----\nYWJj\n-----END CERTIFICATE-----" + + +class WorkerTests(unittest.TestCase): + def test_profiles_are_shared_normalized_and_unsupported_fail(self): + self.assertTrue(hasattr(WORKER, "normalize_profiles"), "worker provides normalized profile contract") + self.assertEqual(WORKER.normalize_profiles([]), ["proxy", "certificate", "unpinning"]) + self.assertEqual(WORKER.normalize_profiles(["certificate", "proxy", "certificate"]), ["proxy", "certificate"]) + for profiles in [["root"], ["system-ca"], ["unpinning"], ["unknown"], "proxy", [None]]: + with self.assertRaises(ValueError): + WORKER.normalize_profiles(profiles) + + def test_builder_serializes_data_and_loads_only_owned_engine_and_bridge(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + ca = root / "ca.pem" + ca.write_text(PEM) + (root / "frida-java-bridge.js").write_text("// bridge\n") + (root / "scope-engine.js").write_text("globalThis.ScopeJavaEngine = {start(c) { globalThis.observed = c; }};\n") + source = WORKER.build_script(root, ca, 17890, ["proxy"]) + result = subprocess.run(["node", "-e", source + "\nconsole.log(JSON.stringify(globalThis.observed))"], capture_output=True, text=True, check=True) + self.assertEqual(json.loads(result.stdout), {"caPem": PEM, "proxyHost": "127.0.0.1", "proxyPort": 17890, "profiles": ["proxy"]}) + ca.write_text(PEM + "`;globalThis.pwned=true;//") + with self.assertRaises(ValueError): + WORKER.build_script(root, ca, 17890, ["proxy"]) + ca.write_text(PEM) + for port in [0, 65536, True, "17890;evil()"]: + with self.assertRaises(ValueError): + WORKER.build_script(root, ca, port, ["proxy"]) + + def test_ready_gate_requires_engine_exact_profiles_and_real_capabilities(self): + self.assertTrue(hasattr(WORKER, "validate_ready"), "worker validates installed capabilities") + good = {"kind": "scope-hooks-ready", "engine": "scope-java-v1", "profiles": ["proxy"], "capabilities": {"proxy": ["java-url"]}} + self.assertEqual(WORKER.validate_ready(good, ["proxy"]), good) + for changes in [{"engine": "other"}, {"profiles": []}, {"capabilities": {}}, {"capabilities": {"proxy": []}}, {"capabilities": {"proxy": ["not-implemented"]}}]: + with self.assertRaises(ValueError): + WORKER.validate_ready({**good, **changes}, ["proxy"]) + + def test_owned_cleanup_kills_exact_pid_before_unload_and_detach(self): + self.assertTrue(hasattr(WORKER, "cleanup_owned"), "worker terminates its instrumented PID") + calls = [] + device = mock.Mock() + device.kill.side_effect = lambda pid: calls.append(("kill", pid)) + script = mock.Mock() + script.unload.side_effect = lambda: calls.append(("unload",)) + session = mock.Mock() + session.detach.side_effect = lambda: calls.append(("detach",)) + WORKER.cleanup_owned(device, 427, script, session, timeout=0.2) + self.assertEqual(calls, [("kill", 427), ("unload",), ("detach",)]) + calls.clear() + WORKER.cleanup_owned(device, None, None, None, timeout=0.2) + self.assertEqual(calls, []) + + def test_cleanup_errors_and_timeouts_are_visible_and_bounded(self): + self.assertTrue(hasattr(WORKER, "cleanup_owned")) + device, script, session = mock.Mock(), mock.Mock(), mock.Mock() + device.kill.side_effect = RuntimeError("permission denied") + with self.assertRaisesRegex(RuntimeError, "kill.*permission denied"): + WORKER.cleanup_owned(device, 427, script, session, timeout=0.05) + script.unload.assert_called_once() + session.detach.assert_called_once() + device.kill.side_effect = lambda _pid: time.sleep(0.3) + start = time.monotonic() + with self.assertRaisesRegex(RuntimeError, "kill.*timed out"): + WORKER.cleanup_owned(device, 427, None, None, timeout=0.02) + self.assertLess(time.monotonic() - start, 0.2) + + def test_cli_rejects_unsupported_profile_before_import_or_device_contact(self): + with mock.patch.object(WORKER.sys, "argv", ["frida_worker", "--profile", "root"]): + with self.assertRaisesRegex(ValueError, "profile"): + WORKER.main() + +class SessionTests(unittest.TestCase): + def fixture(self, behavior): + calls = [] + device, session, script = mock.Mock(), mock.Mock(), mock.Mock() + device.spawn.side_effect = lambda argv: calls.append(("spawn", argv)) or 427 + device.attach.return_value = session + session.create_script.return_value = script + callback = [] + script.on.side_effect = lambda _name, handler: callback.append(handler) + script.load.side_effect = lambda: behavior(callback[0]) + device.kill.side_effect = lambda pid: calls.append(("kill", pid)) + script.unload.side_effect = lambda: calls.append(("unload",)) + session.detach.side_effect = lambda: calls.append(("detach",)) + return device, session, script, calls + + def test_stop_before_ready_cleans_owned_suspended_process_without_ready(self): + stopped = WORKER.threading.Event() + device, session, script, calls = self.fixture(lambda _cb: stopped.set()) + output = [] + with mock.patch.object(WORKER, "emit", side_effect=lambda kind, **fields: output.append({"kind": kind, **fields})): + start = time.monotonic() + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, ready_timeout=30, cleanup_timeout=0.1) + self.assertLess(time.monotonic() - start, 0.5) + self.assertEqual(calls, [("spawn", ["app.test"]), ("kill", 427), ("unload",), ("detach",)]) + device.resume.assert_not_called() + self.assertFalse(any(row["kind"] == "ready" for row in output)) + self.assertEqual(output[0], {"kind": "spawned", "engine": "scope-java-v1", "pid": 427}) + self.assertEqual(output[-1]["processTerminated"], True) + self.assertEqual(output[-1]["pid"], 427) + + def test_late_ready_cannot_revive_hook_error_and_startup_failure_kills_pid(self): + stopped = WORKER.threading.Event() + def behavior(callback): + callback({"type": "send", "payload": {"kind": "scope-hooks-error", "message": "no pinner"}}, None) + callback({"type": "send", "payload": {"kind": "scope-hooks-ready", "engine": "scope-java-v1", "profiles": ["proxy"], "capabilities": {"proxy": ["java-url"]}}}, None) + device, session, script, calls = self.fixture(behavior) + output = [] + with mock.patch.object(WORKER, "emit", side_effect=lambda kind, **fields: output.append({"kind": kind, **fields})): + with self.assertRaisesRegex(RuntimeError, "no pinner"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.1) + self.assertIn(("kill", 427), calls) + self.assertFalse(any(row["kind"] == "ready" for row in output)) + + def test_failed_cleanup_never_emits_clean_stopped(self): + stopped = WORKER.threading.Event() + device, session, script, calls = self.fixture(lambda _cb: stopped.set()) + device.kill.side_effect = RuntimeError("cannot kill") + output = [] + with mock.patch.object(WORKER, "emit", side_effect=lambda kind, **fields: output.append({"kind": kind, **fields})): + with self.assertRaisesRegex(RuntimeError, "cannot kill"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.1) + self.assertFalse(any(row["kind"] == "stopped" for row in output)) + +class DestroyedCleanupTests(unittest.TestCase): + def test_already_destroyed_script_and_detached_session_need_no_second_cleanup(self): + device, script, session = mock.Mock(), mock.Mock(), mock.Mock() + script.is_destroyed = True + session.is_detached = True + script.unload.side_effect = RuntimeError("script has been destroyed") + session.detach.side_effect = RuntimeError("already detached") + WORKER.cleanup_owned(device, 427, script, session, timeout=0.1) + device.kill.assert_called_once_with(427) + script.unload.assert_not_called() + session.detach.assert_not_called() + +class StartupCancellationTests(unittest.TestCase): + fixture = SessionTests.fixture + def test_stop_cancels_inflight_script_load_then_kills_only_owned_pid(self): + stopped = WORKER.threading.Event() + cancelled = WORKER.threading.Event() + class Cancellable: + def __enter__(self): + return self + def __exit__(self, *_args): + pass + def cancel(self): + cancelled.set() + device, session, script, calls = self.fixture(lambda _cb: None) + def blocked_load(): + stopped.set() + if not cancelled.wait(0.5): + raise RuntimeError("startup was not cancelled") + raise RuntimeError("operation cancelled") + script.load.side_effect = blocked_load + with mock.patch.object(WORKER, "emit"): + with self.assertRaisesRegex(RuntimeError, "operation cancelled"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.1, cancellable_factory=Cancellable) + self.assertTrue(cancelled.is_set()) + self.assertIn(("kill", 427), calls) + + +if __name__ == "__main__": + unittest.main() diff --git a/resources/frida/scope-engine.js b/resources/frida/scope-engine.js new file mode 100644 index 0000000..8885432 --- /dev/null +++ b/resources/frida/scope-engine.js @@ -0,0 +1,299 @@ +/* Scope-owned Java interception engine. Uses the documented Frida Java API. + * Coverage is deliberately limited to java.net.URL, SSLContext and OkHttp3. + * Method rollback cannot revoke already-created clients: the worker must end + * its spawned process on Stop or failure. No hostname verifier is replaced. + */ +(function (root) { + 'use strict'; + const PROFILES = ['proxy', 'certificate', 'unpinning']; + const REQUIRED = { proxy: 'java-url', certificate: 'sslcontext-scope-ca', unpinning: 'okhttp3-certificate-pinner' }; + + function normalizeProfiles(profiles) { + if (!Array.isArray(profiles) || profiles.some(p => !PROFILES.includes(p))) { + throw new Error('Unsupported hook profile; supported: proxy, certificate, unpinning'); + } + const selected = profiles.length ? PROFILES.filter(p => profiles.includes(p)) : PROFILES.slice(); + if (selected.includes('unpinning') && !selected.includes('certificate')) { + throw new Error('unpinning profile requires certificate'); + } + return selected; + } + + function createPolicy(validateScopeChain) { + function isScope(chain, context) { + try { validateScopeChain(chain, context); return true; } catch (_) { return false; } + } + return { + checkPins(chain, original) { if (!isScope(chain)) return original(); }, + checkTrust(chain, original, context) { + try { return original(); } catch (rejection) { + if (!isScope(chain, context)) throw rejection; + } + } + }; + } + + function createEngine(config, adapter) { + const profiles = normalizeProfiles(config.profiles); + const undo = []; + const transaction = { + hook(method, implementation) { + const previous = method.implementation; + // Register undo first: a failing setter can have partially installed. + undo.push(() => { method.implementation = previous; }); + method.implementation = implementation; + } + }; + function rollback() { + const errors = []; + while (undo.length) { + try { undo.pop()(); } catch (error) { errors.push(String(error)); } + } + if (errors.length) throw new Error('Hook rollback failed: ' + errors.join('; ').slice(0, 1500)); + } + let installed = false; + return { + rollback, + install() { + if (installed) throw new Error('Engine already installed'); + const capabilities = {}; + try { + for (const profile of profiles) { + const name = 'install' + profile[0].toUpperCase() + profile.slice(1); + const result = adapter[name](transaction); + if (!Array.isArray(result) || !result.includes(REQUIRED[profile])) { + throw new Error('Missing installed capability for profile ' + profile); + } + capabilities[profile] = result; + } + installed = true; + return { kind: 'scope-hooks-ready', engine: 'scope-java-v1', profiles, capabilities }; + } catch (error) { + try { rollback(); } catch (cleanupError) { + throw new Error(String(error) + '; ' + String(cleanupError)); + } + throw error; + } + } + }; + } + + function createJavaAdapter(Java, config) { + let trust; + function use(name) { + try { return Java.use(name); } catch (error) { + throw new Error('Required Java class unavailable: ' + name + ': ' + String(error).slice(0, 300)); + } + } + function overload(klass, method, signature) { + if (!klass[method]) throw new Error('Required method unavailable: ' + method); + try { return klass[method].overload(...signature); } catch (error) { + throw new Error('Required overload unavailable: ' + method + '(' + signature.join(', ') + '): ' + String(error).slice(0, 300)); + } + } + function trustState() { + if (trust) return trust; + const X509 = use('javax.net.ssl.X509TrustManager'); + const Certificate = use('java.security.cert.X509Certificate'); + const Factory = use('javax.net.ssl.TrustManagerFactory'); + const Store = use('java.security.KeyStore'); + const bytes = use('java.lang.String').$new(config.caPem).getBytes('UTF-8'); + const stream = use('java.io.ByteArrayInputStream').$new(bytes); + const ca = Java.cast(use('java.security.cert.CertificateFactory').getInstance('X.509').generateCertificate(stream), Certificate); + ca.checkValidity(); + if (ca.getBasicConstraints() < 0) throw new Error('Scope certificate must be a CA'); + const store = Store.getInstance(Store.getDefaultType()); + store.load(null, null); // Empty, never the Android/system trust store. + store.setCertificateEntry('scope', ca); + const factory = Factory.getInstance(Factory.getDefaultAlgorithm()); + factory.init(store); + const managers = factory.getTrustManagers(); + let manager; + for (let i = 0; i < managers.length; i++) { + if (X509.class.isInstance(managers[i])) { manager = Java.cast(managers[i], X509); break; } + } + if (!manager) throw new Error('No X509 trust manager for isolated Scope CA'); + let Extended; + try { Extended = Java.use('javax.net.ssl.X509ExtendedTrustManager'); } catch (_) { /* Android before API 24. */ } + trust = { manager, ca, Factory, X509, Certificate, Extended }; + return trust; + } + function certificates(chain) { + if (!chain) throw new Error('Peer chain is empty'); + const values = typeof chain.toArray === 'function' ? chain.toArray() : chain; + if (!values.length) throw new Error('Peer chain is empty'); + const { Certificate } = trustState(); + const result = []; + for (let i = 0; i < values.length; i++) { + if (!Certificate.class.isInstance(values[i])) throw new Error('Peer chain contains a non-X509 certificate'); + result.push(Java.cast(values[i], Certificate)); + } + return result; + } + function validateScopeChain(chain, context) { + const state = trustState(); + const values = certificates(chain); + if (values[0].getBasicConstraints() >= 0) throw new Error('Peer chain must start with a server leaf certificate'); + // Check every supplied certificate, including trust anchors. Trust manager + // performs path construction, signature, usage and algorithm validation. + state.ca.checkValidity(); + values.forEach(cert => cert.checkValidity()); + const authType = context ? context.authType : String(values[0].getPublicKey().getAlgorithm()); + const peer = Java.array('java.security.cert.X509Certificate', values); + if (context && context.type) { + if (!state.Extended || !state.Extended.class.isInstance(state.manager)) throw new Error('Isolated Scope manager cannot preserve extended TLS context'); + const manager = Java.cast(state.manager, state.Extended); + manager.checkServerTrusted.overload('[Ljava.security.cert.X509Certificate;', 'java.lang.String', context.type).call(manager, peer, authType, context.transport); + } else { + state.manager.checkServerTrusted(peer, authType); + } + } + const policy = createPolicy(validateScopeChain); + + function installCertificate(transaction) { + const state = trustState(); + const init = overload(use('javax.net.ssl.SSLContext'), 'init', ['[Ljavax.net.ssl.KeyManager;', '[Ljavax.net.ssl.TrustManager;', 'java.security.SecureRandom']); + function wrapperClass(extended) { + const methods = { + getAcceptedIssuers() { + const original = this.delegate.value.getAcceptedIssuers(); + return Java.array('java.security.cert.X509Certificate', Array.from(original).concat([state.ca])); + } + }; + for (const name of ['checkClientTrusted', 'checkServerTrusted']) { + const implementations = []; + for (const type of extended ? [null, 'java.net.Socket', 'javax.net.ssl.SSLEngine'] : [null]) { + implementations.push({ + returnType: 'void', + argumentTypes: ['[Ljava.security.cert.X509Certificate;', 'java.lang.String'].concat(type ? [type] : []), + implementation(chain, authType, transport) { + const delegate = Java.cast(this.delegate.value, extended ? state.Extended : state.X509); + const original = () => type ? delegate[name](chain, authType, transport) : delegate[name](chain, authType); + if (name === 'checkClientTrusted') return original(); + return policy.checkTrust(chain, original, { authType, type, transport }); + } + }); + } + methods[name] = implementations; + } + return Java.registerClass({ + name: extended ? 'dev.scope.runtime.ScopeExtendedTrustManager' : 'dev.scope.runtime.ScopeTrustManager', + ...(extended ? { superClass: state.Extended } : { implements: [state.X509] }), + fields: { delegate: 'javax.net.ssl.X509TrustManager' }, + methods + }); + } + const BasicWrapper = wrapperClass(false); + const ExtendedWrapper = state.Extended && state.Extended.class.isInstance(state.manager) ? wrapperClass(true) : null; + transaction.hook(init, function (keys, managers, random) { + if (managers === null) { + const defaults = state.Factory.getInstance(state.Factory.getDefaultAlgorithm()); + // Select the KeyStore overload explicitly; null otherwise ambiguous. + defaults.init.overload('java.security.KeyStore').call(defaults, null); + managers = defaults.getTrustManagers(); + } + let count = 0; + const wrapped = Array.from(managers, manager => { + if (!state.X509.class.isInstance(manager)) return manager; + const extended = state.Extended && state.Extended.class.isInstance(manager); + if (extended && !ExtendedWrapper) throw new Error('Cannot preserve X509ExtendedTrustManager context with this Scope manager'); + const wrapper = (extended ? ExtendedWrapper : BasicWrapper).$new(); + wrapper.delegate.value = Java.cast(manager, state.X509); + count++; + return wrapper; + }); + if (!count) throw new Error('SSLContext has no X509 trust manager to extend'); + return init.call(this, keys, Java.array('javax.net.ssl.TrustManager', wrapped), random); + }); + return ['sslcontext-scope-ca']; + } + + function installProxy(transaction) { + const URL = use('java.net.URL'); + const Proxy = use('java.net.Proxy'); + const address = use('java.net.InetSocketAddress').$new(config.proxyHost, config.proxyPort); + const proxy = Proxy.$new(use('java.net.Proxy$Type').HTTP.value, address); + const plain = overload(URL, 'openConnection', []); + const explicit = overload(URL, 'openConnection', ['java.net.Proxy']); + function http(url) { return ['http', 'https'].includes(String(url.getProtocol())); } + transaction.hook(plain, function () { return http(this) ? explicit.call(this, proxy) : plain.call(this); }); + transaction.hook(explicit, function (selected) { return explicit.call(this, http(this) ? proxy : selected); }); + const capabilities = ['java-url']; + let Builder; + try { Builder = Java.use('okhttp3.OkHttpClient$Builder'); } catch (_) { return capabilities; } + const build = overload(Builder, 'build', []); + const setProxy = overload(Builder, 'proxy', ['java.net.Proxy']); + transaction.hook(build, function () { setProxy.call(this, proxy); return build.call(this); }); + capabilities.push('okhttp3-proxy'); + return capabilities; + } + + function installUnpinning(transaction) { + trustState(); + const Pinner = use('okhttp3.CertificatePinner'); + let count = 0; + let Supplier; + for (const name of ['check', 'check$okhttp']) { + if (!Pinner[name]) continue; + for (const method of Pinner[name].overloads) { + const args = method.argumentTypes.map(type => type.className); + const ordinary = name === 'check' && args.length === 2 && args[0] === 'java.lang.String' && ['java.util.List', '[Ljava.security.cert.Certificate;'].includes(args[1]); + const lazy = name === 'check$okhttp' && args.length === 2 && args[0] === 'java.lang.String' && args[1] === 'kotlin.jvm.functions.Function0'; + if (!ordinary && !lazy) throw new Error('Unsupported OkHttp pinner overload: ' + name + '(' + args.join(', ') + ')'); + if (lazy && !Supplier) { + Supplier = Java.registerClass({ + name: 'dev.scope.runtime.ScopePeerChainSupplier', + implements: [use('kotlin.jvm.functions.Function0')], + fields: { delegate: 'kotlin.jvm.functions.Function0', chain: 'java.lang.Object', evaluated: 'boolean' }, + methods: { + invoke: { + returnType: 'java.lang.Object', argumentTypes: [], + implementation() { + if (!this.evaluated.value) { + this.chain.value = this.delegate.value.invoke(); + this.evaluated.value = true; + } + return this.chain.value; + } + } + } + }); + } + transaction.hook(method, function (host, peer) { + if (lazy) { + const supplier = Supplier.$new(); + supplier.delegate.value = peer; + // Let the original decide whether pins need a chain at all. Cache + // its actual input so a second supplier result cannot grant trust. + try { return method.call(this, host, supplier); } catch (rejection) { + if (!supplier.evaluated.value) throw rejection; + const chain = Java.cast(supplier.chain.value, use('java.util.List')); + return policy.checkPins(chain, () => { throw rejection; }); + } + } + return policy.checkPins(peer, () => method.call(this, host, peer)); + }); + count++; + } + } + if (!count) throw new Error('No supported okhttp3.CertificatePinner overloads'); + return ['okhttp3-certificate-pinner']; + } + return { validateScopeChain, installProxy, installCertificate, installUnpinning }; + } + + function start(config) { + try { + normalizeProfiles(config.profiles); + if (config.proxyHost !== '127.0.0.1' || !Number.isInteger(config.proxyPort) || config.proxyPort < 1 || config.proxyPort > 65535) throw new Error('Invalid loopback proxy configuration'); + if (!root.Java || !root.Java.available) throw new Error('Frida Java runtime is unavailable'); + root.Java.perform(() => { + try { root.send(createEngine(config, createJavaAdapter(root.Java, config)).install()); } + catch (error) { root.send({ kind: 'scope-hooks-error', engine: 'scope-java-v1', message: String(error).slice(0, 2000) }); } + }); + } catch (error) { root.send({ kind: 'scope-hooks-error', engine: 'scope-java-v1', message: String(error).slice(0, 2000) }); } + } + const api = { normalizeProfiles, createPolicy, createEngine, createJavaAdapter, start }; + if (typeof module !== 'undefined' && module.exports) module.exports = api; + else root.ScopeJavaEngine = api; +})(globalThis); diff --git a/resources/frida/tests/scope-engine.test.cjs b/resources/frida/tests/scope-engine.test.cjs new file mode 100644 index 0000000..b66c3a2 --- /dev/null +++ b/resources/frida/tests/scope-engine.test.cjs @@ -0,0 +1,254 @@ +'use strict'; +// Java/Frida doubles exercise Scope policy and API wiring, not Android execution. +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const vm = require('node:vm'); +let engine; +try { + const context = { module: { exports: {} } }; + vm.runInNewContext(fs.readFileSync(require('node:path').join(__dirname, '../scope-engine.js'), 'utf8'), context); + engine = context.module.exports; +} catch (error) { if (error.code !== 'ENOENT') throw error; } +const api = () => { assert.ok(engine, 'Scope-owned Java engine is implemented'); return engine; }; +const plain = value => JSON.parse(JSON.stringify(value)); + +test('profiles default, deduplicate and reject unsupported or missing dependencies', () => { + const { normalizeProfiles } = api(); + assert.deepEqual(plain(normalizeProfiles([])), ['proxy', 'certificate', 'unpinning']); + assert.deepEqual(plain(normalizeProfiles(['certificate', 'proxy', 'certificate'])), ['proxy', 'certificate']); + for (const value of [['root'], ['system-ca'], ['other'], ['unpinning'], 'proxy', [null]]) { + assert.throws(() => normalizeProfiles(value), /profile|certificate/i); + } +}); + +test('pins skip only a chain accepted by the isolated CA validator; preserve original rejection identity', () => { + const { createPolicy } = api(); + const valid = {}, unrelated = {}, expired = {}, malformed = {}; + const rejection = new Error('original pin mismatch'); + const checked = []; + const policy = createPolicy(chain => { checked.push(chain); if (chain !== valid) throw new Error('PKIX rejection'); }); + let originalCalls = 0; + const original = () => { originalCalls++; throw rejection; }; + assert.equal(policy.checkPins(valid, original), undefined); + for (const chain of [unrelated, expired, malformed, null]) { + assert.throws(() => policy.checkPins(chain, original), error => error === rejection); + } + assert.equal(originalCalls, 4); + assert.equal(checked.length, 5); +}); + +test('trust delegates original decisions and preserves original errors for invalid Scope chains', () => { + const { createPolicy } = api(); + const valid = {}, rejection = new Error('original trust rejection'); + const policy = createPolicy(chain => { if (chain !== valid) throw new Error('invalid'); }); + assert.equal(policy.checkTrust({}, () => 'original acceptance'), 'original acceptance'); + assert.equal(policy.checkTrust(valid, () => { throw rejection; }), undefined); + assert.throws(() => policy.checkTrust({}, () => { throw rejection; }), error => error === rejection); +}); + +test('missing requested capabilities fail transactionally and restore earlier hooks', () => { + const { createEngine } = api(); + const method = { implementation: 'original' }; + const adapter = { + installProxy(transaction) { transaction.hook(method, () => 'proxy'); return ['java-url']; }, + installCertificate() { throw new Error('SSLContext.init overload missing'); }, + }; + assert.throws(() => createEngine({ profiles: ['proxy', 'certificate'] }, adapter).install(), /SSLContext.init/); + assert.equal(method.implementation, 'original'); + assert.throws(() => createEngine({ profiles: ['proxy'] }, { installProxy() { return []; } }).install(), /capabilit/i); +}); + +test('readiness reports exactly installed requested capabilities and rollback is explicit', () => { + const { createEngine } = api(); + const method = { implementation: null }; + const instance = createEngine({ profiles: ['proxy'] }, { + installProxy(transaction) { transaction.hook(method, () => 'proxy'); return ['java-url']; }, + }); + assert.deepEqual(plain(instance.install()), { kind: 'scope-hooks-ready', engine: 'scope-java-v1', profiles: ['proxy'], capabilities: { proxy: ['java-url'] } }); + instance.rollback(); + assert.equal(method.implementation, null); +}); + +function javaFixture() { + const calls = []; + const ca = { checkValidity() { calls.push('ca-valid'); }, getBasicConstraints() { return 1; } }; + const scopeTrust = { checkServerTrusted(chain, auth) { calls.push(['validate', chain, auth]); if (chain.some(c => c.unrelated)) throw new Error('bad signature'); } }; + const x509 = { class: { isInstance(value) { return value === scopeTrust; } } }; + const store = { load(stream, pass) { calls.push(['empty-store', stream, pass]); }, setCertificateEntry(name, cert) { calls.push(['anchor', name, cert]); } }; + const classes = { + 'java.lang.String': { $new(value) { return { getBytes() { return value; } }; } }, + 'java.io.ByteArrayInputStream': { $new() { return {}; } }, + 'java.security.cert.CertificateFactory': { getInstance() { return { generateCertificate() { return ca; } }; } }, + 'java.security.KeyStore': { getDefaultType() { return 'BKS'; }, getInstance() { return store; } }, + 'javax.net.ssl.TrustManagerFactory': { getDefaultAlgorithm() { return 'PKIX'; }, getInstance() { return { init(value) { calls.push(['tm-init', value]); }, getTrustManagers() { return [scopeTrust]; } }; } }, + 'javax.net.ssl.X509TrustManager': x509, + 'java.security.cert.X509Certificate': { class: { isInstance(value) { return Boolean(value && value.checkValidity); } } }, + }; + const Java = { use(name) { if (!classes[name]) throw new Error('Missing class ' + name); return classes[name]; }, cast(value) { return value; }, array(type, value) { return value; } }; + return { Java, calls, ca, scopeTrust, store, classes }; +} +function leaf(options = {}) { return { checkValidity() { if (options.expired) throw new Error('expired'); }, getBasicConstraints() { return -1; }, getPublicKey() { return { getAlgorithm() { return 'RSA'; } }; }, ...options }; } + +test('Java validator uses empty keystore with only Scope CA and validates peer signatures and validity', () => { + const { createJavaAdapter } = api(); + const fixture = javaFixture(); + const adapter = createJavaAdapter(fixture.Java, { caPem: 'public certificate', proxyHost: '127.0.0.1', proxyPort: 17890 }); + const good = leaf(); + adapter.validateScopeChain([good]); + assert.deepEqual(fixture.calls.find(call => Array.isArray(call) && call[0] === 'empty-store'), ['empty-store', null, null]); + assert.deepEqual(fixture.calls.filter(call => Array.isArray(call) && call[0] === 'anchor'), [['anchor', 'scope', fixture.ca]]); + assert.deepEqual(plain(fixture.calls.find(call => Array.isArray(call) && call[0] === 'validate')), ['validate', [{}], 'RSA']); + assert.throws(() => adapter.validateScopeChain([leaf({ unrelated: true }), fixture.ca]), /bad signature/); + assert.throws(() => adapter.validateScopeChain([leaf({ expired: true })]), /expired/); + assert.throws(() => adapter.validateScopeChain([fixture.ca]), /leaf/); + assert.throws(() => adapter.validateScopeChain([]), /empty|chain/i); + assert.throws(() => adapter.validateScopeChain([{}]), /certificate/i); +}); + +test('real adapter fails closed on missing Java classes and missing OkHttp pinner', () => { + const { createJavaAdapter, createEngine } = api(); + const fixture = javaFixture(); + const adapter = createJavaAdapter(fixture.Java, { caPem: 'public certificate', proxyHost: '127.0.0.1', proxyPort: 17890 }); + assert.throws(() => createEngine({ profiles: ['proxy'] }, adapter).install(), /java.net|Missing class/); + assert.throws(() => adapter.installUnpinning({ hook() {} }), /okhttp3.CertificatePinner/); +}); + +function method(signature, original = () => undefined) { + return { argumentTypes: signature.map(className => ({ className })), implementation: null, call(receiver, ...args) { return original.apply(receiver, args); } }; +} +function dispatcher(...methods) { + return { overloads: methods, overload(...signature) { + const found = methods.find(m => JSON.stringify(m.argumentTypes.map(t => t.className)) === JSON.stringify(signature)); + if (!found) throw new Error('overload missing'); + return found; + } }; +} +function tlsFixture(extended = false) { + const f = javaFixture(); + const calls = []; + const rejection = new Error('original rejected'); + const original = { extended, getAcceptedIssuers() { return []; }, checkClientTrusted(...args) { calls.push(['client', ...args]); }, checkServerTrusted(...args) { calls.push(['server', ...args]); throw rejection; } }; + f.classes['javax.net.ssl.X509TrustManager'].class.isInstance = value => value === f.scopeTrust || value === original; + f.classes['javax.net.ssl.X509ExtendedTrustManager'] = { class: { isInstance: value => Boolean(value.extended) } }; + f.scopeTrust.extended = extended; + let wrapped; + const init = method(['[Ljavax.net.ssl.KeyManager;', '[Ljavax.net.ssl.TrustManager;', 'java.security.SecureRandom'], function (_keys, managers) { wrapped = managers[0]; }); + f.classes['javax.net.ssl.SSLContext'] = { init: dispatcher(init) }; + let definition; + f.Java.registerClass = spec => { + definition = spec; + return { $new() { + const object = { delegate: { value: null } }; + for (const [name, entry] of Object.entries(spec.methods)) { + object[name] = Array.isArray(entry) ? (...args) => entry.find(m => m.argumentTypes.length === args.length && (args.length !== 3 || m.argumentTypes[2] === args[2].type)).implementation.apply(object, args) : entry.bind(object); + } + return object; + } }; + }; + return { ...f, init, original, calls, rejection, get wrapped() { return wrapped; }, get definition() { return definition; } }; +} + +test('SSLContext adds Scope trust while preserving client checks, original errors and issuer inputs', () => { + const { createJavaAdapter, createEngine } = api(); + const f = tlsFixture(); + createEngine({ profiles: ['certificate'] }, createJavaAdapter(f.Java, { caPem: 'public certificate' })).install(); + f.init.implementation(null, [f.original], null); + const valid = [leaf()]; + assert.equal(f.wrapped.checkServerTrusted(valid, 'RSA'), undefined); + assert.throws(() => f.wrapped.checkServerTrusted([leaf({ unrelated: true })], 'RSA'), e => e === f.rejection); + f.wrapped.checkClientTrusted(valid, 'RSA'); + assert.equal(f.calls.at(-1)[0], 'client'); + assert.equal(f.calls.at(-1)[1], valid); + assert.equal(f.wrapped.getAcceptedIssuers()[0], f.ca); +}); + +test('extended trust preserves socket and engine context and scope context rejection', () => { + const { createJavaAdapter, createEngine } = api(); + const f = tlsFixture(true); + const contexts = []; + const basicValidation = f.scopeTrust.checkServerTrusted; + f.scopeTrust.checkServerTrusted = dispatcher( + method(['[Ljava.security.cert.X509Certificate;', 'java.lang.String'], basicValidation), + method(['[Ljava.security.cert.X509Certificate;', 'java.lang.String', 'java.net.Socket'], (_chain, _auth, context) => { contexts.push(context); throw new Error('hostname mismatch'); }), + method(['[Ljava.security.cert.X509Certificate;', 'java.lang.String', 'javax.net.ssl.SSLEngine'], (_chain, _auth, context) => { contexts.push(context); }) + ); + createEngine({ profiles: ['certificate'] }, createJavaAdapter(f.Java, { caPem: 'public certificate' })).install(); + f.init.implementation(null, [f.original], null); + assert.equal(f.definition.superClass, f.classes['javax.net.ssl.X509ExtendedTrustManager']); + const socket = { type: 'java.net.Socket' }, tlsEngine = { type: 'javax.net.ssl.SSLEngine' }; + assert.throws(() => f.wrapped.checkServerTrusted([leaf()], 'RSA', socket), e => e === f.rejection); + assert.equal(f.wrapped.checkServerTrusted([leaf()], 'RSA', tlsEngine), undefined); + assert.equal(contexts[0], socket); + assert.equal(contexts[1], tlsEngine); + assert.equal(f.calls[0][3], socket); + assert.equal(f.calls[1][3], tlsEngine); +}); + +test('URL proxy hooks redirect HTTP only and rollback restores original implementations', () => { + const { createJavaAdapter, createEngine } = api(); + const f = javaFixture(); + const scopeProxy = {}; + const plainMethod = method([], () => 'original non-http'); + const explicit = method(['java.net.Proxy'], selected => selected); + f.classes['java.net.URL'] = { openConnection: dispatcher(plainMethod, explicit) }; + f.classes['java.net.Proxy'] = { $new(_type, address) { assert.equal(address.host, '127.0.0.1'); assert.equal(address.port, 17890); return scopeProxy; } }; + f.classes['java.net.Proxy$Type'] = { HTTP: { value: 'HTTP' } }; + f.classes['java.net.InetSocketAddress'] = { $new(host, port) { return { host, port }; } }; + const instance = createEngine({ profiles: ['proxy'] }, createJavaAdapter(f.Java, { proxyHost: '127.0.0.1', proxyPort: 17890 })); + instance.install(); + assert.equal(plainMethod.implementation.call({ getProtocol: () => 'https' }), scopeProxy); + assert.equal(plainMethod.implementation.call({ getProtocol: () => 'file' }), 'original non-http'); + const custom = {}; + assert.equal(explicit.implementation.call({ getProtocol: () => 'file' }, custom), custom); + instance.rollback(); + assert.equal(plainMethod.implementation, null); + assert.equal(explicit.implementation, null); +}); + +test('OkHttp actual hook preserves unrelated pin exception and rejects unsupported overload transactionally', () => { + const { createJavaAdapter, createEngine } = api(); + const f = javaFixture(); + const rejection = new Error('pin mismatch'); + const check = method(['java.lang.String', 'java.util.List'], () => { throw rejection; }); + f.classes['okhttp3.CertificatePinner'] = { check: dispatcher(check) }; + const adapter = createJavaAdapter(f.Java, { caPem: 'public certificate' }); + // Certificate profile orchestration is tested separately; install the actual pin adapter here. + const instance = createEngine({ profiles: ['certificate', 'unpinning'] }, { installCertificate: () => ['sslcontext-scope-ca'], installUnpinning: adapter.installUnpinning }); + instance.install(); + assert.equal(check.implementation('example.test', [leaf()]), undefined); + assert.throws(() => check.implementation('example.test', [leaf({ unrelated: true }), f.ca]), e => e === rejection); + instance.rollback(); + f.classes['okhttp3.CertificatePinner'].check.overloads.push(method(['java.lang.String', 'unknown.Chain'])); + assert.throws(() => instance.install(), /already installed/); + const second = createEngine({ profiles: ['certificate', 'unpinning'] }, { installCertificate: () => ['sslcontext-scope-ca'], installUnpinning: adapter.installUnpinning }); + assert.throws(() => second.install(), /Unsupported OkHttp/); + assert.equal(check.implementation, null); +}); + +test('Kotlin pin hook validates the exact original supplier result once and preserves no-pin acceptance', () => { + const { createJavaAdapter, createEngine } = api(); + const f = javaFixture(); + f.classes['kotlin.jvm.functions.Function0'] = {}; + f.classes['java.util.List'] = {}; + f.Java.registerClass = spec => ({ $new() { + const object = {}; + for (const field of Object.keys(spec.fields)) object[field] = { value: null }; + for (const [name, entry] of Object.entries(spec.methods)) object[name] = (typeof entry === 'function' ? entry : entry.implementation).bind(object); + return object; + } }); + const rejection = new Error('pin mismatch'); + const check = method(['java.lang.String', 'kotlin.jvm.functions.Function0'], function (host, peer) { if (host === 'no-pins.test') return; peer.invoke(); throw rejection; }); + f.classes['okhttp3.CertificatePinner'] = { 'check$okhttp': dispatcher(check) }; + const adapter = createJavaAdapter(f.Java, { caPem: 'public certificate' }); + createEngine({ profiles: ['certificate', 'unpinning'] }, { installCertificate: () => ['sslcontext-scope-ca'], installUnpinning: adapter.installUnpinning }).install(); + let calls = 0; + const peer = { invoke() { calls++; return calls === 1 ? [leaf({ unrelated: true })] : [leaf()]; } }; + assert.equal(check.implementation('no-pins.test', peer), undefined); + assert.equal(calls, 0); + assert.throws(() => check.implementation('pinned.test', peer), e => e === rejection); + assert.equal(calls, 1); + assert.equal(check.implementation('pinned.test', { invoke: () => [leaf()] }), undefined); + const supplierError = new Error('chain cleaner rejected'); + assert.throws(() => check.implementation('pinned.test', { invoke() { throw supplierError; } }), e => e === supplierError); +}); From 8a06c5bf41a92ead29c1a7c4d9eb721e413ef708 Mon Sep 17 00:00:00 2001 From: Mateusz Dalke <150788324+dalkedev@users.noreply.github.com> Date: Sat, 7 Nov 2026 08:13:04 +0000 Subject: [PATCH 03/18] fix: preserve Java exceptions and current process ownership --- resources/device/frida_worker.py | 81 +++++++++++++++++--- resources/device/tests/test_frida_worker.py | 84 +++++++++++++++++++++ resources/frida/scope-engine.js | 36 ++++++--- resources/frida/tests/scope-engine.test.cjs | 78 +++++++++++++++++-- 4 files changed, 251 insertions(+), 28 deletions(-) diff --git a/resources/device/frida_worker.py b/resources/device/frida_worker.py index 2dbf8c5..10a932c 100644 --- a/resources/device/frida_worker.py +++ b/resources/device/frida_worker.py @@ -86,14 +86,69 @@ def run(): return value -def cleanup_owned(device, pid, script, session, timeout=3, process_not_found=()): +class ProcessOwnership: + """Current local evidence about one spawn, not an atomic remote PID handle.""" + + def __init__(self): + self.exited = threading.Event() + self.replaced = threading.Event() + self.lock = threading.Lock() + self.pending_kill = None + self.kill_dispatched = False + self.kill_uncertain = False + + def observe(self, reason): + pending = None + with self.lock: + if reason == "process-terminated": + self.exited.set() + elif reason == "process-replaced": + self.replaced.set() + if self.kill_dispatched: + self.kill_uncertain = True + pending = self.pending_kill + if pending is not None: + pending.cancel() + + def kill_current(self, operation, cancellable): + try: + with cancellable if cancellable is not None else nullcontext(): + with self.lock: + if self.replaced.is_set(): + raise RuntimeError("Process ownership lost: original image was replaced; refusing kill") + if self.exited.is_set(): + return + self.pending_kill = cancellable + self.kill_dispatched = True + # Check inside the destructive operation's worker thread, rather + # than relying on the caller's earlier ownership snapshot. + if self.replaced.is_set(): + raise RuntimeError("Process ownership lost before kill dispatch; refusing kill") + if self.exited.is_set(): + return + operation() + finally: + with self.lock: + self.pending_kill = None + self.kill_dispatched = False + uncertain = self.kill_uncertain + if uncertain: + raise RuntimeError("Kill completion is unknown after in-flight process replacement; cancellation cannot prove remote outcome") + + +def cleanup_owned(device, pid, script, session, timeout=3, process_not_found=(), ownership=None, cancellable_factory=None): errors = [] + ownership = ownership if ownership is not None else ProcessOwnership() - def kill(): + def kill_pid(): try: device.kill(pid) except process_not_found: - pass # Frida confirms this process already ended. + ownership.observe("process-terminated") + + def kill(): + cancellable = cancellable_factory() if cancellable_factory else None + ownership.kill_current(kill_pid, cancellable) def unload(): if script.is_destroyed is not True: @@ -130,10 +185,10 @@ def detach(): def run_session(device, package, source, profiles, version, stopped, ready_timeout=30, cleanup_timeout=3, process_not_found=(), cancellable_factory=None): profiles = normalize_profiles(profiles) ready = threading.Event() - exited = threading.Event() + ownership = ProcessOwnership() failure = deque(maxlen=2) receipt = [] - replaced = threading.Event() + replaced = ownership.replaced session = script = pid = None def receive(message, _data): @@ -159,10 +214,8 @@ def receive(message, _data): emit("log", message=message.get("stack") or payload or message.get("description") or message) def detached(reason, crash=None): - if reason == "process-terminated": - exited.set() - elif reason == "process-replaced": - replaced.set() + ownership.observe(reason) + if reason == "process-replaced": failure.append("Owned process image was replaced; termination cannot be confirmed safely") emit("detached", message=reason, crash=bool(crash)) stopped.set() @@ -220,13 +273,17 @@ def cancel_startup(): finally: startup_complete.set() try: - cleanup_owned(device, None if exited.is_set() or replaced.is_set() else pid, script, session, cleanup_timeout, process_not_found) + cleanup_owned(device, pid, script, session, cleanup_timeout, process_not_found, ownership, cancellable_factory) except Exception as error: if primary_error: raise RuntimeError(str(primary_error) + "; " + str(error)) from primary_error raise - if not replaced.is_set() and pid is not None: - emit("stopped", engine="scope-java-v1", pid=pid, processTerminated=True, message="Owned application process ended; hooks unloaded and session detached") + if replaced.is_set(): + raise RuntimeError("Owned process image was replaced; cleanup outcome is uncertain") from primary_error + if not attempted_spawn: + emit("stopped", engine="scope-java-v1", pid=None, spawnAttempted=False, cleanupComplete=True, processTerminated=False) + elif pid is not None: + emit("stopped", engine="scope-java-v1", pid=pid, spawnAttempted=True, cleanupComplete=True, processTerminated=True, message="Owned application process ended; hooks unloaded and session detached") if primary_error: raise primary_error diff --git a/resources/device/tests/test_frida_worker.py b/resources/device/tests/test_frida_worker.py index 840e776..9723158 100644 --- a/resources/device/tests/test_frida_worker.py +++ b/resources/device/tests/test_frida_worker.py @@ -174,6 +174,90 @@ def blocked_load(): self.assertTrue(cancelled.is_set()) self.assertIn(("kill", 427), calls) +class ReviewedLifecycleTests(unittest.TestCase): + fixture = SessionTests.fixture + + def test_pre_spawn_stop_has_explicit_safe_no_spawn_receipt(self): + stopped = WORKER.threading.Event() + stopped.set() + device = mock.Mock() + output = [] + with mock.patch.object(WORKER, "emit", side_effect=lambda kind, **fields: output.append({"kind": kind, **fields})): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped) + device.spawn.assert_not_called() + self.assertEqual(output, [{"kind": "stopped", "engine": "scope-java-v1", "pid": None, "spawnAttempted": False, "cleanupComplete": True, "processTerminated": False}]) + + def test_replacement_before_kill_dispatch_never_kills_new_image(self): + stopped = WORKER.threading.Event() + device, session, script, calls = self.fixture(lambda _cb: stopped.set()) + detached = [] + session.on.side_effect = lambda _name, handler: detached.append(handler) + bounded = WORKER.bounded_call + def interleave(label, operation, timeout): + if label == "kill owned application": + detached[0]("process-replaced") + return bounded(label, operation, timeout) + output = [] + with mock.patch.object(WORKER, "bounded_call", side_effect=interleave), mock.patch.object(WORKER, "emit", side_effect=lambda kind, **fields: output.append({"kind": kind, **fields})): + with self.assertRaisesRegex(RuntimeError, "replaced|ownership"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.1) + device.kill.assert_not_called() + self.assertFalse(any(row["kind"] == "stopped" for row in output)) + + def test_replacement_cancels_dispatched_kill_and_retains_uncertainty(self): + stopped = WORKER.threading.Event() + cancelled = WORKER.threading.Event() + class Cancellable: + def __enter__(self): return self + def __exit__(self, *_args): pass + def cancel(self): cancelled.set() + device, session, script, calls = self.fixture(lambda _cb: stopped.set()) + detached = [] + session.on.side_effect = lambda _name, handler: detached.append(handler) + def kill(_pid): + detached[0]("process-replaced") + if not cancelled.wait(0.2): + raise RuntimeError("pending kill was not cancelled") + device.kill.side_effect = kill + with mock.patch.object(WORKER, "emit") as output: + with self.assertRaisesRegex(RuntimeError, "unknown|uncertain"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.3, cancellable_factory=Cancellable) + self.assertTrue(cancelled.is_set()) + self.assertFalse(any(call.args[0] == "stopped" for call in output.call_args_list)) + + +class OwnershipEvidenceTests(unittest.TestCase): + fixture = SessionTests.fixture + + def test_attempted_spawn_without_returned_pid_never_claims_no_spawn_cleanup(self): + stopped = WORKER.threading.Event() + device = mock.Mock() + device.spawn.side_effect = RuntimeError("connection lost during spawn") + with mock.patch.object(WORKER, "emit") as output: + with self.assertRaisesRegex(RuntimeError, "completion is unknown"): + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped) + device.kill.assert_not_called() + self.assertFalse(any(call.args[0] == "stopped" for call in output.call_args_list)) + + def test_observed_exit_before_kill_dispatch_skips_pid_and_acknowledges_termination(self): + stopped = WORKER.threading.Event() + device, session, script, calls = self.fixture(lambda _cb: stopped.set()) + detached = [] + session.on.side_effect = lambda _name, handler: detached.append(handler) + bounded = WORKER.bounded_call + def interleave(label, operation, timeout): + if label == "kill owned application": + detached[0]("process-terminated") + return bounded(label, operation, timeout) + with mock.patch.object(WORKER, "bounded_call", side_effect=interleave), mock.patch.object(WORKER, "emit") as output: + WORKER.run_session(device, "app.test", "source", ["proxy"], "17.9.1", stopped, cleanup_timeout=0.1) + device.kill.assert_not_called() + receipts = [call.kwargs for call in output.call_args_list if call.args[0] == "stopped"] + self.assertEqual(len(receipts), 1) + self.assertIs(receipts[0]["processTerminated"], True) + self.assertIs(receipts[0]["spawnAttempted"], True) + self.assertIs(receipts[0]["cleanupComplete"], True) + if __name__ == "__main__": unittest.main() diff --git a/resources/frida/scope-engine.js b/resources/frida/scope-engine.js index 8885432..65f6c32 100644 --- a/resources/frida/scope-engine.js +++ b/resources/frida/scope-engine.js @@ -19,15 +19,19 @@ return selected; } - function createPolicy(validateScopeChain) { + function createPolicy(validateScopeChain, rejectionTypes = {}) { function isScope(chain, context) { try { validateScopeChain(chain, context); return true; } catch (_) { return false; } } return { - checkPins(chain, original) { if (!isScope(chain)) return original(); }, + checkPins(chain, original) { + try { return original(); } catch (rejection) { + if (!rejectionTypes.isPinRejection || !rejectionTypes.isPinRejection(rejection) || !isScope(chain)) throw rejection; + } + }, checkTrust(chain, original, context) { try { return original(); } catch (rejection) { - if (!isScope(chain, context)) throw rejection; + if (!rejectionTypes.isCertificateRejection || !rejectionTypes.isCertificateRejection(rejection) || !isScope(chain, context)) throw rejection; } } }; @@ -148,16 +152,30 @@ state.manager.checkServerTrusted(peer, authType); } } - const policy = createPolicy(validateScopeChain); + function isJavaException(error, className) { + if (!error || !error.$h) return false; + try { + // The retained bridge attaches the original JNI Throwable handle to + // Java invocation errors. Inspect that object, never message text. + return use(className).class.isInstance(Java.cast(error.$h, use('java.lang.Throwable'))); + } catch (_) { return false; } + } + const policy = createPolicy(validateScopeChain, { + isCertificateRejection: error => isJavaException(error, 'java.security.cert.CertificateException'), + isPinRejection: error => isJavaException(error, 'javax.net.ssl.SSLPeerUnverifiedException') + }); function installCertificate(transaction) { const state = trustState(); const init = overload(use('javax.net.ssl.SSLContext'), 'init', ['[Ljavax.net.ssl.KeyManager;', '[Ljavax.net.ssl.TrustManager;', 'java.security.SecureRandom']); function wrapperClass(extended) { const methods = { - getAcceptedIssuers() { - const original = this.delegate.value.getAcceptedIssuers(); - return Java.array('java.security.cert.X509Certificate', Array.from(original).concat([state.ca])); + getAcceptedIssuers: { + returnType: '[Ljava.security.cert.X509Certificate;', argumentTypes: [], + implementation() { + const original = this.delegate.value.getAcceptedIssuers(); + return Java.array('java.security.cert.X509Certificate', Array.from(original).concat([state.ca])); + } } }; for (const name of ['checkClientTrusted', 'checkServerTrusted']) { @@ -168,7 +186,7 @@ argumentTypes: ['[Ljava.security.cert.X509Certificate;', 'java.lang.String'].concat(type ? [type] : []), implementation(chain, authType, transport) { const delegate = Java.cast(this.delegate.value, extended ? state.Extended : state.X509); - const original = () => type ? delegate[name](chain, authType, transport) : delegate[name](chain, authType); + const original = () => type ? delegate[name].overload('[Ljava.security.cert.X509Certificate;', 'java.lang.String', type).call(delegate, chain, authType, transport) : delegate[name](chain, authType); if (name === 'checkClientTrusted') return original(); return policy.checkTrust(chain, original, { authType, type, transport }); } @@ -266,7 +284,7 @@ // Let the original decide whether pins need a chain at all. Cache // its actual input so a second supplier result cannot grant trust. try { return method.call(this, host, supplier); } catch (rejection) { - if (!supplier.evaluated.value) throw rejection; + if (!supplier.evaluated.value || !isJavaException(rejection, 'javax.net.ssl.SSLPeerUnverifiedException')) throw rejection; const chain = Java.cast(supplier.chain.value, use('java.util.List')); return policy.checkPins(chain, () => { throw rejection; }); } diff --git a/resources/frida/tests/scope-engine.test.cjs b/resources/frida/tests/scope-engine.test.cjs index b66c3a2..b2a9f1d 100644 --- a/resources/frida/tests/scope-engine.test.cjs +++ b/resources/frida/tests/scope-engine.test.cjs @@ -12,6 +12,7 @@ try { } catch (error) { if (error.code !== 'ENOENT') throw error; } const api = () => { assert.ok(engine, 'Scope-owned Java engine is implemented'); return engine; }; const plain = value => JSON.parse(JSON.stringify(value)); +function javaError(javaType, message) { const error = new Error(message); error.$h = { javaType }; return error; } test('profiles default, deduplicate and reject unsupported or missing dependencies', () => { const { normalizeProfiles } = api(); @@ -27,21 +28,21 @@ test('pins skip only a chain accepted by the isolated CA validator; preserve ori const valid = {}, unrelated = {}, expired = {}, malformed = {}; const rejection = new Error('original pin mismatch'); const checked = []; - const policy = createPolicy(chain => { checked.push(chain); if (chain !== valid) throw new Error('PKIX rejection'); }); + const policy = createPolicy(chain => { checked.push(chain); if (chain !== valid) throw new Error('PKIX rejection'); }, { isPinRejection: error => error === rejection }); let originalCalls = 0; const original = () => { originalCalls++; throw rejection; }; assert.equal(policy.checkPins(valid, original), undefined); for (const chain of [unrelated, expired, malformed, null]) { assert.throws(() => policy.checkPins(chain, original), error => error === rejection); } - assert.equal(originalCalls, 4); + assert.equal(originalCalls, 5); assert.equal(checked.length, 5); }); test('trust delegates original decisions and preserves original errors for invalid Scope chains', () => { const { createPolicy } = api(); const valid = {}, rejection = new Error('original trust rejection'); - const policy = createPolicy(chain => { if (chain !== valid) throw new Error('invalid'); }); + const policy = createPolicy(chain => { if (chain !== valid) throw new Error('invalid'); }, { isCertificateRejection: error => error === rejection }); assert.equal(policy.checkTrust({}, () => 'original acceptance'), 'original acceptance'); assert.equal(policy.checkTrust(valid, () => { throw rejection; }), undefined); assert.throws(() => policy.checkTrust({}, () => { throw rejection; }), error => error === rejection); @@ -77,6 +78,9 @@ function javaFixture() { const x509 = { class: { isInstance(value) { return value === scopeTrust; } } }; const store = { load(stream, pass) { calls.push(['empty-store', stream, pass]); }, setCertificateEntry(name, cert) { calls.push(['anchor', name, cert]); } }; const classes = { + 'java.lang.Throwable': {}, + 'java.security.cert.CertificateException': { class: { isInstance: value => value.javaType === 'certificate' } }, + 'javax.net.ssl.SSLPeerUnverifiedException': { class: { isInstance: value => value.javaType === 'pin' } }, 'java.lang.String': { $new(value) { return { getBytes() { return value; } }; } }, 'java.io.ByteArrayInputStream': { $new() { return {}; } }, 'java.security.cert.CertificateFactory': { getInstance() { return { generateCertificate() { return ca; } }; } }, @@ -127,8 +131,12 @@ function dispatcher(...methods) { function tlsFixture(extended = false) { const f = javaFixture(); const calls = []; - const rejection = new Error('original rejected'); + const rejection = javaError('certificate', 'original rejected'); const original = { extended, getAcceptedIssuers() { return []; }, checkClientTrusted(...args) { calls.push(['client', ...args]); }, checkServerTrusted(...args) { calls.push(['server', ...args]); throw rejection; } }; + for (const name of ['checkClientTrusted', 'checkServerTrusted']) { + const dynamic = original[name]; + Object.assign(dynamic, dispatcher(...[null, 'java.net.Socket', 'javax.net.ssl.SSLEngine'].map(type => method(['[Ljava.security.cert.X509Certificate;', 'java.lang.String'].concat(type ? [type] : []), dynamic)))); + } f.classes['javax.net.ssl.X509TrustManager'].class.isInstance = value => value === f.scopeTrust || value === original; f.classes['javax.net.ssl.X509ExtendedTrustManager'] = { class: { isInstance: value => Boolean(value.extended) } }; f.scopeTrust.extended = extended; @@ -141,7 +149,7 @@ function tlsFixture(extended = false) { return { $new() { const object = { delegate: { value: null } }; for (const [name, entry] of Object.entries(spec.methods)) { - object[name] = Array.isArray(entry) ? (...args) => entry.find(m => m.argumentTypes.length === args.length && (args.length !== 3 || m.argumentTypes[2] === args[2].type)).implementation.apply(object, args) : entry.bind(object); + object[name] = Array.isArray(entry) ? (...args) => entry.find(m => m.argumentTypes.length === args.length && (args.length !== 3 || m.argumentTypes[2] === args[2].type)).implementation.apply(object, args) : (typeof entry === 'function' ? entry : entry.implementation).bind(object); } return object; } }; @@ -209,7 +217,7 @@ test('URL proxy hooks redirect HTTP only and rollback restores original implemen test('OkHttp actual hook preserves unrelated pin exception and rejects unsupported overload transactionally', () => { const { createJavaAdapter, createEngine } = api(); const f = javaFixture(); - const rejection = new Error('pin mismatch'); + const rejection = javaError('pin', 'pin mismatch'); const check = method(['java.lang.String', 'java.util.List'], () => { throw rejection; }); f.classes['okhttp3.CertificatePinner'] = { check: dispatcher(check) }; const adapter = createJavaAdapter(f.Java, { caPem: 'public certificate' }); @@ -237,7 +245,7 @@ test('Kotlin pin hook validates the exact original supplier result once and pres for (const [name, entry] of Object.entries(spec.methods)) object[name] = (typeof entry === 'function' ? entry : entry.implementation).bind(object); return object; } }); - const rejection = new Error('pin mismatch'); + const rejection = javaError('pin', 'pin mismatch'); const check = method(['java.lang.String', 'kotlin.jvm.functions.Function0'], function (host, peer) { if (host === 'no-pins.test') return; peer.invoke(); throw rejection; }); f.classes['okhttp3.CertificatePinner'] = { 'check$okhttp': dispatcher(check) }; const adapter = createJavaAdapter(f.Java, { caPem: 'public certificate' }); @@ -252,3 +260,59 @@ test('Kotlin pin hook validates the exact original supplier result once and pres const supplierError = new Error('chain cleaner rejected'); assert.throws(() => check.implementation('pinned.test', { invoke() { throw supplierError; } }), e => e === supplierError); }); + +test('extended accepted issuers registration explicitly implements the X509 array descriptor', () => { + const { createJavaAdapter, createEngine } = api(); + const f = tlsFixture(true); + createEngine({ profiles: ['certificate'] }, createJavaAdapter(f.Java, { caPem: 'public certificate' })).install(); + const descriptor = f.definition.methods.getAcceptedIssuers; + assert.equal(descriptor.returnType, '[Ljava.security.cert.X509Certificate;'); + assert.deepEqual(plain(descriptor.argumentTypes), []); +}); + +test('valid Scope chain cannot suppress unrelated trust or ordinary/lazy pinner errors', () => { + const { createJavaAdapter, createEngine } = api(); + const unrelated = new Error('internal programming failure'); + const f = tlsFixture(); + f.original.checkServerTrusted = () => { throw unrelated; }; + createEngine({ profiles: ['certificate'] }, createJavaAdapter(f.Java, { caPem: 'public certificate' })).install(); + f.init.implementation(null, [f.original], null); + assert.throws(() => f.wrapped.checkServerTrusted([leaf()], 'RSA'), e => e === unrelated); + for (const lazy of [false, true]) { + const p = javaFixture(); + p.classes['kotlin.jvm.functions.Function0'] = {}; + p.classes['java.util.List'] = {}; + p.Java.registerClass = spec => ({ $new() { + const object = {}; + for (const field of Object.keys(spec.fields)) object[field] = { value: null }; + for (const [name, entry] of Object.entries(spec.methods)) object[name] = entry.implementation.bind(object); + return object; + } }); + const signature = ['java.lang.String', lazy ? 'kotlin.jvm.functions.Function0' : 'java.util.List']; + const check = method(signature, (_host, peer) => { if (lazy) peer.invoke(); throw unrelated; }); + p.classes['okhttp3.CertificatePinner'] = { [lazy ? 'check$okhttp' : 'check']: dispatcher(check) }; + const adapter = createJavaAdapter(p.Java, { caPem: 'public certificate' }); + createEngine({ profiles: ['certificate', 'unpinning'] }, { installCertificate: () => ['sslcontext-scope-ca'], installUnpinning: adapter.installUnpinning }).install(); + assert.throws(() => check.implementation('test', lazy ? { invoke: () => [leaf()] } : [leaf()]), e => e === unrelated); + } +}); + +test('nullable extended delegation selects exact original Socket and SSLEngine overloads', () => { + const { createJavaAdapter, createEngine } = api(); + const f = tlsFixture(true); + const selected = []; + for (const name of ['checkClientTrusted', 'checkServerTrusted']) { + const legacy = f.original[name]; + const dynamic = (...args) => legacy(...args); // Baseline dynamic dispatcher cannot distinguish null. + Object.assign(dynamic, dispatcher(...['java.net.Socket', 'javax.net.ssl.SSLEngine'].map(type => method(['[Ljava.security.cert.X509Certificate;', 'java.lang.String', type], (_chain, _auth, transport) => { assert.equal(transport, null); selected.push([name, type]); })))); + f.original[name] = dynamic; + } + createEngine({ profiles: ['certificate'] }, createJavaAdapter(f.Java, { caPem: 'public certificate' })).install(); + f.init.implementation(null, [f.original], null); + for (const name of ['checkClientTrusted', 'checkServerTrusted']) { + for (const type of ['java.net.Socket', 'javax.net.ssl.SSLEngine']) { + f.definition.methods[name].find(entry => entry.argumentTypes[2] === type).implementation.call(f.wrapped, [leaf({ unrelated: true })], 'RSA', null); + } + } + assert.deepEqual(selected, [['checkClientTrusted', 'java.net.Socket'], ['checkClientTrusted', 'javax.net.ssl.SSLEngine'], ['checkServerTrusted', 'java.net.Socket'], ['checkServerTrusted', 'javax.net.ssl.SSLEngine']]); +}); From 0c59a77909f85fd3670304344554bac5574e312d Mon Sep 17 00:00:00 2001 From: Mateusz Dalke <150788324+dalkedev@users.noreply.github.com> Date: Sat, 7 Nov 2026 08:15:29 +0000 Subject: [PATCH 04/18] docs: record reviewed engine lifecycle and integration contract --- .../plans/2026-11-07-owned-android-engine.md | 14 +++++++------- .../2026-11-07-owned-android-engine-design.md | 4 ++-- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/plans/2026-11-07-owned-android-engine.md b/docs/superpowers/plans/2026-11-07-owned-android-engine.md index 24558ef..eb30fcc 100644 --- a/docs/superpowers/plans/2026-11-07-owned-android-engine.md +++ b/docs/superpowers/plans/2026-11-07-owned-android-engine.md @@ -1,6 +1,6 @@ # Scope-owned Android engine Implementation Plan -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking. **Goal:** Replace HTTP Toolkit runtime hooks with an original CA-scoped Android interception engine in one independently reviewed PR. **Architecture:** Original JavaScript hook/policy code runs in the selected app through the existing Python/Frida worker. Rust retains proxy/recovery management and materializes only the new assets. Imported system trust injection and unsupported root profiles are removed explicitly. @@ -33,16 +33,16 @@ **Files:** Create `resources/frida/scope-engine.js`, `resources/frida/tests/scope-engine.test.cjs`, `resources/device/tests/test_frida_worker.py`; modify `resources/device/frida_worker.py`. **Interfaces:** Worker `build_script(root, ca, port, profiles) -> str` remains available. `normalize_profiles(profiles) -> list[str]` is shared by the CLI/builder. New JS exports a testable engine factory/policy boundary and runs through documented Frida `Java` APIs. JSON runtime config uses public CA/loopback proxy/normalized profiles. Readiness payload identifies engine `scope-java-v1`, selected profiles and installed capabilities; Python validates it. Existing Rust JSONL `ready/error/stopped/detached` kinds remain compatible. -- [ ] Write executable JS and Python failing tests for CA-scoped pin decisions, original rejection propagation, original trust delegation, explicit/default/unsupported profiles, missing classes and transactional rollback, hostile config serialization, readiness validation and owned-PID cleanup. -- [ ] Observe RED with expected missing implementation or wrong existing behavior; record exact commands/output in report. -- [ ] Implement original engine (small focused helpers in one runtime file), worker JSON config construction, capability-aware ready gate and finite owned-process cleanup; preserve CLI listing/version and bounded diagnostics. Use the actual Frida API, not a production test stub. Any unimplemented requested capability is an error. -- [ ] Run focused tests plus complete Python device suite; self-review. Do not claim fixture-based adapter tests are device verification. -- [ ] Commit only Task1 files and report exact RED/GREEN and remaining limitations. +- [x] Write executable JS and Python failing tests for CA-scoped pin decisions, original rejection propagation, original trust delegation, explicit/default/unsupported profiles, missing classes and transactional rollback, hostile config serialization, readiness validation and owned-PID cleanup. +- [x] Observe RED with expected missing implementation or wrong existing behavior; record exact commands/output in report. +- [x] Implement original engine (small focused helpers in one runtime file), worker JSON config construction, capability-aware ready gate and finite owned-process cleanup; preserve CLI listing/version and bounded diagnostics. Use the actual Frida API, not a production test stub. Any unimplemented requested capability is an error. +- [x] Run focused tests plus complete Python device suite; self-review. Do not claim fixture-based adapter tests are device verification. +- [x] Commit only Task1 files and report exact RED/GREEN and remaining limitations. ### Task 2: Application integration, migration and release verification **Files:** Modify `crates/scope-device/src/lib.rs`, `src/App.tsx`, `README.md`, `docs/ANDROID.md`, `THIRD_PARTY_NOTICES.md`, `.github/workflows/checks.yml`, affected existing tests; remove six imported HTK runtime JS/config files and `resources/device/install-system-ca.template.sh`; retain Frida bridge/licenses. Add `docs/ANDROID_ENGINE.md` with explicit coverage/evidence. -**Interfaces:** Consume Task1 `scope-engine.js`, `normalize_profiles` and JSONL engine/capability payload. Rust rejects unsupported profiles before ADB/root probes; ASSETS materializes only `frida/scope-engine.js`, existing Frida bridge and worker. Remove automatic system CA installation, preserve cleanup of old recovery ca_hash. UI removes unsupported root option and explains Stop ends the instrumented process. +**Interfaces:** Consume Task1 `scope-engine.js`, `normalize_profiles` and JSONL engine/capability payload. Rust persists worker-spawned target PID/pending-cleanup before readiness where observable and requires explicit successful termination acknowledgment (or conservative proof that the recorded process no longer exists) before clearing recovery; worker timeout/nonzero/missing acknowledgment must remain an error. New recovery fields are backward-compatible serde defaults. Startup failure allows bounded worker teardown before kill. Rust rejects unsupported profiles before ADB/root probes; ASSETS materializes only `frida/scope-engine.js`, existing Frida bridge and worker. Remove automatic system CA installation, preserve cleanup of old recovery ca_hash. UI removes unsupported root option and explains Stop ends the instrumented process. - [ ] Add regression tests for early unsupported profile rejection, asset construction and legacy owned-CA recovery behavior where affected; observe RED. - [ ] Wire original engine and remove imported runtime source/injection path. Preserve unrelated Rust proxy and recovery semantics. diff --git a/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md b/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md index 8504754..b3fee4e 100644 --- a/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md +++ b/docs/superpowers/specs/2026-11-07-owned-android-engine-design.md @@ -18,7 +18,7 @@ Supported profiles are `proxy`, `certificate`, `unpinning`; an empty selection m The Python worker validates finite profile, PEM and port inputs before device work. Source construction uses JSON serialization for runtime data, not source-text replacement. The engine reports per-capability installation results. Readiness requires successful installation of every requested profile, not merely entry into Java.perform. Unsupported classes/overloads or hook errors produce actionable bounded diagnostics and no success claim. Installation failure rolls back installed method implementations. -Process-created proxy/trust objects can outlive method unhooking. Therefore Stop terminates the exact process spawned by this worker, then unloads/detaches; it does not promise that unloading alone restores the running app. It must not kill an unrelated process. Cleanup errors remain visible, and no clean-stop receipt is emitted after failed cleanup. Existing Rust recovery of owned ADB/server state and legacy CA removal remains available after restart. +Process-created proxy/trust objects can outlive method unhooking. Therefore Stop terminates the exact process spawned by this worker, then unloads/detaches; it does not promise that unloading alone restores the running app. It must not kill an unrelated process. Cleanup errors remain visible, and no clean-stop receipt is emitted after failed cleanup. The worker emits a spawned-PID event before attach and an explicit termination acknowledgment only after cleanup. Rust preserves pending target-process cleanup in recovery records; it must not convert a killed/failed worker into a successful Stop. A saved owned PID can be checked for absence before clearing an interrupted session; ambiguous state is an actionable error, never a reason to kill an arbitrary or reused PID. Existing Rust recovery of owned ADB/server state and legacy CA removal remains available after restart. ## Compatibility changes @@ -34,4 +34,4 @@ Real Android/Frida/OkHttp TLS, app-stop restoration and API/ABI compatibility ar - https://frida.re/docs/javascript-api/#java - https://developer.android.com/privacy-and-security/security-config -- https://square.github.io/okhttp/features/https/ +- https://github.com/square/okhttp From b2967e4c95f50de10686eabf3beec3ae8a1b1351 Mon Sep 17 00:00:00 2001 From: Mateusz Dalke <150788324+dalkedev@users.noreply.github.com> Date: Sat, 7 Nov 2026 08:29:13 +0000 Subject: [PATCH 05/18] Integrate Scope-owned Android Java engine and conservative recovery --- .github/workflows/checks.yml | 1 + README.md | 6 +- THIRD_PARTY_NOTICES.md | 40 +- crates/scope-device/src/diagnostics.rs | 2 + crates/scope-device/src/lib.rs | 929 ++++++++++++++---- docs/ANDROID.md | 35 +- docs/ANDROID_ENGINE.md | 36 + docs/ARCHITECTURE.md | 8 +- docs/VERIFICATION.md | 4 + .../device/install-system-ca.template.sh | 93 -- .../android-certificate-unpinning-fallback.js | 267 ----- .../android/android-certificate-unpinning.js | 539 ---------- .../android/android-disable-root-detection.js | 400 -------- .../frida/android/android-proxy-override.js | 98 -- .../android-system-certificate-injection.js | 88 -- resources/frida/config-template.js | 255 ----- src/App.breakpoints.test.tsx | 20 + src/App.tsx | 24 +- 18 files changed, 856 insertions(+), 1989 deletions(-) create mode 100644 docs/ANDROID_ENGINE.md delete mode 100644 resources/device/install-system-ca.template.sh delete mode 100644 resources/frida/android/android-certificate-unpinning-fallback.js delete mode 100644 resources/frida/android/android-certificate-unpinning.js delete mode 100644 resources/frida/android/android-disable-root-detection.js delete mode 100644 resources/frida/android/android-proxy-override.js delete mode 100644 resources/frida/android/android-system-certificate-injection.js delete mode 100644 resources/frida/config-template.js diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 41ff35d..06203f4 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -49,6 +49,7 @@ jobs: - run: cargo fmt --all -- --check - run: cargo test --workspace --locked - run: npm test -- --run + - run: node --test resources/frida/tests/scope-engine.test.cjs - run: node --test scripts/release.test.mjs - run: python -m unittest discover -s resources/device/tests - run: npm audit --audit-level=high diff --git a/README.md b/README.md index 7375133..d94fb12 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ npm run desktop Discover connected devices and installed apps over ADB. Use the **device-wide proxy** path, or instrument a **selected process with Frida** and explicit hook profiles. Read-only diagnostics explain authorization, root, ABI, runtime/server versions, mappings and recovery state before you connect. -Frida is an optional, separate runtime installation. Pinning and native networking support depend on the target; selecting an app in proxy mode does not prove per-app attribution. **Stop** restores recorded Scope-owned state and preserves external changes. [Setup, supported paths and recovery →](docs/ANDROID.md) · [Diagnostic reference →](docs/DIAGNOSTICS.md) +Frida is an optional, separate runtime installation. The experimental Scope-owned engine supports selected Java URL, SSLContext and OkHttp3 seams; native TLS and custom pinners are unsupported. **Stop ends the instrumented app process** before restoring recorded Scope-owned state and preserving external changes. System proxy mode requires manually configured app trust for HTTPS and never installs a system CA. [Setup, supported paths and recovery →](docs/ANDROID.md) · [Diagnostic reference →](docs/DIAGNOSTICS.md) ## An agent can read the evidence. You set the scope. @@ -124,7 +124,7 @@ The same engine powers the native inspector and MCP. Real HTTP/TLS fixtures test | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | macOS arm64 | Local UI/package evidence is recorded by version; 1.3.0 interactive validation is recorded separately by scenario. Ad-hoc signed; not Apple-notarized. | | Windows x64 / Linux x64 | CI builds, packages and headless MCP/capture smoke pass. Interactive UI and device validation remain open. Intel macOS is unverified. | -| Android / Frida | Earlier smoke covered one rooted Android 16 arm64 emulator. The 1.0 diagnostics changes use fixtures; arbitrary apps and TLS stacks are unverified. | +| Android / Frida | Earlier smoke covered the previous hook backend on one emulator. The current Scope-owned engine has local simulated tests only; real Android/Frida/OkHttp TLS and Stop acceptance are NOT RUN. | | Capture | Bodies are buffered up to **16 MiB each**. Streaming, WebSocket frames, trailers and HTTP3 are not implemented. | | Archives & analysis | Records currently load into memory. Large-session streaming/export is planned. Observed schemas are not complete API contracts. | | Dependencies | A Linux glib unsoundness advisory remains tracked in [#11](https://github.com/dalkedev/scope/issues/11). See the full [dependency record](docs/DEPENDENCIES.md). | @@ -145,6 +145,6 @@ Every change goes through maintainer review and required platform checks. [Owner Scope logo and brand assets

-An independent project by **[dalkedev](https://github.com/dalkedev)**. Scope includes attributed HTTP Toolkit interception scripts and Frida integration; their original licenses and notices remain intact. HTTP Toolkit does not need to be installed. +An independent project by **[dalkedev](https://github.com/dalkedev)**. Scope uses its original Java interception engine with Frida and an unchanged, attributed Frida Java bridge. The bridge distribution provenance and third-party licenses remain in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). **AGPL-3.0-or-later** · [License](LICENSE) · [Third-party notices](THIRD_PARTY_NOTICES.md) · [Brand assets](docs/BRAND.md) diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 8874a2c..251b190 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -1,5 +1,3 @@ -Scope includes scripts from the user's SculptTechProject/bypass-scripts-adb repository. The notices below preserve that distribution's provenance. Scope's Rust device manager and Python worker are new integration code; they do not depend on an installed HTTP Toolkit instance. The original JavaScript files and CA template are vendored unchanged. Scope substitutes its own public CA and chosen proxy port only in runtime copies. The runtime CA template also uses a Scope-specific temporary directory and certificate filename. - # Third-party notices The project is distributed under **AGPL-3.0-or-later**; see [LICENSE](LICENSE). @@ -33,24 +31,9 @@ hashes. Both directories must be retained in corresponding source archives. GLib's original license and authorship remain applicable; linked native GNOME libraries have their own license terms described in GLib's `COPYRIGHT`. -## HTTP Toolkit interception scripts - -Files: `resources/frida/config-template.js` and `resources/frida/android/*.js`. - -- Authors: Tim Perry and HTTP Toolkit contributors; `android-disable-root-detection.js` also credits Riyad Mondal. -- License: **AGPL-3.0-or-later**. The original notices remain in each source file; the full license is included in [LICENSE](LICENSE). -- Upstream project: [frida-interception-and-unpinning](https://github.com/httptoolkit/frida-interception-and-unpinning). -- Exact distribution source: [HTTP Toolkit Server v1.26.1, overrides/frida](https://github.com/httptoolkit/httptoolkit-server/tree/1abef0adc96d07bd64377e85e847025a55bac2ac/overrides/frida), commit `1abef0adc96d07bd64377e85e847025a55bac2ac`. -- Local changes: `config.js` was renamed to `config-template.js`. The JavaScript content is otherwise unchanged from that distribution. Runtime certificate and proxy substitutions are written to the ignored `.runtime/` directory, rather than these source files. - -## HTTP Toolkit system CA installation template +## Original Scope engine and historical migration -File: `resources/device/install-system-ca.template.sh`. - -- Authors: Tim Perry and HTTP Toolkit contributors. -- License: **AGPL-3.0-or-later**; see [LICENSE](LICENSE). -- Source: the `injectSystemCertificate` shell template in [HTTP Toolkit Server v1.26.1, adb-commands.ts](https://github.com/httptoolkit/httptoolkit-server/blob/1abef0adc96d07bd64377e85e847025a55bac2ac/src/interceptors/android/adb-commands.ts). -- Modified on **2026-09-30**: extracted the embedded shell script into a standalone template, added provenance and license comments, and retained `${certificatePath}` and `${injectionScriptPath}` as substitutions supplied by the launcher. Any later changes should keep this notice up to date. +`resources/frida/scope-engine.js` and the Python worker are original Scope integration code under the project AGPL-3.0-or-later license. The current runtime no longer includes the six HTTP Toolkit hook/config files or its extracted system-CA installation template. The project previously distributed them from the owner's `SculptTechProject/bypass-scripts-adb` repository, originating in HTTP Toolkit Server v1.26.1 (`1abef0adc96d07bd64377e85e847025a55bac2ac`). Their historical copyright, provenance and AGPL notices remain in public Git history. No history rewrite or project license change is part of this migration. This is not a legally certified clean-room claim. ## Frida Java bridge @@ -83,21 +66,9 @@ The helper license texts were copied verbatim from the published `@frida/buffer@ The GNU Library GPL text was obtained from [GNU](https://www.gnu.org/licenses/old-licenses/lgpl-2.0.txt). The wxWindows license and exception were obtained from [Frida Gum 17.9.1 COPYING](https://github.com/frida/frida-gum/blob/17.9.1/COPYING), preserving that license section verbatim. -The extracted CA installation template has SHA-256 `201fa259470c9fa4e1b70dd3a020e7807f762bf838346ae2f8297f62586eb67b` after the modifications listed above. - -## Vendored JavaScript integrity - -These SHA-256 values identify the unmodified JavaScript copied from HTTP Toolkit Server v1.26.1. The renamed configuration template has the same bytes as upstream `config.js`. +## Retained JavaScript integrity -| File | SHA-256 | -| --- | --- | -| `resources/frida/android/android-certificate-unpinning-fallback.js` | `5190e449ae590e896f362a5751267199c2bef5417f69635715705def00462cd7` | -| `resources/frida/android/android-certificate-unpinning.js` | `21cfb61cc492c557fc55e2b504a2e5e8f679b3515bff824acb12b14975381d9b` | -| `resources/frida/android/android-disable-root-detection.js` | `25e91287ab38111a188eb5786fa47586a252c711f0e401b9dc6a7c727600fda8` | -| `resources/frida/android/android-proxy-override.js` | `aed9941c65bd9d23789f9e54e11d37cd95ab83c5818f0c39a36e4581b05ad688` | -| `resources/frida/android/android-system-certificate-injection.js` | `f751f783f30fc9b296bf7f05692f27faabb31a37dafd46c31cd756bab98f8641` | -| `resources/frida/config-template.js` | `5afad65ed908006eb1af99aa6da3a73359036bdaa870fe32bf654695fa95d02a` | -| `resources/frida/frida-java-bridge.js` | `88f0ca1fb34d2bd7b589f3d4c44aa0077780473530922887a53311ec55d9c214` | +The unchanged retained bridge distributed by HTTP Toolkit Server v1.26.1 has SHA-256 `88f0ca1fb34d2bd7b589f3d4c44aa0077780473530922887a53311ec55d9c214`. The exact bridge distribution source and corresponding source are listed above; retention does not make it original Scope code. ## Optional packaged Frida runtime @@ -126,8 +97,7 @@ The main Frida licenses incorporate GNU Library GPL 2 or later, with an explicit exception permitting binary object code distribution under the distributor's own terms. The [GNU Library GPL text](resources/device/licenses/LGPL-2.0.txt) is included alongside those exceptions. Scope does not modify Frida, CPython or -PyInstaller. The JavaScript interception scripts remain subject to the separate -AGPL notices above. +PyInstaller. The original Scope engine remains under the project AGPL license. The older Python 3.9 developer environment optionally installs `typing_extensions` 4.16.0; its [license](resources/device/licenses/typing_extensions-4.16.0-LICENSE.txt) diff --git a/crates/scope-device/src/diagnostics.rs b/crates/scope-device/src/diagnostics.rs index 46b122e..7cc5854 100644 --- a/crates/scope-device/src/diagnostics.rs +++ b/crates/scope-device/src/diagnostics.rs @@ -1075,6 +1075,8 @@ mod tests { server_pid: Some(481), ca_hash: None, root_shell: true, + target_spawn_pending: false, + target_pid: None, } } fn elf(path: &Path, machine: u16, class: u8, endian: u8) { diff --git a/crates/scope-device/src/lib.rs b/crates/scope-device/src/lib.rs index a0d9c29..88b901d 100644 --- a/crates/scope-device/src/lib.rs +++ b/crates/scope-device/src/lib.rs @@ -1,5 +1,4 @@ use anyhow::{bail, Context, Result}; -use md5::{Digest, Md5}; use serde::{Deserialize, Serialize}; use std::{ collections::HashMap, @@ -75,11 +74,15 @@ struct Recovery { server_pid: Option, ca_hash: Option, root_shell: bool, + #[serde(default)] + target_spawn_pending: bool, + #[serde(default)] + target_pid: Option, } struct Active { recovery: Recovery, - worker: Option, + worker: Option, } struct Inner { runtime: PathBuf, @@ -88,6 +91,7 @@ struct Inner { operation: AsyncMutex<()>, active: AsyncMutex>, worker_generation: AtomicU64, + recovery_io: Mutex<()>, } #[derive(Clone)] pub struct DeviceManager { @@ -118,6 +122,7 @@ impl DeviceManager { operation: AsyncMutex::new(()), active: AsyncMutex::new(None), worker_generation: AtomicU64::new(0), + recovery_io: Mutex::new(()), }), } } @@ -261,12 +266,7 @@ impl DeviceManager { if !["proxy", "frida"].contains(&mode.as_str()) { bail!("Mode must be proxy or frida"); } - for value in &profile { - if !["proxy", "certificate", "unpinning", "root", "system-ca"].contains(&value.as_str()) - { - bail!("Unknown instrumentation profile: {value}"); - } - } + let profile = normalize_profiles(profile)?; if proxy_port == 0 { bail!("Proxy port must be greater than zero"); } @@ -350,6 +350,8 @@ impl DeviceManager { server_pid: None, ca_hash: None, root_shell, + target_spawn_pending: false, + target_pid: None, }; *self.inner.status.lock().unwrap() = DeviceStatus { running: false, @@ -367,33 +369,18 @@ impl DeviceManager { self.log(format!("ADB reverse: device 127.0.0.1:{proxy_port} → Scope localhost:{proxy_port}")); let worker = if mode == "proxy" { self.log("Proxy mode is DEVICE-WIDE. The selected package is launched, but Android may route other applications through this proxy."); - if rooted { - let hash = certificate_hash(&ca_cert)?; - let existing = self.root_command(&state, &format!("cat /apex/com.android.conscrypt/cacerts/{hash}.0 2>/dev/null || cat /system/etc/security/cacerts/{hash}.0 2>/dev/null || true")).await?; - if !existing.trim().is_empty() && existing.trim() != fs::read_to_string(&ca_cert)?.trim() { - bail!("An existing Android CA uses the same subject hash. Scope will not overwrite it"); - } - if existing.trim().is_empty() { - // Record the exact certificate before attempting injection so partial setup is recoverable. - state.ca_hash = Some(hash); - self.save_recovery(&state)?; - self.install_ca(&state, &ca_cert).await?; - } else { - self.log("Scope CA was already present; it will be preserved on Stop"); - } - } else { - self.log("No root: HTTP capture is available. HTTPS requires the app to trust Scope's CA manually; certificate pinning remains active."); - } + self.log("HTTPS requires manually configured application trust. System proxy mode does not install a CA or bypass pinning."); self.command(&state.serial, &["shell", "settings", "put", "global", "http_proxy", &format!("127.0.0.1:{proxy_port}")]).await?; self.launch(&state).await?; - if state.ca_hash.is_some() { self.verify_ca(&state, &ca_cert).await?; } (None, None) } else { self.log("Frida mode instruments the selected application process only; Java HTTP stacks are supported. Native TLS, child processes and QUIC are not guaranteed."); let (abi, version) = frida_preflight.as_ref().context("Missing Frida preflight")?; let address = self.prepare_frida_server(&mut state, abi, version).await?; self.save_recovery(&state)?; - let (worker, live) = self.spawn_worker(&state, &ca_cert, &profile, address, generation).await?; + let worker = self.spawn_worker(&mut state, &ca_cert, &profile, address, generation).await?; + let live = worker.live.clone(); + state = worker.observation.lock().unwrap().recovery.clone(); (Some(worker), Some(live)) }; Ok::<_, anyhow::Error>(worker) @@ -419,7 +406,11 @@ impl DeviceManager { } Err(error) => { self.log(format!("Setup failed: {error:#}")); - let cleanup = self.cleanup(&state).await; + let cleanup = if state.target_spawn_pending { + Err(anyhow::anyhow!("Target process cleanup is unconfirmed; recovery retained. Press Stop to reconcile the recorded PID, or manually reconcile an unknown spawn")) + } else { + self.cleanup(&state).await + }; let mut status = self.inner.status.lock().unwrap(); status.running = false; status.message = match cleanup { @@ -477,64 +468,6 @@ impl DeviceManager { Ok(()) } - async fn install_ca(&self, state: &Recovery, cert: &std::path::Path) -> Result<()> { - let hash = state.ca_hash.as_ref().context("Missing CA hash")?; - let remote_cert = format!("/data/local/tmp/scope-{hash}.0"); - let remote_script = "/data/local/tmp/scope-install-ca.sh"; - let script = include_str!("../../../resources/device/install-system-ca.template.sh") - .replace("htk-ca-copy", "scope-ca-copy") - .replace("${certificatePath}", &remote_cert) - .replace("${injectionScriptPath}", remote_script); - // The template moves the input basename into cacerts: it must be .0. - let script = script.replace( - &format!("mv {remote_cert} /system/etc/security/cacerts/"), - &format!("cp {remote_cert} /system/etc/security/cacerts/{hash}.0"), - ); - let path = self.inner.runtime.join("install-system-ca.sh"); - fs::write(&path, script)?; - self.command( - &state.serial, - &["push", &cert.to_string_lossy(), &remote_cert], - ) - .await?; - self.command( - &state.serial, - &["push", &path.to_string_lossy(), remote_script], - ) - .await?; - let result = self - .root_command(state, &format!("sh {remote_script}")) - .await?; - if !result.contains("System cert successfully injected") { - bail!("System CA injection was not confirmed"); - } - self.log(format!( - "Temporary Scope CA {hash}.0 injected into Android trust stores; removed on Stop" - )); - Ok(()) - } - - async fn verify_ca(&self, state: &Recovery, cert: &PathBuf) -> Result<()> { - let pid = self - .command(&state.serial, &["shell", "pidof", &state.package]) - .await?; - let pid = pid - .split_whitespace() - .next() - .context("Launched app PID was not found")?; - if !pid.bytes().all(|c| c.is_ascii_digit()) { - bail!("Invalid application PID"); - } - let hash = state.ca_hash.as_ref().context("Missing CA hash")?; - let command = format!("nsenter --mount=/proc/{pid}/ns/mnt -- cat /apex/com.android.conscrypt/cacerts/{hash}.0 || nsenter --mount=/proc/{pid}/ns/mnt -- cat /system/etc/security/cacerts/{hash}.0"); - let installed = self.root_command(state, &command).await?; - if installed.trim() != fs::read_to_string(cert)?.trim() { - bail!("Scope CA was not visible in the selected app's mount namespace"); - } - self.log("Scope CA verified in the selected application's mount namespace"); - Ok(()) - } - async fn prepare_frida_server( &self, state: &mut Recovery, @@ -633,12 +566,12 @@ impl DeviceManager { async fn spawn_worker( &self, - state: &Recovery, + state: &mut Recovery, cert: &PathBuf, profile: &[String], address: Option, generation: u64, - ) -> Result<(Child, Arc)> { + ) -> Result { let mut command = worker_command(&self.inner.runtime); command .args([ @@ -655,7 +588,7 @@ impl DeviceManager { if let Some(address) = address { command.args(["--address", &address]); } - for profile in profile.iter().filter(|v| v.as_str() != "system-ca") { + for profile in profile { command.args(["--profile", profile]); } command @@ -665,55 +598,118 @@ impl DeviceManager { .kill_on_drop(true); #[cfg(windows)] command.creation_flags(0x08000000); - let mut child = command.spawn().context("Cannot start Frida worker; set SCOPE_PYTHON to a Python executable with frida installed")?; + // Persist uncertainty before a worker can attempt a remote spawn, including a host crash. + state.target_spawn_pending = true; + self.save_recovery(state)?; + let mut child = match command.spawn() { + Ok(child) => child, + Err(error) => { + state.target_spawn_pending = false; + self.save_recovery(state)?; + return Err(error).context("Cannot start Frida worker; set SCOPE_PYTHON to a Python executable with frida installed"); + } + }; let stdout = child.stdout.take().context("Missing Frida worker stdout")?; let stderr = child.stderr.take().context("Missing Frida worker stderr")?; let (sender, receiver) = oneshot::channel(); let live = Arc::new(AtomicBool::new(true)); - let worker_live = live.clone(); - tokio::spawn(monitor_worker_output( + let observation = Arc::new(Mutex::new(WorkerObservation::new( + state.clone(), + profile.to_vec(), + ))); + let monitor = tokio::spawn(monitor_worker_output( self.clone(), stdout, generation, - worker_live, + live.clone(), sender, + observation.clone(), )); let manager = self.clone(); tokio::spawn(async move { let mut lines = BufReader::new(stderr).lines(); while let Ok(Some(line)) = lines.next_line().await { - manager.log(line); + if manager.inner.worker_generation.load(Ordering::SeqCst) == generation { + manager.log(line); + } } }); + let mut worker = Worker { + child, + live, + observation, + monitor, + }; match tokio::time::timeout(Duration::from_secs(40), receiver).await { - Ok(Ok(Ok(()))) => Ok((child, live)), + Ok(Ok(Ok(()))) => Ok(worker), result => { - drop(child.stdin.take()); - let _ = child.kill().await; - match result { - Ok(Ok(Err(message))) => bail!(message), - _ => bail!("Frida worker did not confirm hook readiness within 40 seconds"), + let teardown = self.finish_worker(&mut worker).await; + *state = worker.observation.lock().unwrap().recovery.clone(); + self.invalidate_worker(); + if teardown.is_ok() { + state.target_spawn_pending = false; + self.save_recovery(state)?; } + let cause = match result { + Ok(Ok(Err(message))) => message, + _ => "Frida worker did not confirm hook readiness within 40 seconds".into(), + }; + bail!( + "{cause}; teardown: {}", + teardown + .err() + .map(|e| e.to_string()) + .unwrap_or_else(|| "confirmed".into()) + ); + } + } + } + + // Never holds operation/recovery locks while awaiting stdout. The monitor must drain the final receipt. + async fn finish_worker(&self, worker: &mut Worker) -> Result<()> { + if let Some(mut stdin) = worker.child.stdin.take() { + let _ = stdin.write_all(b"stop\n").await; + } + let exit = match tokio::time::timeout(Duration::from_secs(12), worker.child.wait()).await { + Ok(result) => result.context("Cannot reap Frida worker"), + Err(_) => { + let _ = worker.child.kill().await; + Err(anyhow::anyhow!("Frida worker exceeded 12-second cleanup grace; target termination is unconfirmed")) } + }; + let drained = tokio::time::timeout(Duration::from_secs(2), &mut worker.monitor).await; + if !matches!(drained, Ok(Ok(()))) { + worker.monitor.abort(); + bail!("Frida worker output did not drain; target termination is unconfirmed"); } + let exit = exit?; + if !exit.success() { + bail!("Frida worker failed ({exit}); recovery retained"); + } + let observed = worker.observation.lock().unwrap(); + observed.confirm_stopped()?; + Ok(()) + } + + fn invalidate_worker(&self) { + let _io = self.inner.recovery_io.lock().unwrap(); + self.inner.worker_generation.fetch_add(1, Ordering::SeqCst); } pub async fn stop(&self) -> Result { let _operation = self.inner.operation.lock().await; - // Ignore late stdout/EOF notifications from a worker being intentionally stopped. - self.inner.worker_generation.fetch_add(1, Ordering::SeqCst); let active = self.inner.active.lock().await.take(); let state = if let Some(mut active) = active { - if let Some(mut child) = active.worker.take() { - if let Some(mut stdin) = child.stdin.take() { - let _ = stdin.write_all(b"stop\n").await; - } - if tokio::time::timeout(Duration::from_secs(5), child.wait()) - .await - .is_err() - { - let _ = child.kill().await; + if let Some(mut worker) = active.worker.take() { + let result = self.finish_worker(&mut worker).await; + active.recovery = worker.observation.lock().unwrap().recovery.clone(); + self.invalidate_worker(); + if let Err(error) = result { + self.cleanup_failed(&active.recovery, &error); + return Err(error); } + active.recovery.target_spawn_pending = false; + self.save_recovery(&active.recovery)?; } Some(active.recovery) } else if self.recovery_path().is_file() { @@ -725,24 +721,70 @@ impl DeviceManager { }; if let Some(state) = state { if let Err(error) = self.cleanup(&state).await { - let mut status = self.inner.status.lock().unwrap(); - status.running = false; - status.message = format!( - "Cleanup incomplete. Reconnect {} and press Stop again: {error:#}", - state.serial - ); + self.cleanup_failed(&state, &error); return Err(error); } } let mut status = self.inner.status.lock().unwrap(); status.running = false; - status.message = "Device settings restored; saved captures remain available".into(); + status.message = "Device settings restored; instrumented process ended when applicable; saved captures remain available".into(); Ok(status.clone()) } + fn cleanup_failed(&self, state: &Recovery, error: &anyhow::Error) { + let mut status = self.inner.status.lock().unwrap(); + status.running = false; + status.message = format!( + "Cleanup incomplete. Reconnect {} and press Stop again: {error:#}", + state.serial + ); + } + + async fn reconcile_target(&self, state: &Recovery) -> Result<()> { + if !state.target_spawn_pending { + return Ok(()); + } + let pid = state.target_pid.context("Spawn was attempted without a recorded PID. Manually reconcile the app/process on the device; Scope cannot safely infer or kill it. Recovery retained")?; + if pid == 0 { + bail!("Invalid recorded target PID; recovery retained"); + } + // Require a readable root procfs view. A package pid lookup or worker exit cannot establish absence. + let probe = format!( + r#"if [ "$(id -u)" != 0 ] || [ ! -r /proc/1/stat ] || [ ! -r /proc/self/stat ]; then + echo UNKNOWN +elif [ -d /proc/{pid} ]; then + echo PRESENT +elif mounts=$(cat /proc/mounts 2>/dev/null) && entries=$(ls /proc 2>/dev/null); then + case "$mounts" in + *hidepid=*) echo UNKNOWN;; + *) found=0; init=0 + for entry in $entries; do + [ "$entry" = "{pid}" ] && found=1 + [ "$entry" = 1 ] && init=1 + done + if [ "$found" = 1 ]; then echo PRESENT + elif [ "$init" = 1 ] && [ ! -e /proc/{pid} ]; then echo ABSENT + else echo UNKNOWN; fi;; + esac +else echo UNKNOWN; fi"# + ); + let result = self + .root_command(state, &probe) + .await + .context("Cannot inspect recorded target process; recovery retained")?; + if result.trim() != "ABSENT" { + bail!("Recorded target PID {pid} is present or its state is inaccessible. Scope will not kill a saved or reused PID; manually reconcile it and retry Stop. Recovery retained"); + } + self.log(format!( + "Recorded target PID {pid} is absent in the observed root procfs view" + )); + Ok(()) + } + async fn cleanup(&self, state: &Recovery) -> Result<()> { // Validate every persisted field before the first cleanup command. diagnostics::validate_recovery(state)?; + self.reconcile_target(state).await?; let mut errors = vec![]; if let Some(old) = &state.previous_proxy { let current = self @@ -874,6 +916,7 @@ impl DeviceManager { if !errors.is_empty() { bail!("{}", errors.join("; ")); } + let _io = self.inner.recovery_io.lock().unwrap(); if self.recovery_path().exists() { fs::remove_file(self.recovery_path())?; } @@ -908,6 +951,11 @@ impl DeviceManager { ) } fn save_recovery(&self, state: &Recovery) -> Result<()> { + let _io = self.inner.recovery_io.lock().unwrap(); + self.write_recovery(state) + } + + fn write_recovery(&self, state: &Recovery) -> Result<()> { fs::create_dir_all(&self.inner.runtime)?; let temporary = self.inner.runtime.join("device-recovery.tmp"); fs::write(&temporary, serde_json::to_vec_pretty(state)?)?; @@ -916,7 +964,7 @@ impl DeviceManager { } fn materialize(&self) -> Result<()> { - fs::create_dir_all(self.inner.runtime.join("frida/android"))?; + fs::create_dir_all(self.inner.runtime.join("frida"))?; for (path, content) in ASSETS { fs::write(self.inner.runtime.join(path), content)?; } @@ -924,57 +972,232 @@ impl DeviceManager { } } +const ENGINE: &str = "scope-java-v1"; + +fn normalize_profiles(profiles: Vec) -> Result> { + let profiles = if profiles.is_empty() { + vec!["proxy".into(), "certificate".into(), "unpinning".into()] + } else { + profiles + }; + let mut selected = Vec::new(); + for profile in profiles { + if !["proxy", "certificate", "unpinning"].contains(&profile.as_str()) { + bail!("Unsupported instrumentation profile: {profile}"); + } + if !selected.contains(&profile) { + selected.push(profile); + } + } + if selected.iter().any(|p| p == "unpinning") && !selected.iter().any(|p| p == "certificate") { + bail!("unpinning profile requires certificate"); + } + // Match the worker/engine canonical order while retaining exactly the explicit selection. + Ok(["proxy", "certificate", "unpinning"] + .into_iter() + .filter(|profile| selected.iter().any(|selected| selected == profile)) + .map(String::from) + .collect()) +} + +#[derive(Deserialize)] +#[serde(tag = "kind", rename_all = "camelCase")] +enum WorkerEvent { + Spawned { + engine: String, + pid: u32, + }, + Ready { + engine: String, + pid: u32, + profiles: Vec, + capabilities: HashMap>, + }, + Stopped { + engine: String, + pid: Option, + #[serde(rename = "spawnAttempted")] + spawn_attempted: bool, + #[serde(rename = "cleanupComplete")] + cleanup_complete: bool, + #[serde(rename = "processTerminated")] + process_terminated: bool, + }, + Error { + message: String, + }, + Detached { + message: String, + }, + Log { + message: serde_json::Value, + }, +} + +struct Worker { + child: Child, + live: Arc, + observation: Arc>, + monitor: tokio::task::JoinHandle<()>, +} +struct WorkerObservation { + recovery: Recovery, + profiles: Vec, + spawned: bool, + terminal: bool, + failure: Option, + stopped: bool, +} +impl WorkerObservation { + fn new(recovery: Recovery, profiles: Vec) -> Self { + Self { + recovery, + profiles, + spawned: false, + terminal: false, + failure: None, + stopped: false, + } + } + fn confirm_stopped(&self) -> Result<()> { + if let Some(error) = &self.failure { + bail!("{error}; recovery retained"); + } + if !self.stopped { + bail!("Missing matching target-termination acknowledgment; recovery retained"); + } + Ok(()) + } + fn ready( + &self, + engine: &str, + pid: u32, + profiles: &[String], + capabilities: &HashMap>, + ) -> bool { + !self.terminal + && self.spawned + && engine == ENGINE + && self.recovery.target_pid == Some(pid) + && profiles == self.profiles + && capabilities.len() == profiles.len() + && profiles.iter().all(|profile| { + let required = match profile.as_str() { + "proxy" => "java-url", + "certificate" => "sslcontext-scope-ca", + "unpinning" => "okhttp3-certificate-pinner", + _ => return false, + }; + capabilities + .get(profile) + .is_some_and(|installed| installed.iter().any(|item| item == required)) + }) + } +} + async fn monitor_worker_output( manager: DeviceManager, stdout: R, generation: u64, live: Arc, sender: oneshot::Sender>, + observation: Arc>, ) { let mut sender = Some(sender); let mut lines = BufReader::new(stdout).lines(); while let Ok(Some(line)) = lines.next_line().await { - if let Ok(value) = serde_json::from_str::(&line) { - match value.get("kind").and_then(|v| v.as_str()).unwrap_or("log") { - "ready" => { - if let Some(sender) = sender.take() { - let _ = sender.send(Ok(())); - } - } - "error" => { - let message = value - .get("message") - .and_then(|v| v.as_str()) - .unwrap_or("Frida worker failed") - .to_string(); - if let Some(sender) = sender.take() { - let _ = sender.send(Err(message.clone())); + let _io = manager.inner.recovery_io.lock().unwrap(); + if manager.inner.worker_generation.load(Ordering::SeqCst) != generation { + continue; + } + let mut observed = observation.lock().unwrap(); + let event = serde_json::from_str::(&line); + let mut error = None; + match event { + Ok(WorkerEvent::Spawned { engine, pid }) + if engine == ENGINE && pid > 0 && !observed.spawned && !observed.terminal => + { + observed.recovery.target_pid = Some(pid); + observed.spawned = true; + if manager.recovery_path().is_file() { + if let Err(failure) = manager.write_recovery(&observed.recovery) { + error = Some(format!("Cannot persist spawned target PID: {failure}")); } - manager.log(message); - live.store(false, Ordering::SeqCst); - manager.worker_ended(generation, "Frida worker reported an error; press Stop to restore the recorded device state"); + } else { + error = + Some("Recovery record disappeared before target PID persistence".into()); } - "stopped" | "detached" => { + } + Ok(WorkerEvent::Ready { + engine, + pid, + profiles, + capabilities, + }) => { + if observed.ready(&engine, pid, &profiles, &capabilities) { if let Some(sender) = sender.take() { - let detail = value - .get("message") - .and_then(|value| value.as_str()) - .unwrap_or("device session ended"); - let _ = sender.send(Err(format!("Frida detached before hook readiness: {detail}. Check Android compatibility and the selected app's process"))); + let _ = sender.send(Ok(())); } - live.store(false, Ordering::SeqCst); - manager.worker_ended( - generation, - "Frida worker detached; press Stop to restore the recorded device state", + } else { + error = Some( + "Invalid engine readiness/capability receipt or terminal session".into(), ); } - _ => manager.log( - value - .get("message") - .map(|v| v.as_str().unwrap_or("Frida diagnostic")) - .unwrap_or("Frida diagnostic"), - ), } + Ok(WorkerEvent::Stopped { + engine, + pid, + spawn_attempted, + cleanup_complete, + process_terminated, + }) => { + let valid = engine == ENGINE + && cleanup_complete + && if spawn_attempted { + observed.spawned + && pid == observed.recovery.target_pid + && process_terminated + } else { + !observed.spawned && pid.is_none() && !process_terminated + }; + if valid { + observed.stopped = true; + } else { + error = Some("Invalid target-termination acknowledgment".into()); + } + observed.terminal = true; + if let Some(sender) = sender.take() { + let _ = sender.send(Err("Frida stopped before hook readiness".into())); + } + } + Ok(WorkerEvent::Error { message }) => error = Some(message), + Ok(WorkerEvent::Detached { message }) => { + observed.terminal = true; + if let Some(sender) = sender.take() { + let _ = sender.send(Err(format!( + "Frida detached before hook readiness: {message}" + ))); + } + } + Ok(WorkerEvent::Log { message }) => { + manager.log(message.as_str().unwrap_or("Frida diagnostic")) + } + _ => error = Some("Invalid Frida worker protocol receipt".into()), + } + if let Some(message) = error { + observed.terminal = true; + observed.failure = Some(message.clone()); + if let Some(sender) = sender.take() { + let _ = sender.send(Err(message.clone())); + } + manager.log(message); + } + if observed.terminal { + live.store(false, Ordering::SeqCst); + manager.worker_ended( + generation, + "Frida worker ended; press Stop to reconcile the recorded device state", + ); } } if let Some(sender) = sender.take() { @@ -1093,55 +1316,375 @@ fn frida_arch(abi: &str) -> Option<&'static str> { } } -fn certificate_hash(path: &PathBuf) -> Result { - let pem = pem::parse(fs::read(path)?).context("Cannot parse Scope public CA")?; - let (_, cert) = x509_parser::parse_x509_certificate(pem.contents()) - .map_err(|e| anyhow::anyhow!("Cannot parse Scope CA: {e}"))?; - let hash = Md5::digest(cert.subject().as_raw()); - Ok(format!( - "{:08x}", - u32::from_le_bytes(hash[..4].try_into().unwrap()) - )) -} - const ASSETS: &[(&str, &str)] = &[ ( "frida_worker.py", include_str!("../../../resources/device/frida_worker.py"), ), ( - "frida/config-template.js", - include_str!("../../../resources/frida/config-template.js"), + "frida/scope-engine.js", + include_str!("../../../resources/frida/scope-engine.js"), ), ( "frida/frida-java-bridge.js", include_str!("../../../resources/frida/frida-java-bridge.js"), ), - ( - "frida/android/android-proxy-override.js", - include_str!("../../../resources/frida/android/android-proxy-override.js"), - ), - ( - "frida/android/android-system-certificate-injection.js", - include_str!("../../../resources/frida/android/android-system-certificate-injection.js"), - ), - ( - "frida/android/android-certificate-unpinning.js", - include_str!("../../../resources/frida/android/android-certificate-unpinning.js"), - ), - ( - "frida/android/android-certificate-unpinning-fallback.js", - include_str!("../../../resources/frida/android/android-certificate-unpinning-fallback.js"), - ), - ( - "frida/android/android-disable-root-detection.js", - include_str!("../../../resources/frida/android/android-disable-root-detection.js"), - ), ]; #[cfg(test)] mod tests { use super::*; + fn test_observation() -> Arc> { + Arc::new(Mutex::new(WorkerObservation::new( + Recovery { + serial: "fixture-01".into(), + package: "com.scope.demo".into(), + mode: "frida".into(), + port: 17890, + previous_proxy: None, + previous_reverse: None, + forward_port: None, + server_pid: None, + ca_hash: None, + root_shell: true, + target_spawn_pending: true, + target_pid: None, + }, + vec!["proxy".into()], + ))) + } + #[test] + fn old_recovery_defaults_do_not_invent_target_ownership() { + let state: Recovery = serde_json::from_str(r#"{"serial":"fixture-01","package":"com.scope.demo","mode":"proxy","port":17890,"previousProxy":null,"previousReverse":null,"forwardPort":null,"serverPid":null,"caHash":"1234abcd","rootShell":true}"#).unwrap(); + assert!(!state.target_spawn_pending); + assert_eq!(state.target_pid, None); + assert_eq!(state.ca_hash.as_deref(), Some("1234abcd")); + } + + #[test] + fn explicit_profiles_remain_exact_and_defaults_are_compatible() { + assert_eq!( + normalize_profiles(vec![]).unwrap(), + ["proxy", "certificate", "unpinning"] + ); + assert_eq!( + normalize_profiles(vec!["proxy".into(), "proxy".into()]).unwrap(), + ["proxy"] + ); + assert_eq!( + normalize_profiles(vec!["certificate".into()]).unwrap(), + ["certificate"] + ); + assert_eq!( + normalize_profiles(vec!["certificate".into(), "proxy".into()]).unwrap(), + ["proxy", "certificate"] + ); + } + + #[cfg(unix)] + #[tokio::test] + async fn worker_stop_drains_and_requires_matching_successful_receipts() { + // Run actual child pipes and the production monitor/teardown, including final-line races. + let owned = r#"{"kind":"stopped","engine":"scope-java-v1","pid":42,"spawnAttempted":true,"cleanupComplete":true,"processTerminated":true}"#; + let safe = r#"{"kind":"stopped","engine":"scope-java-v1","pid":null,"spawnAttempted":false,"cleanupComplete":true,"processTerminated":false}"#; + let spawned = r#"{"kind":"spawned","engine":"scope-java-v1","pid":42}"#; + let wrong = owned.replace("42", "43"); + let error = r#"{"kind":"error","message":"cleanup uncertain"}"#; + for (spawn, receipt, exit, success) in [ + (spawned, owned.to_owned(), 0, true), + ("", safe.to_owned(), 0, true), + (spawned, "".into(), 0, false), + (spawned, wrong, 0, false), + (spawned, safe.to_owned(), 0, false), + (spawned, owned.to_owned(), 1, false), + (spawned, format!("{error}\n{owned}"), 0, false), + ] { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + let observation = test_observation(); + manager + .save_recovery(&observation.lock().unwrap().recovery) + .unwrap(); + let before_stop = if spawn.is_empty() { + String::new() + } else { + format!("printf '%s\\n' {}; ", adb::quote(spawn)) + }; + let script = format!( + "{before_stop}read reply; printf '%s\\n' {}; exit {exit}", + adb::quote(&receipt) + ); + let mut child = Command::new("sh") + .arg("-c") + .arg(script) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .unwrap(); + let stdout = child.stdout.take().unwrap(); + let live = Arc::new(AtomicBool::new(true)); + let (sender, _receiver) = oneshot::channel(); + let monitor = tokio::spawn(monitor_worker_output( + manager.clone(), + stdout, + 0, + live.clone(), + sender, + observation.clone(), + )); + let mut worker = Worker { + child, + live, + observation, + monitor, + }; + let result = manager.finish_worker(&mut worker).await; + assert_eq!(result.is_ok(), success, "{receipt}: {result:?}"); + let saved: Recovery = + serde_json::from_slice(&fs::read(manager.recovery_path()).unwrap()).unwrap(); + assert!( + saved.target_spawn_pending, + "receipt alone must not clear recovery before exit/drain validation" + ); + if !spawn.is_empty() { + assert_eq!(saved.target_pid, Some(42)); + } + } + } + + #[cfg(unix)] + #[tokio::test] + async fn full_worker_cleanup_budget_is_allowed_before_killing() { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + let observation = test_observation(); + manager + .save_recovery(&observation.lock().unwrap().recovery) + .unwrap(); + let mut child = Command::new("sh").args(["-c", "read reply; sleep 9; printf '%s\\n' '{\"kind\":\"stopped\",\"engine\":\"scope-java-v1\",\"pid\":null,\"spawnAttempted\":false,\"cleanupComplete\":true,\"processTerminated\":false}'"]).stdin(Stdio::piped()).stdout(Stdio::piped()).spawn().unwrap(); + let stdout = child.stdout.take().unwrap(); + let live = Arc::new(AtomicBool::new(true)); + let (sender, _receiver) = oneshot::channel(); + let monitor = tokio::spawn(monitor_worker_output( + manager.clone(), + stdout, + 0, + live.clone(), + sender, + observation.clone(), + )); + let mut worker = Worker { + child, + live, + observation, + monitor, + }; + manager.finish_worker(&mut worker).await.unwrap(); + } + + #[cfg(unix)] + #[tokio::test] + async fn recovery_only_accepts_observed_known_pid_absence_and_preserves_legacy_ca_cleanup() { + use std::os::unix::fs::PermissionsExt; + for (pid, proc_state, succeeds) in [ + (Some(42), "ABSENT", true), + (Some(42), "PRESENT", false), + (Some(42), "UNKNOWN", false), + (None, "ABSENT", false), + ] { + let directory = tempfile::tempdir().unwrap(); + let log = directory.path().join("commands.log"); + let adb_path = directory.path().join("fake-adb"); + fs::write(&adb_path, format!("#!/bin/sh\nprintf '%s\\n' \"$*\" >> {}\ncase \"$*\" in\n*'/proc/42'*) echo {proc_state};;\n*) exit 0;;\nesac\n", adb::quote(&log.to_string_lossy()))).unwrap(); + fs::set_permissions(&adb_path, fs::Permissions::from_mode(0o755)).unwrap(); + let mut manager = DeviceManager::new(directory.path().into()); + Arc::get_mut(&mut manager.inner).unwrap().adb = adb_path; + let mut state = test_observation().lock().unwrap().recovery.clone(); + state.target_pid = pid; + state.ca_hash = Some("1234abcd".into()); + manager.save_recovery(&state).unwrap(); + assert_eq!(manager.stop().await.is_ok(), succeeds); + assert_eq!(manager.recovery_path().exists(), !succeeds); + let commands = fs::read_to_string(&log).unwrap_or_default(); + assert!(!commands.contains("kill 42")); + assert!( + !commands.contains("pidof com.scope.demo"), + "package PID lookup cannot prove target termination" + ); + if succeeds { + assert!(commands.contains("rm -f /system/etc/security/cacerts/1234abcd.0")); + } else { + assert!(!commands.contains("rm -f")); + } + } + } + + #[cfg(unix)] + #[tokio::test] + async fn inaccessible_pid_listed_in_procfs_is_not_treated_as_absent() { + use std::os::unix::fs::PermissionsExt; + let directory = tempfile::tempdir().unwrap(); + for (name, source) in [ + ("id", "#!/bin/sh\necho 0\n".to_owned()), + ("ls", "#!/bin/sh\nprintf '1\\n4294967295\\n'\n".to_owned()), + ( + "adb", + format!( + "#!/bin/sh\nPATH={}:$PATH\nexport PATH\nexec sh -c \"$4\"\n", + adb::quote(&directory.path().to_string_lossy()) + ), + ), + ] { + let path = directory.path().join(name); + fs::write(&path, source).unwrap(); + fs::set_permissions(path, fs::Permissions::from_mode(0o755)).unwrap(); + } + let mut manager = DeviceManager::new(directory.path().into()); + Arc::get_mut(&mut manager.inner).unwrap().adb = directory.path().join("adb"); + let mut state = test_observation().lock().unwrap().recovery.clone(); + state.target_pid = Some(u32::MAX); + assert!(manager.reconcile_target(&state).await.is_err(), "a visible directory entry is evidence of presence even if stat cannot observe the target"); + } + + #[tokio::test] + async fn terminal_error_cannot_be_revived_by_ready_or_late_spawn() { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + let observation = test_observation(); + manager + .save_recovery(&observation.lock().unwrap().recovery) + .unwrap(); + let (mut writer, reader) = tokio::io::duplex(4096); + let (sender, receiver) = oneshot::channel(); + let live = Arc::new(AtomicBool::new(true)); + let task = tokio::spawn(monitor_worker_output( + manager.clone(), + reader, + 0, + live.clone(), + sender, + observation.clone(), + )); + writer.write_all(b"{\"kind\":\"error\",\"message\":\"failed\"}\n{\"kind\":\"spawned\",\"engine\":\"scope-java-v1\",\"pid\":42}\n{\"kind\":\"ready\",\"engine\":\"scope-java-v1\",\"pid\":42,\"profiles\":[\"proxy\"],\"capabilities\":{\"proxy\":[\"java-url\"]}}\n").await.unwrap(); + drop(writer); + assert!(receiver.await.unwrap().is_err()); + task.await.unwrap(); + assert!(!live.load(Ordering::SeqCst)); + assert!(!observation.lock().unwrap().spawned); + let saved: Recovery = + serde_json::from_slice(&fs::read(manager.recovery_path()).unwrap()).unwrap(); + assert!(saved.target_spawn_pending); + assert_eq!(saved.target_pid, None); + } + + #[tokio::test] + async fn unsupported_profiles_fail_before_proxy_or_device_probes() { + for profiles in [ + vec!["root"], + vec!["system-ca"], + vec!["unknown"], + vec!["unpinning"], + ] { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + let error = manager + .start( + "fixture-01".into(), + "com.scope.demo".into(), + "frida".into(), + 1, + directory.path().join("missing-ca"), + profiles.into_iter().map(String::from).collect(), + ) + .await + .unwrap_err() + .to_string(); + assert!( + error.contains("profile") || error.contains("requires certificate"), + "{error}" + ); + assert!(!manager.recovery_path().exists()); + } + } + + #[test] + fn materialized_worker_can_construct_only_the_owned_engine() { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + manager.materialize().unwrap(); + assert!(directory.path().join("frida/scope-engine.js").is_file()); + assert!(directory + .path() + .join("frida/frida-java-bridge.js") + .is_file()); + assert!(!directory.path().join("frida/config-template.js").exists()); + assert!(!directory.path().join("frida/android").exists()); + } + + #[tokio::test] + async fn invalid_readiness_never_confirms_startup() { + for receipt in [ + r#"{"kind":"ready"}"#, + r#"{"kind":"ready","engine":"other","pid":42,"profiles":["proxy"],"capabilities":{"proxy":["java-url"]}}"#, + r#"{"kind":"ready","engine":"scope-java-v1","pid":43,"profiles":["proxy"],"capabilities":{"proxy":["java-url"]}}"#, + r#"{"kind":"ready","engine":"scope-java-v1","pid":42,"profiles":["certificate"],"capabilities":{"certificate":["sslcontext-scope-ca"]}}"#, + r#"{"kind":"ready","engine":"scope-java-v1","pid":42,"profiles":["proxy"],"capabilities":{"proxy":[]}}"#, + ] { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + let observation = test_observation(); + manager + .save_recovery(&observation.lock().unwrap().recovery) + .unwrap(); + let (mut writer, reader) = tokio::io::duplex(4096); + let (sender, receiver) = oneshot::channel(); + let task = tokio::spawn(monitor_worker_output( + manager, + reader, + 0, + Arc::new(AtomicBool::new(true)), + sender, + observation, + )); + writer.write_all(format!("{{\"kind\":\"spawned\",\"engine\":\"scope-java-v1\",\"pid\":42}}\n{receipt}\n").as_bytes()).await.unwrap(); + drop(writer); + assert!(receiver.await.unwrap().is_err(), "{receipt}"); + task.await.unwrap(); + } + } + + #[tokio::test] + async fn late_generation_cannot_replace_or_recreate_recovery() { + for saved in [true, false] { + let directory = tempfile::tempdir().unwrap(); + let manager = DeviceManager::new(directory.path().into()); + manager.inner.worker_generation.store(2, Ordering::SeqCst); + if saved { + manager + .save_recovery(&test_observation().lock().unwrap().recovery) + .unwrap(); + } + let before = fs::read(manager.recovery_path()).ok(); + let (mut writer, reader) = tokio::io::duplex(4096); + let (sender, _receiver) = oneshot::channel(); + let task = tokio::spawn(monitor_worker_output( + manager.clone(), + reader, + 1, + Arc::new(AtomicBool::new(true)), + sender, + test_observation(), + )); + writer + .write_all(b"{\"kind\":\"spawned\",\"engine\":\"scope-java-v1\",\"pid\":42}\n") + .await + .unwrap(); + drop(writer); + task.await.unwrap(); + assert_eq!(fs::read(manager.recovery_path()).ok(), before); + } + } + #[cfg(unix)] #[tokio::test] async fn cleanup_checks_live_mapping_ownership_before_mutations() { @@ -1175,6 +1718,8 @@ mod tests { server_pid: None, ca_hash: None, root_shell: false, + target_spawn_pending: false, + target_pid: None, }) .unwrap(); manager.stop().await.unwrap(); @@ -1226,8 +1771,12 @@ mod tests { 1, live.clone(), sender, + test_observation(), )); - writer.write_all(b"{\"kind\":\"ready\"}\n").await.unwrap(); + manager + .save_recovery(&test_observation().lock().unwrap().recovery) + .unwrap(); + writer.write_all(b"{\"kind\":\"spawned\",\"engine\":\"scope-java-v1\",\"pid\":42}\n{\"kind\":\"ready\",\"engine\":\"scope-java-v1\",\"pid\":42,\"profiles\":[\"proxy\"],\"capabilities\":{\"proxy\":[\"java-url\"]}}\n").await.unwrap(); assert!(receiver.await.unwrap().is_ok()); drop(writer); task.await.unwrap(); @@ -1249,6 +1798,7 @@ mod tests { 1, Arc::new(AtomicBool::new(true)), sender, + test_observation(), )); drop(writer); task.await.unwrap(); @@ -1266,6 +1816,7 @@ mod tests { 0, Arc::new(AtomicBool::new(true)), sender, + test_observation(), )); writer .write_all(b"{\"kind\":\"detached\",\"message\":\"process-terminated\"}\n") @@ -1310,6 +1861,8 @@ mod tests { server_pid: None, ca_hash: None, root_shell: false, + target_spawn_pending: false, + target_pid: None, }; let mut manager = DeviceManager::new(directory.path().to_owned()); Arc::get_mut(&mut manager.inner).unwrap().adb = directory.path().join("missing-adb"); @@ -1335,6 +1888,8 @@ mod tests { server_pid: None, ca_hash: None, root_shell: false, + target_spawn_pending: false, + target_pid: None, }; manager.save_recovery(&state).unwrap(); let error = manager.stop().await.unwrap_err().to_string(); diff --git a/docs/ANDROID.md b/docs/ANDROID.md index 3d9acc7..099f3db 100644 --- a/docs/ANDROID.md +++ b/docs/ANDROID.md @@ -1,14 +1,14 @@ # Android interception -Scope's Android adapter uses its own localhost proxy and public CA. HTTP Toolkit does not need to be installed or running. +Scope's Android adapter uses its own localhost proxy, public CA and experimental Java interception engine on Frida. The retained Java bridge has separate third-party provenance. [Engine coverage and evidence](ANDROID_ENGINE.md). ## What is implemented - Real ADB device enumeration, including offline/unauthorized states, model and a read-only root check. - Installed packages and running state. If an existing Frida server is available, app display labels come from Frida; otherwise the exact package ID is shown as the name. -- **System proxy:** Android's global HTTP proxy, a matching ADB reverse mapping and, on rooted devices, temporary system CA injection. The selected application is restarted. This mode is **device-wide**, so traffic from other apps may appear. -- **Frida:** a separate Python worker spawns and attaches to the selected application. Bundled JavaScript profiles configure the app proxy, inject Scope's CA, disable supported certificate pinning and optionally hide selected root indicators. -- Stop restores the previous Android proxy and reverse mapping, detaches hooks, removes Scope's temporary CA and stops only a server process spawned by Scope. +- **System proxy:** Android's global HTTP proxy and a matching ADB reverse mapping. The selected application is restarted. This mode is **device-wide**, so traffic from other apps may appear. HTTPS requires manually configured application trust; Scope does not install a system CA. +- **Frida:** a separate Python worker spawns and attaches to the selected application. Original Scope Java hooks configure supported proxy seams, add trust for chains validated against Scope's CA, and conditionally bypass supported OkHttp3 pins for those chains. Hostname verification remains active. +- **Frida Stop ends the exact instrumented app process**, unloads/detaches and restores recorded owned mappings/server state. System proxy Stop restores recorded proxy/reverse state. Older recovery records retain cleanup of their Scope-owned CA. - A saved recovery file allows Stop to retry after a disconnected device or interrupted session. Starting another session is refused until that cleanup succeeds. - The 1.0 **Diagnostics** workspace and `device_diagnostics` MCP tool provide read-only authorization/root/ABI/runtime/mapping/recovery preflight with actionable fixes. See [diagnostic boundaries and troubleshooting](DIAGNOSTICS.md). @@ -104,14 +104,15 @@ The locally tested optional worker used CPython 3.12.13 from python-build-standa ### Hook profiles -| ID | Behavior | -| ------------- | ----------------------------------------------------------------------------- | -| `proxy` | Redirect supported Java HTTP proxy configuration to Scope. | -| `certificate` | Inject Scope's public CA into supported Java trust managers. | -| `unpinning` | Disable supported Java certificate pinning checks, including a fallback hook. | -| `root` | Hide selected root indicators from the instrumented app. | +| ID | Supported behavior | +| --- | --- | +| `proxy` | Subsequent Java URL HTTP(S) connections and available OkHttp3 builders use the Scope listener. | +| `certificate` | Subsequent SSLContext initialization wraps original X509 trust managers; separate Java PKIX validation admits only valid chains to the supplied Scope CA. | +| `unpinning` | Enumerated OkHttp3 CertificatePinner List, Certificate[] and Kotlin Function0 overloads may accept a Scope-validated chain after an original pin rejection. Requires `certificate`. | -An empty profile array selects `proxy`, `certificate`, `unpinning`. Explicit profiles select exactly those hooks. Root hiding is opt-in. The UI and worker report hook startup diagnostics; readiness means the Java callbacks ran, not that every possible network stack is covered. +An omitted/empty API or CLI profile array selects `proxy`, `certificate`, `unpinning` for compatibility. Explicit selections remain exact. The UI disables Start for an empty selection; deselecting certificate also deselects/disables unpinning. `root`, `system-ca` and unknown profiles fail before device probes. No root hiding is provided. + +Readiness requires the exact engine, owned PID, requested profiles and installed capability receipts. An unavailable supported OkHttp pinner causes default-all startup to fail; choose explicit proxy/certificate profiles for the supported available seams. A callback alone cannot establish readiness or network coverage. ### Compatibility boundaries @@ -119,19 +120,19 @@ The source repository recorded Android 16 ART/GC crashes with Frida 17.9.1. This The bundled profiles target **Java** networking. Native TLS implementations, QUIC/HTTP3, child processes, anti-instrumentation and app-specific pinning may need additional work. Do not infer complete capture from an empty request list. -The version pin is the imported repository's integration baseline, not a claim that it supports every current Android release. Python/server upgrades should be tested together with the Java bridge bundle. +The Frida version pin is the existing integration baseline, not a claim that it supports every current Android release. Python/server upgrades should be tested together with the Java bridge bundle. ## Cleanup and recovery -Scope writes `device-recovery.json` under its runtime directory before changing the device. It contains the selected serial/package, previous proxy/reverse state, temporary CA hash and owned forwarding/server identifiers. +Scope writes `device-recovery.json` under its runtime directory before changing the device. It contains the selected serial/package, previous proxy/reverse state, legacy temporary CA hash, owned forwarding/server identifiers and pending spawned-process cleanup/PID. The spawned PID is persisted before readiness; an attempted spawn without a returned PID remains explicitly uncertain. Press Stop while the device is connected. If cleanup fails, the file remains and the log explains that reconnection is needed. Reconnect the **same device**, then press Stop again. Do not delete the recovery file as a substitute for restoring device settings. Stop verifies that current proxy and ADB mappings still match Scope's recorded values before restoring them. External changes are preserved and logged. Invalid saved fields are rejected before cleanup commands; failed inspection leaves recovery available for a retry. -Rooted CA injection uses temporary mount overlays, following the upstream Android 14+ namespace handling. Stop removes Scope's certificate from those shared stores and the selected app. The remaining mount overlays disappear on reboot. The adapter does not unmount shared trust stores underneath other apps. +New sessions never inject system CAs. For older recovery records only, Stop still removes the recorded Scope-owned certificate from shared stores and the selected app. Existing temporary mount overlays disappear on reboot; Scope does not unmount shared trust stores underneath other apps. -Frida detaches hooks on Stop and leaves the selected app running. Restart that app to return its full runtime state to a clean baseline. A server binary downloaded/copied for Scope remains available for the next session; its process is stopped. +Frida Stop terminates the owned app process because created client/TLS objects can outlive method unhooking. The worker has 12 seconds for bounded cleanup, followed by stdout drain. A successful worker exit alone cannot prove termination: Rust requires a matching successful acknowledgment. Failed/timed-out workers and missing acknowledgments retain recovery. After a restart, only observed absence of the recorded PID in a readable root procfs view allows automatic reconciliation. A present or inaccessible PID remains an error and is never killed from saved state. Unknown attempted-spawn completion requires manual process/device reconciliation; Scope cannot safely discover it by package name. Preserve the recovery file until that reconciliation and owned setting/mapping restoration are complete. A server binary downloaded/copied for Scope remains for the next session; only its owned process is stopped. ## Local Android demo @@ -166,10 +167,12 @@ Set `SCOPE_DEVICE_RUNTIME` to an absolute directory to keep the smoke-test recov ## Source and licenses -The JavaScript, bridge and original CA template are preserved with their original copyright notices. See [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md) and the included license files. Runtime config substitutes Scope's own CA and selected proxy port; an installed HTTP Toolkit service or certificate is never read. +The runtime uses original `scope-engine.js` plus the unchanged Frida Java bridge. Six imported HTTP Toolkit hook/config files and the CA installation template were removed; source history and retained license/provenance notices remain. Runtime data is JSON-serialized. This migration changes neither the project AGPL license nor the existing Rust proxy/CA/upstream verification. See [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md). ## Recorded local smoke test +**Historical backend receipt:** the following 2026-11-02 results used the removed hook backend and CA-injection path. They do not validate the current Scope-owned engine or its process-ending Stop contract. Current real Android/Frida/OkHttp TLS and Stop acceptance: **NOT RUN**. + On 2026-11-02, the macOS build was checked against an existing rooted Android 16 arm64 emulator using only the included `com.scope.demo` application: - The APK built offline, signed, installed and launched successfully. diff --git a/docs/ANDROID_ENGINE.md b/docs/ANDROID_ENGINE.md new file mode 100644 index 0000000..183a234 --- /dev/null +++ b/docs/ANDROID_ENGINE.md @@ -0,0 +1,36 @@ +# Scope-owned Android Java engine + +The experimental `scope-java-v1` engine uses documented Frida/Java APIs and the unchanged third-party Frida Java bridge. It replaces the imported HTTP Toolkit runtime hooks. The project remains AGPL-3.0-or-later; [notices and bridge provenance](../THIRD_PARTY_NOTICES.md) remain applicable. No legally certified clean-room claim is made. + +## Coverage + +| Profile | Supported seam | Limits | +| --- | --- | --- | +| `proxy` | Java URL openConnection overloads and available OkHttp3 Builder.build | Existing clients, other Java clients and native connections are not covered. | +| `certificate` | Subsequent SSLContext.init, original basic/extended X509 managers | Existing/default contexts are not covered. Original socket/SSLEngine delegation and hostname verification remain active. | +| `unpinning` | Enumerated OkHttp3 CertificatePinner List, Certificate[] and Kotlin Function0 signatures | Requires certificate profile; missing/unknown pinner capabilities fail startup. Custom/obfuscated/native pinners are unsupported. | + +A separate empty KeyStore contains only the supplied Scope CA, and a real Java TrustManagerFactory performs chain validation. CA constraints, certificate validity, server-leaf checks and original Java exception types bound acceptance. A valid Scope chain can override an original certificate/pin rejection only at the supported seams; unrelated failures and non-Scope chain decisions remain unchanged. No trust-all manager, hostname bypass or expiry bypass is installed. Native TLS, Cronet, QUIC, Flutter, child processes and root hiding are unsupported. + +System proxy mode remains device-wide and requires manually configured application trust for HTTPS. Neither mode injects a system CA. Legacy recovery records retain removal of their recorded owned CA. + +## Lifecycle + +The worker serializes runtime data as JSON, validates profiles before device contact and requires every requested capability before ready. Rust persists attempted-spawn uncertainty before launching the worker and the exact spawned PID before accepting ready. Terminal errors/detach/EOF cannot revive readiness; a prior worker generation cannot write a newer session's state. + +Stop ends the exact process spawned by the active worker, then unloads/detaches. This is necessary because created proxy/TLS objects can outlive unhooking. Rust allows 12 seconds for the worker's bounded nine-second cleanup plus scheduling overhead, then drains stdout. A matching successful cleanup/termination receipt and successful worker exit are required before clearing pending cleanup. A receipt proving Stop occurred before any attempted spawn is also accepted without claiming termination. + +A failed/timed-out worker or missing acknowledgment retains recovery. After restart, only observed absence of a saved PID in a readable root procfs view can establish that it ended. Present/inaccessible PID state and unknown attempted-spawn completion require manual reconciliation; saved target PIDs are never killed. Existing mapping ownership checks preserve external proxy/reverse/forward changes. + +## Evidence and acceptance + +| Gate | Current evidence | +| --- | --- | +| Java policy and simulated Frida adapter | Executable Node suite covers original validation/pinning controls, capability rollback and supported API wiring. | +| Worker lifecycle and source construction | Executable Python suite covers profiles, JSON config, readiness, cancellation and owned-process cleanup. | +| Rust application lifecycle | Local child pipes and fixture ADB cover typed receipts, output drain, recovery, profile preflight and legacy cleanup. | +| UI | Selection/dependency regressions and production frontend build are local checks. | +| Real Android/Frida/OkHttp TLS and hostname controls | **NOT RUN** | +| Actual app termination/restoration and API/ABI compatibility | **NOT RUN** | + +Simulated tests do not prove Android execution, cryptography or compatibility. Historical emulator observations used the previous backend. The source candidate must remain draft until real-device acceptance is separately recorded. No native UI/device evidence is inferred from a headless build. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8444f4a..ac85801 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -27,8 +27,8 @@ flowchart LR | `crates/scope-mcp` | stdio MCP protocol and twelve tool schemas; calls the desktop's local bridge | | `src-tauri` | Desktop lifecycle, native dialogs, command dispatch, MCP access policy and local demo server | | `src` | React inspector, sessions, device setup, analysis, comparison and MCP settings | -| `resources/device` | Frida worker, setup helpers and certificate injection template | -| `resources/frida` | Attributed upstream JavaScript hook profiles and Java bridge | +| `resources/device` | Frida worker, runtime setup helpers and retained third-party licenses | +| `resources/frida` | Original Scope Java engine and attributed unchanged Frida Java bridge | | `examples/android-demo` | Small Java/Android localhost test target | | `scripts/smoke.mjs` | Real-process MCP/capture/persistence/redaction smoke test | @@ -60,6 +60,8 @@ is a local operational journal, not a tamper-proof compliance audit system. System proxy mode is device-wide. Frida mode spawns/attaches to the selected package through an external worker and uses supported Java hooks. The worker runs outside the Tauri process to isolate instrumentation failures. It uses a standalone executable when bundled, or a Python fallback. Host and device Frida versions must match. +Frida Stop terminates the exact process spawned by the worker, because created proxy/TLS objects can survive unhooking. Typed capability readiness follows durable spawned-PID recording. A successful exit and matching cleanup receipt are required; worker failure or missing acknowledgment retains target recovery. After restart, only observed absence of a recorded PID in a readable root procfs view can reconcile it automatically; unknown spawn completion requires manual reconciliation. Saved target PIDs are never killed. System proxy mode does not install certificates; legacy owned-CA removal remains supported. + A failed cleanup retains its recovery record and blocks a conflicting new start. Reconnect the same device and retry Stop. Cleanup removes Scope-owned resources and restores recorded settings; it does not stop unrelated Frida servers. [Android details](ANDROID.md) ## MCP boundary @@ -90,6 +92,6 @@ The application settings expose the actual data directory. `--data-dir` or `SCOP ## Verification and design choices -Tests exercise actual local HTTP and CA-signed HTTPS, rather than only proxy parsers. The release smoke launches the actual desktop engine and MCP executable, tests access/redaction, and reopens the same SQLite archive. Native UI and one rooted Android emulator were checked separately. [Recorded results and limits](VERIFICATION.md) +Tests exercise actual local HTTP and CA-signed HTTPS, rather than only proxy parsers. The release smoke launches the actual desktop engine and MCP executable, tests access/redaction, and reopens the same SQLite archive. Native UI and one rooted Android emulator were checked separately for earlier versions. These historical receipts do not validate the current Scope-owned engine; its real-device acceptance remains NOT RUN. [Recorded results and limits](VERIFICATION.md) Scope 1.0.0 does not capture WebSocket frames, HTTP trailers, QUIC/HTTP3 or arbitrary TCP/UDP. Windows/Linux runtime parity and physical Android support need evidence from those environments. New networking-stack support should include a minimal reproducible client/server fixture and a cleanup test. diff --git a/docs/VERIFICATION.md b/docs/VERIFICATION.md index 64d74cf..938c2bb 100644 --- a/docs/VERIFICATION.md +++ b/docs/VERIFICATION.md @@ -1,5 +1,9 @@ # Recorded verification +## Scope-owned Android engine boundary — 2026-11-07 + +The current source replaces the imported hook backend with Scope-owned Java hooks and ends the owned app process on Frida Stop. Local executable Node/Python policy/adapter and Rust lifecycle tests use simulated Java/Frida or local child processes; they do not execute Android TLS. Real Android/Frida/OkHttp TLS, hostname controls, API/ABI compatibility and Stop restoration acceptance are **NOT RUN**. Earlier emulator receipts below describe the previous backend and remain historical. See [current engine coverage](ANDROID_ENGINE.md). + ## Endpoint manual editing candidate — 2026-11-06 Scope 1.3.0 adds a `pause` Traffic rule with request, response or both-stage diff --git a/resources/device/install-system-ca.template.sh b/resources/device/install-system-ca.template.sh deleted file mode 100644 index f3f017a..0000000 --- a/resources/device/install-system-ca.template.sh +++ /dev/null @@ -1,93 +0,0 @@ -# Derived from HTTP Toolkit Server 1.26.1 injectSystemCertificate. -# Source: https://github.com/httptoolkit/httptoolkit-server -# SPDX-License-Identifier: AGPL-3.0-or-later -# Copyright Tim Perry and HTTP Toolkit contributors. -# Modified 2026-09-30: extracted as a standalone template with certificate/script path placeholders. -set -e # Fail on error - -echo "\n---\nInjecting certificate:" - -# Create a separate temp directory, to hold the current certificates -# Without this, when we add the mount we can't read the current certs anymore. -mkdir -p /data/local/tmp/htk-ca-copy -chmod 700 /data/local/tmp/htk-ca-copy -rm -rf /data/local/tmp/htk-ca-copy/* - -# Copy out the existing certificates -if [ -d "/apex/com.android.conscrypt/cacerts" ]; then - cp /apex/com.android.conscrypt/cacerts/* /data/local/tmp/htk-ca-copy/ -else - cp /system/etc/security/cacerts/* /data/local/tmp/htk-ca-copy/ -fi - -# Create the in-memory mount on top of the system certs folder -mount -t tmpfs tmpfs /system/etc/security/cacerts - -# Copy the existing certs back into the tmpfs mount, so we keep trusting them -mv /data/local/tmp/htk-ca-copy/* /system/etc/security/cacerts/ - -# Copy our new cert in, so we trust that too -mv ${certificatePath} /system/etc/security/cacerts/ - -# Update the perms & selinux context labels, so everything is as readable as before -chown root:root /system/etc/security/cacerts/* -chmod 644 /system/etc/security/cacerts/* - -chcon u:object_r:system_file:s0 /system/etc/security/cacerts/ -chcon u:object_r:system_file:s0 /system/etc/security/cacerts/* - -echo 'System cacerts setup completed' - -# Deal with the APEX overrides in Android 14+, which need injecting into each namespace: -if [ -d "/apex/com.android.conscrypt/cacerts" ]; then - echo 'Injecting certificates into APEX cacerts' - - # When the APEX manages cacerts, we need to mount them at that path too. We can't do - # this globally as APEX mounts are namespaced per process, so we need to inject a - # bind mount for this directory into every mount namespace. - - # First we mount for the shell itself, for completeness and so we can see this - # when we check for correct installation on later runs - mount --bind /system/etc/security/cacerts /apex/com.android.conscrypt/cacerts - - # First we get the Zygote process(es), which launch each app - ZYGOTE_PID=$(pidof zygote || true) - ZYGOTE64_PID=$(pidof zygote64 || true) - Z_PIDS="$ZYGOTE_PID $ZYGOTE64_PID" - # N.b. some devices appear to have both, some have >1 of each (!) - - # Apps inherit the Zygote's mounts at startup, so we inject here to ensure all newly - # started apps will see these certs straight away: - for Z_PID in $Z_PIDS; do - if [ -n "$Z_PID" ]; then - nsenter --mount=/proc/$Z_PID/ns/mnt -- \ - /bin/mount --bind /system/etc/security/cacerts /apex/com.android.conscrypt/cacerts - fi - done - - echo 'Zygote APEX certificates remounted' - - # Then we inject the mount into all already running apps, so they see these certs immediately. - - # Get the PID of every process whose parent is one of the Zygotes: - APP_PIDS=$( - echo $Z_PIDS | \ - xargs -n1 ps -o 'PID' -P | \ - grep -v PID - ) - - # Inject into the mount namespace of each of those apps: - for PID in $APP_PIDS; do - nsenter --mount=/proc/$PID/ns/mnt -- \ - /bin/mount --bind /system/etc/security/cacerts /apex/com.android.conscrypt/cacerts & - done - wait # Launched in parallel - wait for completion here - - echo "APEX certificates remounted for $(echo $APP_PIDS | wc -w) apps" -fi - -# Delete the temp cert directory & this script itself -rm -r /data/local/tmp/htk-ca-copy -rm ${injectionScriptPath} - -echo "System cert successfully injected\n---\n" diff --git a/resources/frida/android/android-certificate-unpinning-fallback.js b/resources/frida/android/android-certificate-unpinning-fallback.js deleted file mode 100644 index 2a2641f..0000000 --- a/resources/frida/android/android-certificate-unpinning-fallback.js +++ /dev/null @@ -1,267 +0,0 @@ -/************************************************************************************************** - * - * Once we've set up the configuration and certificate, and then disabled all the pinning - * techniques we're aware of, we add one last touch: a fallback hook, designed to spot and handle - * unknown unknowns. - * - * This can also be useful for heavily obfuscated apps, where 3rd party libraries are obfuscated - * sufficiently that our hooks no longer recognize the methods we care about. - * - * To handle this, we watch for methods that throw known built-in TLS errors (these are *very* - * widely used, and always recognizable as they're defined natively), and then subsequently patch - * them for all future calls. Whenever a method throws this, we attempt to recognize it from - * signatures alone, and automatically hook it. - * - * These are very much a fallback! They might not work! They almost certainly won't work on the - * first request, so applications will see at least one failure. Even when they fail though, they - * will at least log the method that's failing, so this works well as a starting point for manual - * reverse engineering. If this does fail and cause problems, you may want to skip this script - * and use only the known-good patches provided elsewhere. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * - *************************************************************************************************/ - -// Capture the full fields or methods from a Frida class reference via JVM reflection: -const getFields = (cls) => getFridaValues(cls, cls.class.getDeclaredFields()); -const getMethods = (cls) => getFridaValues(cls, cls.class.getDeclaredMethods()); - -// Take a Frida class + JVM reflection result, and turn it into a clear list -// of names -> Frida values (field or method references) -const getFridaValues = (cls, values) => values.map((value) => - [value.getName(), cls[value.getName()]] -); - -Java.perform(function () { - try { - const X509TrustManager = Java.use("javax.net.ssl.X509TrustManager"); - const defaultTrustManager = getCustomX509TrustManager(); // Defined in the unpinning script - const certBytes = Java.use("java.lang.String").$new(CERT_PEM).getBytes(); - const trustedCACert = buildX509CertificateFromBytes(certBytes); // Ditto - - const isX509TrustManager = (cls, methodName) => - methodName === 'checkServerTrusted' && - X509TrustManager.class.isAssignableFrom(cls.class); - - // There are two standard methods that X509TM implementations might override. We confirm we're - // matching the methods we expect by double-checking against the argument types: - const BASE_METHOD_ARGUMENTS = [ - '[Ljava.security.cert.X509Certificate;', - 'java.lang.String' - ]; - const EXTENDED_METHOD_ARGUMENTS = [ - '[Ljava.security.cert.X509Certificate;', - 'java.lang.String', - 'java.lang.String' - ]; - - const isOkHttpCheckMethod = (errorMessage, method) => - errorMessage.startsWith("Certificate pinning failure!" + "\n Peer certificate chain:") && - method.argumentTypes.length === 2 && - method.argumentTypes[0].className === 'java.lang.String'; - - const isAppmattusOkHttpInterceptMethod = (errorMessage, method) => { - if (errorMessage !== 'Certificate transparency failed') return; - - // Takes a single OkHttp chain argument: - if (method.argumentTypes.length !== 1) return; - - // The method must take an Interceptor.Chain, for which we need to - // call chain.proceed(chain.request()) to return a Response type. - // To do that, we effectively pattern match our way through all the - // related types to work out what's what: - - const chainType = Java.use(method.argumentTypes[0].className); - const responseTypeName = method.returnType.className; - - const matchedChain = matchOkHttpChain(chainType, responseTypeName); - return !!matchedChain; - }; - - const isMetaPinningMethod = (errorMessage, method) => - method.argumentTypes.length === 1 && - method.argumentTypes[0].className === 'java.util.List' && - method.returnType.className === 'void' && - errorMessage.includes('pinning error'); - - const matchOkHttpChain = (cls, expectedReturnTypeName) => { - // Find the chain.proceed() method: - const methods = getMethods(cls); - const matchingMethods = methods.filter(([_, method]) => - method.returnType.className === expectedReturnTypeName - ); - if (matchingMethods.length !== 1) return; - - const [proceedMethodName, proceedMethod] = matchingMethods[0]; - if (proceedMethod.argumentTypes.length !== 1) return; - - const argumentTypeName = proceedMethod.argumentTypes[0].className; - - // Find the chain.request private field (.request() getter can be - // optimized out, so we read the field directly): - const fields = getFields(cls); - const matchingFields = fields.filter(([_, field]) => - field.fieldReturnType?.className === argumentTypeName - ); - if (matchingFields.length !== 1) return; - - const [requestFieldName] = matchingFields[0]; - - return { - proceedMethodName, - requestFieldName - }; - }; - - const buildUnhandledErrorPatcher = (errorClassName, originalConstructor) => { - return function (errorArg) { - try { - console.log('\n !!! --- Unexpected TLS failure --- !!!'); - - // This may be a message, or an cause, or plausibly maybe other types? But - // stringifying gives something consistently message-shaped, so that'll do. - const errorMessage = errorArg?.toString() ?? ''; - - // Parse the stack trace to work out who threw this error: - const stackTrace = Java.use('java.lang.Thread').currentThread().getStackTrace(); - const exceptionStackIndex = stackTrace.findIndex(stack => - stack.getClassName() === errorClassName - ); - const callingFunctionStack = stackTrace[exceptionStackIndex + 1]; - - const className = callingFunctionStack.getClassName(); - const methodName = callingFunctionStack.getMethodName(); - - const errorTypeName = errorClassName.split('.').slice(-1)[0]; - console.log(` ${errorTypeName}: ${errorMessage}`); - console.log(` Thrown by ${className}->${methodName}`); - - const callingClass = Java.use(className); - const callingMethod = callingClass[methodName]; - - callingMethod.overloads.forEach((failingMethod) => { - if (failingMethod.implementation) { - console.warn(' Already patched - but still failing!') - return; // Already patched by Frida - skip it - } - - // Try to spot known methods (despite obfuscation) and disable them: - if (isOkHttpCheckMethod(errorMessage, failingMethod)) { - // See okhttp3.CertificatePinner patches in unpinning script: - failingMethod.implementation = () => { - if (DEBUG_MODE) console.log(` => Fallback OkHttp patch`); - }; - console.log(` [+] ${className}->${methodName} (fallback OkHttp patch)`); - } else if (isAppmattusOkHttpInterceptMethod(errorMessage, failingMethod)) { - // See Appmattus CertificateTransparencyInterceptor patch in unpinning script: - const chainType = Java.use(failingMethod.argumentTypes[0].className); - const responseTypeName = failingMethod.returnType.className; - const okHttpChain = matchOkHttpChain(chainType, responseTypeName); - failingMethod.implementation = (chain) => { - if (DEBUG_MODE) console.log(` => Fallback Appmattus+OkHttp patch`); - const proceed = chain[okHttpChain.proceedMethodName].bind(chain); - const request = chain[okHttpChain.requestFieldName].value; - return proceed(request); - }; - console.log(` [+] ${className}->${methodName} (fallback Appmattus+OkHttp patch)`); - } else if (isX509TrustManager(callingClass, methodName)) { - const argumentTypes = failingMethod.argumentTypes.map(t => t.className); - const returnType = failingMethod.returnType.className; - - if ( - argumentTypes.length === 2 && - argumentTypes.every((t, i) => t === BASE_METHOD_ARGUMENTS[i]) && - returnType === 'void' - ) { - // For the base method, just check against the default: - failingMethod.implementation = (certs, authType) => { - if (DEBUG_MODE) console.log(` => Fallback X509TrustManager patch of ${ - className - } base method`); - - const defaultTrustManager = getCustomX509TrustManager(); // Defined in the unpinning script - defaultTrustManager.checkServerTrusted(certs, authType); - }; - console.log(` [+] ${className}->${methodName} (fallback X509TrustManager base patch)`); - } else if ( - argumentTypes.length === 3 && - argumentTypes.every((t, i) => t === EXTENDED_METHOD_ARGUMENTS[i]) && - returnType === 'java.util.List' - ) { - // For the extended method, we just ignore the hostname, and if the certs are good - // (i.e they're ours), then we say the whole chain is good to go: - failingMethod.implementation = function (certs, authType, _hostname) { - if (DEBUG_MODE) console.log(` => Fallback X509TrustManager patch of ${ - className - } extended method`); - - try { - defaultTrustManager.checkServerTrusted(certs, authType); - } catch (e) { - console.error('Default TM threw:', e); - } - return Java.use('java.util.Arrays').asList(certs); - }; - console.log(` [+] ${className}->${methodName} (fallback X509TrustManager ext patch)`); - } else { - console.warn(` [ ] Skipping unrecognized checkServerTrusted signature in class ${ - callingClass.class.getName() - }`); - } - } else if (isMetaPinningMethod(errorMessage, failingMethod)) { - failingMethod.implementation = function (certs) { - if (DEBUG_MODE) console.log(` => Fallback patch for meta proxygen pinning`); - for (const cert of certs.toArray()) { - if (cert.equals(trustedCACert)) { - return; // Our own cert - all good - } - } - - if (DEBUG_MODE) { - console.warn(' Meta unpinning fallback found only untrusted certificates'); - } - // Fall back to normal logic, in case of passthrough or similar - return failingMethod.call(this, certs); - } - - console.log(` [+] ${className}->${methodName} (Meta proxygen pinning fallback patch)`); - } else { - console.error(' [ ] Unrecognized TLS error - this must be patched manually'); - return; - // Later we could try to cover other cases here - automatically recognizing other - // OkHttp interceptors for example, or potentially other approaches, but we need - // to do so carefully to avoid disabling TLS checks entirely. - } - }); - } catch (e) { - console.log(' [ ] Failed to automatically patch failure'); - console.warn(e); - } - - return originalConstructor.call(this, ...arguments); - } - }; - - // These are the exceptions we watch for and attempt to auto-patch out after they're thrown: - [ - 'javax.net.ssl.SSLPeerUnverifiedException', - 'java.security.cert.CertificateException' - ].forEach((errorClassName) => { - const ErrorClass = Java.use(errorClassName); - ErrorClass.$init.overloads.forEach((overload) => { - overload.implementation = buildUnhandledErrorPatcher( - errorClassName, - overload - ); - }); - }) - - console.log('== Unpinning fallback auto-patcher installed =='); - } catch (err) { - console.error(err); - console.error(' !!! --- Unpinning fallback auto-patcher installation failed --- !!!'); - } - -}); \ No newline at end of file diff --git a/resources/frida/android/android-certificate-unpinning.js b/resources/frida/android/android-certificate-unpinning.js deleted file mode 100644 index 4df1c91..0000000 --- a/resources/frida/android/android-certificate-unpinning.js +++ /dev/null @@ -1,539 +0,0 @@ -/************************************************************************************************** - * - * This script defines a large set of targeted certificate unpinning hooks: matching specific - * methods in certain classes, and transforming their behaviour to ensure that restrictions to - * TLS trust are disabled. - * - * This does not disable TLS protections completely - each hook is designed to disable only - * *additional* restrictions, and to explicitly trust the certificate provided as CERT_PEM in the - * config.js configuration file, preserving normal TLS protections wherever possible, even while - * allowing for controlled MitM of local traffic. - * - * The file consists of a few general-purpose methods, then a data structure declaratively - * defining the classes & methods to match, and how to transform them, and then logic at the end - * which uses this data structure, applying the transformation for each found match to the - * target process. - * - * For more details on what was matched, and log output when each hooked method is actually used, - * enable DEBUG_MODE in config.js, and watch the Frida output after running this script. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * - *************************************************************************************************/ - -function buildX509CertificateFromBytes(certBytes) { - const ByteArrayInputStream = Java.use('java.io.ByteArrayInputStream'); - const CertFactory = Java.use('java.security.cert.CertificateFactory'); - const certFactory = CertFactory.getInstance("X.509"); - return certFactory.generateCertificate(ByteArrayInputStream.$new(certBytes)); -} - -function getCustomTrustManagerFactory() { - // This is the one X509Certificate that we want to trust. No need to trust others (we should capture - // _all_ TLS traffic) and risky to trust _everything_ (risks interception between device & proxy, or - // worse: some traffic being unintercepted & sent as HTTPS with TLS effectively disabled over the - // real web - potentially exposing auth keys, private data and all sorts). - const certBytes = Java.use("java.lang.String").$new(CERT_PEM).getBytes(); - const trustedCACert = buildX509CertificateFromBytes(certBytes); - - // Build a custom TrustManagerFactory with a KeyStore that trusts only this certificate: - - const KeyStore = Java.use("java.security.KeyStore"); - const keyStore = KeyStore.getInstance(KeyStore.getDefaultType()); - keyStore.load(null); - keyStore.setCertificateEntry("ca", trustedCACert); - - const TrustManagerFactory = Java.use("javax.net.ssl.TrustManagerFactory"); - const customTrustManagerFactory = TrustManagerFactory.getInstance( - TrustManagerFactory.getDefaultAlgorithm() - ); - customTrustManagerFactory.init(keyStore); - - return customTrustManagerFactory; -} - -function getCustomX509TrustManager() { - const customTrustManagerFactory = getCustomTrustManagerFactory(); - const trustManagers = customTrustManagerFactory.getTrustManagers(); - - const X509TrustManager = Java.use('javax.net.ssl.X509TrustManager'); - - const x509TrustManager = trustManagers.find((trustManager) => { - return trustManager.class.isAssignableFrom(X509TrustManager.class); - }); - - // We have to cast it explicitly before Frida will allow us to use the X509 methods: - return Java.cast(x509TrustManager, X509TrustManager); -} - -// Some standard hook replacements for various cases: -const NO_OP = () => {}; -const RETURN_TRUE = () => true; -const CHECK_OUR_TRUST_MANAGER_ONLY = () => { - const trustManager = getCustomX509TrustManager(); - return (certs, authType) => { - trustManager.checkServerTrusted(certs, authType); - }; -}; - -const PINNING_FIXES = { - // --- Native HttpsURLConnection - - 'javax.net.ssl.HttpsURLConnection': [ - { - methodName: 'setDefaultHostnameVerifier', - replacement: () => NO_OP - }, - { - methodName: 'setSSLSocketFactory', - replacement: () => NO_OP - }, - { - methodName: 'setHostnameVerifier', - replacement: () => NO_OP - }, - ], - - // --- Native SSLContext - - 'javax.net.ssl.SSLContext': [ - { - methodName: 'init', - overload: ['[Ljavax.net.ssl.KeyManager;', '[Ljavax.net.ssl.TrustManager;', 'java.security.SecureRandom'], - replacement: (targetMethod) => { - const customTrustManagerFactory = getCustomTrustManagerFactory(); - - // When constructor is called, replace the trust managers argument: - return function (keyManager, _providedTrustManagers, secureRandom) { - return targetMethod.call(this, - keyManager, - customTrustManagerFactory.getTrustManagers(), // Override their trust managers - secureRandom - ); - } - } - } - ], - - // --- Native Conscrypt CertPinManager - - 'com.android.org.conscrypt.CertPinManager': [ - { - methodName: 'isChainValid', - replacement: () => RETURN_TRUE - }, - { - methodName: 'checkChainPinning', - replacement: () => NO_OP - } - ], - - // --- Native pinning configuration loading (used for configuration by many libraries) - - 'android.security.net.config.NetworkSecurityConfig': [ - { - methodName: '$init', - overload: '*', - replacement: (targetMethod) => { - const PinSet = Java.use('android.security.net.config.PinSet'); - const EMPTY_PINSET = PinSet.EMPTY_PINSET.value; - return function () { - // Always ignore the 2nd 'pins' PinSet argument entirely: - arguments[2] = EMPTY_PINSET; - targetMethod.call(this, ...arguments); - } - } - } - ], - - // --- Native HostnameVerification override (n.b. Android contains its own vendored OkHttp v2!) - - 'com.android.okhttp.internal.tls.OkHostnameVerifier': [ - { - methodName: 'verify', - overload: [ - 'java.lang.String', - 'javax.net.ssl.SSLSession' - ], - replacement: (targetMethod) => { - // Our trust manager - this trusts *only* our extra CA - const trustManager = getCustomX509TrustManager(); - - return function (hostname, sslSession) { - try { - const certs = sslSession.getPeerCertificates(); - - // https://stackoverflow.com/a/70469741/68051 - const authType = "RSA"; - - // This throws if the certificate isn't trusted (i.e. if it's - // not signed by our extra CA specifically): - trustManager.checkServerTrusted(certs, authType); - - // If the cert is from our CA, great! Skip hostname checks entirely. - return true; - } catch (e) {} // Ignore errors and fallback to default behaviour - - // We fallback to ensure that connections with other CAs (e.g. direct - // connections allowed past the proxy) validate as normal. - return targetMethod.call(this, ...arguments); - } - } - } - ], - - 'com.android.okhttp.Address': [ - { - methodName: '$init', - overload: [ - 'java.lang.String', - 'int', - 'com.android.okhttp.Dns', - 'javax.net.SocketFactory', - 'javax.net.ssl.SSLSocketFactory', - 'javax.net.ssl.HostnameVerifier', - 'com.android.okhttp.CertificatePinner', - 'com.android.okhttp.Authenticator', - 'java.net.Proxy', - 'java.util.List', - 'java.util.List', - 'java.net.ProxySelector' - ], - replacement: (targetMethod) => { - const defaultHostnameVerifier = Java.use("com.android.okhttp.internal.tls.OkHostnameVerifier") - .INSTANCE.value; - const defaultCertPinner = Java.use("com.android.okhttp.CertificatePinner") - .DEFAULT.value; - - return function () { - // Override arguments, to swap any custom check params (widely used - // to add stricter rules to TLS verification) with the defaults instead: - arguments[5] = defaultHostnameVerifier; - arguments[6] = defaultCertPinner; - - targetMethod.call(this, ...arguments); - } - } - }, - // Almost identical patch, but for Nougat and older. In these versions, the DNS argument - // isn't passed here, so the arguments to patch changes slightly: - { - methodName: '$init', - overload: [ - 'java.lang.String', - 'int', - // No DNS param - 'javax.net.SocketFactory', - 'javax.net.ssl.SSLSocketFactory', - 'javax.net.ssl.HostnameVerifier', - 'com.android.okhttp.CertificatePinner', - 'com.android.okhttp.Authenticator', - 'java.net.Proxy', - 'java.util.List', - 'java.util.List', - 'java.net.ProxySelector' - ], - replacement: (targetMethod) => { - const defaultHostnameVerifier = Java.use("com.android.okhttp.internal.tls.OkHostnameVerifier") - .INSTANCE.value; - const defaultCertPinner = Java.use("com.android.okhttp.CertificatePinner") - .DEFAULT.value; - - return function () { - // Override arguments, to swap any custom check params (widely used - // to add stricter rules to TLS verification) with the defaults instead: - arguments[4] = defaultHostnameVerifier; - arguments[5] = defaultCertPinner; - - targetMethod.call(this, ...arguments); - } - } - } - ], - - // --- OkHttp v3 - - 'okhttp3.CertificatePinner': [ - { - methodName: 'check', - overload: ['java.lang.String', 'java.util.List'], - replacement: () => NO_OP - }, - { - methodName: 'check', - overload: ['java.lang.String', 'java.security.cert.Certificate'], - replacement: () => NO_OP - }, - { - methodName: 'check', - overload: ['java.lang.String', '[Ljava.security.cert.Certificate;'], - replacement: () => NO_OP - }, - { - methodName: 'check$okhttp', - replacement: () => NO_OP - }, - ], - - // --- SquareUp OkHttp (< v3) - - 'com.squareup.okhttp.CertificatePinner': [ - { - methodName: 'check', - overload: ['java.lang.String', 'java.security.cert.Certificate'], - replacement: () => NO_OP - }, - { - methodName: 'check', - overload: ['java.lang.String', 'java.util.List'], - replacement: () => NO_OP - } - ], - - // --- Trustkit (https://github.com/datatheorem/TrustKit-Android/) - - 'com.datatheorem.android.trustkit.pinning.PinningTrustManager': [ - { - methodName: 'checkServerTrusted', - replacement: CHECK_OUR_TRUST_MANAGER_ONLY - } - ], - - // --- Appcelerator (https://github.com/tidev/appcelerator.https) - - 'appcelerator.https.PinningTrustManager': [ - { - methodName: 'checkServerTrusted', - replacement: CHECK_OUR_TRUST_MANAGER_ONLY - } - ], - - // --- PhoneGap sslCertificateChecker (https://github.com/EddyVerbruggen/SSLCertificateChecker-PhoneGap-Plugin) - - 'nl.xservices.plugins.sslCertificateChecker': [ - { - methodName: 'execute', - overload: ['java.lang.String', 'org.json.JSONArray', 'org.apache.cordova.CallbackContext'], - replacement: () => (_action, _args, context) => { - context.success("CONNECTION_SECURE"); - return true; - } - // This trusts _all_ certs, but that's fine - this is used for checks of independent test - // connections, rather than being a primary mechanism to secure the app's TLS connections. - } - ], - - // --- IBM WorkLight - - 'com.worklight.wlclient.api.WLClient': [ - { - methodName: 'pinTrustedCertificatePublicKey', - getMethod: (WLClientCls) => WLClientCls.getInstance().pinTrustedCertificatePublicKey, - overload: '*' - } - ], - - 'com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning': [ - { - methodName: 'verify', - overload: '*', - replacement: () => NO_OP - } - // This covers at least 4 commonly used WorkLight patches. Oddly, most sets of hooks seem - // to return true for 1/4 cases, which must be wrong (overloads must all have the same - // return type) but also it's very hard to find any modern (since 2017) references to this - // class anywhere including WorkLight docs, so it may no longer be relevant anyway. - ], - - 'com.worklight.androidgap.plugin.WLCertificatePinningPlugin': [ - { - methodName: 'execute', - overload: '*', - replacement: () => RETURN_TRUE - } - ], - - // --- CWAC-Netsecurity (unofficial back-port pinner for Android<4.2) CertPinManager - - 'com.commonsware.cwac.netsecurity.conscrypt.CertPinManager': [ - { - methodName: 'isChainValid', - overload: '*', - replacement: () => RETURN_TRUE - } - ], - - // --- Netty - - 'io.netty.handler.ssl.util.FingerprintTrustManagerFactory': [ - { - methodName: 'checkTrusted', - replacement: () => NO_OP - } - ], - - // --- Cordova / PhoneGap Advanced HTTP Plugin (https://github.com/silkimen/cordova-plugin-advanced-http) - - // Modern version: - 'com.silkimen.cordovahttp.CordovaServerTrust': [ - { - methodName: '$init', - replacement: (targetMethod) => function () { - // Ignore any attempts to set trust to 'pinned'. Default settings will trust - // our cert because of the separate system-certificate injection step. - if (arguments[0] === 'pinned') { - arguments[0] = 'default'; - } - - return targetMethod.call(this, ...arguments); - } - } - ], - - // --- Appmattus Cert Transparency (https://github.com/appmattus/certificatetransparency/) - - 'com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier': [ - { - methodName: 'verify', - replacement: () => RETURN_TRUE - // This is not called unless the cert passes basic trust checks, so it's safe to blindly accept. - } - ], - - 'com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor': [ - { - methodName: 'intercept', - replacement: () => (a) => a.proceed(a.request()) - // This is not called unless the cert passes basic trust checks, so it's safe to blindly accept. - } - ], - - 'com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager': [ - { - methodName: 'checkServerTrusted', - overload: ['[Ljava.security.cert.X509Certificate;', 'java.lang.String'], - replacement: CHECK_OUR_TRUST_MANAGER_ONLY, - methodName: 'checkServerTrusted', - overload: ['[Ljava.security.cert.X509Certificate;', 'java.lang.String', 'java.lang.String'], - replacement: () => { - const trustManager = getCustomX509TrustManager(); - return (certs, authType, _hostname) => { - // We ignore the hostname - if the certs are good (i.e they're ours), then the - // whole chain is good to go. - trustManager.checkServerTrusted(certs, authType); - return Java.use('java.util.Arrays').asList(certs); - }; - } - } - ] - -}; - -const getJavaClassIfExists = (clsName) => { - try { - return Java.use(clsName); - } catch { - return undefined; - } -} - -Java.perform(function () { - if (DEBUG_MODE) console.log('\n === Disabling all recognized unpinning libraries ==='); - - const classesToPatch = Object.keys(PINNING_FIXES); - - classesToPatch.forEach((targetClassName) => { - const TargetClass = getJavaClassIfExists(targetClassName); - if (!TargetClass) { - // We skip patches for any classes that don't seem to be present. This is common - // as not all libraries we handle are necessarily used. - if (DEBUG_MODE) console.log(`[ ] ${targetClassName} *`); - return; - } - - const patches = PINNING_FIXES[targetClassName]; - - let patchApplied = false; - - patches.forEach(({ methodName, getMethod, overload, replacement }) => { - const namedTargetMethod = getMethod - ? getMethod(TargetClass) - : TargetClass[methodName]; - - const methodDescription = `${methodName}${ - overload === '*' - ? '(*)' - : overload - ? '(' + overload.map((argType) => { - // Simplify arg names to just the class name for simpler logs: - const argClassName = argType.split('.').slice(-1)[0]; - if (argType.startsWith('[L')) return `${argClassName}[]`; - else return argClassName; - }).join(', ') + ')' - // No overload: - : '' - }` - - let targetMethodImplementations = []; - try { - if (namedTargetMethod) { - if (!overload) { - // No overload specified - targetMethodImplementations = [namedTargetMethod]; - } else if (overload === '*') { - // Targetting _all_ overloads - targetMethodImplementations = namedTargetMethod.overloads; - } else { - // Or targetting a specific overload: - targetMethodImplementations = [namedTargetMethod.overload(...overload)]; - } - } - } catch (e) { - // Overload not present - } - - - // We skip patches for any methods that don't seem to be present. This is rarer, but does - // happen due to methods that only appear in certain library versions or whose signatures - // have changed over time. - if (targetMethodImplementations.length === 0) { - if (DEBUG_MODE) console.log(`[ ] ${targetClassName} ${methodDescription}`); - return; - } - - targetMethodImplementations.forEach((targetMethod, i) => { - const patchName = `${targetClassName} ${methodDescription}${ - targetMethodImplementations.length > 1 ? ` (${i})` : '' - }`; - - try { - const newImplementation = replacement(targetMethod); - if (DEBUG_MODE) { - // Log each hooked method as it's called: - targetMethod.implementation = function () { - console.log(` => ${patchName}`); - return newImplementation.apply(this, arguments); - } - } else { - targetMethod.implementation = newImplementation; - } - - if (DEBUG_MODE) console.log(`[+] ${patchName}`); - patchApplied = true; - } catch (e) { - // In theory, errors like this should never happen - it means the patch is broken - // (e.g. some dynamic patch building fails completely) - console.error(`[!] ERROR: ${patchName} failed: ${e}`); - } - }) - }); - - if (!patchApplied) { - console.warn(`[!] Matched class ${targetClassName} but could not patch any methods`); - } - }); - - console.log('== Certificate unpinning completed =='); -}); \ No newline at end of file diff --git a/resources/frida/android/android-disable-root-detection.js b/resources/frida/android/android-disable-root-detection.js deleted file mode 100644 index aa5a204..0000000 --- a/resources/frida/android/android-disable-root-detection.js +++ /dev/null @@ -1,400 +0,0 @@ -/************************************************************************************************** - * - * This script defines a large set of root detection bypasses for Android. Hooks included here - * block detection of many known root indicators, including file paths, package names, commands, - * notably binaries, and system properties. - * - * Enable DEBUG_MODE to see debug output for each bypassed check. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * SPDX-FileCopyrightText: Riyad Mondal - * - *************************************************************************************************/ - -(() => { - let loggedRootDetectionWarning = false; - function logFirstRootDetection() { - if (!loggedRootDetectionWarning) { - console.log(" => Blocked possible root detection checks. Enable DEBUG_MODE for more details."); - loggedRootDetectionWarning = true; - } - } - - const LIB_C = Process.findModuleByName("libc.so"); - - const BUILD_FINGERPRINT_REGEX = /^([\w.-]+\/[\w.-]+\/[\w.-]+):([\w.]+\/[\w.-]+\/[\w.-]+):(\w+\/[\w,.-]+)$/; - - const CONFIG = { - secureProps: { - "ro.secure": "1", - "ro.debuggable": "0", - "ro.build.type": "user", - "ro.build.tags": "release-keys" - } - }; - - const ROOT_INDICATORS = { - paths: new Set([ - "/data/local/bin/su", - "/data/local/su", - "/data/local/xbin/su", - "/dev/com.koushikdutta.superuser.daemon/", - "/sbin/su", - "/su/bin/su", - "/system/bin/su", - "/system/xbin/su", - "/system/sbin/su", - "/vendor/bin/su", - "/data/adb/su/bin/su", - "/system/bin/failsafe/su", - "/system/bin/.ext/.su", - "/system/bin/.ext/su", - "/system/bin/failsafe/su", - "/system/sd/xbin/su", - "/system/usr/we-need-root/su", - "/cache/su", - "/data/su", - "/dev/su", - "/data/adb/magisk", - "/sbin/.magisk", - "/cache/.disable_magisk", - "/dev/.magisk.unblock", - "/cache/magisk.log", - "/data/adb/magisk.img", - "/data/adb/magisk.db", - "/data/adb/magisk_simple", - "/init.magisk.rc", - "/system/app/Superuser.apk", - "/system/etc/init.d/99SuperSUDaemon", - "/system/xbin/daemonsu", - "/system/xbin/ku.sud", - "/data/adb/ksu", - "/data/adb/ksud", - "/system/xbin/busybox", - "/system/app/Kinguser.apk" - ]), - - packages: new Set([ - "com.noshufou.android.su", - "com.noshufou.android.su.elite", - "eu.chainfire.supersu", - "com.koushikdutta.superuser", - "com.thirdparty.superuser", - "com.yellowes.su", - "com.koushikdutta.rommanager", - "com.koushikdutta.rommanager.license", - "com.dimonvideo.luckypatcher", - "com.chelpus.lackypatch", - "com.ramdroid.appquarantine", - "com.ramdroid.appquarantinepro", - "com.topjohnwu.magisk", - "me.weishu.kernelsu" - ]), - - commands: new Set([ - "su", - "which su", - "whereis su", - "locate su", - "find / -name su", - "mount", - "magisk", - "/system/bin/su", - "/system/xbin/su", - "/sbin/su", - "/su/bin/su" - ]), - - binaries: new Set([ - "su", - "busybox", - "magisk", - "supersu", - "ksud", - "daemonsu" - ]) - }; - - function bypassNativeFileCheck() { - const fopen = LIB_C.findExportByName("fopen"); - if (fopen) { - Interceptor.attach(fopen, { - onEnter(args) { - this.path = args[0].readUtf8String(); - }, - onLeave(retval) { - if (retval.toInt32() !== 0) { - const path = this.path.toLowerCase(); - if (ROOT_INDICATORS.paths.has(this.path) || path.includes("magisk") || path.includes("/su") || path.endsWith("/su")) { - if (DEBUG_MODE) { - console.log(`Blocked possible root-detection: fopen ${this.path}`); - } else logFirstRootDetection(); - retval.replace(ptr(0x0)); - } - } - } - }); - } - - const access = LIB_C.findExportByName("access"); - if (access) { - Interceptor.attach(access, { - onEnter(args) { - this.path = args[0].readUtf8String(); - }, - onLeave(retval) { - if (retval.toInt32() === 0) { - const path = this.path.toLowerCase(); - if (ROOT_INDICATORS.paths.has(this.path) || path.includes("magisk") || path.includes("/su") || path.endsWith("/su")) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: access ${this.path}`); - } else logFirstRootDetection(); - retval.replace(ptr(-1)); - } - } - } - }); - } - - const stat = LIB_C.findExportByName("stat"); - if (stat) { - Interceptor.attach(stat, { - onEnter(args) { - this.path = args[0].readUtf8String(); - }, - onLeave(retval) { - const path = this.path.toLowerCase(); - if (ROOT_INDICATORS.paths.has(this.path) || path.includes("magisk") || path.includes("/su") || path.endsWith("/su")) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: stat ${this.path}`); - } else logFirstRootDetection(); - retval.replace(ptr(-1)); - } - } - }); - } - - const lstat = LIB_C.findExportByName("lstat"); - if (lstat) { - Interceptor.attach(lstat, { - onEnter(args) { - this.path = args[0].readUtf8String(); - }, - onLeave(retval) { - const path = this.path.toLowerCase(); - if (ROOT_INDICATORS.paths.has(this.path) || path.includes("magisk") || path.includes("/su") || path.endsWith("/su")) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: lstat ${this.path}`); - } else logFirstRootDetection(); - retval.replace(ptr(-1)); - } - } - }); - } - } - - function bypassJavaFileCheck() { - function isRootIndicatorFile(file) { - const path = file.getAbsolutePath(); - const filename = file.getName(); - return ROOT_INDICATORS.paths.has(path) || - path.includes("magisk") || - filename === "su"; - } - - const UnixFileSystem = Java.use("java.io.UnixFileSystem"); - UnixFileSystem.checkAccess.implementation = function(file, access) { - if (isRootIndicatorFile(file)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: filesystem access check for ${file.getAbsolutePath()}`); - } else logFirstRootDetection(); - return false; - } - return this.checkAccess(file, access); - }; - - const File = Java.use("java.io.File"); - File.exists.implementation = function() { - if (isRootIndicatorFile(this)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: file exists check for ${this.getAbsolutePath()}`); - } else logFirstRootDetection(); - return false; - } - return this.exists(); - }; - - File.length.implementation = function() { - if (isRootIndicatorFile(this)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: file length check for ${this.getAbsolutePath()}`); - } else logFirstRootDetection(); - return 0; - } - return this.length(); - }; - - const FileInputStream = Java.use("java.io.FileInputStream"); - FileInputStream.$init.overload('java.io.File').implementation = function(file) { - if (isRootIndicatorFile(file)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: file stream for ${file.getAbsolutePath()}`); - } else logFirstRootDetection(); - throw Java.use("java.io.FileNotFoundException").$new(file.getAbsolutePath()); - } - return this.$init(file); - }; - } - - function setProp() { - const Build = Java.use("android.os.Build"); - - // We do a little work to make the minimum changes required to hide in the BUILD fingerprint, - // but otherwise keep matching the real device wherever possible. - const realFingerprint = Build.FINGERPRINT.value; - - const fingerprintMatch = BUILD_FINGERPRINT_REGEX.exec(realFingerprint); - let fixedFingerprint; - if (fingerprintMatch) { - let [, device, versions, tags] = BUILD_FINGERPRINT_REGEX.exec(realFingerprint); - tags = 'user/release-keys'; // Should always be the case in production builds - if (device.includes('generic') || device.includes('sdk') || device.includes('lineage')) { - device = 'google/raven/raven'; - } - - fixedFingerprint = `${device}:${versions}:${tags}`; - } else { - console.warn(`Unexpected BUILD fingerprint format: ${realFingerprint}`); - // This should never happen in theory (the format is standard), but just in case, - // we use this fallback fingerprint: - fixedFingerprint = "google/crosshatch/crosshatch:10/QQ3A.200805.001/6578210:user/release-keys"; - } - - const fields = { - "TAGS": "release-keys", - "TYPE": "user", - "FINGERPRINT": fixedFingerprint - }; - - Object.entries(fields).forEach(([field, value]) => { - const fieldObj = Build.class.getDeclaredField(field); - fieldObj.setAccessible(true); - fieldObj.set(null, value); - }); - - const system_property_get = LIB_C.findExportByName("__system_property_get"); - if (system_property_get) { - Interceptor.attach(system_property_get, { - onEnter(args) { - this.key = args[0].readCString(); - this.ret = args[1]; - }, - onLeave(retval) { - const secureValue = CONFIG.secureProps[this.key]; - if (secureValue !== undefined) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: system_property_get ${this.key}`); - } else logFirstRootDetection(); - const valuePtr = Memory.allocUtf8String(secureValue); - Memory.copy(this.ret, valuePtr, secureValue.length + 1); - } - } - }); - } - - const Runtime = Java.use('java.lang.Runtime'); - Runtime.exec.overload('java.lang.String').implementation = function(cmd) { - if (cmd.startsWith("getprop ")) { - const prop = cmd.split(" ")[1]; - if (CONFIG.secureProps[prop]) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: getprop ${prop}`); - } else logFirstRootDetection(); - return null; - } - } - return this.exec(cmd); - }; - } - - function bypassRootPackageCheck() { - const ApplicationPackageManager = Java.use("android.app.ApplicationPackageManager"); - - ApplicationPackageManager.getPackageInfo.overload('java.lang.String', 'int').implementation = function(str, i) { - if (ROOT_INDICATORS.packages.has(str)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: package info for ${str}`); - } else logFirstRootDetection(); - str = "invalid.example.nonexistent.package"; - } - return this.getPackageInfo(str, i); - }; - - ApplicationPackageManager.getInstalledPackages.overload('int').implementation = function(flags) { - const packages = this.getInstalledPackages(flags); - const packageList = packages.toArray(); - const filteredPackages = packageList.filter(pkg => !ROOT_INDICATORS.packages.has(pkg.packageName?.value)); - return Java.use("java.util.ArrayList").$new(Java.use("java.util.Arrays").asList(filteredPackages)); - }; - } - - function bypassShellCommands() { - const ProcessBuilder = Java.use('java.lang.ProcessBuilder'); - ProcessBuilder.command.overload('java.util.List').implementation = function(commands) { - const cmdArray = commands.toArray(); - if (cmdArray.length > 0) { - const cmd = cmdArray[0].toString(); - if (ROOT_INDICATORS.commands.has(cmd) || (cmdArray.length > 1 && ROOT_INDICATORS.binaries.has(cmdArray[1].toString()))) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: ProcessBuilder with ${cmdArray.join(' ')}`); - } else logFirstRootDetection(); - return this.command(Java.use("java.util.Arrays").asList([""])); - } - } - return this.command(commands); - }; - - const Runtime = Java.use('java.lang.Runtime'); - Runtime.exec.overload('[Ljava.lang.String;').implementation = function(cmdArray) { - if (cmdArray.length > 0) { - const cmd = cmdArray[0]; - if (ROOT_INDICATORS.commands.has(cmd) || (cmdArray.length > 1 && ROOT_INDICATORS.binaries.has(cmdArray[1]))) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: Runtime.exec for ${cmdArray.join(' ')}`); - } else logFirstRootDetection(); - return this.exec([""]); - } - } - return this.exec(cmdArray); - }; - - const ProcessImpl = Java.use("java.lang.ProcessImpl"); - ProcessImpl.start.implementation = function(cmdArray, env, dir, redirects, redirectErrorStream) { - if (cmdArray.length > 0) { - const cmd = cmdArray[0].toString(); - const arg = cmdArray.length > 1 ? cmdArray[1].toString() : ""; - - if (ROOT_INDICATORS.commands.has(cmd) || ROOT_INDICATORS.binaries.has(arg)) { - if (DEBUG_MODE) { - console.debug(`Blocked possible root detection: ProcessImpl.start for ${cmdArray.join(' ')}`); - } else logFirstRootDetection(); - return ProcessImpl.start.call(this, [Java.use("java.lang.String").$new("")], env, dir, redirects, redirectErrorStream); - } - } - return ProcessImpl.start.call(this, cmdArray, env, dir, redirects, redirectErrorStream); - }; - } - - try { - bypassNativeFileCheck(); - bypassJavaFileCheck(); - setProp(); - bypassRootPackageCheck(); - bypassShellCommands(); - console.log("== Disabled Android root detection =="); - } catch (error) { - console.error("\n !!! Error setting up root detection bypass !!!", error); - } -})(); diff --git a/resources/frida/android/android-proxy-override.js b/resources/frida/android/android-proxy-override.js deleted file mode 100644 index d4248e6..0000000 --- a/resources/frida/android/android-proxy-override.js +++ /dev/null @@ -1,98 +0,0 @@ -/************************************************************************************************** - * - * The first step in intercepting HTTP & HTTPS traffic is to set the default proxy settings, - * telling the app that all requests should be sent via our HTTP proxy. - * - * In this script, we set that up via a few different mechanisms, which cumulatively should - * ensure that all connections are sent via the proxy, even if they attempt to use their - * own custom proxy configurations to avoid this. - * - * Despite that, this still only covers well behaved apps - it's still possible for apps - * to send network traffic directly if they're determined to do so, or if they're built - * with a framework that does not do this by default (Flutter is notably in this category). - * To handle those less tidy cases, we manually capture traffic to recognized target ports - * in the native connect() hook script. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * - *************************************************************************************************/ - -Java.perform(() => { - // Set default JVM system properties for the proxy address. Notably these are used - // to initialize WebView configuration. - Java.use('java.lang.System').setProperty('http.proxyHost', PROXY_HOST); - Java.use('java.lang.System').setProperty('http.proxyPort', PROXY_PORT.toString()); - Java.use('java.lang.System').setProperty('https.proxyHost', PROXY_HOST); - Java.use('java.lang.System').setProperty('https.proxyPort', PROXY_PORT.toString()); - - Java.use('java.lang.System').clearProperty('http.nonProxyHosts'); - Java.use('java.lang.System').clearProperty('https.nonProxyHosts'); - - // Some Android internals attempt to reset these settings to match the device configuration. - // We block that directly here: - const controlledSystemProperties = [ - 'http.proxyHost', - 'http.proxyPort', - 'https.proxyHost', - 'https.proxyPort', - 'http.nonProxyHosts', - 'https.nonProxyHosts' - ]; - Java.use('java.lang.System').clearProperty.implementation = function (property) { - if (controlledSystemProperties.includes(property)) { - if (DEBUG_MODE) console.log(`Ignoring attempt to clear ${property} system property`); - return this.getProperty(property); - } - return this.clearProperty(...arguments); - } - Java.use('java.lang.System').setProperty.implementation = function (property) { - if (controlledSystemProperties.includes(property)) { - if (DEBUG_MODE) console.log(`Ignoring attempt to override ${property} system property`); - return this.getProperty(property); - } - return this.setProperty(...arguments); - } - - // Configure the app's proxy directly, via the app connectivity manager service: - const ConnectivityManager = Java.use('android.net.ConnectivityManager'); - const ProxyInfo = Java.use('android.net.ProxyInfo'); - ConnectivityManager.getDefaultProxy.implementation = () => ProxyInfo.$new(PROXY_HOST, PROXY_PORT, ''); - // (Not clear if this works 100% - implying there are ConnectivityManager subclasses handling this) - - console.log(`== Proxy system configuration overridden to ${PROXY_HOST}:${PROXY_PORT} ==`); - - // Configure the proxy indirectly, by overriding the return value for all ProxySelectors everywhere: - const Collections = Java.use('java.util.Collections'); - const ProxyType = Java.use('java.net.Proxy$Type'); - const InetSocketAddress = Java.use('java.net.InetSocketAddress'); - const ProxyCls = Java.use('java.net.Proxy'); // 'Proxy' is reserved in JS - - const targetProxy = ProxyCls.$new( - ProxyType.HTTP.value, - InetSocketAddress.$new(PROXY_HOST, PROXY_PORT) - ); - const getTargetProxyList = () => Collections.singletonList(targetProxy); - - const ProxySelector = Java.use('java.net.ProxySelector'); - - // Find every implementation of ProxySelector by quickly scanning method signatures, and - // then checking whether each match actually implements java.net.ProxySelector: - const proxySelectorClasses = Java.enumerateMethods('*!select(java.net.URI): java.util.List/s') - .flatMap((matchingLoader) => matchingLoader.classes - .map((classData) => Java.use(classData.name)) - .filter((Cls) => ProxySelector.class.isAssignableFrom(Cls.class)) - ); - - // Replace the 'select' of every implementation, so they all send traffic to us: - proxySelectorClasses.forEach(ProxySelectorCls => { - if (DEBUG_MODE) { - console.log('Rewriting', ProxySelectorCls.toString()); - } - ProxySelectorCls.select.implementation = () => getTargetProxyList() - }); - - console.log(`== Proxy configuration overridden to ${PROXY_HOST}:${PROXY_PORT} ==`); -}); - diff --git a/resources/frida/android/android-system-certificate-injection.js b/resources/frida/android/android-system-certificate-injection.js deleted file mode 100644 index 5b30422..0000000 --- a/resources/frida/android/android-system-certificate-injection.js +++ /dev/null @@ -1,88 +0,0 @@ -/************************************************************************************************** - * - * Once we have captured traffic (once it's being sent to our proxy port) the next step is - * to ensure any clients using TLS (HTTPS) trust our CA certificate, to allow us to intercept - * encrypted connections successfully. - * - * This script does so by attaching to the internals of Conscrypt (the Android SDK's standard - * TLS implementation) and pre-adding our certificate to the 'already trusted' cache, so that - * future connections trust it implicitly. This ensures that all normal uses of Android APIs - * for HTTPS & TLS will allow interception. - * - * This does not handle all standalone certificate pinning techniques - where the application - * actively rejects certificates that are trusted by default on the system. That's dealt with - * in the separate certificate unpinning script. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * - *************************************************************************************************/ - -Java.perform(() => { - // First, we build a JVM representation of our certificate: - const String = Java.use("java.lang.String"); - const ByteArrayInputStream = Java.use('java.io.ByteArrayInputStream'); - const CertFactory = Java.use('java.security.cert.CertificateFactory'); - - let cert; - try { - const certFactory = CertFactory.getInstance("X.509"); - const certBytes = String.$new(CERT_PEM).getBytes(); - cert = certFactory.generateCertificate(ByteArrayInputStream.$new(certBytes)); - } catch (e) { - console.error('Could not parse provided certificate PEM!'); - console.error(e); - Java.use('java.lang.System').exit(1); - } - - // Then we hook TrustedCertificateIndex. This is used for caching known trusted certs within Conscrypt - - // by prepopulating all instances, we ensure that all TrustManagerImpls (and potentially other - // things) automatically trust our certificate specifically (without disabling validation entirely). - // This should apply to Android v7+ - previous versions used SSLContext & X509TrustManager. - [ - 'com.android.org.conscrypt.TrustedCertificateIndex', - 'org.conscrypt.TrustedCertificateIndex', // Might be used (com.android is synthetic) - unclear - 'org.apache.harmony.xnet.provider.jsse.TrustedCertificateIndex' // Used in Apache Harmony version of Conscrypt - ].forEach((TrustedCertificateIndexClassname, i) => { - let TrustedCertificateIndex; - try { - TrustedCertificateIndex = Java.use(TrustedCertificateIndexClassname); - } catch (e) { - if (i === 0) { - throw new Error(`${TrustedCertificateIndexClassname} not found - could not inject system certificate`); - } else { - // Other classnames are optional fallbacks - if (DEBUG_MODE) { - console.log(`[ ] Skipped cert injection for ${TrustedCertificateIndexClassname} (not present)`); - } - return; - } - } - - TrustedCertificateIndex.$init.overloads.forEach((overload) => { - overload.implementation = function () { - this.$init(...arguments); - // Index our cert as already trusted, right from the start: - this.index(cert); - } - }); - - TrustedCertificateIndex.reset.overloads.forEach((overload) => { - overload.implementation = function () { - const result = this.reset(...arguments); - // Index our cert in here again, since the reset removes it: - this.index(cert); - return result; - }; - }); - - if (DEBUG_MODE) console.log(`[+] Injected cert into ${TrustedCertificateIndexClassname}`); - }); - - // This effectively adds us to the system certs, and also defeats quite a bit of basic certificate - // pinning too! It auto-trusts us in any implementation that uses TrustManagerImpl (Conscrypt) as - // the underlying cert checking component. - - console.log('== System certificate trust injected =='); -}); \ No newline at end of file diff --git a/resources/frida/config-template.js b/resources/frida/config-template.js deleted file mode 100644 index 42326bf..0000000 --- a/resources/frida/config-template.js +++ /dev/null @@ -1,255 +0,0 @@ -/************************************************************************************************** - * - * This file defines various config parameters, used later within the other scripts. - * - * In all cases, you'll want to set CERT_PEM and likely PROXY_HOST and PROXY_PORT. - * - * Source available at https://github.com/httptoolkit/frida-interception-and-unpinning/ - * SPDX-License-Identifier: AGPL-3.0-or-later - * SPDX-FileCopyrightText: Tim Perry - * - *************************************************************************************************/ - -// Put your CA certificate data here in PEM format: -const CERT_PEM = `-----BEGIN CERTIFICATE----- - -[!! Put your CA certificate data here, in PEM format !!] - ------END CERTIFICATE-----`; - -// Put your intercepting proxy's address here: -const PROXY_HOST = '127.0.0.1'; -const PROXY_PORT = 8000; - -// If you like, set to to true to enable extra logging: -const DEBUG_MODE = false; - -// If you find issues with non-HTTP traffic being captured (due to the -// native connect hook script) you can add ports here to exempt traffic -// on that port from being redirected. Note that this will only affect -// traffic captured by the raw connection hook - for apps using the -// system HTTP proxy settings, traffic on these ports will still be -// sent via the proxy and intercepted despite this setting. -const IGNORED_NON_HTTP_PORTS = []; - -// As HTTP/3 is often not well supported by MitM proxies, by default it -// is blocked entirely, so all outgoing UDP connections to port 443 -// will fail. If this is set to false, they will instead be left unintercepted. -const BLOCK_HTTP3 = true; - -// Set this to true if your proxy supports SOCKS5 connections. -// This makes it possible for native-connect-hook to redirect -// non-HTTP traffic through your proxy (to view it raw, and -// avoid breaking non-HTTP traffic en route). -const PROXY_SUPPORTS_SOCKS5 = false; - - -// ---------------------------------------------------------------------------- -// You don't need to modify any of the below, it just checks and applies some -// of the configuration that you've entered above. -// ---------------------------------------------------------------------------- - - -if (DEBUG_MODE) { - // Add logging just for clean output & to separate reloads: - console.log('\n*** Starting scripts ***'); - if (globalThis.Java?.available) { - Java.perform(() => { - setTimeout(() => console.log('*** Scripts completed ***\n'), 5); - // (We assume that nothing else will take more than 5ms, but app startup - // probably will, so this should separate script & runtime logs) - }); - } else { - setTimeout(() => console.log('*** Scripts completed ***\n'), 5); - // (We assume that nothing else will take more than 5ms, but app startup - // probably will, so this should separate script & runtime logs) - } -} else { - console.log(''); // Add just a single newline, for minimal clarity -} - -// Check the certificate (without literally including the instruction phrasing -// here, as that can be confusing for some users): -if (CERT_PEM.match(/\[!!.* CA certificate data .* !!\]/)) { - throw new Error('No certificate was provided' + - '\n\n' + - 'You need to set CERT_PEM in the Frida config script ' + - 'to the contents of your CA certificate.' - ); -} - - - -// ---------------------------------------------------------------------------- -// Don't modify any of the below unless you know what you're doing! -// This section defines various utilities & calculates some constants which may -// be used by later scripts elsewhere in this project. -// ---------------------------------------------------------------------------- - - - -// As web atob & Node.js Buffer aren't available, we need to reimplement base64 decoding -// in pure JS. This is a quick rough implementation without much error handling etc! - -// Base64 character set (plus padding character =) and lookup: -const BASE64_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/='; -const BASE64_LOOKUP = new Uint8Array(123); -for (let i = 0; i < BASE64_CHARS.length; i++) { - BASE64_LOOKUP[BASE64_CHARS.charCodeAt(i)] = i; -} - - -/** - * Take a base64 string, and return the raw bytes - * @param {string} input - * @returns Uint8Array - */ -function decodeBase64(input) { - // Calculate the length of the output buffer based on padding: - let outputLength = Math.floor((input.length * 3) / 4); - if (input[input.length - 1] === '=') outputLength--; - if (input[input.length - 2] === '=') outputLength--; - - const output = new Uint8Array(outputLength); - let outputPos = 0; - - // Process each 4-character block: - for (let i = 0; i < input.length; i += 4) { - const a = BASE64_LOOKUP[input.charCodeAt(i)]; - const b = BASE64_LOOKUP[input.charCodeAt(i + 1)]; - const c = BASE64_LOOKUP[input.charCodeAt(i + 2)]; - const d = BASE64_LOOKUP[input.charCodeAt(i + 3)]; - - // Assemble into 3 bytes: - const chunk = (a << 18) | (b << 12) | (c << 6) | d; - - // Add each byte to the output buffer, unless it's padding: - output[outputPos++] = (chunk >> 16) & 0xff; - if (input.charCodeAt(i + 2) !== 61) output[outputPos++] = (chunk >> 8) & 0xff; - if (input.charCodeAt(i + 3) !== 61) output[outputPos++] = chunk & 0xff; - } - - return output; -} - -/** - * Take a single-certificate PEM string, and return the raw DER bytes - * @param {string} input - * @returns Uint8Array - */ -function pemToDer(input) { - const pemLines = input.split('\n'); - if ( - pemLines[0] !== '-----BEGIN CERTIFICATE-----' || - pemLines[pemLines.length- 1] !== '-----END CERTIFICATE-----' - ) { - throw new Error( - 'Your certificate should be in PEM format, starting & ending ' + - 'with a BEGIN CERTIFICATE & END CERTIFICATE header/footer' - ); - } - - const base64Data = pemLines.slice(1, -1).map(l => l.trim()).join(''); - if ([...base64Data].some(c => !BASE64_CHARS.includes(c))) { - throw new Error( - 'Your certificate should be in PEM format, containing only ' + - 'base64 data between a BEGIN & END CERTIFICATE header/footer' - ); - } - - return decodeBase64(base64Data); -} - -const CERT_DER = pemToDer(CERT_PEM); - -// Right now this API is a bit funky - the callback will be called with a Frida Module instance -// if the module is properly detected, but may be called with just { name, path, base, size } -// in some cases (e.g. shared libraries loaded from inside an APK on Android). Works OK right now, -// as it's not widely used but needs improvement in future if we extend this. -function waitForModule(moduleName, callback) { - if (Array.isArray(moduleName)) { - moduleName.forEach(module => waitForModule(module, callback)); - } - - try { - const module = Process.getModuleByName(moduleName) - module.ensureInitialized(); - callback(module); - return; - } catch (e) { - try { - const module = Module.load(moduleName); - callback(module); - return; - } catch (e) {} - } - - MODULE_LOAD_CALLBACKS[moduleName] = callback; -} - -const getModuleName = (nameOrPath) => { - const endOfPath = nameOrPath.lastIndexOf('/'); - return nameOrPath.slice(endOfPath + 1); -}; - -const MODULE_LOAD_CALLBACKS = {}; -new ApiResolver('module').enumerateMatches('exports:linker*!*dlopen*').forEach((dlopen) => { - Interceptor.attach(dlopen.address, { - onEnter(args) { - const moduleArg = args[0].readCString(); - if (moduleArg) { - this.path = moduleArg; - this.moduleName = getModuleName(moduleArg); - } - }, - onLeave(retval) { - if (!this.path || !retval || retval.isNull()) return; - if (!MODULE_LOAD_CALLBACKS[this.moduleName]) return; - - let module = Process.findModuleByName(this.moduleName) - ?? Process.findModuleByAddress(retval); - if (!module) { - // Some modules are loaded in ways that mean Frida can't detect them, and - // can't look them up by name (notably when loading libraries from inside an - // APK on Android). To handle this, we can use dlsym to look up an example - // symbol and find the underlying module details directly, where possible. - module = getAnonymousModule(this.moduleName, this.path, retval); - if (!module) return; - } - - Object.keys(MODULE_LOAD_CALLBACKS).forEach((key) => { - if (this.moduleName === key) { - if (module) { - MODULE_LOAD_CALLBACKS[key](module); - delete MODULE_LOAD_CALLBACKS[key]; - } - } - }); - } - }); -}); - -const getAnonymousModule = (name, path, handle) => { - const dlsymAddr = Module.findGlobalExportByName('dlsym'); - if (!dlsymAddr) { - console.error(`[!] Cannot find dlsym, cannot get anonymous module info for ${name}`); - return; - } - - const dlsym = new NativeFunction(dlsymAddr, 'pointer', ['pointer', 'pointer']); - - // Handle here is the return value from dlopen - but in this scenario, it's just an - // opaque handle into to 'soinfo' data that other methods can use to get the - // real pointer to parts of the module, like so: - const onLoadPointer = dlsym(handle, Memory.allocUtf8String('JNI_OnLoad')); - - // Once we have an actual pointer, we can get the range that holds it: - const range = Process.getRangeByAddress(onLoadPointer); - - return { - base: range.base, - size: range.size, - name, - path, - } -}; \ No newline at end of file diff --git a/src/App.breakpoints.test.tsx b/src/App.breakpoints.test.tsx index 49b5219..44cd558 100644 --- a/src/App.breakpoints.test.tsx +++ b/src/App.breakpoints.test.tsx @@ -139,6 +139,26 @@ describe('Proxy listener settings', () => { fireEvent.click(screen.getByRole('button', { name: 'Start interception' })); }); } + it('requires an explicit Frida selection and certificate for unpinning', async () => { + androidAvailable(); + await mount(); + await act(async () => { + fireEvent.click(screen.getByRole('button', { name: /Connect a device Android/ })); + }); + await act(async () => { + fireEvent.click(screen.getByRole('button', { name: /Demo app com.example.demo/ })); + }); + fireEvent.click(screen.getByRole('button', { name: /Frida Optional instrumentation/ })); + fireEvent.click(screen.getByRole('checkbox', { name: 'Trust the Scope CA' })); + expect(screen.getByRole('checkbox', { name: 'TLS certificate unpinning' })).not.toBeChecked(); + expect(screen.getByRole('checkbox', { name: 'TLS certificate unpinning' })).toBeDisabled(); + fireEvent.click(screen.getByRole('checkbox', { name: 'Proxy configuration' })); + expect(screen.getByRole('button', { name: 'Start interception' })).toBeDisabled(); + expect(mocks.start_device).not.toHaveBeenCalled(); + expect( + screen.queryByRole('checkbox', { name: 'Root detection hooks' }), + ).not.toBeInTheDocument(); + }); it('uses the configured port when demo traffic starts an idle listener', async () => { idleListener(); await mount(); diff --git a/src/App.tsx b/src/App.tsx index f8295f0..f6819a2 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -1651,6 +1651,7 @@ function DevicePanel({ ); const start = () => void perform('device', async () => { + if (mode === 'frida' && profiles.length === 0) return; let p = proxy; if (!p.running) { const port = validatedListenerPort(portDraft); @@ -1783,14 +1784,17 @@ function DevicePanel({

Device-wide capture. Android's proxy setting applies to the whole device. Other apps may appear in this session; choosing an app launches it and - does not isolate its traffic. + does not isolate its traffic. HTTPS requires manually configured app trust; no + system CA is installed.

) : (

Select the bundled hook profiles for an app you own or are authorized to test. - Frida support depends on Android and application versions. + Experimental Java URL/OkHttp hooks only. Stop ends the instrumented app process. + Unpinning requires the Scope CA profile; native TLS and custom pinners are + unsupported.

@@ -1803,15 +1807,19 @@ function DevicePanel({ ['certificate', 'Trust the Scope CA'], ['proxy', 'Proxy configuration'], ['unpinning', 'TLS certificate unpinning'], - ['root', 'Root detection hooks'], ].map(([id, label]) => (