Sharique - #4
Open
Sharique-Raza wants to merge 49 commits into
Open
Conversation
Root cause: stale server process running old dist/index.mjs without DATABASE_URL. PostgresUserRepository.findActorByUsername() query was valid — connection pool could not initialize without the environment variable. Changes: - Add regression test #32 validating DevelopmentActorAuthenticator wiring, DATABASE_URL guard, PORT requirement, and persistent context contracts - Add UUID parameter validation to cases.ts and investigations.ts routes (malformed UUID was causing 500 instead of 400) - Add .env.example documenting required/optional environment variables - Add opencode.json to .gitignore for API key safety - Update Phase 5 final validation and project status documentation Validation: - 32/32 tests pass - Typecheck pass (4/4 workspace projects) - OpenAPI codegen pass (orval v8.23.0) - Build pass (2.1MB bundle) - git diff --check pass - Clean database replay: 6/6 migrations, idempotency verified - PostgreSQL E2E: case→investigation→evidence→graph→intelligence chain - RBAC: dual-role (INVESTIGATOR/SUPERVISOR) verified - Case isolation: cross-case access denied (404) - Audit immutability: UPDATE denied (error 23514) - Security: SQL injection safe, malformed UUID → 400, no secrets in errors - Legacy /api/* endpoints: healthz, dashboard, cases all 200 - Phase 4 regression: graph with filters → 200 External gates (not software defects): - Etherscan V2: IMPLEMENTED_PENDING_LIVE_VALIDATION - Esplora: IMPLEMENTED_PENDING_LIVE_VALIDATION - TronGrid: IMPLEMENTED_PENDING_LIVE_VALIDATION - crypto-wallet-address-labels: DATASET_PENDING_APPROVAL - Accuracy: INSUFFICIENT_GROUND_TRUTH
… collected Live collection from https://blockstream.info/api using Bitcoin genesis address (1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa): - 25 transactions collected and persisted - 49 inputs, 50 outputs in PostgreSQL - Full RBAC flow: case→authorize→collect→graph→intelligence→audit - 11 audit events for complete chain - Clustering returned INSUFFICIENT_DATA (correct for single-address) - VASP analysis returned INSUFFICIENT_EVIDENCE (correct)
Provider fixes:
- Etherscan V2: add 250ms inter-request delay + detect API-level rate limits
- TronGrid: use /wallet/gettransactionbyid (POST) instead of /v1/transactions/{hash} (404)
- Collection: serialize provider calls (was Promise.all → rate limit on free tiers)
- Repository: truncate contract_interactions input_data to 512 chars (btree overflow)
Live validation results:
- Bitcoin/Esplora: 25 txs from genesis address via blockstream.info/api
- Ethereum/Etherscan V2: 283 txs + 100 token transfers from Ethereum Foundation
- TRON/TronGrid: 200 txs + 100 TRC-20 transfers
All 32 tests pass, typecheck clean across 4 projects.
Classifications based on actual code inspection: - 5 CLEAN_ROOM_IMPLEMENTED (clustering, graph/exposure, evidence, forensics) - 1 DATASET_PENDING_APPROVAL (address labels — governance blocker only) - 1 REFERENCE_ONLY (OpenAML — risk signals exist in fusion pipeline) - 2 OUT_OF_SCOPE (MEV, Chainabuse) - 3 LIVE_VALIDATED providers (Esplora, Etherscan, TronGrid) No software defects. 32/32 tests pass.
OpenAML formally classified as REFERENCE_ONLY per phase5-plan.md line 28: 'Later governed AML/risk research | Separate model/data/evaluation governance' Final verification: - Typecheck: 4/4 PASS - Tests: 32/32 PASS - Build: 2.1MB PASS - git diff --check: PASS All software requirements met. Two non-software blockers remain: - DATASET_PENDING_APPROVAL (governance) - INSUFFICIENT_GROUND_TRUTH (data) Tag v0.5.0-phase5 preserved. Git history not rewritten. Phase 6 not started.
Phase 6.0 architecture audit complete. Documents created: - phase6-scope.md: capability boundaries and non-goals - phase6-architecture.md: service/provider/repository extension design - phase6-roadmap.md: milestone dependency graph with hard validation checkpoints - phase6-security-model.md: auth, RBAC, case isolation, API security - phase6-data-model.md: new tables, indexes, migration strategy - phase6-provider-matrix.md: provider selection per chain with verification requirements - phase6-validation-strategy.md: test categories, evaluation, calibration, release gates No code changes. Phase 5 baseline preserved.
New providers: - BscScanBnbProvider: BNB Chain via BscScan API (chain=BNB_CHAIN) - PolygonScanProvider: Polygon via PolygonScan API (chain=POLYGON) - SolanaRpcProvider: Solana via JSON-RPC 2.0 (chain=SOLANA, NOT EVM) New normalizers: - bnbWallet/bnbTransaction/bnbTokenTransfer (EVM pattern, independent chain ID) - polygonWallet/polygonTransaction/polygonTokenTransfer (EVM pattern) - solanaWallet/solanaTransaction/solanaTokenTransfer (chain-specific) Solana preserves native forensic detail: - signature, slot, blockTime, program_id, instruction_index - inner_instruction_index, account_keys, SPL token transfers Extended: - ProviderRouter dispatches all 6 chains - CashnetConfig: BSCSCAN_API_KEY, POLYGONSCAN_API_KEY, SOLANA_RPC_URL - Migration: chain-specific indexes + COLLECTION_* permissions Validation: typecheck 4/4, tests 32/32, build PASS, git diff --check PASS Live status: IMPLEMENTED_PENDING_LIVE_VALIDATION (no API keys configured)
New services: - AMLRiskIndicatorService: deterministic, versioned risk indicators - rapid-in-out, fan-in/fan-out, burst activity - counterparty concentration, high-value anomaly - Each indicator: type, version, evidence, confidence, explanation - Risk indicator != suspicious person, != confirmed fraud - RiskTypologyFramework: named AML pattern matching - RAPID_MOVEMENT, STRUCTURING, LAYERING, HIGH_RISK_EXPOSURE, CONCENTRATION - Confidence levels: CANDIDATE, LIKELY, REVIEW_REQUIRED - Every match explicitly requires human review New migration: - risk_analysis_runs, risk_indicators, risk_indicator_evidence tables - risk_typologies with versioned definitions - RISK_ANALYZE, RISK_READ permissions with RBAC mapping Validation: typecheck 4/4, tests 32/32, build PASS
…ation Phase 6.3 - Advanced Graph: - GraphFeatureService: degree, volume, temporal, concentration features - CommunityDetectionService: deterministic Union-Find connected components - Bounded execution (max nodes/edges/time) - graph_features table migration Phase 6.4 - DeFi/MEV: - DeFiInteractionService: known DEX router identification - MEVDetectionService: sandwich + arbitrage candidates (historical only) - Known routers: Uniswap, PancakeSwap, SushiSwap, QuickSwap - defi_protocol_interactions + mev_candidates table migration - Every MEV result: CANDIDATE/LIKELY/REVIEW_REQUIRED Phase 6.5 - Evaluation: - Binary metrics: precision, recall, F1, FPR, FNR, specificity, sensitivity - Ranking metrics: Top-K, MRR - Calibration: Brier score, ECE, reliability diagram bins - False positive categorization - Score type labeling (ORDINAL/RANKING/HEURISTIC/CALIBRATED) - ACCURACY = INSUFFICIENT_GROUND_TRUTH until independent dataset exists Validation: typecheck 4/4, tests 32/32, build PASS
Phase 6.6 - Production Hardening: - JWTAuthenticator: generic OIDC/JWT (RS256/ES256), JWKS cache w/ TTL - Issuer allowlist, audience verification, expiry + clock skew - Provider-neutral (no Auth0/Keycloak hardcoded) - Security middleware: secure headers (HSTS/CSP/X-Frame-Options/nosniff) - Token bucket rate limiting, request ID injection - CORS with configurable origins, request size limits - Secret redaction for log safety - ReportGenerator: structured forensic reports with sections - FACTS, OBSERVATIONS, INFERENCES, ASSESSMENTS, CONTRADICTIONS - Mandatory disclaimer: scores != probabilities - Contradictions NEVER suppressed - Method version provenance chain - Production RBAC: SENIOR_INVESTIGATOR, REVIEWER, AUDITOR roles - REPORT_GENERATE, REPORT_EXPORT, AUDIT_EXPORT permissions - forensic_reports table + audit immutability trigger - Dockerfile: multi-stage, non-root, health check, alpine - docker-compose: PostgreSQL + API with statement timeouts - CI: typecheck → test → build → migration verify → security audit Phase 6 Test Suite (14 new tests, 46 total): - Typology matching + empty indicators - Community detection + bounded execution - DeFi router identification + chain filtering - MEV sandwich detection - Binary metrics, calibration, false positive analysis - Secret redaction, report generation - Multi-chain provider routing (all 6 chains) Validation: typecheck 4/4, tests 46/46, build PASS, git diff --check PASS
Defects discovered and fixed: 1. MIGRATION RUNNER: Phase 6 migrations (multichain, risk, graph, defi, production) were NOT registered in lib/db/src/migrate.ts. They existed on disk but would NOT execute on deployment. Now registered. 2. READINESS PROBE: /readyz endpoint was missing. /healthz existed as liveness probe but Kubernetes/orchestrator readiness probe was absent. Added /readyz with database configuration check. Validation: typecheck 4/4, tests 46/46, build PASS, git diff --check PASS
… 6 intelligence services
…ediate Solana provider Provider migrations: - BNB Chain: Replace BscScanBnbProvider (Etherscan V2) with NodeRealBnbProvider using nr_getAssetTransfers (JSON-RPC 2.0). Independent BNB_NODEREAL_API_KEY credential. Preserves getTransactions, getTokenTransfers, getInternalTransactions. - Polygon: Replace PolygonScanProvider with PolygonBlockscoutProvider using the public Blockscout REST API. Independent configuration via POLYGON_BLOCKSCOUT_*. - Solana: Add contract interaction extraction, fix apiKey to read from config instead of direct process.env access. Deleted legacy providers: - bscscan-provider.ts (replaced by nodereal-provider.ts) - polygonscan-provider.ts (replaced by blockscout-provider.ts) Configuration: - Add noderealBnb and polygon Blockscout provider config blocks. - Remove obsolete bscscan and polygonscan config entries. - Ethereum/Etherscan remains completely independent and unchanged. Tests and validation: - Update phase3.test.ts for NodeReal and Blockscout provider tests. - Add isolated read-only live validator scripts for all 6 chains. - All 68 unit tests pass. Typecheck clean. git diff --check clean. No secrets, credentials, database dumps, or .env files are included.
Collaborator
|
blan blah ........... |
Co-authored-by: subhammohanty092-netizen <233521796+subhammohanty092-netizen@users.noreply.github.com>
Collaborator
|
blahj blah |
- P0.8: Provider Runtime Testing report - P0.9: E2E Pipeline Testing report - P0.10: API Runtime Testing report - P0.11: Database Runtime load test report - P0.12: Typology verification report (debunked 22/22 claim) - P0.13: VASP attribution fusion verification - P0.14: Synthetic/Live boundary verification - P0.15: Purged legacy Python folders (services, models, tests, migrations) - Generated final verification status report
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 6 - blockchain readiness.