From 606061d3197a9d87e022922ffeab4dd9c593e018 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Mon, 27 Jul 2026 03:26:07 +0200 Subject: [PATCH 01/10] Add btcd endpoint functional tests. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New endpoints/test_btcd_rpc.py covering the btcd JSON-RPC/websocket compatibility interface, modeled on test_bitcoind_rpc.py's request/response pattern and test_electrum_subscriptions.py's persistent-connection/ notification-reading pattern. No new pip dependency: the websocket client (WebSocketConnection) is a minimal stdlib-only RFC 6455 implementation (handshake, masked text frames, multi-frame message reassembly), matching this suite's existing reference for hand-rolled wire protocol over a library (see Electrum's raw-socket client). Tests are split into two kinds: - Real assertions (no marker) — authenticate, session, notifyblocks/stopnotifyblocks, chain methods over http post, deprecated methods and stop staying rejected, response envelope behavior. A failure here is a regression. - @pytest.mark.xfail(strict=False) — development targets for what isn't implemented yet: chain methods bridged into the ws connection itself (phase B), the wired not-yet-implemented stubs (notifynewtransactions, loadtxfilter, rescanblocks, ...), and the not-yet-wired phase B/C methods (getinfo, getdifficulty, createrawtransaction, ...). Expected to fail now; flips to XPASS once implemented, which is the cue to tighten the assertion and drop the marker. conftest.py: --btcd-host/--btcd-port/--btcd-username/--btcd-password options and btcd_config fixture. utils.py: DEFAULT_BTCD_PORT. README.md: btcd section (config sample, options, coverage, debug/trigger env vars) alongside the existing per-endpoint documentation. --- endpoints/README.md | 91 +++ endpoints/conftest.py | 39 ++ endpoints/test_btcd_rpc.py | 548 ++++++++++++++++++ endpoints/utils.py | 1 + include/bitcoin/server.hpp | 2 + .../bitcoin/server/channels/channel_btcd.hpp | 80 +++ include/bitcoin/server/server_node.hpp | 2 + include/bitcoin/server/settings.hpp | 3 + 8 files changed, 766 insertions(+) create mode 100644 endpoints/test_btcd_rpc.py create mode 100644 include/bitcoin/server/channels/channel_btcd.hpp diff --git a/endpoints/README.md b/endpoints/README.md index 0110e899f..06e12f0cd 100644 --- a/endpoints/README.md +++ b/endpoints/README.md @@ -41,6 +41,11 @@ bind = 0.0.0.0:8332 bind = [::]:8332 connections = 10 +[btcd] +bind = 0.0.0.0:8334 +bind = [::]:8334 +connections = 10 + [electrum] bind = 0.0.0.0:50001 bind = [::]:50001 @@ -80,6 +85,9 @@ pytest test_native.py # Test only bitcoind RPC pytest test_bitcoind_rpc.py +# Test only btcd JSON-RPC/websocket +pytest test_btcd_rpc.py + # Test only Electrum protocol pytest test_electrum.py ``` @@ -130,6 +138,34 @@ pytest test_bitcoind_rpc.py \ - `--bitcoind-auth` — Enable authentication (cookie file) - `--bitcoind-cookie` — Path to cookie file (default: /mnt/core/.cookie) +### btcd JSON-RPC/WebSocket Options + +```bash +pytest test_btcd_rpc.py \ + --btcd-host=localhost \ + --btcd-port=8334 +``` + +**With basic auth / ws `authenticate` credentials configured:** + +```bash +pytest test_btcd_rpc.py \ + --btcd-username=user \ + --btcd-password=pass +``` + +**Available options:** +- `--btcd-host` — Host for btcd JSON-RPC/websocket (default: localhost) +- `--btcd-port` — Port for btcd JSON-RPC/websocket (default: 8334) +- `--btcd-username` / `--btcd-password` — Credentials to test `authenticate` + against, if the server has `btcd.username`/`btcd.password` configured + (default: none — the credential-specific tests are skipped) + +No extra Python dependency is required: the websocket client is a small +stdlib-only implementation (see `WebSocketConnection` in `test_btcd_rpc.py`), +matching this suite's existing preference for hand-rolled wire protocol over +Electrum's raw TCP handling. + ### Electrum Protocol Options ```bash @@ -221,6 +257,60 @@ pytest test_bitcoind_rpc.py \ pytest test_bitcoind_rpc.py -k "getblock" ``` +### test_btcd_rpc.py — btcd JSON-RPC/WebSocket Compatibility + +Tests the btcd-compatible endpoint: JSON-RPC 1.0 over a persistent websocket +connection (session/notification/filter/admin extension methods) plus plain +http post (chain methods inherited from `protocol_bitcoind_rpc`, same shape +as `bitcoind`). See `docs/btcd-endpoint.md` for the full design and phase +scope this suite tracks against. + +Unlike the other endpoint suites, this one is explicitly split into two +kinds of test: +- **No `xfail` marker** — asserts real, currently-implemented behavior. A + failure here is a regression. +- **`@pytest.mark.xfail(strict=False)`** — describes intended behavior that + isn't implemented (or not yet reachable over ws) yet. Expected to fail + today; flips to XPASS once implemented, which is the cue to tighten the + assertion and remove the marker. Use `pytest -m xfail -rx` to see the + current development checklist. + +**Coverage:** +- ✅ Session management — `authenticate` (no-op / credential match / credential + mismatch), `session` +- ✅ Block subscription — `notifyblocks`/`stopnotifyblocks` (ack), a real + `blockconnected` push-notification test (positional `[hash, height, time]`, + verified against upstream btcd's actual wire format) +- ✅ Chain methods over http post (positive control — `getbestblockhash`, + `getblockcount`, `getblockhash`, `getblockheader`, `gettxout`, + `getrawtransaction`) +- ✅ Deprecated methods stay rejected — `notifyreceived`, `stopnotifyreceived`, + `notifyspent`, `stopnotifyspent`, `rescan`; `stop` (permanent, no + dev-target xfail — these are guarded against ever silently "working") +- ✅ Response envelope — id echo across requests, unknown-method error + without dropping the connection +- 🚧 (xfail, phase B) Chain methods bridged into the *ws* connection itself +- 🚧 (xfail, phase B) `notifynewtransactions`/`stopnotifynewtransactions`, + `loadtxfilter`, `rescanblocks` +- 🚧 (xfail, phase B/C) `getcurrentnet`, `getdifficulty`, `getinfo`, + `getnettotals`, `getnetworkhashps`, `createrawtransaction`, + `decoderawtransaction`, `decodescript`, `validateaddress`, `help` + +**Example test runs:** + +```bash +pytest test_btcd_rpc.py +pytest test_btcd_rpc.py --btcd-port=8334 +pytest test_btcd_rpc.py -k "not xfail" # only currently-real behavior +pytest test_btcd_rpc.py -m xfail -rx # what's left to implement +BTCD_DEBUG=1 pytest test_btcd_rpc.py -s -k session + +# blockconnected waits for a real block; trigger one instead of waiting out +# the timeout (regtest/testnet), or just let it wait on a synced mainnet node +BTCD_TRIGGER_BLOCK="bitcoin-cli generatetoaddress 1
" \ + pytest test_btcd_rpc.py -k blockconnected --subscription-timeout=120 +``` + ### test_electrum.py — Electrum Protocol Tests Electrum Protocol JSON-RPC over TCP. All assertions are spec-driven — they @@ -508,6 +598,7 @@ Each test module supports a debug environment variable that enables pretty-print | `test_electrum.py` | `ELECTRUM_DEBUG=1` | `>>>` JSON-RPC payload / `<<<` response with elapsed time | | `test_native.py` | `NATIVE_DEBUG=1` | `>>> GET ` / `<<<` response JSON with elapsed time | | `test_bitcoind_rpc.py` | `BITCOIND_DEBUG=1` | `>>>` JSON-RPC payload / `<<<` response with elapsed time | +| `test_btcd_rpc.py` | `BTCD_DEBUG=1` | `>>>` JSON-RPC payload / `<<<` response over websocket | ```bash # Electrum — pretty-print all requests and responses diff --git a/endpoints/conftest.py b/endpoints/conftest.py index f28ce0a7a..8edfcd2de 100644 --- a/endpoints/conftest.py +++ b/endpoints/conftest.py @@ -65,6 +65,32 @@ def pytest_addoption(parser): help="Port for bitcoind REST (default: 8332)" ) + # btcd options + parser.addoption( + "--btcd-host", + action="store", + default="localhost", + help="Host for btcd JSON-RPC/websocket (default: localhost)" + ) + parser.addoption( + "--btcd-port", + action="store", + default="8334", + help="Port for btcd JSON-RPC/websocket (default: 8334)" + ) + parser.addoption( + "--btcd-username", + action="store", + default=None, + help="Username for btcd basic auth / ws authenticate (default: none configured)" + ) + parser.addoption( + "--btcd-password", + action="store", + default=None, + help="Password for btcd basic auth / ws authenticate (default: none configured)" + ) + # Electrum options parser.addoption( "--electrum-host", @@ -197,6 +223,19 @@ def bitcoind_rest_config(request): } +@pytest.fixture(scope="session") +def btcd_config(request): + """Configuration for btcd JSON-RPC/websocket tests.""" + return { + "host": request.config.getoption("--btcd-host"), + "port": int(request.config.getoption("--btcd-port")), + "username": request.config.getoption("--btcd-username"), + "password": request.config.getoption("--btcd-password"), + "timeout": float(request.config.getoption("--timeout")), + "subscription_timeout": float(request.config.getoption("--subscription-timeout")), + } + + @pytest.fixture(scope="session") def electrum_config(request): """Configuration for Electrum protocol tests.""" diff --git a/endpoints/test_btcd_rpc.py b/endpoints/test_btcd_rpc.py new file mode 100644 index 000000000..7e402546c --- /dev/null +++ b/endpoints/test_btcd_rpc.py @@ -0,0 +1,548 @@ +""" +Tests for libbitcoin-server btcd JSON-RPC/websocket compatibility interface. + +btcd speaks JSON-RPC 1.0 over a persistent websocket connection (preferred, +required for the session/notification extension methods) or plain HTTP POST +(same request/response shape as the bitcoind endpoint, for the chain methods +inherited from protocol_bitcoind_rpc). See docs/btcd-endpoint.md for the full +design and phased scope. + +This suite is split into what's real today and what's a development target: + + - Tests with no xfail marker assert real, currently-implemented behavior. + A failure here is a regression. + - Tests marked `@pytest.mark.xfail(strict=False)` describe the *intended* + behavior of something not yet implemented (or not yet reachable over + ws). They're expected to fail now. When a method is implemented, the + test starts passing (reported as XPASS, not a hard failure since + strict=False) -- that's the cue to tighten the assertion and drop the + marker. + +Run with: + pytest test_btcd_rpc.py + pytest test_btcd_rpc.py --btcd-host=localhost --btcd-port=8334 + pytest test_btcd_rpc.py -k "not xfail" # only currently-real behavior + pytest test_btcd_rpc.py -m xfail -rx # see what's left to implement +""" + +import base64 +import hashlib +import json +import os +import select +import socket +import struct +import time +import warnings +from typing import Any, Optional + +import pytest +import requests + +from utils import ReferenceData, TestConfig + +_WS_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" + + +# ─── Minimal stdlib websocket client (RFC 6455, text frames only) ───────────── +# No external websocket dependency is added here, matching the rest of this +# suite's preference for hand-rolled wire protocol over a client library (see +# ElectrumConnection in test_electrum_subscriptions.py for the same approach +# applied to Electrum's newline-delimited TCP protocol). + +class WebSocketConnection: + """Bare-bones RFC 6455 client: handshake + masked text frame I/O.""" + + def __init__(self, host: str, port: int, target: str = "/", + connect_timeout: float = 5.0): + self.sock = socket.create_connection((host, port), timeout=connect_timeout) + self.sock.settimeout(None) # blocking; timeouts handled via select() + self._buf = b"" + self._handshake(host, target) + + def _handshake(self, host: str, target: str) -> None: + key = base64.b64encode(os.urandom(16)).decode() + request = ( + f"GET {target} HTTP/1.1\r\n" + f"Host: {host}\r\n" + "Upgrade: websocket\r\n" + "Connection: Upgrade\r\n" + f"Sec-WebSocket-Key: {key}\r\n" + "Sec-WebSocket-Version: 13\r\n" + "\r\n" + ) + self.sock.sendall(request.encode("ascii")) + + headers = self._read_headers() + lines = headers.splitlines() + if not lines or " 101 " not in lines[0]: + raise ConnectionError(f"websocket upgrade rejected: {lines[:1]!r}") + + expected = base64.b64encode( + hashlib.sha1((key + _WS_GUID).encode("ascii")).digest() + ).decode() + accept = None + for line in lines[1:]: + name, _, value = line.partition(":") + if name.strip().lower() == "sec-websocket-accept": + accept = value.strip() + if accept != expected: + raise ConnectionError("websocket handshake accept key mismatch") + + def _read_headers(self) -> str: + while b"\r\n\r\n" not in self._buf: + chunk = self.sock.recv(4096) + if not chunk: + raise ConnectionError("connection closed during ws handshake") + self._buf += chunk + head, _, rest = self._buf.partition(b"\r\n\r\n") + self._buf = rest + return head.decode("iso-8859-1") + + def close(self) -> None: + try: + self.sock.close() + except OSError: + pass + + def _recv_exact(self, size: int, timeout_s: Optional[float]) -> Optional[bytes]: + while len(self._buf) < size: + if timeout_s is not None: + ready, _, _ = select.select([self.sock], [], [], timeout_s) + if not ready: + return None + chunk = self.sock.recv(65536) + if not chunk: + return None + self._buf += chunk + data, self._buf = self._buf[:size], self._buf[size:] + return data + + def send_text(self, message: str) -> None: + payload = message.encode("utf-8") + length = len(payload) + mask_key = os.urandom(4) + masked = bytes(b ^ mask_key[i % 4] for i, b in enumerate(payload)) + + header = bytearray([0x80 | 0x1]) # FIN=1, opcode=text + if length < 126: + header.append(0x80 | length) + elif length < 65536: + header.append(0x80 | 126) + header += struct.pack(">H", length) + else: + header.append(0x80 | 127) + header += struct.pack(">Q", length) + header += mask_key + + self.sock.sendall(bytes(header) + masked) + + def recv_message(self, timeout_s: Optional[float] = None) -> Optional[str]: + """Read one complete (possibly multi-frame) text message.""" + parts: list[bytes] = [] + deadline = time.monotonic() + timeout_s if timeout_s is not None else None + + while True: + remaining = None + if deadline is not None: + remaining = deadline - time.monotonic() + if remaining <= 0: + return None + + head = self._recv_exact(2, remaining) + if head is None: + return None + + fin = bool(head[0] & 0x80) + opcode = head[0] & 0x0F + masked = bool(head[1] & 0x80) + length = head[1] & 0x7F + + if length == 126: + ext = self._recv_exact(2, remaining) + if ext is None: + return None + length = struct.unpack(">H", ext)[0] + elif length == 127: + ext = self._recv_exact(8, remaining) + if ext is None: + return None + length = struct.unpack(">Q", ext)[0] + + mask_key = b"" + if masked: + mask_key = self._recv_exact(4, remaining) + if mask_key is None: + return None + + payload = self._recv_exact(length, remaining) if length else b"" + if payload is None: + return None + if masked: + payload = bytes(b ^ mask_key[i % 4] for i, b in enumerate(payload)) + + if opcode == 0x8: # close + return None + if opcode in (0x9, 0xA): # ping/pong: not a message, keep reading + continue + + parts.append(payload) + if fin: + return b"".join(parts).decode("utf-8", errors="replace") + + +# ─── btcd JSON-RPC 1.0 connection ────────────────────────────────────────────── + +class BtcdConnection: + """A single btcd websocket connection with JSON-RPC 1.0 request/response.""" + + def __init__(self, host: str, port: int, connect_timeout: float): + self.ws = WebSocketConnection(host, port, "/", connect_timeout) + self._next_id = 0 + + def close(self) -> None: + self.ws.close() + + def raw_rpc(self, method: str, params: Optional[list] = None, + timeout_s: float = 10.0) -> dict: + """Send a request and return the full parsed response, error or not.""" + payload = { + "jsonrpc": "1.0", + "id": self._next_id, + "method": method, + "params": params if params is not None else [], + } + self._next_id += 1 + + if os.getenv("BTCD_DEBUG"): + print(">>>", json.dumps(payload, indent=2), flush=True) + + self.ws.send_text(json.dumps(payload)) + raw = self.ws.recv_message(timeout_s) + if raw is None: + pytest.fail(f"{method}: no response (connection closed or timed out)") + + if os.getenv("BTCD_DEBUG"): + print(f"<<< {method}:", raw, flush=True) + + try: + return json.loads(raw) + except json.JSONDecodeError as e: + pytest.fail(f"Invalid JSON from server for {method}: {raw!r} -> {e}") + + def send_rpc(self, method: str, params: Optional[list] = None, + timeout_s: float = 10.0) -> dict: + """Like raw_rpc, but xfail()s on a valid json-rpc error response.""" + data = self.raw_rpc(method, params, timeout_s) + if "error" in data and data["error"] is not None: + pytest.xfail(f"Server sent valid error response: {data['error']}") + return data + + def read_notification(self, timeout_s: float) -> Optional[dict]: + """Read the next server-pushed notification (has 'method', no 'id').""" + deadline = time.monotonic() + timeout_s + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + return None + raw = self.ws.recv_message(remaining) + if raw is None: + return None + try: + obj = json.loads(raw) + except json.JSONDecodeError: + continue + if isinstance(obj, dict) and "method" in obj and "id" not in obj: + return obj + + +def http_rpc(config: dict, method: str, params: Optional[list] = None) -> dict: + """Plain http post json-rpc call to the same btcd endpoint (no ws upgrade). + + The chain methods inherited from protocol_bitcoind_rpc are reachable this + way today even though they aren't yet bridged into the ws dispatcher. + """ + payload = { + "jsonrpc": "2.0", + "id": 0, + "method": method, + "params": params if params is not None else [], + } + url = f"http://{config['host']}:{config['port']}/" + response = requests.post( + url, json=payload, + headers={"Content-Type": "application/json", "Connection": "close"}, + timeout=config.get("timeout", TestConfig.DEFAULT_RPC_TIMEOUT), + ) + response.raise_for_status() + return response.json() + + +# ─── Fixture ────────────────────────────────────────────────────────────────── + +@pytest.fixture +def conn(btcd_config: dict) -> BtcdConnection: + """Fresh btcd websocket connection per test.""" + host, port = btcd_config["host"], btcd_config["port"] + timeout = btcd_config.get("timeout", TestConfig.DEFAULT_SOCKET_TIMEOUT) + try: + c = BtcdConnection(host, port, connect_timeout=timeout) + except (OSError, ConnectionError) as exc: + pytest.skip(f"Cannot connect to btcd at {host}:{port}: {exc}") + yield c + c.close() + + +# ═══════════════════════════════════════════════════════════════════════════════ +# SESSION MANAGEMENT (implemented) +# ═══════════════════════════════════════════════════════════════════════════════ + +def test_authenticate_no_credentials_configured(conn, btcd_config): + """authenticate is a no-op success when the server has no configured + username/password, regardless of what's sent.""" + if btcd_config.get("username"): + pytest.skip("server has credentials configured; see " + "test_authenticate_with_configured_credentials") + response = conn.send_rpc("authenticate", ["anyuser", "anypass"]) + assert response.get("error") is None + + +def test_authenticate_with_configured_credentials(conn, btcd_config): + """When btcd.username/password are configured, authenticate must accept + the matching credentials and reject a mismatch (closing the connection). + """ + username = btcd_config.get("username") + password = btcd_config.get("password") + if not username: + pytest.skip("--btcd-username not set; server has no configured " + "credential to test against") + + response = conn.send_rpc("authenticate", [username, password]) + assert response.get("error") is None + + +def test_authenticate_wrong_password_rejected(btcd_config): + """A wrong password must be rejected and the connection closed.""" + username = btcd_config.get("username") + if not username: + pytest.skip("--btcd-username not set; nothing to mismatch against") + + host, port = btcd_config["host"], btcd_config["port"] + timeout = btcd_config.get("timeout", TestConfig.DEFAULT_SOCKET_TIMEOUT) + try: + c = BtcdConnection(host, port, connect_timeout=timeout) + except (OSError, ConnectionError) as exc: + pytest.skip(f"Cannot connect to btcd at {host}:{port}: {exc}") + + try: + data = c.raw_rpc("authenticate", [username, "definitely-wrong"]) + assert data.get("error") is not None + finally: + c.close() + + +def test_session_returns_id(conn): + """session returns an object containing an 'id' field (the channel + identifier), usable by clients to detect a reconnect to a fresh server. + """ + response = conn.send_rpc("session") + result = response.get("result") + assert isinstance(result, dict) + assert "id" in result + + +# ═══════════════════════════════════════════════════════════════════════════════ +# BLOCK SUBSCRIPTION (implemented) +# ═══════════════════════════════════════════════════════════════════════════════ + +def test_notifyblocks_acknowledges(conn): + response = conn.send_rpc("notifyblocks") + assert response.get("error") is None + + +def test_stopnotifyblocks_acknowledges(conn): + conn.send_rpc("notifyblocks") + response = conn.send_rpc("stopnotifyblocks") + assert response.get("error") is None + + +@pytest.mark.slow +def test_blockconnected_notification(conn, btcd_config): + """ + Subscribe via notifyblocks and wait for a real blockconnected push. + + Notification format (verified against btcsuite/btcd/btcjson): + {"method": "blockconnected", "params": [hash, height, time]} + + Requires a live node with a new block arriving during the timeout window. + Set BTCD_TRIGGER_BLOCK to a shell command that produces one (e.g. bumps + a regtest node), or just wait out the window on a synced mainnet node. + """ + sub_timeout = btcd_config.get("subscription_timeout", 60.0) + + ack = conn.send_rpc("notifyblocks") + assert ack.get("error") is None + + cmd = os.getenv("BTCD_TRIGGER_BLOCK") + if cmd: + os.system(cmd) + else: + print(f"\n Waiting up to {sub_timeout:.0f}s for a blockconnected " + "notification (set BTCD_TRIGGER_BLOCK to trigger one).", + flush=True) + + notif = conn.read_notification(timeout_s=sub_timeout) + if notif is None: + pytest.skip(f"No blockconnected notification within {sub_timeout:.0f}s. " + "Increase --subscription-timeout or set BTCD_TRIGGER_BLOCK.") + + assert notif.get("method") == "blockconnected" + params = notif.get("params", []) + assert isinstance(params, list) and len(params) == 3, ( + f"blockconnected params must be [hash, height, time], got {params!r}" + ) + block_hash, height, block_time = params + assert isinstance(block_hash, str) and len(block_hash) == 64 + assert isinstance(height, int) and height > 0 + assert isinstance(block_time, int) and block_time > 0 + + +# ═══════════════════════════════════════════════════════════════════════════════ +# STANDARD CHAIN METHODS -- reachable today only via plain http post. +# Development target: bridge these into the ws dispatcher (phase B). +# ═══════════════════════════════════════════════════════════════════════════════ + +CHAIN_METHODS = [ + ("getbestblockhash", []), + ("getblockcount", []), + ("getblockhash", [ReferenceData.GENESIS_HEIGHT]), + ("getblockheader", [ReferenceData.GENESIS_HASH]), + ("gettxout", [ReferenceData.KNOWN_TX_HASH, 0, False]), + ("getrawtransaction", [ReferenceData.FIRST_TX_HASH, 0]), +] + + +@pytest.mark.parametrize("method,params", CHAIN_METHODS) +def test_chain_method_over_http_post(btcd_config, method, params): + """Positive control: the chain logic itself works today, reachable over + plain http post to the same endpoint (unchanged from bitcoind).""" + data = http_rpc(btcd_config, method, params) + if data.get("error") is not None: + pytest.xfail(f"Server sent valid error response: {data['error']}") + assert "result" in data + + +@pytest.mark.xfail(reason="phase B: standard chain methods not yet bridged " + "into the ws dispatcher (see docs/btcd-endpoint.md)", + strict=False) +@pytest.mark.parametrize("method,params", CHAIN_METHODS) +def test_chain_method_over_websocket(conn, method, params): + """Development target: once bridged, chain methods should also work over + the same ws connection used for session/notifyblocks/etc -- this is what + a real lnd/btcwallet client needs (it can't open a second plain-http + connection once it has upgraded to ws).""" + response = conn.send_rpc(method, params) + assert "result" in response + + +# ═══════════════════════════════════════════════════════════════════════════════ +# WIRED BUT NOT YET IMPLEMENTED (phase B/C development targets) +# ═══════════════════════════════════════════════════════════════════════════════ +# These methods are present in interfaces/btcd.hpp's method table today, but +# every handler unconditionally returns not_implemented. Each xfail here is a +# checklist item: implement the handler and the test flips to XPASS. + +NOT_YET_IMPLEMENTED_STUBS = [ + ("notifynewtransactions", [False]), + ("stopnotifynewtransactions", []), + ("loadtxfilter", [False, [], []]), + ("rescanblocks", [[]]), +] + +# Deprecated upstream (superseded by loadtxfilter/rescanblocks) but still +# wired -- included so a regression can't silently start "working" in a way +# that contradicts the deliberate scope decision to leave these stubbed. +DEPRECATED_STUBS = [ + ("notifyreceived", [[]]), + ("stopnotifyreceived", [[]]), + ("notifyspent", [[]]), + ("stopnotifyspent", [[]]), + ("rescan", ["", [""], [""], ""]), +] + + +@pytest.mark.xfail(reason="phase B: wired stub, handler not yet implemented", + strict=False) +@pytest.mark.parametrize("method,params", NOT_YET_IMPLEMENTED_STUBS) +def test_stub_not_yet_implemented(conn, method, params): + data = conn.raw_rpc(method, params) + assert data.get("error") is None, ( + f"{method} still returns not_implemented: {data.get('error')}" + ) + + +@pytest.mark.parametrize("method,params", DEPRECATED_STUBS) +def test_deprecated_method_stays_not_implemented(conn, method, params): + """Regression guard, not a development target: these are deliberately + never implemented (superseded upstream by loadtxfilter/rescanblocks).""" + data = conn.raw_rpc(method, params) + assert data.get("error") is not None + + +def test_stop_always_not_implemented(conn): + """Regression guard, not a development target: no secure remote-shutdown + path exists, so stop is permanently rejected regardless of auth state.""" + data = conn.raw_rpc("stop") + assert data.get("error") is not None + + +# ═══════════════════════════════════════════════════════════════════════════════ +# NOT YET WIRED AT ALL (phase B/C development targets) +# ═══════════════════════════════════════════════════════════════════════════════ +# These aren't in interfaces/btcd.hpp's method table yet, so today they hit +# "unexpected method". Each is a checklist item for the generic-tooling +# compatibility phase (see docs/btcd-endpoint.md, Phase C). + +NOT_YET_WIRED = [ + ("getcurrentnet", []), + ("getdifficulty", []), + ("getinfo", []), + ("getnettotals", []), + ("getnetworkhashps", []), + ("createrawtransaction", [[], {}]), + ("decoderawtransaction", ["00"]), + ("decodescript", [""]), + ("validateaddress", [ReferenceData.EXAMPLE_ADDRESS]), + ("help", []), +] + + +@pytest.mark.xfail(reason="phase B/C: method not yet wired in interfaces/btcd.hpp", + strict=False) +@pytest.mark.parametrize("method,params", NOT_YET_WIRED) +def test_method_not_yet_wired(conn, method, params): + data = conn.raw_rpc(method, params) + assert data.get("error") is None, ( + f"{method} still unwired: {data.get('error')}" + ) + + +# ═══════════════════════════════════════════════════════════════════════════════ +# RESPONSE ENVELOPE / ERROR HANDLING (implemented) +# ═══════════════════════════════════════════════════════════════════════════════ + +def test_response_id_matches_request(conn): + r0 = conn.send_rpc("session") + r1 = conn.send_rpc("notifyblocks") + assert r0.get("id") == 0 + assert r1.get("id") == 1 + + +def test_unknown_method_errors_without_dropping_connection(conn): + """An unrecognized method must return a json-rpc error and keep the ws + connection open for the next request (verified by the follow-up call).""" + unknown = conn.raw_rpc("nosuchmethod") + assert unknown.get("error") is not None + + follow_up = conn.send_rpc("session") + assert follow_up.get("error") is None diff --git a/endpoints/utils.py b/endpoints/utils.py index affca9444..f828e39dd 100644 --- a/endpoints/utils.py +++ b/endpoints/utils.py @@ -136,6 +136,7 @@ class TestConfig: DEFAULT_BITCOIND_RPC_PORT = 8332 DEFAULT_BITCOIND_REST_PORT = 8332 DEFAULT_ELECTRUM_PORT = 50001 + DEFAULT_BTCD_PORT = 8334 DEFAULT_STRATUM_V1_PORT = 3333 DEFAULT_STRATUM_V2_PORT = 3336 diff --git a/include/bitcoin/server.hpp b/include/bitcoin/server.hpp index 669f2e2ed..5b39999f9 100644 --- a/include/bitcoin/server.hpp +++ b/include/bitcoin/server.hpp @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -49,6 +50,7 @@ #include #include #include +#include #include #include #include diff --git a/include/bitcoin/server/channels/channel_btcd.hpp b/include/bitcoin/server/channels/channel_btcd.hpp new file mode 100644 index 000000000..e5d978ade --- /dev/null +++ b/include/bitcoin/server/channels/channel_btcd.hpp @@ -0,0 +1,80 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#ifndef LIBBITCOIN_SERVER_CHANNELS_CHANNEL_BTCD_HPP +#define LIBBITCOIN_SERVER_CHANNELS_CHANNEL_BTCD_HPP + +#include +#include +#include +#include + +namespace libbitcoin { +namespace server { + +class BCS_API channel_btcd + : public server::channel, + public network::channel_http, + protected network::tracker +{ +public: + typedef std::shared_ptr ptr; + + inline channel_btcd(const network::logger& log, + const network::socket::ptr& socket, uint64_t identifier, + const node::configuration& config, const options_t& options) NOEXCEPT + : server::channel(log, socket, identifier, config), + network::channel_http(log, socket, identifier, config.network, options), + network::tracker(log) + { + } + + /// True once the client has authenticated over the ws 'authenticate' + /// handshake (or basic auth was not configured, requiring none). + inline bool authenticated() const NOEXCEPT + { + return authenticated_; + } + + inline void set_authenticated(bool value) NOEXCEPT + { + authenticated_ = value; + } + +protected: + using value_type = network::http::body::value_type; + + /// Preselect the json-rpc request parser for websocket frames, so that + /// each btcd ws message is parsed the same way as an http post body. + inline value_type websocket_body() const NOEXCEPT override + { + // There is no forwarding constructor so assign and move. + network::http::body::value_type value{}; + value = network::rpc::request{}; + return value; + } + +private: + // This is protected by strand. + bool authenticated_{}; +}; + +} // namespace server +} // namespace libbitcoin + +#endif diff --git a/include/bitcoin/server/server_node.hpp b/include/bitcoin/server/server_node.hpp index 604783c28..de89f39c6 100644 --- a/include/bitcoin/server/server_node.hpp +++ b/include/bitcoin/server/server_node.hpp @@ -72,6 +72,7 @@ class BCS_API server_node virtual session_admin::ptr attach_admin_session() NOEXCEPT; virtual session_native::ptr attach_native_session() NOEXCEPT; virtual session_bitcoind::ptr attach_bitcoind_session() NOEXCEPT; + virtual session_btcd::ptr attach_btcd_session() NOEXCEPT; virtual session_electrum::ptr attach_electrum_session() NOEXCEPT; virtual session_stratum_v1::ptr attach_stratum_v1_session() NOEXCEPT; virtual session_stratum_v2::ptr attach_stratum_v2_session() NOEXCEPT; @@ -84,6 +85,7 @@ class BCS_API server_node void start_admin(const code& ec, const result_handler& handler) NOEXCEPT; void start_native(const code& ec, const result_handler& handler) NOEXCEPT; void start_bitcoind(const code& ec, const result_handler& handler) NOEXCEPT; + void start_btcd(const code& ec, const result_handler& handler) NOEXCEPT; void start_electrum(const code& ec, const result_handler& handler) NOEXCEPT; void start_stratum_v1(const code& ec, const result_handler& handler) NOEXCEPT; void start_stratum_v2(const code& ec, const result_handler& handler) NOEXCEPT; diff --git a/include/bitcoin/server/settings.hpp b/include/bitcoin/server/settings.hpp index 985b3367e..e27438486 100644 --- a/include/bitcoin/server/settings.hpp +++ b/include/bitcoin/server/settings.hpp @@ -167,6 +167,9 @@ class BCS_API settings /// bitcoind compat interface (http/s, stateless json-rpc-v2) network::settings::http_server bitcoind{ "bitcoind" }; + /// btcd compat interface (http/s + websocket, json-rpc-v1) + network::settings::http_server btcd{ "btcd" }; + /// electrum compat interface (tcp/s, json-rpc-v2) electrum_server electrum{ "electrum" }; From 9b79dd32760e10e23c940fecb9bb0e239e89e04a Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Mon, 27 Jul 2026 03:34:13 +0200 Subject: [PATCH 02/10] Add btcd Boost.Test suite. New test/protocols/btcd/{btcd_rpc.cpp,btcd_setup_fixture.{hpp,cpp}}, in the style of test/protocols/bitcoind: a real websocket connection to a mock ten-block store, covering authenticate, session, notifyblocks/stopnotifyblocks, every not_implemented stub, response id echo across multiple requests on one connection, and unknown-method handling (json-rpc error reply, connection stays alive for the next request). --- test/protocols/btcd/btcd_rpc.cpp | 130 ++++++++++++++++++++ test/protocols/btcd/btcd_setup_fixture.cpp | 133 +++++++++++++++++++++ test/protocols/btcd/btcd_setup_fixture.hpp | 74 ++++++++++++ 3 files changed, 337 insertions(+) create mode 100644 test/protocols/btcd/btcd_rpc.cpp create mode 100644 test/protocols/btcd/btcd_setup_fixture.cpp create mode 100644 test/protocols/btcd/btcd_setup_fixture.hpp diff --git a/test/protocols/btcd/btcd_rpc.cpp b/test/protocols/btcd/btcd_rpc.cpp new file mode 100644 index 000000000..1f2b81d65 --- /dev/null +++ b/test/protocols/btcd/btcd_rpc.cpp @@ -0,0 +1,130 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include "../../test.hpp" +#include "btcd_setup_fixture.hpp" + +using namespace system; + +namespace { + +bool has_error(const boost::json::value& response) NOEXCEPT +{ + return response.is_object() && response.as_object().contains("error") && + !response.at("error").is_null(); +} + +bool has_result(const boost::json::value& response) NOEXCEPT +{ + return response.is_object() && response.as_object().contains("result") && + !response.at("result").is_null(); +} + +} // namespace + +BOOST_FIXTURE_TEST_SUITE(btcd_rpc_tests, btcd_ten_block_setup_fixture) + +// session management +// ---------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(btcd_rpc__authenticate__no_auth_configured__null_result) +{ + // Without basic auth configured, authenticate is a no-op success. + const auto response = rpc("authenticate", R"(["user","pass"])"); + BOOST_REQUIRE(!has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__session__returns_id) +{ + const auto response = rpc("session"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE(response.at("result").is_object()); + BOOST_REQUIRE(response.at("result").as_object().contains("id")); +} + +// block subscription +// ---------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(btcd_rpc__notifyblocks__returns_null_result) +{ + const auto response = rpc("notifyblocks"); + BOOST_REQUIRE(!has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__stopnotifyblocks__returns_null_result) +{ + // Subscribe first so stop is meaningful. + rpc("notifyblocks"); + const auto response = rpc("stopnotifyblocks"); + BOOST_REQUIRE(!has_error(response)); +} + +// Standard chain methods (getblockcount etc.) are inherited from +// protocol_bitcoind_rpc and remain reachable via plain http post on the same +// endpoint (see test/protocols/bitcoind), but are not yet bridged into this +// ws dispatcher -- see the phase B TODO on protocol_btcd_rpc.hpp. + +// not implemented stubs +// ---------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(btcd_rpc__not_implemented__error) +{ + const std::vector> methods + { + { "stop", "[]" }, + { "notifynewtransactions", "[false]" }, + { "stopnotifynewtransactions", "[]" }, + { "loadtxfilter", R"([false,[],[]])" }, + { "rescanblocks", "[[]]" }, + { "notifyreceived", "[[]]" }, + { "stopnotifyreceived", "[[]]" }, + { "notifyspent", "[[]]" }, + { "stopnotifyspent", "[[]]" }, + { "rescan", R"(["",[""],[""],""])" } + }; + + for (const auto& method: methods) + BOOST_REQUIRE_MESSAGE(has_error(rpc(method.first, method.second)), + method.first); +} + +// response envelope +// ---------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(btcd_rpc__response__id_matches_request) +{ + // request_id_ increments each call: first call uses id=0, second id=1. + const auto r0 = rpc("session"); + const auto r1 = rpc("notifyblocks"); + BOOST_REQUIRE_EQUAL(r0.at("id").as_int64(), 0); + BOOST_REQUIRE_EQUAL(r1.at("id").as_int64(), 1); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__unknown_method__error_keeps_connection_alive) +{ + // Unrecognized method returns a json-rpc error rather than dropping the + // ws connection (matches real btcd, verified by the follow-up call). + const auto unknown = rpc("nosuchmethod"); + BOOST_REQUIRE(has_error(unknown)); + + const auto follow_up = rpc("session"); + BOOST_REQUIRE(!has_error(follow_up)); +} + +BOOST_AUTO_TEST_SUITE_END() diff --git a/test/protocols/btcd/btcd_setup_fixture.cpp b/test/protocols/btcd/btcd_setup_fixture.cpp new file mode 100644 index 000000000..9b7a0398b --- /dev/null +++ b/test/protocols/btcd/btcd_setup_fixture.cpp @@ -0,0 +1,133 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include "../../test.hpp" +#include "../../mocks/blocks.hpp" +#include "btcd_setup_fixture.hpp" +#include +#include + +using namespace system; +using namespace boost::beast; + +BC_PUSH_WARNING(NO_THROW_IN_NOEXCEPT) + +btcd_setup_fixture::btcd_setup_fixture(const initializer& setup) + : config_ + { + system::chain::selection::mainnet, + test::web_pages, + test::web_pages + }, + store_ + { + [&]() NOEXCEPT -> const database::settings& + { + config_.database.path = TEST_DIRECTORY; + return config_.database; + }() + }, + query_{ store_ }, log_{}, + server_{ query_, config_, log_ } +{ + test::clear(test::directory); + + auto& database_settings = config_.database; + auto& network_settings = config_.network; + auto& node_settings = config_.node; + auto& btcd = config_.server.btcd; + + btcd.binds = { { BTCD_ENDPOINT } }; + btcd.connections = 1; + database_settings.interval_depth = 2; + node_settings.delay_inbound = false; + node_settings.minimum_fee_rate = 99.0; + network_settings.inbound.connections = 0; + network_settings.outbound.connections = 0; + + // Create and populate the store. + auto ec = store_.create([](auto, auto) {}); + BOOST_REQUIRE_MESSAGE(!ec, ec.message()); + setup(query_); + + // Run the server. + std::promise running{}; + server_.run([&](const code& ec) NOEXCEPT + { + running.set_value(ec); + }); + + // Block until server is running. + ec = running.get_future().get(); + BOOST_REQUIRE_MESSAGE(!ec, ec.message()); + + socket_.connect(btcd.binds.back().to_endpoint()); + + network::boost_code wec{}; + websocket_.text(true); + websocket_.handshake("localhost", "/", wec); + BOOST_REQUIRE_MESSAGE(!wec, wec.message()); +} + +btcd_setup_fixture::~btcd_setup_fixture() +{ + network::boost_code ec{}; + websocket_.close(websocket::close_code::normal, ec); + + // Expected and harmless during fixture teardown. + if (ec && + ec != boost::beast::websocket::error::closed && + ec != boost::asio::error::operation_aborted) + { + BOOST_WARN_MESSAGE(false, ec.message()); + } + + server_.close(); + ec = store_.close([](auto, auto){}); + BOOST_WARN_MESSAGE(!ec, ec.message()); + test::clear(test::directory); +} + +BC_POP_WARNING() + +boost::json::value btcd_setup_fixture::rpc(std::string_view method, + std::string_view params) +{ + std::ostringstream body{}; + body << R"({"jsonrpc":"1.0","id":)" << request_id_++ + << R"(,"method":")" << method + << R"(","params":)" << params << "}"; + + network::boost_code ec{}; + websocket_.write(net::buffer(body.str()), ec); + BOOST_REQUIRE_MESSAGE(!ec, ec.message()); + + flat_buffer buffer{}; + websocket_.read(buffer, ec); + BOOST_REQUIRE_MESSAGE(!ec, ec.message()); + return test::parse_json(buffers_to_string(buffer.data())); +} + +boost::json::value btcd_setup_fixture::receive_notification() +{ + flat_buffer buffer{}; + network::boost_code ec{}; + websocket_.read(buffer, ec); + BOOST_REQUIRE_MESSAGE(!ec, ec.message()); + return test::parse_json(buffers_to_string(buffer.data())); +} diff --git a/test/protocols/btcd/btcd_setup_fixture.hpp b/test/protocols/btcd/btcd_setup_fixture.hpp new file mode 100644 index 000000000..744dd78b2 --- /dev/null +++ b/test/protocols/btcd/btcd_setup_fixture.hpp @@ -0,0 +1,74 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#ifndef LIBBITCOIN_SERVER_TEST_PROTOCOLS_BTCD_BTCD_SETUP_FIXTURE +#define LIBBITCOIN_SERVER_TEST_PROTOCOLS_BTCD_BTCD_SETUP_FIXTURE + +#include "../../test.hpp" +#include "../../mocks/blocks.hpp" + +#define BTCD_ENDPOINT "127.0.0.1:65004" + +struct btcd_setup_fixture +{ + using initializer = std::function; + + DELETE_COPY_MOVE(btcd_setup_fixture); + explicit btcd_setup_fixture(const initializer& setup); + ~btcd_setup_fixture(); + + // JSON-RPC 1.0 over websocket. params must be a json array (json-rpc-v1 + // does not support 'null' or omitted params; use "[]" for no arguments). + // Returns the parsed json-rpc response object (result/error). + boost::json::value rpc(std::string_view method, + std::string_view params = "[]"); + + // Read one further (unprompted) server push, e.g. a blockconnected + // notification. Returns the parsed json-rpc notification object. + boost::json::value receive_notification(); + +protected: + configuration config_; + test::store_t store_; + test::query_t query_; + +private: + using tcp_stream = boost::beast::tcp_stream; + using websocket_stream = boost::beast::websocket::stream; + + network::logger log_; + server::server_node server_; + boost::asio::io_context io_{}; + tcp_stream socket_{ io_.get_executor() }; + websocket_stream websocket_{ socket_ }; + int request_id_{}; +}; + +struct btcd_ten_block_setup_fixture + : btcd_setup_fixture +{ + inline btcd_ten_block_setup_fixture() + : btcd_setup_fixture([](test::query_t& query) + { + return test::setup_ten_block_store(query); + }) + { + } +}; + +#endif From 2d9349ae9265215b584a9e4e6ba90bc5c83bf353 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Mon, 27 Jul 2026 03:41:21 +0200 Subject: [PATCH 03/10] Add btcd-compatible JSON-RPC/WebSocket endpoint (phase A). Adds channel_btcd (channel_http + per-connection auth state, preselects the json-rpc request parser for websocket frames) and protocol_btcd_rpc (inherits protocol_bitcoind_rpc for the standard chain handlers, adds a second dispatcher for btcd's session/notification/filter/admin extension methods over dispatch_websocket). Live: authenticate (validates against configured btcd.username/password, or no-op success if none configured), session (returns the channel identifier), notifyblocks/stopnotifyblocks (chase::organized/reorganized -> blockconnected/blockdisconnected, positional [hash, height, time] per real btcd wire format). Stubbed not_implemented: stop (no secure remote-shutdown path exists), notifynewtransactions/stopnotifynewtransactions (no mempool in v4), loadtxfilter/rescanblocks (phase B), the deprecated notifyreceived/ notifyspent/rescan family. Extends interfaces/btcd.hpp with a stop method. Wiring: channels.hpp, protocols.hpp, sessions.hpp, server_node.hpp/.cpp (start_btcd chained after bitcoind), parser.cpp ([btcd] config section mirroring [bitcoind]), builds/gnu/Makefile.am. Auth over websocket: basic auth is a per-http-request header, which ws data frames cannot carry (only the upgrade handshake has headers, and that request never reaches the auth check -- see channel_http's error::upgraded short-circuit). channel_btcd::unauthorized() bypasses the inherited per-request check for websocket traffic; the plain http post path keeps it unchanged. Enforcement over ws moves into dispatch_websocket instead: every method other than authenticate is rejected until the connection has authenticated. handle_authenticate's reject path also no longer calls stop() synchronously alongside the error send -- that raced the async write and could close the connection before the error response flushed. It's now threaded through as the SEND completion reason, so the close only happens once the write actually finishes. Known gap: standard chain methods (getblockcount etc.) remain reachable only via plain http post on the same endpoint, not yet bridged into the ws dispatcher -- see the phase B TODO on protocol_btcd_rpc.hpp. --- builds/gnu/Makefile.am | 5 + docs/btcd-endpoint.md | 247 +++++++++ .../bitcoin/server/channels/channel_btcd.hpp | 12 + include/bitcoin/server/channels/channels.hpp | 3 + include/bitcoin/server/interfaces/btcd.hpp | 14 +- .../server/protocols/protocol_btcd_rpc.hpp | 176 +++++++ .../bitcoin/server/protocols/protocols.hpp | 5 +- include/bitcoin/server/sessions/sessions.hpp | 2 + src/parser.cpp | 87 ++++ src/protocols/btcd/protocol_btcd_rpc.cpp | 493 ++++++++++++++++++ src/server_node.cpp | 21 + 11 files changed, 1059 insertions(+), 6 deletions(-) create mode 100644 docs/btcd-endpoint.md create mode 100644 include/bitcoin/server/protocols/protocol_btcd_rpc.hpp create mode 100644 src/protocols/btcd/protocol_btcd_rpc.cpp diff --git a/builds/gnu/Makefile.am b/builds/gnu/Makefile.am index feda0b550..6d17b4db5 100644 --- a/builds/gnu/Makefile.am +++ b/builds/gnu/Makefile.am @@ -64,6 +64,7 @@ src_libbitcoin_server_la_SOURCES = \ ${srcdir}/../../src/protocols/bitcoind/protocol_bitcoind_rest.cpp \ ${srcdir}/../../src/protocols/bitcoind/protocol_bitcoind_rpc.cpp \ ${srcdir}/../../src/protocols/bitcoind/protocol_bitcoind_rpc_json.cpp \ + ${srcdir}/../../src/protocols/btcd/protocol_btcd_rpc.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum_addresses.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum_fees.cpp \ @@ -108,6 +109,7 @@ include_bitcoin_server_channelsdir = \ include_bitcoin_server_channels_HEADERS = \ ${srcdir}/../../include/bitcoin/server/channels/channel.hpp \ + ${srcdir}/../../include/bitcoin/server/channels/channel_btcd.hpp \ ${srcdir}/../../include/bitcoin/server/channels/channel_electrum.hpp \ ${srcdir}/../../include/bitcoin/server/channels/channel_http.hpp \ ${srcdir}/../../include/bitcoin/server/channels/channel_stratum_v1.hpp \ @@ -155,6 +157,7 @@ include_bitcoin_server_protocols_HEADERS = \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_admin.hpp \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_bitcoind_rest.hpp \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_bitcoind_rpc.hpp \ + ${srcdir}/../../include/bitcoin/server/protocols/protocol_btcd_rpc.hpp \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_electrum.hpp \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_electrum_version.hpp \ ${srcdir}/../../include/bitcoin/server/protocols/protocol_html.hpp \ @@ -217,6 +220,8 @@ test_libbitcoin_server_test_SOURCES = \ ${srcdir}/../../test/protocols/bitcoind/bitcoind_rest.cpp \ ${srcdir}/../../test/protocols/bitcoind/bitcoind_rpc.cpp \ ${srcdir}/../../test/protocols/bitcoind/bitcoind_setup_fixture.cpp \ + ${srcdir}/../../test/protocols/btcd/btcd_rpc.cpp \ + ${srcdir}/../../test/protocols/btcd/btcd_setup_fixture.cpp \ ${srcdir}/../../test/protocols/electrum/electrum_addresses.cpp \ ${srcdir}/../../test/protocols/electrum/electrum_disabled.cpp \ ${srcdir}/../../test/protocols/electrum/electrum_fees.cpp \ diff --git a/docs/btcd-endpoint.md b/docs/btcd-endpoint.md new file mode 100644 index 000000000..5188d1443 --- /dev/null +++ b/docs/btcd-endpoint.md @@ -0,0 +1,247 @@ +# btcd-Compatible Endpoint + +This document specifies the `btcd` JSON-RPC/WebSocket compatibility endpoint for +libbitcoin-server, modeled on the existing `bitcoind` endpoint (see +`src/protocols/bitcoind/`, added in #795) and on +[btcd's JSON-RPC API](https://btcd.readthedocs.io/en/latest/json_rpc_api.html). + +## Background + +btcd is a Bitcoin full node written in Go that intentionally separates wallet and +chain services into independent processes: unlike Core/bitcoind, a direct +connection to btcd exposes only chain RPCs, never wallet methods. That matches +libbitcoin-server's own scope (no wallet), which is what makes btcd a good second +compatibility target after bitcoind. + +btcd exposes two transports on the same port (default `8334`): + +- **HTTP POST**, one JSON-RPC request/response per connection (same shape as the + existing `bitcoind` endpoint). +- **WebSocket** (`/ws`), btcd's preferred transport: a single connection carries + many requests plus asynchronous push notifications (new blocks, filtered + transactions, rescan progress). + +Real btcd authenticates via TLS + HTTP Basic Auth, with two credential tiers: +`rpcuser`/`rpcpass` (full access) and `rpclimituser`/`rpclimitpass` (restricted to +a fixed allowlist of chain-read methods). **Decision: this endpoint uses a single +credential tier** (reusing `network::settings::http_server`'s existing +`username`/`password`/`authenticated()`, same as `bitcoind.*`). Sensitive methods +(`stop`, mining/peer-management) are stubbed to `not_implemented` unconditionally, +so a limited-user tier adds no security value here; it can be revisited if a +future phase implements a real `stop` path. + +## Architecture + +``` +network::protocol_http +└── server::protocol_bitcoind_rpc + └── server::protocol_btcd_rpc (new) + +network::channel_http +└── server::channel_http + └── server::channel_btcd (new) + +server::session_btcd = session_server (new) +``` + +`protocol_btcd_rpc` inherits `protocol_bitcoind_rpc`, which already implements the +chain-read/rawtx handlers both APIs share (`getbestblockhash`, `getblock`, +`getblockhash`, `getblockheader`, `getblockcount`, `gettxout`, `getrawtransaction`, +`sendrawtransaction`, ...). It adds a second dispatcher for `interface::btcd` +(btcd-only methods) and overrides `network::protocol_http::dispatch_websocket()` — +the extension point `protocol_http` already exposes for handling an HTTP→WS +upgrade — to route btcd's WS-only notification/subscription methods. + +`channel_btcd` extends `channel_http` with an `authenticated_` flag, set once the +WS `authenticate` command (or HTTP Basic Auth header) succeeds. + +Notification/subscription state (watched blocks, watched scripts/outpoints) +follows the pattern already established in `protocol_electrum` +(`src/protocols/electrum/protocol_electrum.cpp`): a small map guarded by a +dedicated notification strand, populated by `node::chase` event callbacks, fed to +clients via `do_`/`complete_*` handlers. Model the btcd notifier on that +shape rather than inventing a new one. + +New files: + +- `include/bitcoin/server/channels/channel_btcd.hpp` +- `include/bitcoin/server/protocols/protocol_btcd_rpc.hpp` +- `include/bitcoin/server/interfaces/btcd.hpp` (method table, see + `interfaces/bitcoind_rpc.hpp` for the pattern) +- `src/protocols/btcd/protocol_btcd_rpc.cpp` +- `src/protocols/btcd/protocol_btcd_rpc_subscribe.cpp` +- `test/protocols/btcd/btcd_rpc.cpp` + +Modified: `channels.hpp`, `protocols.hpp`, `sessions.hpp`, `settings.hpp`, +`server_node.hpp`/`.cpp`, `parser.cpp` (all following the `bitcoind` wiring as a +template). + +## Method Scope + +### Already covered (inherited from `protocol_bitcoind_rpc`) + +`getbestblockhash`, `getblock`, `getblockhash`, `getblockheader`, `getblockcount`, +`gettxout`, `getrawtransaction`, `sendrawtransaction`. No btcd-specific work +needed; response shapes already match Core/btcd conventions closely enough for +Phase A. **Caveat verified during Phase A implementation**: these are +currently reachable only via plain http post to the same endpoint (unchanged +from `bitcoind`), not yet over the ws connection itself. `dispatch_websocket` +only routes to the btcd-only extension dispatcher; bridging the inherited +handlers' post-oriented private send path (which caches the original +`http::request` for header derivation, unavailable for a ws frame) into a +ws-reachable form is real work, scheduled for Phase B. Until then, a +ws-connected client (required for `authenticate`/`session`/`notifyblocks`) +cannot also call `getblockcount` etc. on that same connection — a real +lnd/btcwallet client would need this bridged to be useful in practice. + +### Phase A — wiring + +`channel_btcd`, `protocol_btcd_rpc`, `authenticate`, `session` (must be +functionally correct, not stubbed — see [lnd Compatibility](#lnd-compatibility)), +`notifyblocks`/`stopnotifyblocks`, `not_implemented` stubs for every other btcd +method, Boost.Test suite. + +### Phase B — lnd-critical, promoted ahead of generic filler + +`getcurrentnet` (network-match check lnd/btcwallet performs once at connect), +`loadtxfilter`, `rescanblocks` (guard behind `address_enabled`, same guard used +by the address-indexing-dependent electrum methods). None of these are on the +critical path for a generic btcd client — they're promoted here specifically +because `btcwallet`'s `chain.RPCClient` and lnd's own `chainntnfs/btcdnotify` +depend on them (see [lnd Compatibility](#lnd-compatibility)). + +### Phase C — remaining plain RPC methods (generic btcd-tooling compatibility) + +`getdifficulty`, `getinfo`, `getnettotals`, `getnetworkhashps`, +`createrawtransaction`, `decoderawtransaction`, `decodescript`, +`validateaddress`, `help`. No specific named consumer — these round out +compatibility for generic btcd-speaking tooling (block explorers, mining-pool +stats, monitoring dashboards, tx-building/debug scripts), not lnd. + +### Permanently stubbed (`not_implemented`, all phases) + +- **`stop`** — no secure remote-shutdown path exists in libbitcoin-server; always + returns `not_implemented` regardless of auth tier or config. +- **`notifynewtransactions`**, and the deprecated WS methods `notifyreceived`, + `stopnotifyreceived`, `notifyspent`, `stopnotifyspent`, `rescan` (all superseded + upstream by `loadtxfilter`/`rescanblocks`) — no mempool exists in v4, revisit in + v5. +- **`submitblock`** — future-phase only, not in current scope. +- Mining (`getgenerate`, `gethashespersec`, `getmininginfo`, `setgenerate`) and + peer-management (`addnode`, `getaddednodeinfo`, `getconnectioncount`, + `getpeerinfo`) — out of scope; libbitcoin-server is not a miner and peer + management is a node-config concern, not an RPC surface here. + +## lnd Compatibility + +btcd's own websocket API is one of lnd's three supported chain backends (the +others being `bitcoind` via ZMQ, and `neutrino`). Hooking an `lnd` node up to +libbitcoin-server through this endpoint means satisfying the calls +`btcwallet`'s `chain.RPCClient` (`btcsuite/btcwallet/chain/btcd.go`) and lnd's own +`chainntnfs/btcdnotify` package make against a btcd-style server. Verified against +both sources: + +**Plain RPC** (all either inherited from bitcoind or trivial to add): +`getbestblockhash`/`getblockcount` (chain tip), `getblockhash`, `getblockheader` +(chain-view / `IsCurrent` checks), `getblock` (verbose, block scanning), +`getrawtransaction`, `sendrawtransaction` (broadcasting funding/commitment/ +sweep/justice transactions), `gettxout` (UTXO-existence checks), `getcurrentnet` +(network-match validation performed once at connect — scheduled in Phase B). + +**WebSocket notification surface — the real gap.** Both `btcwallet`'s RPCClient +and lnd's `btcdnotify` depend on: + +- `NotifyBlocks` → `OnBlockConnected`/`OnBlockDisconnected` — covered by Phase A. +- A working `session` on connect — `btcwallet` uses the session id to detect a + reconnect to a fresh server instance, which must trigger re-registration of all + filters/rescans. If this is left as a stub, reconnect handling is undefined; it + should be implemented for real in Phase A, not deferred. +- **`loadtxfilter`** (address/outpoint watch-list registration) plus + block-filtered notification delivery (`OnFilteredBlockConnected`/ + `OnRedeemingTx`/`OnRecvTx` equivalents) — this is the mechanism both the wallet + and lnd's own notifier use to detect channel-funding confirmations and + on-chain spends of channel outputs (breach/force-close detection). It is not + optional legacy behavior; modern `btcwallet`/`lnd` don't fall back to the + deprecated `notifyreceived`/`notifyspent` path. This is why it's scheduled in + Phase B, ahead of Phase C's generic filler methods — it's more load-bearing + for lnd than any Phase C method. +- **`rescan`/`rescanblocks`** — used at wallet startup (birthday scan) and after + any downtime to replay historical blocks against the loaded filter. Also + scheduled in Phase B for the same reason. + +**Hard blocker: mempool.** Both sources confirm unconfirmed-transaction awareness +is an exercised, not merely optional, code path: `OnRecvTx`/`OnRedeemingTx` fire +for mempool-resident transactions (distinguished by a nil `BlockDetails`), and +lnd's notifier has a dedicated mempool-spend lookup path +(`LookupInputMempoolSpend`) used for fast breach/force-close reaction ahead of +confirmation. **libbitcoin-server v4 has no mempool**, so this notification class +cannot be implemented now — confirmed-only fallback is straightforward (fire the +same callbacks only on block connection), but it means lnd would lose +unconfirmed-spend awareness and any zero-conf-style UX. Final safety in Lightning +still rests on confirmation + CSV/CLTV timeout margins, so this is a +responsiveness/UX degradation rather than a correctness break, but it's a real, +user-visible limitation worth stating plainly rather than glossing over. Revisit +once a v5 mempool exists. + +**Not needed for lnd**: fee estimation — lnd's default fee estimator for the +`btcd` backend is a web API (`chainfee.WebAPIEstimator`), not an RPC call to the +node, so `estimatesmartfee`-equivalent support is not on the critical path. + +## Configuration + +New `[btcd]` section, mirroring `[bitcoind]`'s shape in `parser.cpp`/ +`settings.hpp` (`bind`, `safe`, `cert_auth`, `cert_path`, `key_path`, `key_pass`, +`username`, `password`, `connections`, `inactivity_minutes`, +`expiration_minutes`, `minimum_buffer`, `maximum_request`, `host`, `origin`, +`allow_opaque_origin`). No dedicated `stop`-gating config key: `stop` has no +conditional path at all right now (always `not_implemented`, unconditionally), +so a flag controlling it would be dead configuration. If a real guarded `stop` +path is ever implemented, add the gating key then. + +Default port: `8334` (matches upstream btcd). + +## Code Conventions (apply from the outset, per #795 review feedback) + +The `bitcoind` PR (#795) went through several review rounds with @evoskuil before +merging; apply these proactively on the btcd PR to avoid repeating the same +cycle: + +- No inline (header-implemented) non-template method bodies — put them in + `.cpp` files; keeps build/rebuild times down for a developer-facing lib. +- Protocol-specific json/serialization helpers are protected static methods on + the base protocol class (inherited by subclasses), not free functions in the + `server` namespace. Split into a `_json.cpp` for organization if it grows. +- Pass `hash_cptr` and other small shared_ptr objects as `const&`, never by + value — a copy needlessly bumps the refcount and hits a shared lock. +- Prefer `to_shared(std::move(x))` over manual move-then-construct. +- Use `std::from_chars` (C++17) directly; add an explicit leading-zero guard if + canonical parsing matters (`from_chars` alone accepts `"01"`). +- Arithmetic: `add1()`, `floored_subtract()` — never raw `-`, the store is + concurrent and `top >= height` is not assured — `possible_sign_cast<>`, + `possible_wide_cast<>`, `is_lesser()` over raw casts, + `ceilinged_add()`/`is_overflow()` for additions that could overflow, + `std::next()` instead of pointer arithmetic. +- Cache constants that are invariant for the process lifetime (e.g. genesis + hash lookups) as function-local `static`; use `constexpr base16_hash("...")` + instead of runtime `encode_hash`/string comparisons in hot paths. +- Use the query layer's existing fluent methods instead of re-deriving values: + `get_confirmed_headers` + `get_wire_header` instead of hand-looping + `chain::header` objects, `get_tx_height(height, link)` alone instead of extra + containment checks, the stored context field (single `get_context` query) for + median-time-past instead of recomputing over 11 headers. +- Stream into one pre-sized buffer (`stream::out::fast` + a typed writer) + instead of looping and copying per item. Don't mask a per-item store failure + inside such a loop as "not found" — that's `database::error::integrity` + (`send_internal_server_error`), a different failure class. +- Naming: `send_text()`/`send_json()`/`to_data()`/`to_text()` — "data" means + byte array, "text" means base16 string; avoid "hex"/"bin" abbreviations. +- Pure type→json serialization belongs in **libbitcoin-system** as a + `chain/json/` serializer (see `system/chain/json/block.hpp`+`.cpp`+ + test), not hand-rolled long-term in the protocol layer. File a system-repo PR + if a new btcd-only json shape is needed, rather than keeping it local. +- Tests: Boost.Test only for Unit/Component/Functional tiers, no external + dependencies (no Python) — that's `test/protocols/btcd/`. Data-driven loops + only over genuine external vectors; no line-wrapping; precompute `constexpr` + hashes/expectations at file scope. The Python suite under `endpoints/` is a + separate, optional **acceptance** tier (run against a live compiled node), + not a merge requirement. diff --git a/include/bitcoin/server/channels/channel_btcd.hpp b/include/bitcoin/server/channels/channel_btcd.hpp index e5d978ade..c36fce257 100644 --- a/include/bitcoin/server/channels/channel_btcd.hpp +++ b/include/bitcoin/server/channels/channel_btcd.hpp @@ -69,6 +69,18 @@ class BCS_API channel_btcd return value; } + /// Basic auth is a per-http-request header, which websocket data frames + /// structurally cannot carry (only the upgrade handshake has headers, + /// and that request never reaches this check -- see channel_http's + /// error::upgraded short-circuit). Once upgraded, auth is instead + /// enforced in-band by the ws 'authenticate' method (see + /// protocol_btcd_rpc::dispatch_websocket); the plain http post path + /// (e.g. inherited chain methods) keeps the normal per-request check. + inline bool unauthorized(const network::http::request& request) NOEXCEPT override + { + return websocket() ? false : network::channel_http::unauthorized(request); + } + private: // This is protected by strand. bool authenticated_{}; diff --git a/include/bitcoin/server/channels/channels.hpp b/include/bitcoin/server/channels/channels.hpp index 0a3cf0d6a..8698a76f9 100644 --- a/include/bitcoin/server/channels/channels.hpp +++ b/include/bitcoin/server/channels/channels.hpp @@ -20,6 +20,7 @@ #define LIBBITCOIN_SERVER_CHANNELS_CHANNELS_HPP #include +#include #include #include #include @@ -39,6 +40,7 @@ network::channel │ └── [server::channel_electrum] ├── channel_http │ ├── [server::channel_http] +│ └── [server::channel_btcd] └── channel_peer └── [node::channel_peer] @@ -51,5 +53,6 @@ server::channel → node::channel ├── channel_stratum_v1 → network::channel_rpc ├── channel_electrum → network::channel_rpc ├── channel_http → network::channel_http +├── channel_btcd → network::channel_http */ diff --git a/include/bitcoin/server/interfaces/btcd.hpp b/include/bitcoin/server/interfaces/btcd.hpp index 616cb9055..989926c83 100644 --- a/include/bitcoin/server/interfaces/btcd.hpp +++ b/include/bitcoin/server/interfaces/btcd.hpp @@ -49,6 +49,9 @@ struct btcd_methods method<"loadtxfilter", boolean_t, value_t, value_t>{ "reload", "addresses", "outpoints" }, method<"rescanblocks", value_t>{ "blockhashes" }, + /// Admin (not_implemented; no secure remote-shutdown path exists). + method<"stop">{}, + /// Deprecated (address/outpoint filtering). method<"notifyreceived", value_t>{ "addresses" }, method<"stopnotifyreceived", value_t>{ "addresses" }, @@ -72,11 +75,12 @@ struct btcd_methods using stop_notify_new_transactions = at<5>; using load_tx_filter = at<6>; using rescan_blocks = at<7>; - using notify_received = at<8>; - using stop_notify_received = at<9>; - using notify_spent = at<10>; - using stop_notify_spent = at<11>; - using rescan = at<12>; + using stop = at<8>; + using notify_received = at<9>; + using stop_notify_received = at<10>; + using notify_spent = at<11>; + using stop_notify_spent = at<12>; + using rescan = at<13>; }; } // namespace interface diff --git a/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp new file mode 100644 index 000000000..89e0e6e46 --- /dev/null +++ b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp @@ -0,0 +1,176 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#ifndef LIBBITCOIN_SERVER_PROTOCOLS_PROTOCOL_BTCD_RPC_HPP +#define LIBBITCOIN_SERVER_PROTOCOLS_PROTOCOL_BTCD_RPC_HPP + +#include +#include +#include +#include +#include +#include + +namespace libbitcoin { +namespace server { + +/// btcd-compatible json-rpc-v1 over websocket. Inherits the standard chain +/// handlers (getblockcount, getblock, etc.) from protocol_bitcoind_rpc, which +/// remain reachable via the inherited http post path (unchanged). Adds the +/// btcd-only extension methods (session/notification/filter/admin) over +/// websocket via a second dispatcher (dispatch_websocket, below). +/// TODO (phase B): bridge the standard chain handlers into the ws dispatcher +/// too, so a single ws connection can also reach them (real btcd/btcwallet +/// clients expect this); requires reconciling protocol_bitcoind_rpc's +/// post-oriented private send path with this class's ws-oriented senders. +class BCS_API protocol_btcd_rpc + : public server::protocol_bitcoind_rpc, + protected network::tracker +{ +public: + // Replace channel_t so session_server creates channel_btcd instances. + using channel_t = channel_btcd; + + typedef std::shared_ptr ptr; + using btcd_interface = interface::btcd; + using btcd_dispatcher = network::rpc::dispatcher; + + inline protocol_btcd_rpc(const auto& session, + const network::channel::ptr& channel, + const options_t& options) NOEXCEPT + : server::protocol_bitcoind_rpc(session, channel, options), + network::tracker(session->log), + options_(options), + btcd_channel_(std::dynamic_pointer_cast(channel)) + { + } + + /// Configuration options (for basic-auth credential comparison). + inline const options_t& options() const NOEXCEPT + { + return options_; + } + + void start() NOEXCEPT override; + void stopping(const code& ec) NOEXCEPT override; + +protected: + /// Dispatch btcd ws frames (standard chain requests dispatch normally + /// over http post via the inherited handle_receive_post). + void dispatch_websocket( + const network::http::request& request) NOEXCEPT override; + + /// Handlers (session/handshake). + bool handle_authenticate(const code& ec, btcd_interface::authenticate, + const std::string& username, const std::string& password) NOEXCEPT; + bool handle_session(const code& ec, btcd_interface::session) NOEXCEPT; + + /// Handlers (block subscription). + bool handle_notify_blocks(const code& ec, + btcd_interface::notify_blocks) NOEXCEPT; + bool handle_stop_notify_blocks(const code& ec, + btcd_interface::stop_notify_blocks) NOEXCEPT; + + /// Handlers (mempool subscription, not_implemented pending v5 mempool). + bool handle_notify_new_transactions(const code& ec, + btcd_interface::notify_new_transactions, bool verbose) NOEXCEPT; + bool handle_stop_notify_new_transactions(const code& ec, + btcd_interface::stop_notify_new_transactions) NOEXCEPT; + + /// Handlers (address/outpoint filtering, not_implemented pending phase B). + bool handle_load_tx_filter(const code& ec, + btcd_interface::load_tx_filter, bool reload, + const network::rpc::value_t& addresses, + const network::rpc::value_t& outpoints) NOEXCEPT; + bool handle_rescan_blocks(const code& ec, + btcd_interface::rescan_blocks, + const network::rpc::value_t& blockhashes) NOEXCEPT; + + /// Handler (admin, permanently not_implemented). + bool handle_stop(const code& ec, btcd_interface::stop) NOEXCEPT; + + /// Handlers (deprecated, permanently not_implemented). + bool handle_notify_received(const code& ec, + btcd_interface::notify_received, + const network::rpc::value_t& addresses) NOEXCEPT; + bool handle_stop_notify_received(const code& ec, + btcd_interface::stop_notify_received, + const network::rpc::value_t& addresses) NOEXCEPT; + bool handle_notify_spent(const code& ec, + btcd_interface::notify_spent, + const network::rpc::value_t& outpoints) NOEXCEPT; + bool handle_stop_notify_spent(const code& ec, + btcd_interface::stop_notify_spent, + const network::rpc::value_t& outpoints) NOEXCEPT; + bool handle_rescan(const code& ec, btcd_interface::rescan, + const std::string& beginblock, const network::rpc::value_t& addresses, + const network::rpc::value_t& outpoints, + const std::string& endblock) NOEXCEPT; + + /// Chase event subscription (block connect/disconnect for notifyblocks). + bool handle_chase(const code& ec, node::chase event_, + node::event_value value) NOEXCEPT; + void do_block_connected(node::header_t link) NOEXCEPT; + void do_block_disconnected(node::header_t link) NOEXCEPT; + + /// Senders (btcd ws envelope, distinct id/version cache from the http + /// json-rpc-v2 senders inherited from protocol_bitcoind_rpc). + void send_btcd_result(network::rpc::value_option&& result, + size_t size_hint) NOEXCEPT; + void send_btcd_error(const code& ec) NOEXCEPT; + void send_btcd_error(const code& ec, size_t size_hint) NOEXCEPT; + + /// Sends the error and closes the channel once the write completes (not + /// synchronously) -- use for cases (e.g. failed authenticate) where the + /// channel must not survive the error, without racing the async send. + void send_btcd_error(const code& ec, size_t size_hint, + const code& close_reason) NOEXCEPT; + + void send_btcd_notification(const std::string& method, + network::rpc::array_t&& params, size_t size_hint) NOEXCEPT; + +private: + template + inline void btcd_subscribe(Method&& method, Args&&... args) NOEXCEPT + { + btcd_dispatcher_.subscribe(BIND_SHARED(method, args)); + } + + // Senders. + void send_btcd_rpc(network::rpc::response_t&& model, size_t size_hint, + const code& close_reason=network::error::success) NOEXCEPT; + + // These are thread safe. + std::atomic_bool subscribed_blocks_{}; + + // These are protected by strand. + btcd_dispatcher btcd_dispatcher_{}; + network::rpc::version btcd_version_{}; + network::rpc::id_option btcd_id_{}; + + // These are thread safe. + const options_t& options_; + + // This is mostly thread safe, and used in a thread safe manner. + const channel_btcd::ptr btcd_channel_; +}; + +} // namespace server +} // namespace libbitcoin + +#endif diff --git a/include/bitcoin/server/protocols/protocols.hpp b/include/bitcoin/server/protocols/protocols.hpp index 5a7242544..0f4d6bc9f 100644 --- a/include/bitcoin/server/protocols/protocols.hpp +++ b/include/bitcoin/server/protocols/protocols.hpp @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -49,6 +50,7 @@ network::protocol │ └── [server::protocol_rpc] ├── protocol_http │ ├── [server::protocol_bitcoind_rpc] +│ │ └── [server::protocol_btcd_rpc] │ └── [server::protocol_http] └── protocol_peer ├── [node::protocol_peer] @@ -93,6 +95,7 @@ server::protocol → node::protocol │ ├── protocol_admin │ └── protocol_native └── protocol_bitcoind_rpc - └── protocol_bitcoind_rest + ├── protocol_bitcoind_rest + └── protocol_btcd_rpc */ diff --git a/include/bitcoin/server/sessions/sessions.hpp b/include/bitcoin/server/sessions/sessions.hpp index 8f6a90b5b..6b143a392 100644 --- a/include/bitcoin/server/sessions/sessions.hpp +++ b/include/bitcoin/server/sessions/sessions.hpp @@ -32,6 +32,7 @@ namespace server { using session_admin = session_server; using session_native = session_server; using session_bitcoind = session_server; +using session_btcd = session_server; using session_stratum_v1 = session_server; using session_stratum_v2 = session_server; using session_electrum = session_handshake ╞══ session_native = server::session_server ╞══ session_bitcoind = server::session_server + ╞══ session_btcd = server::session_server ╞══ session_stratum_v1 = server::session_server ╞══ session_stratum_v2 = server::session_server └── server::session_handshake<...Protocols> diff --git a/src/parser.cpp b/src/parser.cpp index 4647e12e1..0136ef817 100644 --- a/src/parser.cpp +++ b/src/parser.cpp @@ -1093,6 +1093,93 @@ options_metadata parser::load_settings() THROWS "Allow requests from opaque origin (see CORS), multiple allowed, defaults to false." ) + /* [btcd] */ + ( + "btcd.bind", + value(&configured.server.btcd.binds), + "IP address to bind, multiple allowed, defaults to empty (disabled)." + ) + ( + "btcd.safe", + value(&configured.server.btcd.safes), + "IP address to secure bind, multiple allowed, defaults to empty (disabled)." + ) + ( + "btcd.cert_auth", + value(&configured.server.btcd.cert_auth), + "The certificate authority directory (*.PEM), enables client authentication." + ) + ( + "btcd.cert_path", + value(&configured.server.btcd.cert_path), + "The path to the server certificate file (.PEM), defaults to unused." + ) + ( + "btcd.key_path", + value(&configured.server.btcd.key_path), + "The path to the server private key file (.PEM), defaults to unused." + ) + ( + "btcd.key_pass", + value(&configured.server.btcd.key_pass), + "The password to decrypt the server private key file (.PEM), optional." + ) + ( + "btcd.username", + value(&configured.server.btcd.username), + "The basic authorization username (not secure), also used for the ws 'authenticate' method." + ) + ( + "btcd.password", + value(&configured.server.btcd.password), + "The basic authorization password (not secure), also used for the ws 'authenticate' method." + ) + ( + "btcd.connections", + value(&configured.server.btcd.connections), + "The required maximum number of connections, defaults to '0'." + ) + ( + "btcd.inactivity_minutes", + value(&configured.server.btcd.inactivity_minutes), + "The idle timeout (http/ws keep-alive), defaults to '10'." + ) + ( + "btcd.expiration_minutes", + value(&configured.server.btcd.expiration_minutes), + "The idle timeout (http/ws keep-alive), defaults to '60'." + ) + ( + "btcd.minimum_buffer", + value(&configured.server.btcd.minimum_buffer), + "The minimum retained read buffer size, defaults to '4000000'." + ) + ( + "btcd.maximum_request", + value(&configured.server.btcd.maximum_request), + "The maximum allowed request size, defaults to '4000000'." + ) + ( + "btcd.server", + value(&configured.server.btcd.server), + "The server name (http header), defaults to '" BC_HTTP_SERVER_NAME "'." + ) + ( + "btcd.host", + value(&configured.server.btcd.hosts), + "The host name (http verification), multiple allowed, defaults to empty (disabled)." + ) + ( + "btcd.origin", + value(&configured.server.btcd.origins), + "The allowed origin (see CORS), multiple allowed, defaults to empty (disabled)." + ) + ( + "btcd.allow_opaque_origin", + value(&configured.server.btcd.allow_opaque_origin), + "Allow requests from opaque origin (see CORS), multiple allowed, defaults to false." + ) + /* [electrum] */ ( "electrum.bind", diff --git a/src/protocols/btcd/protocol_btcd_rpc.cpp b/src/protocols/btcd/protocol_btcd_rpc.cpp new file mode 100644 index 000000000..7f7f2e268 --- /dev/null +++ b/src/protocols/btcd/protocol_btcd_rpc.cpp @@ -0,0 +1,493 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include + +#include +#include +#include + +namespace libbitcoin { +namespace server { + +#define CLASS protocol_btcd_rpc +#define SUBSCRIBE_BTCD(method, ...) \ + btcd_subscribe(&CLASS::method, __VA_ARGS__) + +// protocol_bitcoind_rpc declares 'using post = network::http::method::post', +// which shadows network::protocol::post. Qualify explicitly. +#define POST_BTCD(method, ...) \ + this->network::protocol::template post(&CLASS::method, __VA_ARGS__) + +using namespace system; +using namespace network; +using namespace network::rpc; +using namespace std::placeholders; +using namespace boost::json; + +BC_PUSH_WARNING(NO_THROW_IN_NOEXCEPT) +BC_PUSH_WARNING(SMART_PTR_NOT_NEEDED) +BC_PUSH_WARNING(NO_VALUE_OR_CONST_REF_SHARED_PTR) + +// Start. +// ---------------------------------------------------------------------------- + +void protocol_btcd_rpc::start() NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (started()) + return; + + subscribe_chase(BIND(handle_chase, _1, _2, _3)); + + SUBSCRIBE_BTCD(handle_authenticate, _1, _2, _3, _4); + SUBSCRIBE_BTCD(handle_session, _1, _2); + SUBSCRIBE_BTCD(handle_notify_blocks, _1, _2); + SUBSCRIBE_BTCD(handle_stop_notify_blocks, _1, _2); + SUBSCRIBE_BTCD(handle_notify_new_transactions, _1, _2, _3); + SUBSCRIBE_BTCD(handle_stop_notify_new_transactions, _1, _2); + SUBSCRIBE_BTCD(handle_load_tx_filter, _1, _2, _3, _4, _5); + SUBSCRIBE_BTCD(handle_rescan_blocks, _1, _2, _3); + SUBSCRIBE_BTCD(handle_stop, _1, _2); + SUBSCRIBE_BTCD(handle_notify_received, _1, _2, _3); + SUBSCRIBE_BTCD(handle_stop_notify_received, _1, _2, _3); + SUBSCRIBE_BTCD(handle_notify_spent, _1, _2, _3); + SUBSCRIBE_BTCD(handle_stop_notify_spent, _1, _2, _3); + SUBSCRIBE_BTCD(handle_rescan, _1, _2, _3, _4, _5, _6); + + // Base registers all standard chain rpc handlers and starts the listener. + protocol_bitcoind_rpc::start(); +} + +void protocol_btcd_rpc::stopping(const code& ec) NOEXCEPT +{ + BC_ASSERT(stranded()); + btcd_dispatcher_.stop(ec); + unsubscribe_chase(); + protocol_bitcoind_rpc::stopping(ec); +} + +// Websocket dispatch. +// ---------------------------------------------------------------------------- +// btcd extension methods (session/notify/filter/admin) arrive as ws frames +// here. Standard chain methods (getblockcount etc.) remain reachable via +// plain http post on the same endpoint, unchanged, via the inherited +// handle_receive_post. They are NOT yet bridged into this ws dispatcher: that +// requires reconciling protocol_bitcoind_rpc's post-oriented private send +// path (which caches the original http::request for header derivation) with +// this class's ws-oriented senders. Tracked for phase B. + +void protocol_btcd_rpc::dispatch_websocket( + const network::http::request& request) NOEXCEPT +{ + BC_ASSERT(stranded()); + + // channel_btcd::websocket_body() preselects the json-rpc request parser. + if (!request.body().contains()) + { + send_btcd_error(error::invalid_argument); + return; + } + + const auto& body = request.body().get(); + const auto& message = body.message; + + // Cache request context for response building (version + id). + btcd_version_ = message.jsonrpc; + btcd_id_ = message.id; + + // channel_btcd::unauthorized() bypasses the (structurally per-http- + // request-only) basic auth check for the whole ws session, so when a + // credential is configured, enforcement happens here instead: every + // method other than 'authenticate' itself is rejected until it succeeds. + if (options_.authorize() && !btcd_channel_->authenticated() && + message.method != btcd_interface::authenticate::name) + { + const code unauthorized{ network::error::unauthorized }; + send_btcd_error(unauthorized, two * unauthorized.message().size(), + unauthorized); + return; + } + + const auto code = btcd_dispatcher_.notify(message); + if (!code) + return; + + // Unknown method: reply with a json-rpc error and keep the ws connection + // alive (matches real btcd behavior, and the same guard already applied + // to protocol_bitcoind_rpc::handle_receive_post for its own dispatcher). + if (code == network::error::unexpected_method) + send_btcd_error(code); + else + stop(code); +} + +// Handlers (session/handshake). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_authenticate(const code& ec, + btcd_interface::authenticate, const std::string& username, + const std::string& password) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // No-op when no server-side credential is configured. Otherwise the + // username/password must match the configured single-tier credential. + if (!options_.authorize() || + (username == options_.username && password == options_.password)) + { + btcd_channel_->set_authenticated(true); + send_btcd_result({}, 4); + return true; + } + + // Close once the error response has actually been written, rather than + // stopping synchronously here and racing the (async) send. + const code unauthorized{ network::error::unauthorized }; + send_btcd_error(unauthorized, two * unauthorized.message().size(), + unauthorized); + return false; +} + +bool protocol_btcd_rpc::handle_session(const code& ec, + btcd_interface::session) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // The channel identifier is unique per connection for the process + // lifetime, satisfying btcd's reconnect-detection use of session ids. + object_t result{}; + result.emplace("id", value_t{ possible_sign_cast(identifier()) }); + send_btcd_result(value_t{ std::move(result) }, 32); + return true; +} + +// Handlers (block subscription). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_notify_blocks(const code& ec, + btcd_interface::notify_blocks) NOEXCEPT +{ + if (stopped(ec)) + return false; + + subscribed_blocks_.store(true, std::memory_order_relaxed); + send_btcd_result({}, 4); + return true; +} + +bool protocol_btcd_rpc::handle_stop_notify_blocks(const code& ec, + btcd_interface::stop_notify_blocks) NOEXCEPT +{ + if (stopped(ec)) + return false; + + subscribed_blocks_.store(false, std::memory_order_relaxed); + send_btcd_result({}, 4); + return true; +} + +// Handlers (mempool subscription, not_implemented pending v5 mempool). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_notify_new_transactions(const code& ec, + btcd_interface::notify_new_transactions, bool) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_stop_notify_new_transactions(const code& ec, + btcd_interface::stop_notify_new_transactions) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +// Handlers (address/outpoint filtering, not_implemented pending phase B). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_load_tx_filter(const code& ec, + btcd_interface::load_tx_filter, bool, const value_t&, + const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_rescan_blocks(const code& ec, + btcd_interface::rescan_blocks, const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +// Handler (admin, permanently not_implemented). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_stop(const code& ec, + btcd_interface::stop) NOEXCEPT +{ + if (stopped(ec)) return false; + + // No secure remote-shutdown path exists; always rejected regardless of + // configuration or auth state. + send_btcd_error(error::not_implemented); + return true; +} + +// Handlers (deprecated, permanently not_implemented). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_notify_received(const code& ec, + btcd_interface::notify_received, const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_stop_notify_received(const code& ec, + btcd_interface::stop_notify_received, const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_notify_spent(const code& ec, + btcd_interface::notify_spent, const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_stop_notify_spent(const code& ec, + btcd_interface::stop_notify_spent, const value_t&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +bool protocol_btcd_rpc::handle_rescan(const code& ec, + btcd_interface::rescan, const std::string&, const value_t&, + const value_t&, const std::string&) NOEXCEPT +{ + if (stopped(ec)) return false; + send_btcd_error(error::not_implemented); + return true; +} + +// Chase events. +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_chase(const code& ec, node::chase event_, + node::event_value value) NOEXCEPT +{ + // Do not pass ec to stopped, it is not a call status. + if (stopped()) + return false; + + switch (event_) + { + case node::chase::organized: + if (subscribed_blocks_.load(std::memory_order_relaxed)) + { + BC_ASSERT(std::holds_alternative(value)); + POST_BTCD(do_block_connected, std::get(value)); + } + break; + + case node::chase::reorganized: + if (subscribed_blocks_.load(std::memory_order_relaxed)) + { + BC_ASSERT(std::holds_alternative(value)); + POST_BTCD(do_block_disconnected, std::get(value)); + } + break; + + default: + break; + } + + return !stopped(); +} + +void protocol_btcd_rpc::do_block_connected(node::header_t link_value) NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (stopped() || !subscribed_blocks_.load(std::memory_order_relaxed)) + return; + + const database::header_link link{ link_value }; + const auto& query = archive(); + + size_t height{}; + if (!query.get_height(height, link)) + return; + + const auto header = query.get_header(link); + if (!header) + return; + + // btcd 'blockconnected' notification: [hash, height, time] (positional, + // matching btcjson.BlockConnectedNtfn -- the unfiltered notification + // fired by notifyblocks, distinct from loadtxfilter's filtered variant). + array_t params{}; + params.emplace_back(value_t{ encode_hash(query.get_header_key(link)) }); + params.emplace_back(value_t{ possible_sign_cast(height) }); + params.emplace_back(value_t{ possible_sign_cast( + header->timestamp()) }); + + send_btcd_notification("blockconnected", std::move(params), 256); +} + +void protocol_btcd_rpc::do_block_disconnected( + node::header_t link_value) NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (stopped() || !subscribed_blocks_.load(std::memory_order_relaxed)) + return; + + const database::header_link link{ link_value }; + const auto& query = archive(); + + size_t height{}; + if (!query.get_height(height, link)) + return; + + const auto header = query.get_header(link); + if (!header) + return; + + array_t params{}; + params.emplace_back(value_t{ encode_hash(query.get_header_key(link)) }); + params.emplace_back(value_t{ possible_sign_cast(height) }); + params.emplace_back(value_t{ possible_sign_cast( + header->timestamp()) }); + + send_btcd_notification("blockdisconnected", std::move(params), 256); +} + +// Senders. +// ---------------------------------------------------------------------------- + +void protocol_btcd_rpc::send_btcd_result(value_option&& result, + size_t size_hint) NOEXCEPT +{ + BC_ASSERT(stranded()); + send_btcd_rpc( + { + .jsonrpc = btcd_version_, + .id = btcd_id_, + .result = std::move(result) + }, size_hint); +} + +void protocol_btcd_rpc::send_btcd_error(const code& ec) NOEXCEPT +{ + send_btcd_error(ec, two * ec.message().size()); +} + +void protocol_btcd_rpc::send_btcd_error(const code& ec, + size_t size_hint) NOEXCEPT +{ + send_btcd_error(ec, size_hint, error::success); +} + +void protocol_btcd_rpc::send_btcd_error(const code& ec, size_t size_hint, + const code& close_reason) NOEXCEPT +{ + BC_ASSERT(stranded()); + send_btcd_rpc( + { + .jsonrpc = btcd_version_, + .id = btcd_id_, + .error = result_t + { + .code = ec.value(), + .message = ec.message() + } + }, size_hint, close_reason); +} + +void protocol_btcd_rpc::send_btcd_notification(const std::string& method, + array_t&& params, size_t size_hint) NOEXCEPT +{ + BC_ASSERT(stranded()); + using namespace network::http; + static const auto json = from_media_type(media_type::application_json); + + // Server-push notification: json-rpc-v1 request shape, no id. + rpc::request notification{ { .size_hint = size_hint } }; + notification.message.jsonrpc = version::v1; + notification.message.method = method; + notification.message.params = params_t{ std::move(params) }; + + http::response message{ status::ok, 11 }; + message.set(field::content_type, json); + message.body() = std::move(notification); + message.prepare_payload(); + + // NOTIFY does not restart the reader (the ws reader is already active, + // independent of this unprompted push). + NOTIFY(std::move(message), handle_complete, _1, error::success); +} + +// private +void protocol_btcd_rpc::send_btcd_rpc(response_t&& model, size_t size_hint, + const code& close_reason) NOEXCEPT +{ + BC_ASSERT(stranded()); + using namespace network::http; + static const auto json = from_media_type(media_type::application_json); + + btcd_id_.reset(); + btcd_version_ = version::undefined; + + http::response message{ status::ok, 11 }; + message.set(field::content_type, json); + message.body() = rpc::response + { + { .size_hint = size_hint }, std::move(model), + }; + message.prepare_payload(); + + // SEND restarts the ws reader so the next client frame is accepted. + // handle_complete only stops the channel if close_reason is truthy, and + // only after this write actually completes -- never synchronously here. + SEND(std::move(message), handle_complete, _1, close_reason); +} + +BC_POP_WARNING() +BC_POP_WARNING() +BC_POP_WARNING() + +} // namespace server +} // namespace libbitcoin diff --git a/src/server_node.cpp b/src/server_node.cpp index 1a59ff963..cfdf2fce5 100644 --- a/src/server_node.cpp +++ b/src/server_node.cpp @@ -117,6 +117,21 @@ void server_node::start_bitcoind(const code& ec, } attach_bitcoind_session()->start( + std::bind(&server_node::start_btcd, this, _1, handler)); +} + +void server_node::start_btcd(const code& ec, + const result_handler& handler) NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (ec) + { + handler(ec); + return; + } + + attach_btcd_session()->start( std::bind(&server_node::start_electrum, this, _1, handler)); } @@ -185,6 +200,12 @@ session_bitcoind::ptr server_node::attach_bitcoind_session() NOEXCEPT config_.server.bitcoind); } +session_btcd::ptr server_node::attach_btcd_session() NOEXCEPT +{ + return net::attach(*this, config_, + config_.server.btcd); +} + session_electrum::ptr server_node::attach_electrum_session() NOEXCEPT { return net::attach(*this, config_, From 9a6cfd3c6d6bb8ce02adab4159f5bb42e016f29f Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Mon, 27 Jul 2026 05:02:06 +0200 Subject: [PATCH 04/10] Fix btcd endpoint tests for the ws auth-gate change. conn fixture now authenticates up front when the server has credentials configured, matching what a real client must do before calling anything else over ws (every method but authenticate is now rejected until then). Without this, most of the suite degraded to xfail/failed once credentials were actually configured on the server, since none of the individual tests called authenticate themselves. http_rpc() now sends HTTP Basic Auth when credentials are configured, for the same reason on the plain-post path (which keeps the normal per-request check, unlike ws). test_response_id_matches_request no longer hardcodes absolute request ids 0/1: the fixture's own authenticate call consumes an id first when credentials are configured, so only the relative ordering (each id is exactly one more than the last) is a stable guarantee. Verified against a live bs_btcd instance with btcd.username/password configured: 19 passed, 2 skipped, 20 xfailed, 0 failures. --- endpoints/test_btcd_rpc.py | 43 ++++++++++++++++++++++++++++++++++---- 1 file changed, 39 insertions(+), 4 deletions(-) diff --git a/endpoints/test_btcd_rpc.py b/endpoints/test_btcd_rpc.py index 7e402546c..33e84191d 100644 --- a/endpoints/test_btcd_rpc.py +++ b/endpoints/test_btcd_rpc.py @@ -269,9 +269,13 @@ def http_rpc(config: dict, method: str, params: Optional[list] = None) -> dict: "params": params if params is not None else [], } url = f"http://{config['host']}:{config['port']}/" + auth = None + if config.get("username"): + auth = (config["username"], config.get("password") or "") response = requests.post( url, json=payload, headers={"Content-Type": "application/json", "Connection": "close"}, + auth=auth, timeout=config.get("timeout", TestConfig.DEFAULT_RPC_TIMEOUT), ) response.raise_for_status() @@ -282,13 +286,32 @@ def http_rpc(config: dict, method: str, params: Optional[list] = None) -> dict: @pytest.fixture def conn(btcd_config: dict) -> BtcdConnection: - """Fresh btcd websocket connection per test.""" + """Fresh btcd websocket connection per test, authenticated up front if + the server has credentials configured. + + Every method other than 'authenticate' itself is rejected over ws until + the connection has authenticated (see protocol_btcd_rpc::dispatch_ + websocket) -- a real client always does this handshake first, so this + fixture does it here rather than in every single test. + + The authenticate-specific tests (test_authenticate_*) intentionally + don't rely on this and drive their own connections/authenticate calls, + since they're testing that handshake itself. + """ host, port = btcd_config["host"], btcd_config["port"] timeout = btcd_config.get("timeout", TestConfig.DEFAULT_SOCKET_TIMEOUT) try: c = BtcdConnection(host, port, connect_timeout=timeout) except (OSError, ConnectionError) as exc: pytest.skip(f"Cannot connect to btcd at {host}:{port}: {exc}") + + username = btcd_config.get("username") + if username: + auth = c.raw_rpc("authenticate", [username, btcd_config.get("password") or ""]) + if auth.get("error") is not None: + c.close() + pytest.fail(f"authenticate failed in fixture setup: {auth['error']}") + yield c c.close() @@ -309,7 +332,12 @@ def test_authenticate_no_credentials_configured(conn, btcd_config): def test_authenticate_with_configured_credentials(conn, btcd_config): """When btcd.username/password are configured, authenticate must accept - the matching credentials and reject a mismatch (closing the connection). + the matching credentials. The mismatch case is covered separately by + test_authenticate_wrong_password_rejected. + + Note: the conn fixture already authenticated once during setup (it must, + to run anything else) -- this repeats the call explicitly to verify + authenticate itself is idempotent and still reports success. """ username = btcd_config.get("username") password = btcd_config.get("password") @@ -532,10 +560,17 @@ def test_method_not_yet_wired(conn, method, params): # ═══════════════════════════════════════════════════════════════════════════════ def test_response_id_matches_request(conn): + """Each response echoes its own request's id, in order. + + Not asserted as absolute ids 0/1: when credentials are configured the + conn fixture already consumed one id authenticating, so the starting + id here depends on whether that happened -- only the relative order + (each new request's id is exactly one more than the last) is a stable + guarantee. + """ r0 = conn.send_rpc("session") r1 = conn.send_rpc("notifyblocks") - assert r0.get("id") == 0 - assert r1.get("id") == 1 + assert r1.get("id") == r0.get("id") + 1 def test_unknown_method_errors_without_dropping_connection(conn): From 4b6149ec11f6fabc52eb2b232c4f91d6a4159cf1 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Tue, 28 Jul 2026 08:55:55 +0200 Subject: [PATCH 05/10] Finalize btcd auth as an ordinary protocol_btcd_rpc method (no handshake stage). session_handshake (tried in the two preceding commits, now dropped) modeled btcd's auth on electrum/p2p's version negotiation, but that pattern assumes the handshake can only complete by reading a real message. btcd's doesn't: with no credential configured it completes without reading anything, leaving a real async gap between "handshake decided it's done" and protocol_btcd_rpc actually subscribing its handlers. Harmless for websocket (the upgrade round-trip masks it), but a plain HTTP client can and reliably did land a request in that gap. Real btcd has no such gap because it has no handshake protocol at all. Revert session_btcd to session_server (matching bitcoind and real btcd) and move 'authenticate' back into protocol_btcd_rpc as an ordinary extension method, but with the real semantics this time: - Digest-based credential check (sha256("Basic " + base64(user:pass)), same scheme as config::credential) via options_.authorized(), instead of a plaintext username/password comparison. - btcd.username/btcd.password collapse into the same 'btcd.credential = user:pass[:method,...]' shape bitcoind already uses, so a credential can be scoped to a method allowlist -- channel_btcd::permitted() enforces that scope for both http and ws (ws basic-auth-on-upgrade and in-band authenticate are two alternative ways to satisfy it, matching real btcd's own "authenticate is disallowed once basic auth is already established"). - send_btcd_error/send_btcd_rpc thread a close_reason through to handle_complete, so a failed authenticate's error reaches the client before the connection closes, without racing the async send. docs/btcd-endpoint.md documents why the handshake variant was reverted. Boost.Test (640 cases) and the Python endpoint suite pass against a live server both with and without a configured credential. --- docs/btcd-endpoint.md | 38 ++++++++-- endpoints/README.md | 2 +- .../bitcoin/server/channels/channel_btcd.hpp | 63 ++++++++++++---- .../server/protocols/protocol_btcd_rpc.hpp | 59 +++++++++------ include/bitcoin/server/sessions/sessions.hpp | 17 ++++- src/parser.cpp | 11 +-- src/protocols/btcd/protocol_btcd_rpc.cpp | 71 ++++++++++-------- test/protocols/btcd/btcd_rpc.cpp | 72 +++++++++++++++++++ test/protocols/btcd/btcd_setup_fixture.cpp | 8 ++- test/protocols/btcd/btcd_setup_fixture.hpp | 45 +++++++++++- 10 files changed, 304 insertions(+), 82 deletions(-) diff --git a/docs/btcd-endpoint.md b/docs/btcd-endpoint.md index 5188d1443..7db2f76e6 100644 --- a/docs/btcd-endpoint.md +++ b/docs/btcd-endpoint.md @@ -23,12 +23,14 @@ btcd exposes two transports on the same port (default `8334`): Real btcd authenticates via TLS + HTTP Basic Auth, with two credential tiers: `rpcuser`/`rpcpass` (full access) and `rpclimituser`/`rpclimitpass` (restricted to -a fixed allowlist of chain-read methods). **Decision: this endpoint uses a single -credential tier** (reusing `network::settings::http_server`'s existing -`username`/`password`/`authenticated()`, same as `bitcoind.*`). Sensitive methods -(`stop`, mining/peer-management) are stubbed to `not_implemented` unconditionally, -so a limited-user tier adds no security value here; it can be revisited if a -future phase implements a real `stop` path. +a fixed allowlist of chain-read methods). **Decision: this endpoint does not build +a bespoke two-tier scheme.** It reuses `network::settings::http_server`'s +credential model as-is (`btcd.credential = username:password[:method,...]`, same +config shape as `bitcoind.*`), which happens to already support scoping a +credential to a method allowlist generically; `channel_btcd::permitted()` enforces +that scope over both http and ws. Sensitive methods (`stop`, mining/peer +management) are stubbed to `not_implemented` unconditionally regardless of scope, +so this is revisited only if a future phase implements a real `stop` path. ## Architecture @@ -55,6 +57,28 @@ upgrade — to route btcd's WS-only notification/subscription methods. `channel_btcd` extends `channel_http` with an `authenticated_` flag, set once the WS `authenticate` command (or HTTP Basic Auth header) succeeds. +**A `session_handshake` variant of this was +tried and reverted.** The idea was to model btcd's auth on +`protocol_electrum_version`'s handshake pattern: a dedicated protocol object that +gates attachment of the real one. It doesn't fit. `session_handshake` calls +`channel->resume()` immediately after the handshake protocol attaches, on the +assumption that the handshake can only complete by reading a real message +(true for electrum's `server.version`, true for p2p's version message) — but +btcd's handshake, when no credential is configured, completes without reading +anything at all. That created a real async gap between "handshake decided +it's done" and "`protocol_btcd_rpc` has actually subscribed its handlers" — +harmless for a websocket client (the upgrade round-trip masks it), but a +plain HTTP client can land a request in that gap before anything is +listening, and did, reliably, in testing. Real btcd has no such gap because +it has no handshake protocol at all: `session_server` (used here) never calls +`resume()` until *after* `attach_protocols()` has synchronously subscribed +the real handlers — the same shape bitcoind already uses, and the reason +bitcoind never had this problem. `authenticate` is just an ordinary +extension method on `protocol_btcd_rpc`, exactly matching real btcd, where it +is described as "disallowed when basic auth has already been established" — +an alternative to Basic Auth for the one transport that can't carry headers +per-message, not a mandatory step every connection passes through. + Notification/subscription state (watched blocks, watched scripts/outpoints) follows the pattern already established in `protocol_electrum` (`src/protocols/electrum/protocol_electrum.cpp`): a small map guarded by a @@ -191,7 +215,7 @@ node, so `estimatesmartfee`-equivalent support is not on the critical path. New `[btcd]` section, mirroring `[bitcoind]`'s shape in `parser.cpp`/ `settings.hpp` (`bind`, `safe`, `cert_auth`, `cert_path`, `key_path`, `key_pass`, -`username`, `password`, `connections`, `inactivity_minutes`, +`credential`, `connections`, `inactivity_minutes`, `expiration_minutes`, `minimum_buffer`, `maximum_request`, `host`, `origin`, `allow_opaque_origin`). No dedicated `stop`-gating config key: `stop` has no conditional path at all right now (always `not_implemented`, unconditionally), diff --git a/endpoints/README.md b/endpoints/README.md index 06e12f0cd..e18fc676e 100644 --- a/endpoints/README.md +++ b/endpoints/README.md @@ -158,7 +158,7 @@ pytest test_btcd_rpc.py \ - `--btcd-host` — Host for btcd JSON-RPC/websocket (default: localhost) - `--btcd-port` — Port for btcd JSON-RPC/websocket (default: 8334) - `--btcd-username` / `--btcd-password` — Credentials to test `authenticate` - against, if the server has `btcd.username`/`btcd.password` configured + against, if the server has `btcd.credential=username:password` configured (default: none — the credential-specific tests are skipped) No extra Python dependency is required: the websocket client is a small diff --git a/include/bitcoin/server/channels/channel_btcd.hpp b/include/bitcoin/server/channels/channel_btcd.hpp index c36fce257..f58016707 100644 --- a/include/bitcoin/server/channels/channel_btcd.hpp +++ b/include/bitcoin/server/channels/channel_btcd.hpp @@ -40,7 +40,8 @@ class BCS_API channel_btcd const node::configuration& config, const options_t& options) NOEXCEPT : server::channel(log, socket, identifier, config), network::channel_http(log, socket, identifier, config.network, options), - network::tracker(log) + network::tracker(log), + options_(options) { } @@ -51,9 +52,38 @@ class BCS_API channel_btcd return authenticated_; } - inline void set_authenticated(bool value) NOEXCEPT + /// Latch the credential digest that satisfied the in-band 'authenticate' + /// call (see protocol_btcd_rpc::handle_authenticate), so that permitted() + /// can later apply that credential's method scoping to ws traffic. + inline void set_authenticated(const system::hash_digest& digest) NOEXCEPT { - authenticated_ = value; + authenticated_ = true; + authenticated_digest_ = digest; + } + + /// True if the credential that authorized this channel permits the rpc + /// method. Plain http traffic defers entirely to channel_http's own + /// header-latched digest (unchanged from bitcoind). For ws traffic, real + /// btcd recognizes two alternative ways a connection can be authorized -- + /// basic auth already established on the ws upgrade request (also + /// channel_http's header-latched digest, since the upgrade request has + /// real headers even though subsequent ws data frames don't), or the + /// in-band 'authenticate' call (digest latched above); its own docs + /// describe 'authenticate' as "disallowed when basic auth has already + /// been established" -- i.e. an alternative to it, not an additional + /// requirement on top of it. + inline bool permitted(const std::string& method) const NOEXCEPT override + { + if (!websocket()) + return network::channel_http::permitted(method); + + if (!options_.authorize()) + return true; + + if (network::channel_http::permitted(method)) + return true; + + return authenticated_ && options_.permitted(authenticated_digest_, method); } protected: @@ -69,21 +99,28 @@ class BCS_API channel_btcd return value; } - /// Basic auth is a per-http-request header, which websocket data frames - /// structurally cannot carry (only the upgrade handshake has headers, - /// and that request never reaches this check -- see channel_http's - /// error::upgraded short-circuit). Once upgraded, auth is instead - /// enforced in-band by the ws 'authenticate' method (see - /// protocol_btcd_rpc::dispatch_websocket); the plain http post path - /// (e.g. inherited chain methods) keeps the normal per-request check. - inline bool unauthorized(const network::http::request& request) NOEXCEPT override + /// channel_http's authorized() only latches basic-auth from the ws + /// upgrade request (the upgrade request has real headers); subsequent ws + /// data frames are synthesized and structurally cannot carry a + /// per-frame Authorization header. Deferring to the base check here + /// would therefore reject every ws frame once a credential is + /// configured, unless basic auth happened to be presented on the + /// upgrade itself -- permitted() (above) is the real per-method gate for + /// ws traffic, so this base check must never reject a ws frame on its + /// own. The plain http post path (e.g. inherited bitcoind-compatible + /// chain methods) keeps the normal per-request enforcement. + inline bool authorized() const NOEXCEPT override { - return websocket() ? false : network::channel_http::unauthorized(request); + return websocket() ? true : network::channel_http::authorized(); } private: - // This is protected by strand. + // This is thread safe. + const options_t& options_; + + // These are protected by strand. bool authenticated_{}; + system::hash_digest authenticated_digest_{}; }; } // namespace server diff --git a/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp index 89e0e6e46..dc52d5618 100644 --- a/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp +++ b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp @@ -34,6 +34,23 @@ namespace server { /// remain reachable via the inherited http post path (unchanged). Adds the /// btcd-only extension methods (session/notification/filter/admin) over /// websocket via a second dispatcher (dispatch_websocket, below). +/// +/// Attaches immediately on connect (session_btcd = session_server< +/// protocol_btcd_rpc>, no separate handshake protocol/stage) -- matching real +/// btcd's own model, which has no handshake either: HTTP Basic Auth is +/// checked synchronously per plain-http request (channel_http::authorized(), +/// unchanged from bitcoind), and ws 'authenticate' is just an ordinary +/// extension method that lets a client establish the same authorization +/// in-band, for the one transport (websocket data frames) that structurally +/// cannot carry a per-message Authorization header. See docs/btcd-endpoint.md +/// for why an earlier session_handshake +/// design (modeled on electrum/p2p's version negotiation) was reverted: those +/// protocols' handshakes always wait for a real message to complete, but +/// btcd's has no credential-required case to wait for -- modeling it on that +/// pattern introduced an async attach gap real btcd never has, which a plain +/// http client (no upgrade round-trip to mask the gap, unlike ws) could land +/// in before this class ever subscribed its handlers. +/// /// TODO (phase B): bridge the standard chain handlers into the ws dispatcher /// too, so a single ws connection can also reach them (real btcd/btcwallet /// clients expect this); requires reconciling protocol_bitcoind_rpc's @@ -60,12 +77,6 @@ class BCS_API protocol_btcd_rpc { } - /// Configuration options (for basic-auth credential comparison). - inline const options_t& options() const NOEXCEPT - { - return options_; - } - void start() NOEXCEPT override; void stopping(const code& ec) NOEXCEPT override; @@ -75,7 +86,20 @@ class BCS_API protocol_btcd_rpc void dispatch_websocket( const network::http::request& request) NOEXCEPT override; - /// Handlers (session/handshake). + /// Handlers (session/handshake). 'authenticate' is a real, ordinary + /// extension method here (not a separate handshake stage, see class + /// comment): it hashes the given credential the same way + /// config::credential does, checks it via options_.authorized(), and on + /// success latches the digest onto channel_btcd so permitted()'s method + /// scoping can apply to subsequent ws calls. interface::btcd's method + /// table has no notion of "not yet authenticated", so this always runs + /// regardless of current state -- matches real btcd's own tolerance of a + /// repeat authenticate call (only disallowed once basic auth has already + /// been established on the ws upgrade, not enforced here since that + /// case is harmless to allow). A failed attempt ends the session + /// (dispatch_websocket sends the error before stopping), matching real + /// btcd's documented behavior of closing the connection on invalid + /// credentials rather than leaving it open for a retry. bool handle_authenticate(const code& ec, btcd_interface::authenticate, const std::string& username, const std::string& password) NOEXCEPT; bool handle_session(const code& ec, btcd_interface::session) NOEXCEPT; @@ -129,18 +153,17 @@ class BCS_API protocol_btcd_rpc void do_block_disconnected(node::header_t link) NOEXCEPT; /// Senders (btcd ws envelope, distinct id/version cache from the http - /// json-rpc-v2 senders inherited from protocol_bitcoind_rpc). + /// json-rpc-v2 senders inherited from protocol_bitcoind_rpc). The + /// close_reason overloads complete via handle_complete's (ec, reason) + /// idiom: the channel only stops (if reason is truthy) once the write + /// has actually completed, so a failed authenticate's error reaches the + /// client before the connection closes. void send_btcd_result(network::rpc::value_option&& result, size_t size_hint) NOEXCEPT; void send_btcd_error(const code& ec) NOEXCEPT; void send_btcd_error(const code& ec, size_t size_hint) NOEXCEPT; - - /// Sends the error and closes the channel once the write completes (not - /// synchronously) -- use for cases (e.g. failed authenticate) where the - /// channel must not survive the error, without racing the async send. void send_btcd_error(const code& ec, size_t size_hint, const code& close_reason) NOEXCEPT; - void send_btcd_notification(const std::string& method, network::rpc::array_t&& params, size_t size_hint) NOEXCEPT; @@ -153,21 +176,17 @@ class BCS_API protocol_btcd_rpc // Senders. void send_btcd_rpc(network::rpc::response_t&& model, size_t size_hint, - const code& close_reason=network::error::success) NOEXCEPT; + const code& close_reason) NOEXCEPT; // These are thread safe. std::atomic_bool subscribed_blocks_{}; + const options_t& options_; + const channel_btcd::ptr btcd_channel_; // These are protected by strand. btcd_dispatcher btcd_dispatcher_{}; network::rpc::version btcd_version_{}; network::rpc::id_option btcd_id_{}; - - // These are thread safe. - const options_t& options_; - - // This is mostly thread safe, and used in a thread safe manner. - const channel_btcd::ptr btcd_channel_; }; } // namespace server diff --git a/include/bitcoin/server/sessions/sessions.hpp b/include/bitcoin/server/sessions/sessions.hpp index 6b143a392..5c96c7cab 100644 --- a/include/bitcoin/server/sessions/sessions.hpp +++ b/include/bitcoin/server/sessions/sessions.hpp @@ -32,12 +32,20 @@ namespace server { using session_admin = session_server; using session_native = session_server; using session_bitcoind = session_server; -using session_btcd = session_server; using session_stratum_v1 = session_server; using session_stratum_v2 = session_server; using session_electrum = session_handshake; +// No session_handshake here: unlike electrum/p2p, btcd has no negotiation to +// wait for -- real btcd authenticates via plain HTTP Basic Auth (checked +// synchronously per request, same as bitcoind) or an in-band ws +// 'authenticate' method that is just an ordinary extension method, not a +// separate handshake stage (see protocol_btcd_rpc's class comment and +// docs/btcd-endpoint.md for why session_handshake was tried and reverted). +using session_btcd = session_server; + } // namespace server } // namespace libbitcoin @@ -72,12 +80,15 @@ node::session server::session → node::session └── server::session_server<...Protocols> → network::session_server - ╞══ session_admin = server::session_server + ╞══ session_admin = server::session_server ╞══ session_native = server::session_server ╞══ session_bitcoind = server::session_server - ╞══ session_btcd = server::session_server ╞══ session_stratum_v1 = server::session_server ╞══ session_stratum_v2 = server::session_server + ╞══ session_btcd = server::session_server + │ (no handshake stage -- real btcd has no negotiation to wait + │ for; auth is a synchronous per-request/per-connection check, + │ same shape as bitcoind, not a separate protocol) └── server::session_handshake<...Protocols> ╘══ session_electrum = server::session_handshake< protocol_electrum_version, protocol_electrum> diff --git a/src/parser.cpp b/src/parser.cpp index 0136ef817..04e7039eb 100644 --- a/src/parser.cpp +++ b/src/parser.cpp @@ -1125,14 +1125,9 @@ options_metadata parser::load_settings() THROWS "The password to decrypt the server private key file (.PEM), optional." ) ( - "btcd.username", - value(&configured.server.btcd.username), - "The basic authorization username (not secure), also used for the ws 'authenticate' method." - ) - ( - "btcd.password", - value(&configured.server.btcd.password), - "The basic authorization password (not secure), also used for the ws 'authenticate' method." + "btcd.credential", + value(&configured.server.btcd.credentials), + "The 'username:password[:method,...]' authorization (not secure), also used for the ws 'authenticate' method, multiple allowed." ) ( "btcd.connections", diff --git a/src/protocols/btcd/protocol_btcd_rpc.cpp b/src/protocols/btcd/protocol_btcd_rpc.cpp index 7f7f2e268..11721079c 100644 --- a/src/protocols/btcd/protocol_btcd_rpc.cpp +++ b/src/protocols/btcd/protocol_btcd_rpc.cpp @@ -85,13 +85,21 @@ void protocol_btcd_rpc::stopping(const code& ec) NOEXCEPT // Websocket dispatch. // ---------------------------------------------------------------------------- -// btcd extension methods (session/notify/filter/admin) arrive as ws frames -// here. Standard chain methods (getblockcount etc.) remain reachable via -// plain http post on the same endpoint, unchanged, via the inherited -// handle_receive_post. They are NOT yet bridged into this ws dispatcher: that -// requires reconciling protocol_bitcoind_rpc's post-oriented private send -// path (which caches the original http::request for header derivation) with -// this class's ws-oriented senders. Tracked for phase B. +// btcd extension methods (session/notify/filter/admin, including +// authenticate) arrive as ws frames here. Standard chain methods +// (getblockcount etc.) remain reachable via plain http post on the same +// endpoint, unchanged, via the inherited handle_receive_post. They are NOT +// yet bridged into this ws dispatcher: that requires reconciling +// protocol_bitcoind_rpc's post-oriented private send path (which caches the +// original http::request for header derivation) with this class's +// ws-oriented senders. Tracked for phase B. +// +// A credential may be scoped to a subset of methods (see config::credential +// / channel_btcd::permitted()), so each call is checked against that scope -- +// except 'authenticate' itself, which must always reach handle_authenticate +// regardless of current auth state: permitted() requires authentication to +// already be established, so gating 'authenticate' on it would make +// authenticating impossible in the first place. void protocol_btcd_rpc::dispatch_websocket( const network::http::request& request) NOEXCEPT @@ -112,16 +120,10 @@ void protocol_btcd_rpc::dispatch_websocket( btcd_version_ = message.jsonrpc; btcd_id_ = message.id; - // channel_btcd::unauthorized() bypasses the (structurally per-http- - // request-only) basic auth check for the whole ws session, so when a - // credential is configured, enforcement happens here instead: every - // method other than 'authenticate' itself is rejected until it succeeds. - if (options_.authorize() && !btcd_channel_->authenticated() && - message.method != btcd_interface::authenticate::name) + if (message.method != btcd_interface::authenticate::name && + !permitted(message.method)) { - const code unauthorized{ network::error::unauthorized }; - send_btcd_error(unauthorized, two * unauthorized.message().size(), - unauthorized); + send_btcd_error(network::error::unauthorized); return; } @@ -148,22 +150,35 @@ bool protocol_btcd_rpc::handle_authenticate(const code& ec, if (stopped(ec)) return false; - // No-op when no server-side credential is configured. Otherwise the - // username/password must match the configured single-tier credential. - if (!options_.authorize() || - (username == options_.username && password == options_.password)) + // No credential configured: nothing to check against, so a no-op + // success (matches real btcd's own no-auth-required mode -- there is no + // "wrong" answer when authorization was never required in the first + // place). + if (!options_.authorize()) { - btcd_channel_->set_authenticated(true); send_btcd_result({}, 4); return true; } - // Close once the error response has actually been written, rather than - // stopping synchronously here and racing the (async) send. + // Same digest scheme as config::credential (settings::http_server has no + // direct username:password lookup, only digest-keyed authorized()). + const auto digest = sha256_hash( + "Basic " + encode_base64(username + ":" + password)); + + if (options_.authorized(digest)) + { + btcd_channel_->set_authenticated(digest); + send_btcd_result({}, 4); + return true; + } + + // A failed authenticate ends the session (matches real btcd: invalid + // credentials close the connection rather than leave it open for a + // retry) -- but only once the error has actually reached the client. const code unauthorized{ network::error::unauthorized }; send_btcd_error(unauthorized, two * unauthorized.message().size(), unauthorized); - return false; + return true; } bool protocol_btcd_rpc::handle_session(const code& ec, @@ -407,7 +422,7 @@ void protocol_btcd_rpc::send_btcd_result(value_option&& result, .jsonrpc = btcd_version_, .id = btcd_id_, .result = std::move(result) - }, size_hint); + }, size_hint, error::success); } void protocol_btcd_rpc::send_btcd_error(const code& ec) NOEXCEPT @@ -479,9 +494,9 @@ void protocol_btcd_rpc::send_btcd_rpc(response_t&& model, size_t size_hint, }; message.prepare_payload(); - // SEND restarts the ws reader so the next client frame is accepted. - // handle_complete only stops the channel if close_reason is truthy, and - // only after this write actually completes -- never synchronously here. + // handle_complete only stops (if close_reason is truthy) once this write + // has actually completed -- SEND still restarts the ws reader on success, + // matching the base behavior for every other response this class sends. SEND(std::move(message), handle_complete, _1, close_reason); } diff --git a/test/protocols/btcd/btcd_rpc.cpp b/test/protocols/btcd/btcd_rpc.cpp index 1f2b81d65..fa118ad31 100644 --- a/test/protocols/btcd/btcd_rpc.cpp +++ b/test/protocols/btcd/btcd_rpc.cpp @@ -128,3 +128,75 @@ BOOST_AUTO_TEST_CASE(btcd_rpc__unknown_method__error_keeps_connection_alive) } BOOST_AUTO_TEST_SUITE_END() + +// Method-scoped credentials (channel_btcd::permitted()): a credential's +// optional 'user:pass:method,...' suffix restricts which methods it may +// call, enforced over ws by protocol_btcd_rpc::dispatch_websocket. +// ---------------------------------------------------------------------------- + +BOOST_FIXTURE_TEST_SUITE(btcd_scoped_credential_tests, + btcd_scoped_credential_setup_fixture) + +BOOST_AUTO_TEST_CASE(btcd_scoped_credential__listed_method__permitted) +{ + const auto auth = rpc("authenticate", + R"([")" BTCD_TEST_USERNAME R"(", ")" BTCD_TEST_PASSWORD R"("])"); + BOOST_REQUIRE(!has_error(auth)); + + // BTCD_TEST_SCOPED_METHOD ("session") is the credential's only permitted + // method. + const auto session = rpc("session"); + BOOST_REQUIRE(!has_error(session)); +} + +BOOST_AUTO_TEST_CASE(btcd_scoped_credential__unlisted_method__rejected) +{ + const auto auth = rpc("authenticate", + R"([")" BTCD_TEST_USERNAME R"(", ")" BTCD_TEST_PASSWORD R"("])"); + BOOST_REQUIRE(!has_error(auth)); + + // notifyblocks is a real, implemented handler, not scoped by this + // credential -- confirms rejection is permitted()'s doing, not the + // method being unimplemented. + const auto response = rpc("notifyblocks"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_SUITE_END() + +// authenticate (protocol_btcd_rpc::handle_authenticate): exercises the +// credential-configured branch, not covered by btcd_rpc_tests above (whose +// fixture leaves btcd.credential unset, so every one of those connections +// takes the no-op path). +// ---------------------------------------------------------------------------- + +BOOST_FIXTURE_TEST_SUITE(btcd_auth_tests, btcd_credentialed_setup_fixture) + +BOOST_AUTO_TEST_CASE(btcd_auth__correct_credentials__succeeds_then_session_works) +{ + const auto auth = rpc("authenticate", + R"([")" BTCD_TEST_USERNAME R"(", ")" BTCD_TEST_PASSWORD R"("])"); + BOOST_REQUIRE(!has_error(auth)); + + // Connection survives a successful handshake; other methods now work. + const auto session = rpc("session"); + BOOST_REQUIRE(!has_error(session)); +} + +BOOST_AUTO_TEST_CASE(btcd_auth__wrong_password__rejected) +{ + const auto response = rpc("authenticate", + R"([")" BTCD_TEST_USERNAME R"(", "definitely-wrong"])"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_auth__other_method_before_authenticating__rejected) +{ + // 'session' instead of 'authenticate' as the first message: the + // handshake requires authenticate specifically when a credential is + // configured. + const auto response = rpc("session"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_SUITE_END() diff --git a/test/protocols/btcd/btcd_setup_fixture.cpp b/test/protocols/btcd/btcd_setup_fixture.cpp index 9b7a0398b..d19e6b340 100644 --- a/test/protocols/btcd/btcd_setup_fixture.cpp +++ b/test/protocols/btcd/btcd_setup_fixture.cpp @@ -27,7 +27,9 @@ using namespace boost::beast; BC_PUSH_WARNING(NO_THROW_IN_NOEXCEPT) -btcd_setup_fixture::btcd_setup_fixture(const initializer& setup) +btcd_setup_fixture::btcd_setup_fixture(const initializer& setup, + std::string_view username, std::string_view password, + std::string_view methods) : config_ { system::chain::selection::mainnet, @@ -54,6 +56,10 @@ btcd_setup_fixture::btcd_setup_fixture(const initializer& setup) btcd.binds = { { BTCD_ENDPOINT } }; btcd.connections = 1; + if (!username.empty()) + btcd.credentials.emplace_back(std::string{ username } + ":" + + std::string{ password } + + (methods.empty() ? "" : ":" + std::string{ methods })); database_settings.interval_depth = 2; node_settings.delay_inbound = false; node_settings.minimum_fee_rate = 99.0; diff --git a/test/protocols/btcd/btcd_setup_fixture.hpp b/test/protocols/btcd/btcd_setup_fixture.hpp index 744dd78b2..d50f358a9 100644 --- a/test/protocols/btcd/btcd_setup_fixture.hpp +++ b/test/protocols/btcd/btcd_setup_fixture.hpp @@ -29,7 +29,15 @@ struct btcd_setup_fixture using initializer = std::function; DELETE_COPY_MOVE(btcd_setup_fixture); - explicit btcd_setup_fixture(const initializer& setup); + + // username/password configure a single btcd.credential entry (no auth + // required if username is empty), matching the one-tier credential model. + // methods, if non-empty, scopes that credential to a comma-separated + // method list (config::credential's own "user:pass:method,..." syntax); + // empty (the default) leaves the credential unscoped (all methods). + explicit btcd_setup_fixture(const initializer& setup, + std::string_view username={}, std::string_view password={}, + std::string_view methods={}); ~btcd_setup_fixture(); // JSON-RPC 1.0 over websocket. params must be a json array (json-rpc-v1 @@ -71,4 +79,39 @@ struct btcd_ten_block_setup_fixture } }; +#define BTCD_TEST_USERNAME "user" +#define BTCD_TEST_PASSWORD "pass" + +// Configured with a credential but does not authenticate -- for tests that +// exercise the in-band 'authenticate' call itself (success, wrong-password +// rejection, calling another method before authenticating). +struct btcd_credentialed_setup_fixture + : btcd_setup_fixture +{ + inline btcd_credentialed_setup_fixture() + : btcd_setup_fixture([](test::query_t& query) + { + return test::setup_ten_block_store(query); + }, BTCD_TEST_USERNAME, BTCD_TEST_PASSWORD) + { + } +}; + +#define BTCD_TEST_SCOPED_METHOD "session" + +// Configured with a credential scoped to a single method -- for tests that +// verify channel_btcd::permitted() enforces per-method credential scoping +// over ws once authenticated. +struct btcd_scoped_credential_setup_fixture + : btcd_setup_fixture +{ + inline btcd_scoped_credential_setup_fixture() + : btcd_setup_fixture([](test::query_t& query) + { + return test::setup_ten_block_store(query); + }, BTCD_TEST_USERNAME, BTCD_TEST_PASSWORD, BTCD_TEST_SCOPED_METHOD) + { + } +}; + #endif From 7efb852ed2737ce2b6c3d9345d0c471f8d569104 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Tue, 28 Jul 2026 08:57:54 +0200 Subject: [PATCH 06/10] Gate slow/live-event endpoint tests behind --run-slow. Some endpoint tests (e.g. test_blockconnected_notification) wait on a real external event -- an actual new block -- up to --subscription-timeout, which made a normal pytest run slow and made CI/local runs flaky when no block happened to arrive in time. Mark those @pytest.mark.slow and skip them by default; --run-slow opts back in for a real (if longer) end-to-end check. --- endpoints/README.md | 10 ++++++++++ endpoints/conftest.py | 24 ++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/endpoints/README.md b/endpoints/README.md index e18fc676e..4eb77b4a8 100644 --- a/endpoints/README.md +++ b/endpoints/README.md @@ -205,6 +205,16 @@ pytest test_native.py \ --timeout=30 ``` +- `--run-slow` — Include tests marked `@pytest.mark.slow`: these wait on a + real external event (e.g. `test_blockconnected_notification` waits up to + `--subscription-timeout` for an actual new block) and are skipped by + default so a normal run stays fast. + +```bash +# Include slow/live-event tests, e.g. against a node expecting a block soon +pytest test_btcd_rpc.py --run-slow --subscription-timeout=120 +``` + ## Test Organization ### test_native.py — Native REST diff --git a/endpoints/conftest.py b/endpoints/conftest.py index 8edfcd2de..ad965d73b 100644 --- a/endpoints/conftest.py +++ b/endpoints/conftest.py @@ -158,6 +158,30 @@ def pytest_addoption(parser): default="30", help="Default timeout for requests in seconds (default: 30)" ) + parser.addoption( + "--run-slow", + action="store_true", + default=False, + help="Run tests marked @pytest.mark.slow (wait on a real external " + "event, e.g. a live block, up to --subscription-timeout)" + ) + + +def pytest_configure(config): + config.addinivalue_line( + "markers", + "slow: waits on a real external event (e.g. a live block); " + "skipped by default, use --run-slow to include" + ) + + +def pytest_collection_modifyitems(config, items): + if config.getoption("--run-slow"): + return + skip_slow = pytest.mark.skip(reason="needs --run-slow (waits on a real external event)") + for item in items: + if "slow" in item.keywords: + item.add_marker(skip_slow) def pytest_report_header(config): From 176871865fab02e6ee64920d48e4c1d7fc0ecdd2 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Thu, 30 Jul 2026 11:29:19 +0200 Subject: [PATCH 07/10] Bridge, extend, and complete the btcd RPC/websocket endpoint. Reaches standard chain methods over the ws connection (not just plain http post), adds getcurrentnet, implements loadtxfilter with filtered block notifications and rescanblocks (base58 + segwit address/outpoint matching), and wires the remaining generic-tooling Phase C methods (getdifficulty, getinfo, getnettotals, getnetworkhashps, createrawtransaction, decoderawtransaction, decodescript, validateaddress, help). See docs/btcd-endpoint.md for details. --- builds/gnu/Makefile.am | 2 + include/bitcoin/server/interfaces/btcd.hpp | 26 +- .../protocols/protocol_bitcoind_rpc.hpp | 24 + .../server/protocols/protocol_btcd_rpc.hpp | 113 ++++- .../bitcoind/protocol_bitcoind_rpc.cpp | 39 +- src/protocols/btcd/protocol_btcd_rpc.cpp | 101 +++-- .../btcd/protocol_btcd_rpc_filter.cpp | 332 ++++++++++++++ .../btcd/protocol_btcd_rpc_utility.cpp | 413 ++++++++++++++++++ 8 files changed, 1004 insertions(+), 46 deletions(-) create mode 100644 src/protocols/btcd/protocol_btcd_rpc_filter.cpp create mode 100644 src/protocols/btcd/protocol_btcd_rpc_utility.cpp diff --git a/builds/gnu/Makefile.am b/builds/gnu/Makefile.am index 6d17b4db5..653c30b37 100644 --- a/builds/gnu/Makefile.am +++ b/builds/gnu/Makefile.am @@ -65,6 +65,8 @@ src_libbitcoin_server_la_SOURCES = \ ${srcdir}/../../src/protocols/bitcoind/protocol_bitcoind_rpc.cpp \ ${srcdir}/../../src/protocols/bitcoind/protocol_bitcoind_rpc_json.cpp \ ${srcdir}/../../src/protocols/btcd/protocol_btcd_rpc.cpp \ + ${srcdir}/../../src/protocols/btcd/protocol_btcd_rpc_filter.cpp \ + ${srcdir}/../../src/protocols/btcd/protocol_btcd_rpc_utility.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum_addresses.cpp \ ${srcdir}/../../src/protocols/electrum/protocol_electrum_fees.cpp \ diff --git a/include/bitcoin/server/interfaces/btcd.hpp b/include/bitcoin/server/interfaces/btcd.hpp index 989926c83..7df295b81 100644 --- a/include/bitcoin/server/interfaces/btcd.hpp +++ b/include/bitcoin/server/interfaces/btcd.hpp @@ -57,7 +57,21 @@ struct btcd_methods method<"stopnotifyreceived", value_t>{ "addresses" }, method<"notifyspent", value_t>{ "outpoints" }, method<"stopnotifyspent", value_t>{ "outpoints" }, - method<"rescan", string_t, value_t, value_t, optional<""_t>>{ "beginblock", "addresses", "outpoints", "endblock" } + method<"rescan", string_t, value_t, value_t, optional<""_t>>{ "beginblock", "addresses", "outpoints", "endblock" }, + + /// Admin (network magic, checked once by btcwallet/lnd at connect). + method<"getcurrentnet">{}, + + /// Phase C: generic btcd-tooling compatibility (no lnd consumer). + method<"getdifficulty">{}, + method<"getinfo">{}, + method<"getnettotals">{}, + method<"getnetworkhashps", optional<120_u32>, optional<-1_i32>>{ "blocks", "height" }, + method<"createrawtransaction", array_t, object_t, optional<0_u32>>{ "inputs", "outputs", "locktime" }, + method<"decoderawtransaction", string_t>{ "hexstring" }, + method<"decodescript", string_t>{ "hex" }, + method<"validateaddress", string_t>{ "address" }, + method<"help", optional<""_t>>{ "command" } }; template @@ -81,6 +95,16 @@ struct btcd_methods using notify_spent = at<11>; using stop_notify_spent = at<12>; using rescan = at<13>; + using get_current_net = at<14>; + using get_difficulty = at<15>; + using get_info = at<16>; + using get_net_totals = at<17>; + using get_network_hash_ps = at<18>; + using create_raw_transaction = at<19>; + using decode_raw_transaction = at<20>; + using decode_script = at<21>; + using validate_address = at<22>; + using help = at<23>; }; } // namespace interface diff --git a/include/bitcoin/server/protocols/protocol_bitcoind_rpc.hpp b/include/bitcoin/server/protocols/protocol_bitcoind_rpc.hpp index 6f5f067f6..925b722e0 100644 --- a/include/bitcoin/server/protocols/protocol_bitcoind_rpc.hpp +++ b/include/bitcoin/server/protocols/protocol_bitcoind_rpc.hpp @@ -61,6 +61,14 @@ class BCS_API protocol_bitcoind_rpc void handle_receive_post(const code& ec, const post::cptr& post) NOEXCEPT override; + /// Dispatch a chain-rpc message with no post-http context (websocket). + /// Caches id/version (as handle_receive_post does via set_rpc_request) + /// and notifies rpc_dispatcher_ directly; returns error::unexpected_method + /// if the message names a method this dispatcher does not recognize. + /// Callers (e.g. protocol_btcd_rpc::dispatch_websocket) use this to reach + /// the standard chain handlers (getblockcount etc.) over websocket. + code dispatch_rpc(const network::rpc::request_t& message) NOEXCEPT; + /// Handlers. bool handle_get_best_block_hash(const code& ec, rpc_interface::get_best_block_hash) NOEXCEPT; @@ -110,6 +118,22 @@ class BCS_API protocol_bitcoind_rpc rpc_interface::send_raw_transaction, const std::string& hexstring, double maxfeerate) NOEXCEPT; + /// Serialize an object (chain::header, chain::transaction, ...) to a + /// base16 string. Template, so defined here (not *_json.cpp) -- shared + /// with derived classes (e.g. protocol_btcd_rpc's filtered-notification + /// tx serialization), not just this class's own handlers. + template + static std::string to_text(const Object& object, size_t size, + Args&&... args) NOEXCEPT + { + std::string out(two * size, '\0'); + system::stream::out::fast sink{ out }; + system::write::base16::fast writer{ sink }; + object.to_data(writer, std::forward(args)...); + BC_ASSERT(writer); + return out; + } + /// Json context helpers (shared with rest, defined in *_json.cpp). static uint32_t median_time_past(const node::query& query, const database::header_link& link) NOEXCEPT; diff --git a/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp index dc52d5618..e3895fff8 100644 --- a/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp +++ b/include/bitcoin/server/protocols/protocol_btcd_rpc.hpp @@ -21,6 +21,8 @@ #include #include +#include +#include #include #include #include @@ -31,9 +33,12 @@ namespace server { /// btcd-compatible json-rpc-v1 over websocket. Inherits the standard chain /// handlers (getblockcount, getblock, etc.) from protocol_bitcoind_rpc, which -/// remain reachable via the inherited http post path (unchanged). Adds the -/// btcd-only extension methods (session/notification/filter/admin) over -/// websocket via a second dispatcher (dispatch_websocket, below). +/// remain reachable both via the inherited http post path (unchanged) and, +/// bridged, over this class's ws dispatcher (dispatch_websocket falls back to +/// protocol_bitcoind_rpc::dispatch_rpc when the btcd-only dispatcher reports +/// unexpected_method). Adds the btcd-only extension methods (session/ +/// notification/filter/admin) over websocket via a second dispatcher +/// (dispatch_websocket, below). /// /// Attaches immediately on connect (session_btcd = session_server< /// protocol_btcd_rpc>, no separate handshake protocol/stage) -- matching real @@ -51,10 +56,6 @@ namespace server { /// http client (no upgrade round-trip to mask the gap, unlike ws) could land /// in before this class ever subscribed its handlers. /// -/// TODO (phase B): bridge the standard chain handlers into the ws dispatcher -/// too, so a single ws connection can also reach them (real btcd/btcwallet -/// clients expect this); requires reconciling protocol_bitcoind_rpc's -/// post-oriented private send path with this class's ws-oriented senders. class BCS_API protocol_btcd_rpc : public server::protocol_bitcoind_rpc, protected network::tracker @@ -73,7 +74,13 @@ class BCS_API protocol_btcd_rpc : server::protocol_bitcoind_rpc(session, channel, options), network::tracker(session->log), options_(options), - btcd_channel_(std::dynamic_pointer_cast(channel)) + btcd_channel_(std::dynamic_pointer_cast(channel)), + p2kh_(session->system_settings().forks.difficult ? + system::wallet::payment_address::mainnet_p2kh : + system::wallet::payment_address::testnet_p2kh), + p2sh_(session->system_settings().forks.difficult ? + system::wallet::payment_address::mainnet_p2sh : + system::wallet::payment_address::testnet_p2sh) { } @@ -81,8 +88,10 @@ class BCS_API protocol_btcd_rpc void stopping(const code& ec) NOEXCEPT override; protected: - /// Dispatch btcd ws frames (standard chain requests dispatch normally - /// over http post via the inherited handle_receive_post). + /// Dispatch btcd ws frames: tries the btcd-only dispatcher first, then + /// falls back to the inherited chain dispatcher (dispatch_rpc) so + /// standard methods (getblockcount etc.) are also reachable over ws, not + /// only via the inherited handle_receive_post's http post path. void dispatch_websocket( const network::http::request& request) NOEXCEPT override; @@ -104,6 +113,46 @@ class BCS_API protocol_btcd_rpc const std::string& username, const std::string& password) NOEXCEPT; bool handle_session(const code& ec, btcd_interface::session) NOEXCEPT; + /// Handler (network magic; btcwallet/lnd check this once at connect to + /// confirm they're talking to the expected network). + bool handle_get_current_net(const code& ec, + btcd_interface::get_current_net) NOEXCEPT; + + /// Handlers (Phase C: generic btcd-tooling compatibility, no lnd + /// consumer -- see docs/btcd-endpoint.md). + bool handle_get_difficulty(const code& ec, + btcd_interface::get_difficulty) NOEXCEPT; + bool handle_get_info(const code& ec, btcd_interface::get_info) NOEXCEPT; + bool handle_get_net_totals(const code& ec, + btcd_interface::get_net_totals) NOEXCEPT; + bool handle_get_network_hash_ps(const code& ec, + btcd_interface::get_network_hash_ps, uint32_t blocks, + int32_t height) NOEXCEPT; + bool handle_create_raw_transaction(const code& ec, + btcd_interface::create_raw_transaction, + const network::rpc::array_t& inputs, + const network::rpc::object_t& outputs, uint32_t locktime) NOEXCEPT; + bool handle_decode_raw_transaction(const code& ec, + btcd_interface::decode_raw_transaction, + const std::string& hexstring) NOEXCEPT; + bool handle_decode_script(const code& ec, + btcd_interface::decode_script, const std::string& hex) NOEXCEPT; + bool handle_validate_address(const code& ec, + btcd_interface::validate_address, + const std::string& address) NOEXCEPT; + bool handle_help(const code& ec, btcd_interface::help, + const std::string& command) NOEXCEPT; + + /// Address string (base58 p2kh/p2sh or bech32/bech32m p2wpkh/p2wsh/p2tr) + /// to output script, for createrawtransaction. A separate, standalone + /// helper from parse_filter_addresses (which classifies into this + /// class's watch-list hash buckets, not a reusable script) -- kept + /// duplicated rather than forcing a shared abstraction onto already- + /// tested filter-parsing code for a small amount of address-string + /// parsing logic. + code parse_output_script(const std::string& text, + system::chain::script& out) NOEXCEPT; + /// Handlers (block subscription). bool handle_notify_blocks(const code& ec, btcd_interface::notify_blocks) NOEXCEPT; @@ -116,7 +165,15 @@ class BCS_API protocol_btcd_rpc bool handle_stop_notify_new_transactions(const code& ec, btcd_interface::stop_notify_new_transactions) NOEXCEPT; - /// Handlers (address/outpoint filtering, not_implemented pending phase B). + /// Handlers (address/outpoint filtering). loadtxfilter never itself + /// triggers notifications (matching real btcd) -- notifyblocks remains + /// the only thing that arms delivery; once armed, do_block_connected/ + /// do_block_disconnected send filteredblockconnected/disconnected + /// alongside the existing unfiltered blockconnected/disconnected, + /// unconditionally (an empty/never-loaded filter just yields an empty + /// subscribedtxs array, matching real btcd's own behavior). rescanblocks + /// replays the same match logic against explicitly named historical + /// blocks using the already-loaded filter. bool handle_load_tx_filter(const code& ec, btcd_interface::load_tx_filter, bool reload, const network::rpc::value_t& addresses, @@ -125,6 +182,24 @@ class BCS_API protocol_btcd_rpc btcd_interface::rescan_blocks, const network::rpc::value_t& blockhashes) NOEXCEPT; + /// Filter parsing (loadtxfilter). Merges into the existing filter unless + /// reload clears it first. Returns error::invalid_argument if any address + /// fails to parse as either a base58 (p2kh/p2sh) or bech32/bech32m + /// (p2wpkh/p2wsh/p2tr) address, or any outpoint is malformed. + code parse_filter_addresses(bool reload, + const network::rpc::value_t& addresses) NOEXCEPT; + code parse_filter_outpoints(bool reload, + const network::rpc::value_t& outpoints) NOEXCEPT; + + /// Match a block's transactions against the loaded filter: an output + /// paying a watched address, or an input spending a watched outpoint. + /// A matched address-output's own outpoint is added to the watched set + /// (auto-tracking a future spend of it), matching real btcd's + /// wsClientFilter behavior. Returns the hex-encoded (witness) matched + /// transactions, in block order; empty if none matched. + network::rpc::array_t match_filtered_transactions( + const system::chain::block& block) NOEXCEPT; + /// Handler (admin, permanently not_implemented). bool handle_stop(const code& ec, btcd_interface::stop) NOEXCEPT; @@ -182,11 +257,27 @@ class BCS_API protocol_btcd_rpc std::atomic_bool subscribed_blocks_{}; const options_t& options_; const channel_btcd::ptr btcd_channel_; + const uint8_t p2kh_; + const uint8_t p2sh_; // These are protected by strand. btcd_dispatcher btcd_dispatcher_{}; network::rpc::version btcd_version_{}; network::rpc::id_option btcd_id_{}; + + // Tx filter (loadtxfilter), protected by strand -- populated by + // handle_load_tx_filter, read/mutated by match_filtered_transactions + // (do_block_connected/do_block_disconnected, handle_rescan_blocks), all + // of which run stranded (ws dispatch and POST_BTCD-posted chase handlers + // alike). p2kh_/p2wpkh_ share one hash set (both 20-byte hash160 payloads + // but under different script templates, kept separate so a p2kh watch + // never matches a p2wpkh output or vice versa); p2sh_/p2wsh_ likewise. + std::unordered_set pay_key_hashes_{}; + std::unordered_set pay_script_hashes_{}; + std::set pay_witness_key_hash_programs_{}; + std::set pay_witness_script_hash_programs_{}; + std::set pay_witness_taproot_programs_{}; + std::unordered_set filter_outpoints_{}; }; } // namespace server diff --git a/src/protocols/bitcoind/protocol_bitcoind_rpc.cpp b/src/protocols/bitcoind/protocol_bitcoind_rpc.cpp index bbfe2cf6f..99535d1e1 100644 --- a/src/protocols/bitcoind/protocol_bitcoind_rpc.cpp +++ b/src/protocols/bitcoind/protocol_bitcoind_rpc.cpp @@ -162,15 +162,17 @@ void protocol_bitcoind_rpc::handle_receive_post(const code& ec, stop(code); } -template -std::string to_text(const Object& object, size_t size, Args&&... args) NOEXCEPT +// Dispatch a chain-rpc message with no post-http context (websocket). Mirrors +// handle_receive_post's id/version caching (set_rpc_request) and dispatcher +// notify, without the post-specific host/origin/body checks -- those are +// meaningless for a ws frame (channel_btcd::permitted() and the ws upgrade +// itself already gate access) and are handled by the caller. +code protocol_bitcoind_rpc::dispatch_rpc(const request_t& message) NOEXCEPT { - std::string out(two * size, '\0'); - stream::out::fast sink{ out }; - write::base16::fast writer{ sink }; - object.to_data(writer, std::forward(args)...); - BC_ASSERT(writer); - return out; + BC_ASSERT(stranded()); + id_ = message.id; + version_ = message.jsonrpc; + return rpc_dispatcher_.notify(message); } // Handlers. @@ -728,6 +730,27 @@ void protocol_bitcoind_rpc::send_rpc(response_t&& model, BC_ASSERT(stranded()); using namespace http; static const auto json = from_media_type(media_type::application_json); + + // Websocket frames carry no cached post request (dispatch_rpc does not + // set one, unlike handle_receive_post) and have no per-message headers + // to echo (add_common_headers/add_access_control_headers is a post-only + // concept) -- send a minimal response, matching protocol_btcd_rpc's own + // ws senders. + if (websocket()) + { + id_.reset(); + version_ = version::undefined; + http::response message{ status::ok, 11 }; + message.set(field::content_type, json); + message.body() = rpc::response + { + { .size_hint = size_hint }, std::move(model), + }; + message.prepare_payload(); + SEND(std::move(message), handle_complete, _1, error::success); + return; + } + const auto request = reset_rpc_request(); http::response message{ status::ok, request->version() }; add_common_headers(message, *request); diff --git a/src/protocols/btcd/protocol_btcd_rpc.cpp b/src/protocols/btcd/protocol_btcd_rpc.cpp index 11721079c..3943dbff5 100644 --- a/src/protocols/btcd/protocol_btcd_rpc.cpp +++ b/src/protocols/btcd/protocol_btcd_rpc.cpp @@ -58,6 +58,16 @@ void protocol_btcd_rpc::start() NOEXCEPT SUBSCRIBE_BTCD(handle_authenticate, _1, _2, _3, _4); SUBSCRIBE_BTCD(handle_session, _1, _2); + SUBSCRIBE_BTCD(handle_get_current_net, _1, _2); + SUBSCRIBE_BTCD(handle_get_difficulty, _1, _2); + SUBSCRIBE_BTCD(handle_get_info, _1, _2); + SUBSCRIBE_BTCD(handle_get_net_totals, _1, _2); + SUBSCRIBE_BTCD(handle_get_network_hash_ps, _1, _2, _3, _4); + SUBSCRIBE_BTCD(handle_create_raw_transaction, _1, _2, _3, _4, _5); + SUBSCRIBE_BTCD(handle_decode_raw_transaction, _1, _2, _3); + SUBSCRIBE_BTCD(handle_decode_script, _1, _2, _3); + SUBSCRIBE_BTCD(handle_validate_address, _1, _2, _3); + SUBSCRIBE_BTCD(handle_help, _1, _2, _3); SUBSCRIBE_BTCD(handle_notify_blocks, _1, _2); SUBSCRIBE_BTCD(handle_stop_notify_blocks, _1, _2); SUBSCRIBE_BTCD(handle_notify_new_transactions, _1, _2, _3); @@ -86,13 +96,12 @@ void protocol_btcd_rpc::stopping(const code& ec) NOEXCEPT // Websocket dispatch. // ---------------------------------------------------------------------------- // btcd extension methods (session/notify/filter/admin, including -// authenticate) arrive as ws frames here. Standard chain methods -// (getblockcount etc.) remain reachable via plain http post on the same -// endpoint, unchanged, via the inherited handle_receive_post. They are NOT -// yet bridged into this ws dispatcher: that requires reconciling -// protocol_bitcoind_rpc's post-oriented private send path (which caches the -// original http::request for header derivation) with this class's -// ws-oriented senders. Tracked for phase B. +// authenticate) arrive as ws frames here, and are tried first. Standard +// chain methods (getblockcount etc.), inherited from protocol_bitcoind_rpc, +// remain reachable via plain http post on the same endpoint (unchanged, via +// the inherited handle_receive_post) and are also bridged into this ws +// dispatcher via dispatch_rpc, once the btcd-only dispatcher reports +// unexpected_method. // // A credential may be scoped to a subset of methods (see config::credential // / channel_btcd::permitted()), so each call is checked against that scope -- @@ -127,7 +136,15 @@ void protocol_btcd_rpc::dispatch_websocket( return; } - const auto code = btcd_dispatcher_.notify(message); + // Try the btcd-only extension methods first, then fall back to the + // standard chain handlers inherited from protocol_bitcoind_rpc (dispatch_ + // rpc), so a single ws connection can reach both. dispatcher::notify only + // returns unexpected_method for a method-name lookup miss (never for an + // argument mismatch), so this fallback cannot mask a real handler error. + auto code = btcd_dispatcher_.notify(message); + if (code == network::error::unexpected_method) + code = dispatch_rpc(message); + if (!code) return; @@ -195,6 +212,22 @@ bool protocol_btcd_rpc::handle_session(const code& ec, return true; } +bool protocol_btcd_rpc::handle_get_current_net(const code& ec, + btcd_interface::get_current_net) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // network_settings().identifier is the p2p handshake magic (e.g. + // 3652501241 / 0xd9b4bef9 for mainnet) -- the same value real btcd + // returns from getcurrentnet, verified once by btcwallet/lnd at connect + // to confirm they're talking to the expected network. + send_btcd_result( + value_t{ possible_sign_cast(network_settings().identifier) }, + 20); + return true; +} + // Handlers (block subscription). // ---------------------------------------------------------------------------- @@ -239,26 +272,9 @@ bool protocol_btcd_rpc::handle_stop_notify_new_transactions(const code& ec, return true; } -// Handlers (address/outpoint filtering, not_implemented pending phase B). +// Handlers (address/outpoint filtering): see protocol_btcd_rpc_filter.cpp. // ---------------------------------------------------------------------------- -bool protocol_btcd_rpc::handle_load_tx_filter(const code& ec, - btcd_interface::load_tx_filter, bool, const value_t&, - const value_t&) NOEXCEPT -{ - if (stopped(ec)) return false; - send_btcd_error(error::not_implemented); - return true; -} - -bool protocol_btcd_rpc::handle_rescan_blocks(const code& ec, - btcd_interface::rescan_blocks, const value_t&) NOEXCEPT -{ - if (stopped(ec)) return false; - send_btcd_error(error::not_implemented); - return true; -} - // Handler (admin, permanently not_implemented). // ---------------------------------------------------------------------------- @@ -380,6 +396,28 @@ void protocol_btcd_rpc::do_block_connected(node::header_t link_value) NOEXCEPT header->timestamp()) }); send_btcd_notification("blockconnected", std::move(params), 256); + + // btcd 'filteredblockconnected': [height, header, subscribedtxs] + // (matching btcjson.FilteredBlockConnectedNtfn). Sent unconditionally + // alongside blockconnected, same as real btcd (verified against + // rpcwebsocket.go's notificationHandler: both notifyBlockConnected and + // notifyFilteredBlockConnected fire for every notifyblocks client + // regardless of whether a filter was ever loaded) -- an empty/never- + // loaded filter just yields an empty subscribedtxs array. + constexpr auto witness = true; + const auto block = query.get_block(link, witness); + if (!block) + return; + + array_t filtered_params{}; + filtered_params.emplace_back(value_t{ + possible_sign_cast(height) }); + filtered_params.emplace_back(value_t{ + to_text(*header, chain::header::serialized_size()) }); + filtered_params.emplace_back(value_t{ match_filtered_transactions(*block) }); + + send_btcd_notification("filteredblockconnected", + std::move(filtered_params), 256); } void protocol_btcd_rpc::do_block_disconnected( @@ -408,6 +446,17 @@ void protocol_btcd_rpc::do_block_disconnected( header->timestamp()) }); send_btcd_notification("blockdisconnected", std::move(params), 256); + + // btcd 'filteredblockdisconnected': [height, header] (no subscribedtxs + // field -- matching btcjson.FilteredBlockDisconnectedNtfn). + array_t filtered_params{}; + filtered_params.emplace_back(value_t{ + possible_sign_cast(height) }); + filtered_params.emplace_back(value_t{ + to_text(*header, chain::header::serialized_size()) }); + + send_btcd_notification("filteredblockdisconnected", + std::move(filtered_params), 256); } // Senders. diff --git a/src/protocols/btcd/protocol_btcd_rpc_filter.cpp b/src/protocols/btcd/protocol_btcd_rpc_filter.cpp new file mode 100644 index 000000000..b5d78a5e4 --- /dev/null +++ b/src/protocols/btcd/protocol_btcd_rpc_filter.cpp @@ -0,0 +1,332 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include + +#include +#include +#include + +namespace libbitcoin { +namespace server { + +#define CLASS protocol_btcd_rpc + +using namespace system; +using namespace network; +using namespace network::rpc; + +BC_PUSH_WARNING(NO_THROW_IN_NOEXCEPT) +BC_PUSH_WARNING(SMART_PTR_NOT_NEEDED) +BC_PUSH_WARNING(NO_VALUE_OR_CONST_REF_SHARED_PTR) + +// Filter parsing (loadtxfilter). +// ---------------------------------------------------------------------------- +// Addresses/outpoints arrive as generic value_t (not pre-typed by the +// dispatcher, unlike single-valued params), so each is unpacked here via the +// same std::holds_alternative/std::get idiom the dispatcher itself uses +// internally (see network::rpc::dispatcher::get_array/get_object). + +code protocol_btcd_rpc::parse_filter_addresses(bool reload, + const value_t& addresses) NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (reload) + { + pay_key_hashes_.clear(); + pay_script_hashes_.clear(); + pay_witness_key_hash_programs_.clear(); + pay_witness_script_hash_programs_.clear(); + pay_witness_taproot_programs_.clear(); + } + + if (!std::holds_alternative(addresses.value())) + return error::invalid_argument; + + for (const auto& item: std::get(addresses.value())) + { + if (!std::holds_alternative(item.value())) + return error::invalid_argument; + + const auto& text = std::get(item.value()); + + // Try base58 (p2kh/p2sh) first. + const wallet::payment_address base58(text); + if (base58) + { + if (base58.prefix() == p2kh_) + pay_key_hashes_.insert(base58.hash()); + else if (base58.prefix() == p2sh_) + pay_script_hashes_.insert(base58.hash()); + else + return error::invalid_argument; + + continue; + } + + // Otherwise bech32/bech32m (p2wpkh/p2wsh/p2tr). + const wallet::witness_address segwit(text); + if (!segwit) + return error::invalid_argument; + + switch (segwit.identifier()) + { + case wallet::witness_address::program_type::version0_p2kh: + pay_witness_key_hash_programs_.insert(segwit.program()); + break; + case wallet::witness_address::program_type::version0_p2sh: + pay_witness_script_hash_programs_.insert(segwit.program()); + break; + case wallet::witness_address::program_type::version1_taproot: + pay_witness_taproot_programs_.insert(segwit.program()); + break; + default: + return error::invalid_argument; + } + } + + return error::success; +} + +code protocol_btcd_rpc::parse_filter_outpoints(bool reload, + const value_t& outpoints) NOEXCEPT +{ + BC_ASSERT(stranded()); + + if (reload) + filter_outpoints_.clear(); + + if (!std::holds_alternative(outpoints.value())) + return error::invalid_argument; + + for (const auto& item: std::get(outpoints.value())) + { + if (!std::holds_alternative(item.value())) + return error::invalid_argument; + + const auto& fields = std::get(item.value()); + const auto hash_it = fields.find("hash"); + const auto index_it = fields.find("index"); + if (hash_it == fields.end() || index_it == fields.end() || + !std::holds_alternative(hash_it->second.value()) || + !std::holds_alternative(index_it->second.value())) + return error::invalid_argument; + + hash_digest hash{}; + uint32_t index{}; + if (!decode_hash(hash, std::get(hash_it->second.value())) || + !to_integer(index, std::get(index_it->second.value()))) + return error::invalid_argument; + + filter_outpoints_.emplace(hash, index); + } + + return error::success; +} + +// Handlers (address/outpoint filtering). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_load_tx_filter(const code& ec, + btcd_interface::load_tx_filter, bool reload, const value_t& addresses, + const value_t& outpoints) NOEXCEPT +{ + if (stopped(ec)) + return false; + + if (const auto fault = parse_filter_addresses(reload, addresses)) + { + send_btcd_error(fault); + return true; + } + + if (const auto fault = parse_filter_outpoints(reload, outpoints)) + { + send_btcd_error(fault); + return true; + } + + send_btcd_result({}, 4); + return true; +} + +bool protocol_btcd_rpc::handle_rescan_blocks(const code& ec, + btcd_interface::rescan_blocks, const value_t& blockhashes) NOEXCEPT +{ + if (stopped(ec)) + return false; + + if (!std::holds_alternative(blockhashes.value())) + { + send_btcd_error(error::invalid_argument); + return true; + } + + constexpr auto witness = true; + const auto& query = archive(); + array_t discovered{}; + + for (const auto& item: std::get(blockhashes.value())) + { + if (!std::holds_alternative(item.value())) + { + send_btcd_error(error::invalid_argument); + return true; + } + + const auto& text = std::get(item.value()); + hash_digest hash{}; + if (!decode_hash(hash, text)) + { + send_btcd_error(error::invalid_argument); + return true; + } + + const auto block = query.get_block(query.to_header(hash), witness); + if (!block) + { + send_btcd_error(error::not_found); + return true; + } + + auto matched = match_filtered_transactions(*block); + if (!matched.empty()) + { + discovered.emplace_back(value_t{ object_t + { + { "hash", value_t{ text } }, + { "transactions", value_t{ std::move(matched) } } + } }); + } + } + + send_btcd_result(value_t{ std::move(discovered) }, 256); + return true; +} + +// Filter matching. +// ---------------------------------------------------------------------------- +// Manual per-block script inspection, not the persisted address index +// (get_history/address_enabled etc.) -- that index is for whole-chain +// historical lookups by address, the wrong tool for "does this one just- +// connected (or explicitly named, for rescanblocks) block match this small +// in-memory watch-list", and would require the address index to be enabled +// at all. Mirrors real btcd's own per-client wsClientFilter (rpcwebsocket.go: +// notifyForTx/subscribedClients), including auto-tracking: an output that +// matches a watched address has its own outpoint added to filter_outpoints_, +// so a later spend of it also matches without an explicit re-subscribe. + +array_t protocol_btcd_rpc::match_filtered_transactions( + const chain::block& block) NOEXCEPT +{ + BC_ASSERT(stranded()); + constexpr auto witness = true; + + array_t matched{}; + for (const auto& tx: *block.transactions_ptr()) + { + auto hit = false; + + for (const auto& input: *tx->inputs_ptr()) + if (filter_outpoints_.contains(input->point())) + hit = true; + + const auto hash = tx->hash(false); + const auto& outputs = *tx->outputs_ptr(); + for (size_t index = 0; index < outputs.size(); ++index) + { + const auto& script = outputs[index]->script(); + const auto point_index = possible_narrow_cast(index); + + switch (script.output_pattern()) + { + case chain::script_pattern::pay_key_hash: + { + const auto address = wallet::payment_address:: + extract_output(script, p2kh_, p2sh_); + if (address && pay_key_hashes_.contains(address.hash())) + { + hit = true; + filter_outpoints_.emplace(hash, point_index); + } + break; + } + case chain::script_pattern::pay_script_hash: + { + const auto address = wallet::payment_address:: + extract_output(script, p2kh_, p2sh_); + if (address && pay_script_hashes_.contains(address.hash())) + { + hit = true; + filter_outpoints_.emplace(hash, point_index); + } + break; + } + case chain::script_pattern::pay_witness_key_hash: + { + const auto& program = script.witness_program(); + if (program && pay_witness_key_hash_programs_.contains( + *program)) + { + hit = true; + filter_outpoints_.emplace(hash, point_index); + } + break; + } + case chain::script_pattern::pay_witness_script_hash: + { + const auto& program = script.witness_program(); + if (program && pay_witness_script_hash_programs_.contains( + *program)) + { + hit = true; + filter_outpoints_.emplace(hash, point_index); + } + break; + } + case chain::script_pattern::pay_witness_v1_taproot: + { + const auto& program = script.witness_program(); + if (program && pay_witness_taproot_programs_.contains( + *program)) + { + hit = true; + filter_outpoints_.emplace(hash, point_index); + } + break; + } + default: + break; + } + } + + if (hit) + matched.emplace_back(value_t{ to_text(*tx, + tx->serialized_size(witness), witness) }); + } + + return matched; +} + +BC_POP_WARNING() +BC_POP_WARNING() +BC_POP_WARNING() + +} // namespace server +} // namespace libbitcoin diff --git a/src/protocols/btcd/protocol_btcd_rpc_utility.cpp b/src/protocols/btcd/protocol_btcd_rpc_utility.cpp new file mode 100644 index 000000000..a823d86d3 --- /dev/null +++ b/src/protocols/btcd/protocol_btcd_rpc_utility.cpp @@ -0,0 +1,413 @@ +/** + * Copyright (c) 2011-2026 libbitcoin developers + * + * This file is part of libbitcoin. + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ +#include + +#include +#include +#include +#include + +namespace libbitcoin { +namespace server { + +#define CLASS protocol_btcd_rpc + +using namespace system; +using namespace network; +using namespace network::rpc; +using namespace boost::json; + +BC_PUSH_WARNING(NO_THROW_IN_NOEXCEPT) +BC_PUSH_WARNING(SMART_PTR_NOT_NEEDED) +BC_PUSH_WARNING(NO_VALUE_OR_CONST_REF_SHARED_PTR) + +// Phase C: generic btcd-tooling compatibility (no lnd consumer). +// ---------------------------------------------------------------------------- + +bool protocol_btcd_rpc::handle_get_difficulty(const code& ec, + btcd_interface::get_difficulty) NOEXCEPT +{ + if (stopped(ec)) + return false; + + const auto& query = archive(); + const auto header = query.get_header(query.to_confirmed( + query.get_top_confirmed())); + if (!header) + { + send_btcd_error(database::error::integrity); + return true; + } + + send_btcd_result(header->difficulty(), 20); + return true; +} + +bool protocol_btcd_rpc::handle_get_info(const code& ec, + btcd_interface::get_info) NOEXCEPT +{ + if (stopped(ec)) + return false; + + const auto& query = archive(); + const auto top = query.get_top_confirmed(); + const auto header = query.get_header(query.to_confirmed(top)); + if (!header) + { + send_btcd_error(database::error::integrity); + return true; + } + + // Peer-dependent fields (connections) and errors are reported as + // empty/zero -- libbitcoin-server is not a peer-introspection service, + // matching the same honesty convention already used by the inherited + // bitcoind getnetworkinfo handler. + send_btcd_result(object_t + { + { "version", 0 }, + { "protocolversion", 70016 }, + { "blocks", possible_sign_cast(top) }, + { "timeoffset", 0 }, + { "connections", 0 }, + { "proxy", std::string{} }, + { "difficulty", header->difficulty() }, + { "testnet", chain_name(query) != "main" }, + { "relayfee", 0.00001 }, + { "errors", std::string{} } + }, 256); + return true; +} + +bool protocol_btcd_rpc::handle_get_net_totals(const code& ec, + btcd_interface::get_net_totals) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // Byte counters are not tracked by this node; reported as zero rather + // than not_implemented (matches getnetworkinfo's own convention for + // untracked peer-dependent fields). timemillis is real (current time). + send_btcd_result(object_t + { + { "totalbytesrecv", 0 }, + { "totalbytessent", 0 }, + { "timemillis", possible_sign_cast(zulu_time()) * 1000 } + }, 64); + return true; +} + +bool protocol_btcd_rpc::handle_get_network_hash_ps(const code& ec, + btcd_interface::get_network_hash_ps, uint32_t, int32_t height) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // Approximated from the requested height's difficulty and the standard + // 600s target spacing, not a windowed average over 'blocks' -- there is + // no existing big-integer cumulative-work-over-a-range helper in this + // codebase to build a real windowed estimate from, and this is generic- + // tooling scope (no lnd consumer), not consensus-critical. + const auto& query = archive(); + const auto top = query.get_top_confirmed(); + const auto target = (height < 0) ? top : + std::min(static_cast(height), top); + + const auto header = query.get_header(query.to_confirmed(target)); + if (!header) + { + send_btcd_error(database::error::integrity); + return true; + } + + constexpr auto average_seconds_per_block = 600.0; + constexpr auto two_to_the_32 = 4294967296.0; + send_btcd_result( + header->difficulty() * two_to_the_32 / average_seconds_per_block, 20); + return true; +} + +// Raw transaction/script utilities. +// ---------------------------------------------------------------------------- + +// A separate, standalone helper from parse_filter_addresses (see header). +code protocol_btcd_rpc::parse_output_script(const std::string& text, + chain::script& out) NOEXCEPT +{ + const wallet::payment_address base58(text); + if (base58) + { + out = base58.output_script(p2kh_, p2sh_); + return error::success; + } + + const wallet::witness_address segwit(text); + if (segwit) + { + out = segwit.script(); + return error::success; + } + + return error::invalid_argument; +} + +bool protocol_btcd_rpc::handle_create_raw_transaction(const code& ec, + btcd_interface::create_raw_transaction, const array_t& inputs, + const object_t& outputs, uint32_t locktime) NOEXCEPT +{ + if (stopped(ec)) + return false; + + chain::inputs ins{}; + ins.reserve(inputs.size()); + for (const auto& item: inputs) + { + if (!std::holds_alternative(item.value())) + { + send_btcd_error(error::invalid_argument); + return true; + } + + const auto& fields = std::get(item.value()); + const auto txid_it = fields.find("txid"); + const auto vout_it = fields.find("vout"); + if (txid_it == fields.end() || vout_it == fields.end() || + !std::holds_alternative(txid_it->second.value()) || + !std::holds_alternative(vout_it->second.value())) + { + send_btcd_error(error::invalid_argument); + return true; + } + + hash_digest hash{}; + uint32_t vout{}; + if (!decode_hash(hash, std::get(txid_it->second.value())) || + !to_integer(vout, std::get(vout_it->second.value()))) + { + send_btcd_error(error::invalid_argument); + return true; + } + + ins.emplace_back(chain::point{ hash, vout }, chain::script{}, + 0xffffffff_u32); + } + + chain::outputs outs{}; + outs.reserve(outputs.size()); + for (const auto& pair: outputs) + { + if (!std::holds_alternative(pair.second.value())) + { + send_btcd_error(error::invalid_argument); + return true; + } + + chain::script script{}; + if (const auto fault = parse_output_script(pair.first, script)) + { + send_btcd_error(fault); + return true; + } + + const auto btc = std::get(pair.second.value()); + const auto satoshi = static_cast( + std::llround(btc * chain::satoshi_per_bitcoin)); + outs.emplace_back(satoshi, std::move(script)); + } + + const chain::transaction tx{ 1, std::move(ins), std::move(outs), + locktime }; + + // Unsigned, so no witness data yet -- plain (non-witness) serialization. + constexpr auto witness = false; + send_btcd_result(to_text(tx, tx.serialized_size(witness), witness), 400); + return true; +} + +bool protocol_btcd_rpc::handle_decode_raw_transaction(const code& ec, + btcd_interface::decode_raw_transaction, + const std::string& hexstring) NOEXCEPT +{ + if (stopped(ec)) + return false; + + data_chunk data{}; + if (!decode_base16(data, hexstring)) + { + send_btcd_error(error::invalid_argument); + return true; + } + + constexpr auto witness = true; + const chain::transaction tx{ data, witness }; + if (!tx.is_valid()) + { + send_btcd_error(error::invalid_argument); + return true; + } + + // bitcoind(tx) alone (no inject_tx_context) is exactly the bare + // decoderawtransaction shape -- no block-context fields to add for a + // standalone, possibly-unbroadcast transaction. + send_btcd_result(value_from(bitcoind(tx)), + two * tx.serialized_size(witness)); + return true; +} + +bool protocol_btcd_rpc::handle_decode_script(const code& ec, + btcd_interface::decode_script, const std::string& hex) NOEXCEPT +{ + if (stopped(ec)) + return false; + + data_chunk data{}; + if (!decode_base16(data, hex)) + { + send_btcd_error(error::invalid_argument); + return true; + } + + // false: raw script bytes, no serialized length prefix. + const chain::script script{ data, false }; + if (!script.is_valid()) + { + send_btcd_error(error::invalid_argument); + return true; + } + + using namespace chain; + object_t result{}; + result.emplace("asm", value_t{ script.to_string(flags::all_rules, true) }); + + std::string kind{ "nonstandard" }; + switch (script.output_pattern()) + { + case script_pattern::pay_key_hash: + kind = "pubkeyhash"; + break; + case script_pattern::pay_script_hash: + kind = "scripthash"; + break; + case script_pattern::pay_multisig: + kind = "multisig"; + break; + case script_pattern::pay_public_key: + kind = "pubkey"; + break; + case script_pattern::pay_null_data: + kind = "nulldata"; + break; + case script_pattern::pay_witness_key_hash: + kind = "witness_v0_keyhash"; + break; + case script_pattern::pay_witness_script_hash: + kind = "witness_v0_scripthash"; + break; + case script_pattern::pay_witness_v1_taproot: + kind = "witness_v1_taproot"; + break; + default: + break; + } + result.emplace("type", value_t{ kind }); + + if (script.output_pattern() == script_pattern::pay_key_hash || + script.output_pattern() == script_pattern::pay_script_hash) + { + const auto address = wallet::payment_address::extract_output(script, + p2kh_, p2sh_); + if (address) + result.emplace("address", value_t{ address.encoded() }); + } + + // The p2sh-wrapping address of this exact script (i.e. the address that + // pays a redeem script equal to this one), regardless of this script's + // own pattern -- matches real btcd's "p2sh" field. + const wallet::payment_address wrapped{ script, p2sh_ }; + if (wrapped) + result.emplace("p2sh", value_t{ wrapped.encoded() }); + + send_btcd_result(value_t{ std::move(result) }, 256); + return true; +} + +bool protocol_btcd_rpc::handle_validate_address(const code& ec, + btcd_interface::validate_address, const std::string& address) NOEXCEPT +{ + if (stopped(ec)) + return false; + + const wallet::payment_address base58(address); + if (base58) + { + send_btcd_result(object_t + { + { "isvalid", true }, + { "address", value_t{ base58.encoded() } }, + { "isscript", base58.prefix() == p2sh_ }, + { "iswitness", false } + }, 128); + return true; + } + + const wallet::witness_address segwit(address); + if (segwit) + { + send_btcd_result(object_t + { + { "isvalid", true }, + { "address", value_t{ segwit.encoded() } }, + { "isscript", segwit.identifier() == + wallet::witness_address::program_type::version0_p2sh }, + { "iswitness", true }, + { "witness_version", value_t{ + possible_sign_cast(segwit.version()) } }, + { "witness_program", value_t{ encode_base16(segwit.program()) } } + }, 128); + return true; + } + + send_btcd_result(object_t{ { "isvalid", false } }, 32); + return true; +} + +bool protocol_btcd_rpc::handle_help(const code& ec, btcd_interface::help, + const std::string&) NOEXCEPT +{ + if (stopped(ec)) + return false; + + // Minimal, generic-tooling-only implementation: no specific consumer + // (see docs/btcd-endpoint.md), so this lists method names rather than + // building full per-command argument usage text. Chain methods + // (getblockcount etc., inherited from protocol_bitcoind_rpc) are + // documented by bitcoind's own docs, unchanged. + send_btcd_result(std::string{ + "authenticate session getcurrentnet notifyblocks stopnotifyblocks " + "loadtxfilter rescanblocks getdifficulty getinfo getnettotals " + "getnetworkhashps createrawtransaction decoderawtransaction " + "decodescript validateaddress help stop" }, 256); + return true; +} + +BC_POP_WARNING() +BC_POP_WARNING() +BC_POP_WARNING() + +} // namespace server +} // namespace libbitcoin From 35d67ac1e0a1cbf5bf9c8c89b8ba334a877d1803 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Thu, 30 Jul 2026 11:30:49 +0200 Subject: [PATCH 08/10] Add Boost.Test coverage for the btcd endpoint changes. Covers ws-bridged chain methods, getcurrentnet, loadtxfilter/ filteredblockconnected/disconnected/rescanblocks (including a constructed p2kh test block for address/outpoint matching), and the Phase C generic RPC methods. Adds a notify() fixture helper to synthesize chase events without a live p2p sync. --- test/protocols/btcd/btcd_rpc.cpp | 335 ++++++++++++++++++++- test/protocols/btcd/btcd_setup_fixture.cpp | 5 + test/protocols/btcd/btcd_setup_fixture.hpp | 5 + 3 files changed, 339 insertions(+), 6 deletions(-) diff --git a/test/protocols/btcd/btcd_rpc.cpp b/test/protocols/btcd/btcd_rpc.cpp index fa118ad31..d3105e341 100644 --- a/test/protocols/btcd/btcd_rpc.cpp +++ b/test/protocols/btcd/btcd_rpc.cpp @@ -58,6 +58,17 @@ BOOST_AUTO_TEST_CASE(btcd_rpc__session__returns_id) BOOST_REQUIRE(response.at("result").as_object().contains("id")); } +BOOST_AUTO_TEST_CASE(btcd_rpc__getcurrentnet__mainnet_magic) +{ + // btcd_setup_fixture configures system::chain::selection::mainnet, whose + // network_settings().identifier (p2p handshake magic) is 3652501241 + // (0xd9b4bef9) -- the same value real btcd returns from getcurrentnet. + const auto response = rpc("getcurrentnet"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE_EQUAL(response.at("result").as_int64(), 3652501241); +} + // block subscription // ---------------------------------------------------------------------------- @@ -75,10 +86,324 @@ BOOST_AUTO_TEST_CASE(btcd_rpc__stopnotifyblocks__returns_null_result) BOOST_REQUIRE(!has_error(response)); } -// Standard chain methods (getblockcount etc.) are inherited from -// protocol_bitcoind_rpc and remain reachable via plain http post on the same -// endpoint (see test/protocols/bitcoind), but are not yet bridged into this -// ws dispatcher -- see the phase B TODO on protocol_btcd_rpc.hpp. +// Standard chain methods (bridged, B0) +// ---------------------------------------------------------------------------- +// Inherited from protocol_bitcoind_rpc, reachable both via plain http post on +// the same endpoint (see test/protocols/bitcoind) and, bridged, over this ws +// connection (dispatch_websocket falls back to dispatch_rpc on +// unexpected_method from the btcd-only dispatcher). + +BOOST_AUTO_TEST_CASE(btcd_rpc__getblockcount__ten_block_store__nine) +{ + // btcd_ten_block_setup_fixture populates a ten-block confirmed store + // (genesis + 9), so top confirmed height is 9 (matches the equivalent + // bitcoind_rpc__getblockcount__ten_block_store__nine test). + const auto response = rpc("getblockcount"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE_EQUAL(response.at("result").as_int64(), 9); +} + +// Address/outpoint filtering (loadtxfilter, filteredblockconnected/ +// disconnected, rescanblocks) -- implemented, B2. +// ---------------------------------------------------------------------------- +// blocks 1-9 (btcd_ten_block_setup_fixture) are real early mainnet blocks, +// each a single p2pk coinbase with no inter-block spends (first real tx in +// mainnet history isn't until block 170), so none of them can exercise +// loadtxfilter's actual address matching -- block10() below (a controlled +// p2kh output chained after block9) is built for that purpose. + +namespace { + +const short_hash& filter_test_hash() NOEXCEPT +{ + static const short_hash hash{ base16_array( + "0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a") }; + return hash; +} + +const wallet::payment_address& filter_test_address() NOEXCEPT +{ + static const wallet::payment_address address{ filter_test_hash() }; + return address; +} + +const std::string& filter_test_address_text() NOEXCEPT +{ + static const std::string text{ filter_test_address().encoded() }; + return text; +} + +const chain::block& block10() NOEXCEPT +{ + static const chain::block instance + { + [&]() NOEXCEPT + { + using namespace wallet; + const chain::transaction coinbase + { + 1, + chain::inputs{ chain::input + { + chain::point{}, chain::script{}, 0xffffffff + } }, + chain::outputs{ chain::output + { + 50'0000'0000, filter_test_address().output_script( + payment_address::mainnet_p2kh, + payment_address::mainnet_p2sh) + } }, + 0 + }; + + // query.set()/push_confirmed() are raw persistence calls (no + // organize-time consensus validation), so an internally- + // consistent merkle root is not required for this fixture -- + // chain::block::generate_merkle_root() is private besides. + return chain::block + { + chain::header + { + 1, test::block9_hash, null_hash, + test::block9.header().timestamp() + 600, + test::block9.header().bits(), 0 + }, + chain::transactions{ coinbase } + }; + }() + }; + + return instance; +} + +} // namespace + +BOOST_AUTO_TEST_CASE(btcd_rpc__loadtxfilter__valid_address__acknowledges) +{ + const auto response = rpc("loadtxfilter", + "[true,[\"" + filter_test_address_text() + "\"],[]]"); + BOOST_REQUIRE(!has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__loadtxfilter__invalid_address__invalid_argument) +{ + const auto response = rpc("loadtxfilter", + R"([true,["not-an-address"],[]])"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__loadtxfilter__valid_outpoint__acknowledges) +{ + const auto txid = encode_hash( + test::block1.transactions_ptr()->front()->hash(false)); + const auto response = rpc("loadtxfilter", "[true,[],[{\"hash\":\"" + + txid + "\",\"index\":0}]]"); + BOOST_REQUIRE(!has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__loadtxfilter__malformed_outpoint__invalid_argument) +{ + const auto response = rpc("loadtxfilter", R"([true,[],[{"hash":"00"}]])"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__rescanblocks__unknown_hash__not_found) +{ + // 'blockhashes' is one positional arg that is itself an array, so the + // wire params need double-wrapping: [[...]], not [...]. + const auto response = rpc("rescanblocks", "[[\"" + encode_hash(null_hash) + + "\"]]"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__rescanblocks__no_filter_match__empty_result) +{ + rpc("loadtxfilter", "[true,[\"" + filter_test_address_text() + "\"],[]]"); + + const auto response = rpc("rescanblocks", + "[[\"" + encode_hash(test::block1_hash) + "\"]]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE(response.at("result").as_array().empty()); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__rescanblocks__address_match__returns_matched_tx) +{ + BOOST_REQUIRE(query_.set(block10(), database::context{ 0, 10, 0 }, false, + false)); + BOOST_REQUIRE(query_.push_confirmed(query_.to_header(block10().hash()), + true)); + + rpc("loadtxfilter", "[true,[\"" + filter_test_address_text() + "\"],[]]"); + + const auto response = rpc("rescanblocks", + "[[\"" + encode_hash(block10().hash()) + "\"]]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + + const auto& result = response.at("result").as_array(); + BOOST_REQUIRE_EQUAL(result.size(), 1u); + BOOST_REQUIRE_EQUAL(result.front().at("hash").as_string(), + encode_hash(block10().hash())); + BOOST_REQUIRE_EQUAL(result.front().at("transactions").as_array().size(), + 1u); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__filteredblockconnected__address_match__delivered) +{ + rpc("notifyblocks"); + rpc("loadtxfilter", "[true,[\"" + filter_test_address_text() + "\"],[]]"); + + BOOST_REQUIRE(query_.set(block10(), database::context{ 0, 10, 0 }, false, + false)); + BOOST_REQUIRE(query_.push_confirmed(query_.to_header(block10().hash()), + true)); + + notify(node::chase::organized, { 10_u32 }); + + const auto blockconnected = receive_notification(); + BOOST_REQUIRE_EQUAL(blockconnected.at("method").as_string(), + "blockconnected"); + + const auto filtered = receive_notification(); + BOOST_REQUIRE_EQUAL(filtered.at("method").as_string(), + "filteredblockconnected"); + + const auto& params = filtered.at("params").as_array(); + BOOST_REQUIRE_EQUAL(params.size(), 3u); + BOOST_REQUIRE_EQUAL(params[0].as_int64(), 10); + BOOST_REQUIRE_EQUAL(params[2].as_array().size(), 1u); +} + +// Phase C: generic btcd-tooling compatibility (implemented, no lnd consumer) +// ---------------------------------------------------------------------------- + +BOOST_AUTO_TEST_CASE(btcd_rpc__getdifficulty__returns_number) +{ + const auto response = rpc("getdifficulty"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE(response.at("result").is_double()); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__getinfo__ten_block_store__nine) +{ + const auto response = rpc("getinfo"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + const auto& result = response.at("result").as_object(); + BOOST_REQUIRE_EQUAL(result.at("blocks").as_int64(), 9); + BOOST_REQUIRE_EQUAL(result.at("testnet").as_bool(), false); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__getnettotals__returns_object) +{ + const auto response = rpc("getnettotals"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + const auto& result = response.at("result").as_object(); + BOOST_REQUIRE(result.contains("totalbytesrecv")); + BOOST_REQUIRE(result.contains("totalbytessent")); + BOOST_REQUIRE(result.at("timemillis").as_int64() > 0); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__getnetworkhashps__returns_number) +{ + const auto response = rpc("getnetworkhashps"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE(response.at("result").is_double()); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__createrawtransaction__well_formed_hex) +{ + const auto txid = encode_hash( + test::block1.transactions_ptr()->front()->hash(false)); + const auto response = rpc("createrawtransaction", + "[[{\"txid\":\"" + txid + "\",\"vout\":0}],{\"" + + filter_test_address_text() + "\":0.01}]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + + data_chunk data{}; + BOOST_REQUIRE(decode_base16(data, response.at("result").as_string())); + + const chain::transaction tx{ data, false }; + BOOST_REQUIRE(tx.is_valid()); + BOOST_REQUIRE_EQUAL(tx.inputs_ptr()->size(), 1u); + BOOST_REQUIRE_EQUAL(tx.outputs_ptr()->size(), 1u); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__decoderawtransaction__block1_coinbase) +{ + const auto& coinbase = *test::block1.transactions_ptr()->front(); + const auto hex = encode_base16(coinbase.to_data(false)); + + const auto response = rpc("decoderawtransaction", "[\"" + hex + "\"]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + + const auto& result = response.at("result").as_object(); + BOOST_REQUIRE_EQUAL(result.at("txid").as_string(), + encode_hash(coinbase.hash(false))); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__decoderawtransaction__malformed_hex__invalid_argument) +{ + const auto response = rpc("decoderawtransaction", R"(["not-hex"])"); + BOOST_REQUIRE(has_error(response)); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__decodescript__pubkey_script__typed_correctly) +{ + // block1's coinbase output is a p2pk script (early mainnet convention). + const auto& script = + test::block1.transactions_ptr()->front()->outputs_ptr()->front()-> + script(); + const auto hex = encode_base16(script.to_data(false)); + + const auto response = rpc("decodescript", "[\"" + hex + "\"]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + + const auto& result = response.at("result").as_object(); + BOOST_REQUIRE_EQUAL(result.at("type").as_string(), "pubkey"); + BOOST_REQUIRE(result.contains("asm")); + BOOST_REQUIRE(result.contains("p2sh")); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__validateaddress__valid_address__isvalid_true) +{ + const auto response = rpc("validateaddress", + "[\"" + filter_test_address_text() + "\"]"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + + const auto& result = response.at("result").as_object(); + BOOST_REQUIRE_EQUAL(result.at("isvalid").as_bool(), true); + BOOST_REQUIRE_EQUAL(result.at("address").as_string(), + filter_test_address_text()); + BOOST_REQUIRE_EQUAL(result.at("iswitness").as_bool(), false); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__validateaddress__invalid_address__isvalid_false) +{ + const auto response = rpc("validateaddress", R"(["not-an-address"])"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE_EQUAL( + response.at("result").as_object().at("isvalid").as_bool(), false); +} + +BOOST_AUTO_TEST_CASE(btcd_rpc__help__returns_method_list) +{ + const auto response = rpc("help"); + BOOST_REQUIRE(!has_error(response)); + BOOST_REQUIRE(has_result(response)); + BOOST_REQUIRE(response.at("result").as_string().find("getcurrentnet") != + std::string::npos); +} // not implemented stubs // ---------------------------------------------------------------------------- @@ -90,8 +415,6 @@ BOOST_AUTO_TEST_CASE(btcd_rpc__not_implemented__error) { "stop", "[]" }, { "notifynewtransactions", "[false]" }, { "stopnotifynewtransactions", "[]" }, - { "loadtxfilter", R"([false,[],[]])" }, - { "rescanblocks", "[[]]" }, { "notifyreceived", "[[]]" }, { "stopnotifyreceived", "[[]]" }, { "notifyspent", "[[]]" }, diff --git a/test/protocols/btcd/btcd_setup_fixture.cpp b/test/protocols/btcd/btcd_setup_fixture.cpp index d19e6b340..8d5dc3792 100644 --- a/test/protocols/btcd/btcd_setup_fixture.cpp +++ b/test/protocols/btcd/btcd_setup_fixture.cpp @@ -137,3 +137,8 @@ boost::json::value btcd_setup_fixture::receive_notification() BOOST_REQUIRE_MESSAGE(!ec, ec.message()); return test::parse_json(buffers_to_string(buffer.data())); } + +void btcd_setup_fixture::notify(node::chase event_, node::event_value value) +{ + server_.notify(system::error::success, event_, value); +} diff --git a/test/protocols/btcd/btcd_setup_fixture.hpp b/test/protocols/btcd/btcd_setup_fixture.hpp index d50f358a9..7b4fc63e6 100644 --- a/test/protocols/btcd/btcd_setup_fixture.hpp +++ b/test/protocols/btcd/btcd_setup_fixture.hpp @@ -50,6 +50,11 @@ struct btcd_setup_fixture // notification. Returns the parsed json-rpc notification object. boost::json::value receive_notification(); + // Synthesize a node chase event (e.g. a block organized/confirmed after + // a direct query_.set/push_confirmed) without a live p2p sync -- mirrors + // electrum_setup_fixture::notify. + void notify(node::chase event_, node::event_value value=0_u32); + protected: configuration config_; test::store_t store_; From 07d6d6bdcb5fb5b1de7556c577d5a194ff6a086d Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Thu, 30 Jul 2026 11:31:13 +0200 Subject: [PATCH 09/10] Update btcd Python acceptance suite for the endpoint changes. Flips ws-bridged chain methods and getcurrentnet from xfail to real tests, adds loadtxfilter/rescanblocks/filteredblockconnected coverage (the latter marked slow, needs --run-slow) and Phase C method tests, and adds a MAINNET_MAGIC reference constant. --- endpoints/test_btcd_rpc.py | 243 +++++++++++++++++++++++++++++++------ endpoints/utils.py | 3 + 2 files changed, 208 insertions(+), 38 deletions(-) diff --git a/endpoints/test_btcd_rpc.py b/endpoints/test_btcd_rpc.py index 33e84191d..b560e2877 100644 --- a/endpoints/test_btcd_rpc.py +++ b/endpoints/test_btcd_rpc.py @@ -379,6 +379,15 @@ def test_session_returns_id(conn): assert "id" in result +def test_getcurrentnet_returns_network_magic(conn): + """getcurrentnet returns the p2p handshake magic number (network_settings + ().identifier) -- the same value real btcd returns, checked once by + btcwallet/lnd at connect to confirm they're talking to the expected + network (implemented in B1).""" + response = conn.send_rpc("getcurrentnet") + assert response.get("result") == ReferenceData.MAINNET_MAGIC + + # ═══════════════════════════════════════════════════════════════════════════════ # BLOCK SUBSCRIPTION (implemented) # ═══════════════════════════════════════════════════════════════════════════════ @@ -436,9 +445,109 @@ def test_blockconnected_notification(conn, btcd_config): # ═══════════════════════════════════════════════════════════════════════════════ -# STANDARD CHAIN METHODS -- reachable today only via plain http post. -# Development target: bridge these into the ws dispatcher (phase B). +# ADDRESS/OUTPOINT FILTERING (implemented, B2) # ═══════════════════════════════════════════════════════════════════════════════ +# loadtxfilter never itself triggers notifications (matching real btcd) -- +# notifyblocks remains the only thing that arms delivery; once armed, +# filteredblockconnected/disconnected are sent alongside the existing +# unfiltered blockconnected/disconnected. rescanblocks replays the same match +# logic against explicitly named historical blocks using the loaded filter. + +def test_loadtxfilter_valid_address_acknowledges(conn): + response = conn.send_rpc("loadtxfilter", + [True, [ReferenceData.EXAMPLE_ADDRESS], []]) + assert response.get("error") is None + + +def test_loadtxfilter_invalid_address_rejected(conn): + response = conn.raw_rpc("loadtxfilter", [True, ["not-an-address"], []]) + assert response.get("error") is not None + + +def test_loadtxfilter_valid_outpoint_acknowledges(conn): + response = conn.send_rpc("loadtxfilter", + [True, [], [{"hash": ReferenceData.GENESIS_TX_HASH, "index": 0}]]) + assert response.get("error") is None + + +def test_loadtxfilter_malformed_outpoint_rejected(conn): + response = conn.raw_rpc("loadtxfilter", [True, [], [{"hash": "00"}]]) + assert response.get("error") is not None + + +def test_rescanblocks_unknown_hash_rejected(conn): + response = conn.raw_rpc("rescanblocks", [["00" * 32]]) + assert response.get("error") is not None + + +def test_rescanblocks_known_block_no_match_empty_result(conn): + # An arbitrary (real, valid-format) address that does not own the + # genesis coinbase output -- exercises the "no match" path, not a + # specific claim about what the genesis output actually pays. + conn.send_rpc("loadtxfilter", [True, [ReferenceData.EXAMPLE_ADDRESS], []]) + response = conn.send_rpc("rescanblocks", [[ReferenceData.GENESIS_HASH]]) + assert response.get("result") == [] + + +@pytest.mark.slow +def test_filteredblockconnected_notification(conn, btcd_config): + """ + filteredblockconnected must be delivered alongside blockconnected for + every notifyblocks client, even with no address loaded (empty + subscribedtxs) -- verified against btcd's own rpcwebsocket.go: both + notifications fire unconditionally together, the filtered one just + carries an empty list when nothing matches. + + Notification format (verified against btcsuite/btcd/btcjson): + {"method": "filteredblockconnected", "params": [height, header, subscribedtxs]} + """ + sub_timeout = btcd_config.get("subscription_timeout", 60.0) + + ack = conn.send_rpc("notifyblocks") + assert ack.get("error") is None + + cmd = os.getenv("BTCD_TRIGGER_BLOCK") + if cmd: + os.system(cmd) + else: + print(f"\n Waiting up to {sub_timeout:.0f}s for a " + "filteredblockconnected notification (set BTCD_TRIGGER_BLOCK " + "to trigger one).", flush=True) + + # blockconnected and filteredblockconnected are sent back-to-back for the + # same block; read up to two frames to find the filtered one. + notif = conn.read_notification(timeout_s=sub_timeout) + if notif is not None and notif.get("method") != "filteredblockconnected": + notif = conn.read_notification(timeout_s=sub_timeout) + + if notif is None: + pytest.skip(f"No filteredblockconnected notification within " + f"{sub_timeout:.0f}s. Increase --subscription-timeout " + "or set BTCD_TRIGGER_BLOCK.") + + assert notif.get("method") == "filteredblockconnected" + params = notif.get("params", []) + assert isinstance(params, list) and len(params) == 3, ( + "filteredblockconnected params must be [height, header, " + f"subscribedtxs], got {params!r}" + ) + height, header, subscribed_txs = params + assert isinstance(height, int) and height > 0 + assert isinstance(header, str) and len(header) == 160 # 80-byte header + assert isinstance(subscribed_txs, list) + + +# ═══════════════════════════════════════════════════════════════════════════════ +# STANDARD CHAIN METHODS (implemented, bridged into the ws dispatcher -- B0) +# ═══════════════════════════════════════════════════════════════════════════════ +# Inherited from protocol_bitcoind_rpc. Reachable both over plain http post +# (unchanged from bitcoind) and, since B0, over the same ws connection used +# for session/notifyblocks/etc: protocol_btcd_rpc::dispatch_websocket falls +# back to protocol_bitcoind_rpc::dispatch_rpc when the btcd-only dispatcher +# reports "unexpected method". This is what a real lnd/btcwallet client +# needs -- it can't open a second plain-http connection once it has upgraded +# to ws, so authenticate/session/notifyblocks and getblockcount etc. must all +# work on the one connection. CHAIN_METHODS = [ ("getbestblockhash", []), @@ -452,27 +561,41 @@ def test_blockconnected_notification(conn, btcd_config): @pytest.mark.parametrize("method,params", CHAIN_METHODS) def test_chain_method_over_http_post(btcd_config, method, params): - """Positive control: the chain logic itself works today, reachable over - plain http post to the same endpoint (unchanged from bitcoind).""" + """Positive control: the chain logic itself works, reachable over plain + http post to the same endpoint (unchanged from bitcoind).""" data = http_rpc(btcd_config, method, params) if data.get("error") is not None: pytest.xfail(f"Server sent valid error response: {data['error']}") assert "result" in data -@pytest.mark.xfail(reason="phase B: standard chain methods not yet bridged " - "into the ws dispatcher (see docs/btcd-endpoint.md)", - strict=False) @pytest.mark.parametrize("method,params", CHAIN_METHODS) def test_chain_method_over_websocket(conn, method, params): - """Development target: once bridged, chain methods should also work over - the same ws connection used for session/notifyblocks/etc -- this is what - a real lnd/btcwallet client needs (it can't open a second plain-http - connection once it has upgraded to ws).""" + """Standard chain methods now also work over the same ws connection used + for session/notifyblocks/etc (bridged in B0, see + protocol_btcd_rpc::dispatch_websocket / protocol_bitcoind_rpc:: + dispatch_rpc).""" response = conn.send_rpc(method, params) assert "result" in response +def test_btcd_and_chain_method_share_one_websocket_connection(conn): + """The actual point of B0: a single persistent ws connection -- the kind + a real lnd/btcwallet client opens once and keeps -- can reach both a + btcd-only extension method (session) and a standard chain method + (getblockcount) without reconnecting or falling back to plain http post. + """ + session = conn.send_rpc("session") + assert isinstance(session.get("result"), dict) + + block_count = conn.send_rpc("getblockcount") + assert isinstance(block_count.get("result"), int) + + # Same connection still answers a second btcd-only method afterwards. + notify = conn.send_rpc("notifyblocks") + assert notify.get("error") is None + + # ═══════════════════════════════════════════════════════════════════════════════ # WIRED BUT NOT YET IMPLEMENTED (phase B/C development targets) # ═══════════════════════════════════════════════════════════════════════════════ @@ -483,8 +606,6 @@ def test_chain_method_over_websocket(conn, method, params): NOT_YET_IMPLEMENTED_STUBS = [ ("notifynewtransactions", [False]), ("stopnotifynewtransactions", []), - ("loadtxfilter", [False, [], []]), - ("rescanblocks", [[]]), ] # Deprecated upstream (superseded by loadtxfilter/rescanblocks) but still @@ -525,34 +646,80 @@ def test_stop_always_not_implemented(conn): # ═══════════════════════════════════════════════════════════════════════════════ -# NOT YET WIRED AT ALL (phase B/C development targets) +# PHASE C: generic btcd-tooling compatibility (implemented, no lnd consumer) # ═══════════════════════════════════════════════════════════════════════════════ -# These aren't in interfaces/btcd.hpp's method table yet, so today they hit -# "unexpected method". Each is a checklist item for the generic-tooling -# compatibility phase (see docs/btcd-endpoint.md, Phase C). - -NOT_YET_WIRED = [ - ("getcurrentnet", []), - ("getdifficulty", []), - ("getinfo", []), - ("getnettotals", []), - ("getnetworkhashps", []), - ("createrawtransaction", [[], {}]), - ("decoderawtransaction", ["00"]), - ("decodescript", [""]), - ("validateaddress", [ReferenceData.EXAMPLE_ADDRESS]), - ("help", []), -] +# See docs/btcd-endpoint.md for scope/fidelity notes (getnetworkhashps is an +# approximation, getnettotals's byte counters are untracked zeros, help lists +# method names only). +def test_getdifficulty_returns_number(conn): + response = conn.send_rpc("getdifficulty") + assert isinstance(response.get("result"), float) -@pytest.mark.xfail(reason="phase B/C: method not yet wired in interfaces/btcd.hpp", - strict=False) -@pytest.mark.parametrize("method,params", NOT_YET_WIRED) -def test_method_not_yet_wired(conn, method, params): - data = conn.raw_rpc(method, params) - assert data.get("error") is None, ( - f"{method} still unwired: {data.get('error')}" - ) + +def test_getinfo_returns_object(conn): + result = conn.send_rpc("getinfo").get("result") + assert isinstance(result, dict) + assert isinstance(result.get("blocks"), int) + assert result.get("testnet") is False + + +def test_getnettotals_returns_object(conn): + result = conn.send_rpc("getnettotals").get("result") + assert isinstance(result, dict) + assert "totalbytesrecv" in result + assert "totalbytessent" in result + assert result.get("timemillis", 0) > 0 + + +def test_getnetworkhashps_returns_number(conn): + response = conn.send_rpc("getnetworkhashps") + assert isinstance(response.get("result"), (int, float)) + + +def test_createrawtransaction_and_decoderawtransaction_roundtrip(conn): + raw = conn.send_rpc("createrawtransaction", + [[{"txid": ReferenceData.FIRST_TX_HASH, "vout": 0}], + {ReferenceData.EXAMPLE_ADDRESS: 0.01}]) + hexstring = raw.get("result") + assert isinstance(hexstring, str) and len(hexstring) > 0 + + result = conn.send_rpc("decoderawtransaction", [hexstring]).get("result") + assert isinstance(result, dict) + assert len(result.get("vin", [])) == 1 + assert len(result.get("vout", [])) == 1 + + +def test_decoderawtransaction_malformed_hex_rejected(conn): + response = conn.raw_rpc("decoderawtransaction", ["not-hex"]) + assert response.get("error") is not None + + +def test_decodescript_null_data_script(conn): + # OP_RETURN <2-byte push "hi"> -- self-contained, doesn't depend on live + # chain contents. A bare "6a" (OP_RETURN with no push at all) does not + # match the standard null-data template. + result = conn.send_rpc("decodescript", ["6a026869"]).get("result") + assert result.get("type") == "nulldata" + assert "asm" in result + + +def test_validateaddress_valid_address(conn): + result = conn.send_rpc("validateaddress", + [ReferenceData.EXAMPLE_ADDRESS]).get("result") + assert result.get("isvalid") is True + assert result.get("address") == ReferenceData.EXAMPLE_ADDRESS + assert result.get("iswitness") is False + + +def test_validateaddress_invalid_address(conn): + result = conn.send_rpc("validateaddress", ["not-an-address"]).get("result") + assert result.get("isvalid") is False + + +def test_help_returns_method_list(conn): + result = conn.send_rpc("help").get("result") + assert isinstance(result, str) and "getcurrentnet" in result # ═══════════════════════════════════════════════════════════════════════════════ diff --git a/endpoints/utils.py b/endpoints/utils.py index f828e39dd..ff5aac5c1 100644 --- a/endpoints/utils.py +++ b/endpoints/utils.py @@ -122,6 +122,9 @@ class ReferenceData: FILTER_TYPE_BLOOM = "bloom" FILTER_TYPE_COMPACT = "compact" + # p2p handshake magic (network_settings().identifier), mainnet + MAINNET_MAGIC = 3652501241 + class TestConfig: """Default test configuration values.""" From b240401d666173ea59e215519a43b82fe4ac4a86 Mon Sep 17 00:00:00 2001 From: Kevin Abramczyk Date: Thu, 30 Jul 2026 11:31:21 +0200 Subject: [PATCH 10/10] Update btcd endpoint doc for B0-B2/Phase C completion. Marks the ws-bridging gap, getcurrentnet, loadtxfilter/filtered notifications/rescanblocks, and the Phase C filler methods as implemented, corrects the lnd-compatibility notes against btcd's real wire methods (filteredblockconnected/relevanttxaccepted, not the deprecated recvtx/redeemingtx pair), and drops the stale address_enabled guard note for rescanblocks. --- docs/btcd-endpoint.md | 125 +++++++++++++++++++++++++++++------------- 1 file changed, 88 insertions(+), 37 deletions(-) diff --git a/docs/btcd-endpoint.md b/docs/btcd-endpoint.md index 7db2f76e6..d1a7cd234 100644 --- a/docs/btcd-endpoint.md +++ b/docs/btcd-endpoint.md @@ -107,16 +107,19 @@ template). `getbestblockhash`, `getblock`, `getblockhash`, `getblockheader`, `getblockcount`, `gettxout`, `getrawtransaction`, `sendrawtransaction`. No btcd-specific work needed; response shapes already match Core/btcd conventions closely enough for -Phase A. **Caveat verified during Phase A implementation**: these are -currently reachable only via plain http post to the same endpoint (unchanged -from `bitcoind`), not yet over the ws connection itself. `dispatch_websocket` -only routes to the btcd-only extension dispatcher; bridging the inherited -handlers' post-oriented private send path (which caches the original -`http::request` for header derivation, unavailable for a ws frame) into a -ws-reachable form is real work, scheduled for Phase B. Until then, a -ws-connected client (required for `authenticate`/`session`/`notifyblocks`) -cannot also call `getblockcount` etc. on that same connection — a real -lnd/btcwallet client would need this bridged to be useful in practice. +Phase A. **Bridged into the ws dispatcher (B0)**: `dispatch_websocket` tries +the btcd-only extension dispatcher first, then falls back to +`protocol_bitcoind_rpc::dispatch_rpc` (new) when that dispatcher reports +`unexpected_method` — a pure method-name lookup miss, never conflated with an +argument-mismatch error, so the fallback cannot mask a real handler fault. +`send_rpc` (the shared sender all inherited chain handlers funnel through) +branches on `websocket()`: the post path is unchanged (still derives headers +from the cached `http::request`), while the ws path sends a minimal response +with no header enrichment — ws frames are synthesized with no real headers to +echo in the first place (`channel_http::create_request()`), matching +`protocol_btcd_rpc`'s own ws senders. A ws-connected client (required for +`authenticate`/`session`/`notifyblocks`) can now also call `getblockcount` +etc. on that same connection, as a real lnd/btcwallet client needs. ### Phase A — wiring @@ -125,16 +128,28 @@ functionally correct, not stubbed — see [lnd Compatibility](#lnd-compatibility `notifyblocks`/`stopnotifyblocks`, `not_implemented` stubs for every other btcd method, Boost.Test suite. -### Phase B — lnd-critical, promoted ahead of generic filler - -`getcurrentnet` (network-match check lnd/btcwallet performs once at connect), -`loadtxfilter`, `rescanblocks` (guard behind `address_enabled`, same guard used -by the address-indexing-dependent electrum methods). None of these are on the -critical path for a generic btcd client — they're promoted here specifically -because `btcwallet`'s `chain.RPCClient` and lnd's own `chainntnfs/btcdnotify` -depend on them (see [lnd Compatibility](#lnd-compatibility)). - -### Phase C — remaining plain RPC methods (generic btcd-tooling compatibility) +### Phase B — lnd-critical, promoted ahead of generic filler — implemented + +`getcurrentnet` (network-match check lnd/btcwallet performs once at connect — +**B1**: returns `network_settings().identifier`, the same p2p handshake magic +real btcd returns from this call). `loadtxfilter` + filtered block +notifications (`filteredblockconnected`/`filteredblockdisconnected`) + +`rescanblocks` — **B2**: per-connection in-memory watch-list (base58 p2kh/p2sh +via `wallet::payment_address`, bech32/bech32m p2wpkh/p2wsh/p2tr via +`wallet::witness_address` + `chain::script::output_pattern()`, plus outpoints), +matched by direct script inspection of each newly-connected (or, for +`rescanblocks`, explicitly named historical) block's own transactions — not +the persisted address index (`address_enabled`/`get_history`), which is the +wrong tool for "does this one block match this small watch-list" and would +needlessly require the address index to be built at all. An output that +matches a watched address has its own outpoint auto-added to the watched set +(matching real btcd's `wsClientFilter`), so a later spend of it also matches +without an explicit re-subscribe. None of this is on the critical path for a +generic btcd client — it's implemented here specifically because +`btcwallet`'s `chain.RPCClient` and lnd's own `chainntnfs/btcdnotify` depend +on it (see [lnd Compatibility](#lnd-compatibility)). + +### Phase C — remaining plain RPC methods (generic btcd-tooling compatibility) — implemented `getdifficulty`, `getinfo`, `getnettotals`, `getnetworkhashps`, `createrawtransaction`, `decoderawtransaction`, `decodescript`, @@ -142,6 +157,33 @@ depend on them (see [lnd Compatibility](#lnd-compatibility)). compatibility for generic btcd-speaking tooling (block explorers, mining-pool stats, monitoring dashboards, tx-building/debug scripts), not lnd. +Notes on scope/fidelity (all verified against real btcd's `btcjson` +result/command structs before implementing): + +- `createrawtransaction`/`decoderawtransaction`/`decodescript` reuse existing + serializers rather than re-deriving them: `bitcoind(tx)` + (`system::chain::json::transaction`) already produces the bare + decoderawtransaction shape (`inject_tx_context` — block-context fields — + is a separate, additive call, skipped here since a standalone hex tx has no + block context); `chain::script::to_string()` already produces the "asm" + disassembly. There is no existing script-type classifier + (pubkeyhash/scripthash/multisig/etc.) or reverse address encoder anywhere + in the codebase, so `decodescript`'s `type`/`address`/`p2sh` fields and + `validateaddress` are new, built from `chain::script::output_pattern()` and + `wallet::payment_address`/`witness_address` (the same dual-parse approach + as B2's `loadtxfilter`, kept as a separate small helper rather than forcing + a shared abstraction onto that already-tested code). +- `getnetworkhashps` approximates from the requested height's difficulty and + a fixed 600s target spacing, not a true windowed average over the `blocks` + parameter — there is no existing big-integer cumulative-work-over-a-range + helper to build a real windowed estimate from, and this method has no named + consumer, so the simpler approximation was chosen over adding one. +- `getnettotals`'s byte counters are not tracked by this node and are + reported as zero, matching the same honesty convention the inherited + bitcoind `getnetworkinfo` handler already uses for untracked + peer-dependent fields (rather than `not_implemented`). +- `help` lists method names only; no per-command argument usage text. + ### Permanently stubbed (`not_implemented`, all phases) - **`stop`** — no secure remote-shutdown path exists in libbitcoin-server; always @@ -170,7 +212,7 @@ both sources: (chain-view / `IsCurrent` checks), `getblock` (verbose, block scanning), `getrawtransaction`, `sendrawtransaction` (broadcasting funding/commitment/ sweep/justice transactions), `gettxout` (UTXO-existence checks), `getcurrentnet` -(network-match validation performed once at connect — scheduled in Phase B). +(network-match validation performed once at connect — implemented, B1). **WebSocket notification surface — the real gap.** Both `btcwallet`'s RPCClient and lnd's `btcdnotify` depend on: @@ -181,26 +223,35 @@ and lnd's `btcdnotify` depend on: filters/rescans. If this is left as a stub, reconnect handling is undefined; it should be implemented for real in Phase A, not deferred. - **`loadtxfilter`** (address/outpoint watch-list registration) plus - block-filtered notification delivery (`OnFilteredBlockConnected`/ - `OnRedeemingTx`/`OnRecvTx` equivalents) — this is the mechanism both the wallet - and lnd's own notifier use to detect channel-funding confirmations and - on-chain spends of channel outputs (breach/force-close detection). It is not - optional legacy behavior; modern `btcwallet`/`lnd` don't fall back to the - deprecated `notifyreceived`/`notifyspent` path. This is why it's scheduled in - Phase B, ahead of Phase C's generic filler methods — it's more load-bearing - for lnd than any Phase C method. -- **`rescan`/`rescanblocks`** — used at wallet startup (birthday scan) and after - any downtime to replay historical blocks against the loaded filter. Also - scheduled in Phase B for the same reason. + block-filtered notification delivery (the modern wire methods + `filteredblockconnected`/`filteredblockdisconnected`, which drive lnd's + `OnFilteredBlockConnected` client callback — verified against + `btcsuite/btcd/btcjson/chainsvrwsntfns.go` and `rpcclient/notify.go`; the + deprecated per-tx `OnRecvTx`/`OnRedeemingTx` callbacks are fed by a + different, older wire pair (`recvtx`/`redeemingtx`) that modern + `btcwallet`/`lnd` don't use) — **implemented, B2**. This is the mechanism + both the wallet and lnd's own notifier use to detect channel-funding + confirmations and on-chain spends of channel outputs (breach/force-close + detection); not optional legacy behavior. +- **`rescanblocks`** — used at wallet startup (birthday scan) and after any + downtime to replay historical blocks against the loaded filter — + **implemented, B2**, same match logic as live filtered notifications, + applied to explicitly named blocks instead of the newly-connected one. + (The older, deprecated `rescan` — arbitrary address/outpoint scans over an + address-index-backed height range — remains permanently stubbed; modern + `btcwallet`/`lnd` use `rescanblocks` instead.) **Hard blocker: mempool.** Both sources confirm unconfirmed-transaction awareness -is an exercised, not merely optional, code path: `OnRecvTx`/`OnRedeemingTx` fire -for mempool-resident transactions (distinguished by a nil `BlockDetails`), and +is an exercised, not merely optional, code path: `relevanttxaccepted` (the modern +wire method, driving lnd's `OnRelevantTxAccepted` callback; armed by +`notifynewtransactions`, already permanently stubbed — see above) fires for a +mempool-resident transaction matching the loaded filter, and lnd's notifier has a dedicated mempool-spend lookup path (`LookupInputMempoolSpend`) used for fast breach/force-close reaction ahead of -confirmation. **libbitcoin-server v4 has no mempool**, so this notification class -cannot be implemented now — confirmed-only fallback is straightforward (fire the -same callbacks only on block connection), but it means lnd would lose +confirmation. **libbitcoin-server v4 has no mempool**, so `relevanttxaccepted` +cannot be implemented now — the confirmed-only fallback (`filteredblockconnected`, +**implemented, B2**) covers the same address/outpoint watch-list, just one block +later than a mempool-aware node would, so it means lnd would lose unconfirmed-spend awareness and any zero-conf-style UX. Final safety in Lightning still rests on confirmation + CSV/CLTV timeout margins, so this is a responsiveness/UX degradation rather than a correctness break, but it's a real,