diff --git a/docs/en/train/guides/feast-offline-to-online-inference.ipynb b/docs/en/train/guides/feast-offline-to-online-inference.ipynb index 66990c5..6e90df8 100644 --- a/docs/en/train/guides/feast-offline-to-online-inference.ipynb +++ b/docs/en/train/guides/feast-offline-to-online-inference.ipynb @@ -268,6 +268,137 @@ " -d '{\"inputs\":[{\"name\":\"driver_id\",\"shape\":[2],\"datatype\":\"INT64\",\"data\":[1,2]}]}'\n", "echo\n" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 7. Benchmark the online feature service\n", + "\n", + "This smoke benchmark measures the HTTPS Feast online-feature API through the Kubernetes Service, rather than timing the `/health` endpoint. It sends 200 requests at concurrency 20 and checks both HTTP status and that every returned feature status is `PRESENT`. The generated service certificate is cluster-local, so the test client disables certificate verification; use trusted CA verification for production clients.\n", + "\n", + "The request contains three features (`conv_rate`, `acc_rate`, and `avg_daily_trips`) for four driver IDs. Set `FEAST_PERF_REQUESTS` and `FEAST_PERF_CONCURRENCY` to change the test size.\n", + "\n", + "### Recorded x86 result\n", + "\n", + "| Online backend | Connection mode | Throughput | Average | P50 | P95 | HTTP errors | Complete responses |\n", + "| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |\n", + "| SQLite file (baseline) | New HTTPS connection/request | 47.24 req/s | 405.5 ms | 242.9 ms | 1,652.2 ms | 2/200 | 196/200 |\n", + "| Redis 7.2 | New HTTPS connection/request | 138.49 req/s | 139.5 ms | 132.0 ms | 213.8 ms | 0/200 | 200/200 |\n", + "| Redis 7.2 | One persistent connection/worker | 218.69 req/s | 83.2 ms | 77.1 ms | 150.9 ms | 0/200 | 200/200 |\n", + "\n", + "The SQLite result was not stable under concurrency: a separate run reached 86.54 req/s but returned one incomplete response, and later runs logged `sqlite3.InterfaceError: bad parameter or other API misuse`. Redis removed those read errors in this test. These numbers are a development-cluster comparison, not an SLA or capacity limit. Repeat the test with production-sized data, replicas, client connection pooling, and the target traffic shape before setting an SLA.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "set -euo pipefail\n", + "NAMESPACE=\"${FEAST_NAMESPACE:-feast-demo}\"\n", + "FEATURESTORE=\"${FEAST_FEATURESTORE:-demo}\"\n", + "REQUESTS=\"${FEAST_PERF_REQUESTS:-200}\"\n", + "CONCURRENCY=\"${FEAST_PERF_CONCURRENCY:-20}\"\n", + "SERVICE_HOST=\"feast-$FEATURESTORE-online.$NAMESPACE.svc.cluster.local\"\n", + "kubectl exec -i -n \"$NAMESPACE\" \"deploy/feast-$FEATURESTORE\" -c online -- \\\n", + " env PERF_HOST=\"$SERVICE_HOST\" PERF_REQUESTS=\"$REQUESTS\" PERF_CONCURRENCY=\"$CONCURRENCY\" python - <<'PY'\n", + "import concurrent.futures\n", + "import http.client\n", + "import json\n", + "import math\n", + "import os\n", + "import ssl\n", + "import statistics\n", + "import threading\n", + "import time\n", + "\n", + "host = os.environ['PERF_HOST']\n", + "requests = int(os.environ.get('PERF_REQUESTS', '200'))\n", + "concurrency = int(os.environ.get('PERF_CONCURRENCY', '20'))\n", + "body = json.dumps({\n", + " 'features': [\n", + " 'driver_hourly_stats:conv_rate',\n", + " 'driver_hourly_stats:acc_rate',\n", + " 'driver_hourly_stats:avg_daily_trips',\n", + " ],\n", + " 'entities': {'driver_id': [1001, 1002, 1003, 1005]},\n", + "}).encode()\n", + "headers = {'Content-Type': 'application/json', 'Content-Length': str(len(body))}\n", + "context = ssl._create_unverified_context()\n", + "\n", + "def request():\n", + " started = time.perf_counter()\n", + " connection = http.client.HTTPSConnection(host, 443, context=context, timeout=10)\n", + " try:\n", + " connection.request('POST', '/get-online-features', body, headers)\n", + " response = connection.getresponse()\n", + " payload = json.loads(response.read())\n", + " statuses = [\n", + " status\n", + " for result in payload.get('results', [])\n", + " if isinstance(result, dict)\n", + " for status in result.get('statuses', [])\n", + " ]\n", + " complete = len(statuses) == 16 and all(status == 'PRESENT' for status in statuses)\n", + " return (time.perf_counter() - started) * 1000, response.status, complete, None\n", + " except Exception as error:\n", + " return (time.perf_counter() - started) * 1000, None, False, repr(error)\n", + " finally:\n", + " connection.close()\n", + "\n", + "for _ in range(min(10, requests)):\n", + " request()\n", + "barrier = threading.Barrier(concurrency)\n", + "per_worker = [requests // concurrency + (index < requests % concurrency) for index in range(concurrency)]\n", + "\n", + "def worker(index):\n", + " barrier.wait()\n", + " return [request() for _ in range(per_worker[index])]\n", + "\n", + "started = time.perf_counter()\n", + "with concurrent.futures.ThreadPoolExecutor(max_workers=concurrency) as pool:\n", + " results = [item for batch in pool.map(worker, range(concurrency)) for item in batch]\n", + "results = results[:requests]\n", + "elapsed = time.perf_counter() - started\n", + "latencies = sorted(result[0] for result in results)\n", + "errors = [result for result in results if result[1] != 200]\n", + "def percentile(value):\n", + " index = min(len(latencies) - 1, max(0, math.ceil(value * len(latencies)) - 1))\n", + " return latencies[index]\n", + "print(json.dumps({\n", + " 'target': f'https://{host}:443/get-online-features',\n", + " 'requests': len(results),\n", + " 'concurrency': concurrency,\n", + " 'throughput_requests_per_second': round(len(results) / elapsed, 2),\n", + " 'latency_ms': {\n", + " 'avg': round(statistics.mean(latencies), 3),\n", + " 'p50': round(percentile(0.50), 3),\n", + " 'p95': round(percentile(0.95), 3),\n", + " 'p99': round(percentile(0.99), 3),\n", + " },\n", + " 'http_200': len(results) - len(errors),\n", + " 'errors': len(errors),\n", + " 'complete_feature_responses': sum(result[2] for result in results),\n", + " 'error_samples': [result[3] for result in errors[:3]],\n", + "}, indent=2))\n", + "PY\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 8. Production Redis for an online-feature SLA\n", + "\n", + "The benchmark above used a deliberately small, single-replica Redis instance with memory-only storage. Do not use that pattern for a production Feast online service: pod loss removes all materialized values, and it provides no failover, backup, TLS, monitoring, or capacity guarantee.\n", + "\n", + "For production, deploy the platform-managed [Alauda Cache Service for Redis OSS (ACP documentation)](appservice/redis/redis.html) through ACP Data Services. Use that document to choose the Redis architecture and configure durable storage, authentication/TLS, resource sizing, high availability, backup/restore, monitoring, and the read-write access endpoint. Validate connectivity from the Feast namespace before materialization.\n", + "\n", + "For the Feast-side Secret layout and Redis online-store configuration, see [Redis online store + SQL registry](../../../develop/components/feast/quickstart.mdx#redis-online-store--sql-registry). Keep the registry durable as well (for example, SQL-backed PostgreSQL), materialize into the managed Redis service, and size replicas and connection pools against the target p95/p99 latency and availability objectives.\n" + ] } ], "metadata": {