A polyglot (Go, Java, Python) microservices benchmark for software architecture recovery (SAR) tools, built around a food-delivery system. Ground truth ships as an annotated dependency graph: services, datastores and message destinations as nodes, typed endpoint-level edges labelled with the SAR pattern that obscures them and whether each is statically resolvable and/or observed at runtime. See dataset/dataset-schema.md. A grader in grading/ scores tool output against it.
The diagram shows the target system. Not all services are implemented yet; the published dataset describes the current state.
| Path | Contents |
|---|---|
services/ |
The benchmark's microservices |
dataset/ |
Published dependency graph + validation report and their schemas |
grading/ |
Grader and naive baseline |
infra/ |
Tooling (release stamping, canonicalisation) |
proto/ |
gRPC definitions |
docs/ |
Documentation and diagrams |
docker compose up --build
Brings up the entire stack. Jaeger (traces) is available at http://localhost:16686.
Once everything is healthy, you can run the smoke test against the running stack:
python infra/workload/run.py
It exercises each service's /health and a few real endpoints, and exits non-zero on any unexpected status code.
A submission is a JSON file listing the nodes and caller → callee edges your tool recovered:
{
"nodes": ["address", "geocoding", "db:address"],
"edges": [
{"caller": "address", "callee": "geocoding"},
{"caller": "address", "callee": "db:address"}
]
}Services are named after their folder in services/, datastores as db:<name> named after the service that owns them (db:geocoding). Every caller and callee must appear in nodes.
python grading/grader/grader.py results.json [--dbs]
Datastores are ignored unless --dbs is passed. The grader prints the number of correct, missing and additional caller → callee pairs. For a reference point, grading/baselines/baseline.py produces a submission with a naive grep: python grading/baselines/baseline.py services --output results.json.
- The complete synchronous core: the customer, restaurant and courier domains.
- A first end-to-end asynchronous edge: RabbitMQ publish and consume, with trace context carried in the message.
- Authentication between all services.
- Event-driven communication across the system, with ground truth and runtime trace verification for message-based edges.
- Endpoint-level and per-pattern scoring in the grader.
- Ground truth checked by an independent second rater.
- A realistic workload generator for collecting runtime data.
- A stable, citable release archived on Zenodo.
- Reference results for existing architecture recovery tools.
This project's main purpose is to test SAR tools and approaches, so some service calls are deliberately obfuscated. Each obfuscation is realistic on its own, but may be used in an unusual way.
The project currently doesn't encrypt its traffic with TLS.
The auth service uses a committed static private key. This is clearly not a production-ready setup, but it simplifies testing and deployment.
gRPC clients use the default pick_first load balancing, so each client sends all of its traffic to a single replica of a scaled service. This doesn't matter for the current single-instance setup.
Some code, such as the JWT verifier, is deliberately copied into each service instead of shared as a library. Every service then makes the call itself, which gives more and more varied edges than a single shared implementation would. The copies have to be kept in sync by hand.
Apache-2.0, see LICENSE.