From 96dd8f986c06523fdd8cf759df07392ba560f6f2 Mon Sep 17 00:00:00 2001 From: Louie Lu Date: Mon, 28 Sep 2026 14:09:36 -0500 Subject: [PATCH] Leaderboard: update LLM Router to llm-router-g3 (#192) LLM Router moves from 71.26 to 75.69 (#5): gemini-3-flash-preview on every query, robustness 100. The manifest now points at llm-router-g3. Co-Authored-By: Claude Opus 5.5 --- README.md | 24 ++++++++++++------------ leaderboard_manifest.yaml | 2 +- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index a2c6721e..4de8518c 100644 --- a/README.md +++ b/README.md @@ -41,18 +41,18 @@ For more details, please see our [website](https://routeworks.github.io/leaderbo | πŸ₯ˆ | [Sqwish Router](https://www.sqwish.ai/) | πŸ‘€ [@namitha-sqwish](https://github.com/namitha-sqwish) | 76.21 | 79.76 | $0.70 | 9.04 | 23.49 | 94.07 | β€” | 51.67 | | πŸ₯‰ | [Divyam]() | πŸ‘€ [@samikd](https://github.com/samikd) | 75.85 | 78.59 | $0.48 | 4.93 | 16.03 | 93.43 | β€” | 98.33 | | 4 | [Cross-Router]() | πŸ‘€ [@JiaHg](https://github.com/JiaHg) | 75.75 | 78.14 | $0.40 | 17.66 | 45.49 | 90.31 | β€” | 67.14 | -| 5 | [R\*\*2: Reasonometry Router]() | πŸ‘€ [@sashakolpakov](https://github.com/sashakolpakov) | 75.53 | 78.11 | $0.45 | β€” | β€” | β€” | β€” | 93.57 | -| 6 | [SapientAI Auto Router]() | πŸ’Ό Publicis Sapient | 75.41 | 77.89 | $0.43 | 5.93 | 19.99 | 92.00 | β€” | 73.33 | -| 7 | [vLLM‑SR](https://vllm-semantic-router.com/) [[Code]](https://github.com/vllm-project/semantic-router) [[HF]](https://huggingface.co/llm-semantic-router) | πŸŽ“ vLLM SR Team | 74.86 | 77.18 | $0.42 | 16.81 | 25.10 | 89.37 | β€” | 67.62 | -| 8 | [nadir-caliper]() | πŸ‘€ [@doramirdor](https://github.com/doramirdor) | 74.55 | 75.84 | $0.22 | β€” | β€” | β€” | β€” | 79.76 | -| 9 | [AgentForge Router]() | πŸ‘€ [@YangY-Z](https://github.com/YangY-Z) | 74.13 | 74.72 | $0.13 | 17.84 | 52.47 | 98.68 | β€” | 40.48 | -| 10 | [BARouter]() | πŸ‘€ [@zulk2002](https://github.com/zulk2002) | 73.79 | 75.72 | $0.36 | 64.41 | 67.13 | 93.84 | β€” | 68.81 | -| 11 | [Weave Router](https://workweave.ai) | πŸŽ“ Weave | 72.82 | 76.32 | $0.94 | β€” | β€” | β€” | β€” | 100.00 | -| 12 | [Nadir Router](https://github.com/NadirRouter/NadirClaw) | πŸŽ“ NadirRouter | 72.29 | 75.01 | $0.68 | β€” | β€” | β€” | β€” | 25.48 | -| 13 | [OrcaRouter‑Adaptive](https://www.orcarouter.ai/) [[Code]](https://github.com/Continuum-AI-Corp/OrcaRouter-Lite) [[Paper]](https://arxiv.org/abs/2605.30736) [[X]](https://x.com/orcarouter) | πŸŽ“ [Continuum AI](https://www.continuum01.ai/) | 72.08 | 75.54 | $1.00 | β€” | β€” | β€” | β€” | 22.62 | -| 14 | [Hybrid Router]() | πŸ‘€ [@mikemao27](https://github.com/mikemao27) | 72.08 | 71.38 | $0.04 | 89.87 | 94.19 | 92.81 | β€” | 96.67 | -| 15 | [R2-Router](https://arxiv.org/abs/2602.02823/) | πŸŽ“ UCF | 71.60 | 71.23 | $0.06 | 24.51 | 48.70 | 99.85 | β€” | 45.71 | -| 16 | [LLM Router](https://github.com/ypollak2/llm-router) [[PyPI]](https://pypi.org/project/llm-routing/) | πŸ‘€ [@ypollak2](https://github.com/ypollak2) | 71.26 | 72.05 | $0.20 | 18.01 | 20.46 | 89.13 | β€” | 30.00 | +| 5 | [LLM Router](https://github.com/ypollak2/llm-router) [[PyPI]](https://pypi.org/project/llm-routing/) | πŸ‘€ [@ypollak2](https://github.com/ypollak2) | 75.69 | 78.44 | $0.49 | β€” | β€” | β€” | β€” | 100.00 | +| 6 | [R\*\*2: Reasonometry Router]() | πŸ‘€ [@sashakolpakov](https://github.com/sashakolpakov) | 75.53 | 78.11 | $0.45 | β€” | β€” | β€” | β€” | 93.57 | +| 7 | [SapientAI Auto Router]() | πŸ’Ό Publicis Sapient | 75.41 | 77.89 | $0.43 | 5.93 | 19.99 | 92.00 | β€” | 73.33 | +| 8 | [vLLM‑SR](https://vllm-semantic-router.com/) [[Code]](https://github.com/vllm-project/semantic-router) [[HF]](https://huggingface.co/llm-semantic-router) | πŸŽ“ vLLM SR Team | 74.86 | 77.18 | $0.42 | 16.81 | 25.10 | 89.37 | β€” | 67.62 | +| 9 | [nadir-caliper]() | πŸ‘€ [@doramirdor](https://github.com/doramirdor) | 74.55 | 75.84 | $0.22 | β€” | β€” | β€” | β€” | 79.76 | +| 10 | [AgentForge Router]() | πŸ‘€ [@YangY-Z](https://github.com/YangY-Z) | 74.13 | 74.72 | $0.13 | 17.84 | 52.47 | 98.68 | β€” | 40.48 | +| 11 | [BARouter]() | πŸ‘€ [@zulk2002](https://github.com/zulk2002) | 73.79 | 75.72 | $0.36 | 64.41 | 67.13 | 93.84 | β€” | 68.81 | +| 12 | [Weave Router](https://workweave.ai) | πŸŽ“ Weave | 72.82 | 76.32 | $0.94 | β€” | β€” | β€” | β€” | 100.00 | +| 13 | [Nadir Router](https://github.com/NadirRouter/NadirClaw) | πŸŽ“ NadirRouter | 72.29 | 75.01 | $0.68 | β€” | β€” | β€” | β€” | 25.48 | +| 14 | [OrcaRouter‑Adaptive](https://www.orcarouter.ai/) [[Code]](https://github.com/Continuum-AI-Corp/OrcaRouter-Lite) [[Paper]](https://arxiv.org/abs/2605.30736) [[X]](https://x.com/orcarouter) | πŸŽ“ [Continuum AI](https://www.continuum01.ai/) | 72.08 | 75.54 | $1.00 | β€” | β€” | β€” | β€” | 22.62 | +| 15 | [Hybrid Router]() | πŸ‘€ [@mikemao27](https://github.com/mikemao27) | 72.08 | 71.38 | $0.04 | 89.87 | 94.19 | 92.81 | β€” | 96.67 | +| 16 | [R2-Router](https://arxiv.org/abs/2602.02823/) | πŸŽ“ UCF | 71.60 | 71.23 | $0.06 | 24.51 | 48.70 | 99.85 | β€” | 45.71 | | 17 | [cruq-router]() | πŸ‘€ [@nabaruns](https://github.com/nabaruns) | 70.77 | 71.35 | $0.18 | β€” | β€” | β€” | β€” | 81.67 | | 18 | [chuzom-solo-v32]() | πŸ‘€ [@ypollak2](https://github.com/ypollak2) | 70.61 | 70.59 | $0.10 | β€” | β€” | β€” | β€” | 100.00 | | 19 | [Azure-Model-Router](https://ai.azure.com/catalog/models/model-router) [[Web]](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/model-router) | πŸ’Ό Microsoft | 70.42 | 72.94 | $0.73 | β€” | β€” | β€” | β€” | 71.43 | diff --git a/leaderboard_manifest.yaml b/leaderboard_manifest.yaml index 0a2d6c51..9c6a0afc 100644 --- a/leaderboard_manifest.yaml +++ b/leaderboard_manifest.yaml @@ -119,7 +119,7 @@ routers: - readme_name: "LLM Router" website_name: "LLM Router" - prediction: "llm-router" + prediction: "llm-router-g3" category_key: "llm-router" flip_key: "llm-router" website: