Skip to content

Commit 84bc772

Browse files
committed
fix(engine): DEAD 现场复探额度,积分耗尽的 401 不再硬禁用
81d6b36 的收口要求「已探明额度 ≤ 0」,但积分是分钟级烧完的,而周期探额 60 分钟一轮 —— 401 到达时库里往往还是正数,收口条件根本不成立。 实测 2026-11-03 三条 officeace 凭证: - 07:39:37 401,07:36:54 探到 56.66 - 08:00:38 401,00:52:09 探到 100 - 08:25:44 401,08:14:42 探到 51.37 三条全部判 DEAD → 硬禁用 + credential_unavailable;08:40 复探立刻为 0 (新用户福利 4000/4000 + 每日签到 100/100 用满),确认是积分耗尽。 - `_credit_if_exhausted` 改异步:本地证据成立(health == -1 或 quota_remaining <= 0)直接 CREDIT 不出网;证据陈旧(正数 / 未探测)时 `_probe_quota_now` 现场复探一次 `GET /v1/subscription`(只读、不耗积分、 不刷新凭据),写回后按新鲜证据再判。 - 复探仍为正数 / 复探失败 / provider 无 probe_quota / 凭证已并发删除, 一律保持 DEAD:未知 ≠ 耗尽,不能把真坏的 AppKey 静默成「等签到」。 `probe_failed` 不入库,免得一次网络抖动把已探明的正数抹成未知。 - 复探走 `credential_scope`,与推理同一出口(一号一出口);同号并发 401 在 60s 窗口内共用一次复探,避免对着同一账号并发打 N 次订阅接口。 - 五个分类点与 `_stream_error_type` 同判据,尝试行 / 收尾行不会互相矛盾。
1 parent 592fb50 commit 84bc772

8 files changed

Lines changed: 308 additions & 29 deletions

File tree

‎PROPOSAL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383
- **未开通的账号要可行动**:`subscription: null` → 提示「请先在客户端兑换邀请码」,不用含糊的登录失败。
8484
- **三种上游特例**:① 限流走伪 200(HTTP 200 + `data:{"error":…}`,也见过首帧正常、第二帧才错);② `81004`/`81009` 走 403/404 但属模型级,业务码判定先于状态码;③ `max_tokens` 是推理 + 正文总预算,`≥16` 追加 1024 再夹到模型硬上限,`<16` 原样透传,未给不补。
8585
- **积分即主账**:单位 `officeace_points`,`Quota.remaining`/`total` 直接是积分;每笔奖励独立到期。签到 `POST /v1/subscription/bonus/claim` 是写操作,body 必须给 `` + JSON `Content-Type`(空 body 会 500),上游对当日重复领取幂等;收获按 `bonus_skus` 剩余量增量算。
86-
- **不做主动探活 / 不做 credit 估算**:探活要真发推理消耗积分;`cost_coefficient` 与 `tokens_per_point` 叠乘关系官方未说明。
86+
- **不做主动探活 / 不做 credit 估算**:探活要真发推理消耗积分;`cost_coefficient` 与 `tokens_per_point` 叠乘关系官方未说明。(`GET /v1/subscription` 读余额是**只读**、不耗积分,不在此列——它也是 `APIG.1009` 的 DEAD 收口时现场复探用的接口。)
8787
- **单账号单用户**:官方《套餐管理》限制单账号单用户,每人各加自己凭证。
8888

8989
### 2.9 冲突与陷阱

‎README.en.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -522,7 +522,7 @@ Four tabs (channel / model / user / credential) share one table (channel / reque
522522

523523
**Two latency scopes**: successful requests only. Average = `latency_sum ÷ successes` (hourly rollup); percentile from per-request detail (90 days). `NULL` `ttfb_ms` ranks independently in each column. No samples → `—`.
524524

525-
**Failure attribution**: panel below group table, `GET /api/stats/errors` (operator+) aggregates `ok=0` detail by `(channel, model, error_type)` → failure count, **unrescued count**, last failure time (descending, `limit` ≤ 5000). **Failure count = per-attempt**: every upstream rejection lands row, even when retry on another credential succeeds; failure row whose `request_id` also has success row for **same user** counts as rescued; `unrescued = failures − rescued` = **failure attempts never rescued** (a request failing twice counts twice; not number of lost requests). Hard credential failure (DEAD, e.g. 401) also lands a `credential_unavailable` row — **except when that credential's quota is already probed to 0** (officeace returns the same `401 APIG.1009` when its points run out): then it is handled as `CREDIT` (cooled until the next 04:00 CN check-in, not hard-disabled, no such row); an upstream account restriction (CodeBuddy 11140) lands `account_restricted`, cools the account for 2h and stops rotating within that channel, escalating to a whole-channel pause on a second hit in the window; when the upstream says this credential cannot use that model (11102 no such model / 11158 plan not allowed) the row is `model_not_found` and writes an (account, model) negative cache instead of cooling the account; when rotation exhausts without ever picking a credential (all candidates cooling / empty pool) the row records the **candidate channel** rather than `-`; counted as rescued if retry succeeds elsewhere; terminal row not duplicated once rotation exhausted. Click row → detail with *failed only* (`GET /api/stats/events?ok=0`). Admin/operator only. `/metrics` `upstream_errors_total` still in-process counter; not merged.
525+
**Failure attribution**: panel below group table, `GET /api/stats/errors` (operator+) aggregates `ok=0` detail by `(channel, model, error_type)` → failure count, **unrescued count**, last failure time (descending, `limit` ≤ 5000). **Failure count = per-attempt**: every upstream rejection lands row, even when retry on another credential succeeds; failure row whose `request_id` also has success row for **same user** counts as rescued; `unrescued = failures − rescued` = **failure attempts never rescued** (a request failing twice counts twice; not number of lost requests). Hard credential failure (DEAD, e.g. 401) also lands a `credential_unavailable` row — **except when the quota-exhaustion evidence holds** (officeace returns the same `401 APIG.1009` when its points run out): then it is handled as `CREDIT` (cooled until the next 04:00 CN check-in, not hard-disabled, no such row). If the locally probed quota is still positive the executor re-probes that credential once before deciding; only a fresh positive reading or a failed probe keeps it `DEAD`; an upstream account restriction (CodeBuddy 11140) lands `account_restricted`, cools the account for 2h and stops rotating within that channel, escalating to a whole-channel pause on a second hit in the window; when the upstream says this credential cannot use that model (11102 no such model / 11158 plan not allowed) the row is `model_not_found` and writes an (account, model) negative cache instead of cooling the account; when rotation exhausts without ever picking a credential (all candidates cooling / empty pool) the row records the **candidate channel** rather than `-`; counted as rescued if retry succeeds elsewhere; terminal row not duplicated once rotation exhausted. Click row → detail with *failed only* (`GET /api/stats/events?ok=0`). Admin/operator only. `/metrics` `upstream_errors_total` still in-process counter; not merged.
526526

527527
**Row drill-down**: clicking group row filters detail by that row (channel / model / credential; user dimension uses scope), stacking conditions. `GET /api/stats/events` accepts `provider` / `model` / `credential_id` (AND) + `ok=0`.
528528

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -524,7 +524,7 @@ tabs 切四维(渠道 / 模型 / 用户 / 凭证),共用一表(渠道 /
524524

525525
**延迟两类口径**:只统计成功请求。平均 = `latency_sum ÷ 成功数`(小时汇总);分位从逐请求明细算(仅 90 天)。`ttfb_ms` = `NULL` 的两列各自独立排名。无样本 → `—`。
526526

527-
**失败归因**:`GET /api/stats/errors`(operator+)按 `(渠道, 模型, 错误类型)` 聚合窗口内 `ok=0` → 失败次数 / **未救回数** / 最近失败时刻(降序,`limit` ≤ 5000)。**失败数 = 尝试口径**:换号重试每次上游拒绝各落一行;同用户同 `request_id` 另有成功行 = 已救回。`未救回 = 失败数 − 救回数`(同请求多次失败算多次,≠ 受损请求数)。凭证 401 硬禁用(DEAD)也落 `credential_unavailable`(**额度已探明为 0 的凭证除外**:officeace 积分耗尽回的是同一条 `401 APIG.1009`,此时按 `CREDIT` 处理——冷却到次日 04:00 等签到,不硬禁用、不落该归因);账号被上游风控限制(CodeBuddy 11140)落 `account_restricted`,账号级 2h 冷却、当前请求不再换号,同渠道窗口内**第二个不同凭证**也中招时整渠道退避 24h;上游明确说该号用不了这个模型(11102 无此模型 / 11158 套餐不允许)落 `model_not_found`,写 (账号, 模型) 负缓存而非冷却账号;一次都没选中凭证就耗尽时(候选全在冷却 / 池子为空)明细落**候选渠道**而不是 `-`;换号成功算已救回;轮换耗尽不重复落收尾行。点行钻取 +「只看失败」(`GET /api/stats/events?ok=0`)。仅 admin / operator。`/metrics` 的 `upstream_errors_total` 取进程内计数,不合并。
527+
**失败归因**:`GET /api/stats/errors`(operator+)按 `(渠道, 模型, 错误类型)` 聚合窗口内 `ok=0` → 失败次数 / **未救回数** / 最近失败时刻(降序,`limit` ≤ 5000)。**失败数 = 尝试口径**:换号重试每次上游拒绝各落一行;同用户同 `request_id` 另有成功行 = 已救回。`未救回 = 失败数 − 救回数`(同请求多次失败算多次,≠ 受损请求数)。凭证 401 硬禁用(DEAD)也落 `credential_unavailable`(**额度耗尽证据成立的凭证除外**:officeace 积分耗尽回的是同一条 `401 APIG.1009`,此时按 `CREDIT` 处理——冷却到次日 04:00 等签到,不硬禁用、不落该归因;本地额度还是正数时先现场复探一次再判,复探仍为正 / 复探失败才保持 DEAD);账号被上游风控限制(CodeBuddy 11140)落 `account_restricted`,账号级 2h 冷却、当前请求不再换号,同渠道窗口内**第二个不同凭证**也中招时整渠道退避 24h;上游明确说该号用不了这个模型(11102 无此模型 / 11158 套餐不允许)落 `model_not_found`,写 (账号, 模型) 负缓存而非冷却账号;一次都没选中凭证就耗尽时(候选全在冷却 / 池子为空)明细落**候选渠道**而不是 `-`;换号成功算已救回;轮换耗尽不重复落收尾行。点行钻取 +「只看失败」(`GET /api/stats/events?ok=0`)。仅 admin / operator。`/metrics` 的 `upstream_errors_total` 取进程内计数,不合并。
528528

529529
**点行钻取**:点分组表任一行 → 明细按该行过滤(渠道 / 模型 / 凭证;用户维度走作用域参数),条件可叠加。`GET /api/stats/events` 接受 `provider` / `model` / `credential_id`(AND)+ `ok=0`。
530530

0 commit comments

Comments
 (0)