[BUG] 升级 latest 后存储桶写入全部 503，扫描器卡住 / 长时间无法完成

[yh4922]

问题描述

将 RustFS 从旧版本（大约 2026 年 4 月部署，版本估计为 v0.1.7）升级到最新的 rustfs/rustfs:latest 镜像后，所有存储桶的上传操作都返回 503 错误。管理后台（9001 端口）显示存储桶的对象数量和大小均为“未知”，状态页面显示“权威存储用量快照尚不可用”。

重现步骤

1. 原有部署使用 rustfs/rustfs:latest（2026 年 4 月左右，可能为 v0.1.7），通过 Docker Compose 运行：

   • 环境变量：RUSTFS_VOLUMES=/data，RUSTFS_TLS_PATH=/opt/tls/（证书文件夹为空）
   • 使用 Nginx 反向代理处理 TLS 终止

2. 大约 2026 年 8 月 12 日开始，登录时频繁报错“登录失败 set_temp_user”。

3. 拉取最新的 rustfs/rustfs:latest 镜像，并重启容器：

   docker compose down && docker compose up -d

4. 启动时因 TLS/证书问题报错，于是移除 RUSTFS_TLS_PATH 环境变量和证书挂载卷，重新启动成功。

5. 重启后，所有存储桶上传均返回 503。

预期行为

升级后应能正常上传，或在合理的预热时间后自动恢复。

实际行为

• 上传 API 返回：

  • 错误类型：ServiceUnavailable
  • 错误信息：Bucket quota check temporarily unavailable, please retry
  • HTTP 状态码：503

• Admin 管理页面（:9001）：

  • 存储桶列表中的“对象数量”和“大小”显示为“未知”
  • 状态页面显示：“权威存储用量快照尚不可用”
  • “最近成功更新时间”显示为“未知”

• 扫描器状态：页面显示“存储用量与扫描器再扫描中”，截至提交该 Issue 时，扫描器仍在持续运行中，尚未完成首次全量扫描，且没有显示预计完成时间或进度百分比。

环境信息

• 部署方式：Docker Compose
• 镜像：rustfs/rustfs:latest（2026 年 8 月 12 日左右拉取）
• 旧版本：2026 年 4 月部署，大约为 v0.1.7
• Compose 文件：

  version: "3.9"
  
  services:
    rustfs:
      image: rustfs/rustfs:latest
      container_name: rustfs
      ports:
        - "127.0.0.1:9000:9000"
        - "127.0.0.1:9001:9001"
      environment:
        - RUSTFS_VOLUMES=/data
        - RUSTFS_ACCESS_KEY=test
        - RUSTFS_SECRET_KEY=123456
      volumes:
        - /userdata/rustfs/data:/data
      restart: unless-stopped

相关 Issue / PR

该问题似乎与以下已知问题一致：

• #5716（beta.12 升级报告）：从 beta.11 升级后，每个启用配额的存储桶写入都会失败，返回 503 Bucket quota check temporarily unavailable
• #5722（修复）：在权威用量不可用时，允许基于持久化的用量基线进行写入
• #5494（修复）：在扫描器预热期间保持 /datausageinfo 端点可用
• #5818（修复）：提供扫描测得的用量，而非将整个快照置空

待确认问题

1. 扫描器在约 7TB（或类似规模）的数据量下持续扫描 2.5 小时以上仍未完成，这是否属于正常现象？是否存在性能瓶颈或扫描卡死的情况？
2. 扫描器完成第一个完整周期后，上传操作是否会自动恢复，还是需要手动干预（例如移除/重新添加存储桶配额）？
3. 是否有环境变量或配置可以在扫描器预热期间临时绕过配额检查，以缩短生产中断时间？

补充说明

• 证书文件夹（/userdata/rustfs/certs/）为空；TLS 由 Nginx 终止，RustFS 自身未启用 TLS。
• 升级时直接拉取 latest 标签，未指定具体版本号。
• 没有显式配置存储桶配额（据我所知）。

影响范围

生产环境写入服务完全中断：所有上传均失败。数据本身未受影响（数据目录完整），但写入可用性完全被阻塞，需等待扫描完成。由于扫描器迟迟未完成，业务影响时间正在持续延长。

[yh4922]

[cxymds]

rustfs/rustfs#5722

[cxymds]

This issue is related to quota checks failing while the authoritative usage snapshot is unavailable after upgrading from an older version. RustFS 1.0.0-rc.2 includes the corresponding fixes.
We recommend upgrading to the pinned image rustfs/rustfs:1.0.0-rc.2 instead of using the floating latest tag. After upgrading, please verify both regular PUT operations and multipart uploads.
The initial scanner cycle may continue running for some time. However, for existing buckets with a persisted legacy usage baseline, writes should no longer continuously fail with 503 Bucket quota check temporarily unavailable while waiting for the first complete scan.

[yh4922]

我升级了 rustfs/rustfs:1.0.0-rc.2-glibc 镜像 之后 可以正常 上传内容不报 503 的异常了，但是再对象浏览 >存储桶 页面查看的时候
桶内的 对象数量和大小都看不到

[cxymds]

Please provide the following information:
A screenshot of the Status page showing the scanner status, the last completed scan time, and the scan duration.
RustFS logs related to scanner and data usage since the service was started.

[yh4922]

20260819080231.070180-85.rustfs.log

[cxymds]

Thank you for providing the screenshot and logs.

Upgrading to rustfs/rustfs:1.0.0-rc.2-glibc has resolved the original upload failure. The remaining issue with missing object counts and bucket sizes is caused by the data-usage snapshot still being unavailable.

After upgrading from an older release, the legacy .usage.json snapshot is not treated as authoritative because it does not contain the completeness metadata required by the current version. RustFS retains the legacy per-bucket usage as a degraded quota baseline, which allows uploads to proceed, but it does not expose that baseline as authoritative statistics through /accountinfo.

The screenshot shows:

• The current scanner cycle has been running for 4 hours and 33 minutes.
• The last completed scan was 5 days ago.
• Five buckets are still marked as pending dirty buckets.
• The logs report authoritative bucket usage is not available yet.

Therefore, the Console currently receives no authoritative objects or size values and displays --. This does not indicate data loss and does not affect object uploads or reads.

The logs also contain two capacity_scan_timeout warnings and several slow storage commit warnings. These suggest storage or directory-scanning pressure, but the provided file contains only WARN/ERROR entries from a one-hour period, so it is not sufficient to identify the exact reason the scanner is taking this long.

Please let the current scanner cycle finish and check whether the statistics appear afterward. If the scan completes but the values are still unavailable, please provide:

1. A new screenshot showing the completed scanner state.
2. Complete RustFS logs from startup with INFO-level scanner and data-usage messages.
3. The response bodies of /rustfs/admin/v3/datausageinfo and /rustfs/admin/v3/accountinfo.

I will keep the issue open until confirm that the first authoritative snapshot is published successfully.