Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs-site/docs/features/multi-node.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,6 @@ Environment variables: `BACKUPX_AGENT_MASTER`, `BACKUPX_AGENT_TOKEN`, `BACKUPX_A
## Known limitations

- The Master is single-active because it uses embedded SQLite.
- Encrypted backups are Master-only because Agents do not hold the Master encryption key.
- Encrypted remote backups require upgrading both Master and Agent. The Agent receives only a record-specific derived key; the Master key stays on the Master. Newly created remote archives can be decrypted by the owning restore Agent or Master. Legacy Master-only BXENC1 archives must be restored on the Master. Back up the Master configuration encryption key: changing or losing it makes previous encrypted backups unreadable. Use HTTPS for Agent traffic.
- Remote directory browsing is a synchronous queue RPC with a 15-second timeout.
- Claimed commands that stop reporting progress are timed out according to the Master command monitor.
42 changes: 42 additions & 0 deletions docs-site/docs/features/mysql-pitr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
sidebar_position: 5
---

# MySQL binlog backups and point-in-time recovery

Enable **MySQL binlog / point-in-time recovery (PITR)** in a MySQL task. BackupX creates a whole-instance `mysqldump` baseline, then archives all binary logs since that baseline. Each log archive depends only on its full baseline, not on intermediate backup records. Full baselines are refreshed every 24 hours by default (configurable from 1 to 720 hours).

This is logical full backup plus binlog archiving, not XtraBackup or MariaBackup physical incremental backup. Supported servers are MySQL 8.0 / 8.4; install the MySQL 8.0.26+ `mysql`, `mysqldump` and `mysqlbinlog` clients on the execution node. MariaDB is outside this mode's compatibility scope.

## Setup

- Set the database name to `*`: PITR always includes the whole instance to preserve cross-database transactions.
- Enable binary logging and use InnoDB. Avoid DDL during the full dump; nontransactional tables are not guaranteed consistent.
- Use a fixed Master or Agent execution node. Dynamic node pools are unsupported.
- Configure the task schedule for the required archive frequency. A five-minute schedule normally yields an RPO of approximately five minutes plus execution time.
- Grant normal dump privileges plus log-position, brief read-lock and replication-read privileges (typically `REPLICATION CLIENT`, `REPLICATION SLAVE`, and `RELOAD`; consult your server version). Restore requires whole-instance write privileges and permission to set `sql_log_bin`.
- Do not reset binary logs, reset GTIDs or reinitialize the source during the recovery window; doing so invalidates old log coordinates.
- Keep source binlogs longer than the full-backup interval, archive interval and expected outage combined. BackupX does not change source log retention or purge logs. Missing logs, numbering gaps and changed server identity fail the archive. After correcting retention, wait for the next periodic full dump or create a new PITR task.
- Compression, multiple storage targets, retention and encryption use the normal task controls. Remote encryption requires upgrading both Master and Agent. Protect Agent-to-MySQL connections with a trusted network or database TLS tunnel.

The default Alpine Docker image supplies the MariaDB-compatible `mysql-client` package, not Oracle MySQL clients. For PITR, select a fixed Agent with Oracle MySQL 8.0.26+ clients installed, or supply a custom execution image with those tools. The default image's existing full-dump mode is unchanged. See the [Alpine package definition](https://pkgs.alpinelinux.org/package/v3.24/main/x86/mysql-client).

## Restore

Open a successful **MySQL binlog** record and choose Restore. Enter an optional RFC3339 timestamp with a time zone, such as `2026-11-14T12:30:00+08:00`. It must fall between baseline completion and the archive cutoff, excluding the cutoff second. Leave it empty to restore to the captured binary log position.

The UI displays the available interval. Times are converted to UTC; replay uses `mysqlbinlog --stop-datetime`, applying events before the requested second. Subsecond precision is not guaranteed. The window ends at successfully archived logs and excludes later source transactions.

Restore overwrites the **entire instance** currently configured in the task, including system objects and accounts. First point the task at an isolated recovery instance and verify business data. Import is not transactional; failure does not roll back already applied changes. Automated verification is unsupported for PITR; perform regular isolated recovery drills.

The existing `POST /api/backup/records/:id/restore` endpoint accepts an optional `restoreTime` field:

```json
{"restoreTime":"2026-11-14T12:30:00+08:00"}
```

Automatic retention and manual deletion protect full baselines while retained binlog records depend on them. Effective retention can therefore exceed the configured maximum count.

## Encryption and upgrades

Agents receive only a record-specific derived key; the Master key stays on the Master. New encrypted remote artifacts use `BXENC2`, authenticating record identity, chunk order and an explicit end marker. Master can still decrypt legacy `BXENC1` archives; restore those legacy archives on Master. Back up the Master configuration encryption key: losing or replacing it makes old encrypted backups unreadable. Key rotation and automatic migration of old keys are not included.
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,6 @@ backupx agent --help
## 已知限制

- Master 使用内置 SQLite,只支持单活。
- 加密备份仅支持 Master 本机执行,因为 Agent 不持有 Master 加密密钥。
- 远程加密备份需同时升级 Master 与 Agent。Agent 只获取当前备份记录的派生密钥,Master 主密钥不下发。新远程归档可在获授权的恢复 Agent 或 Master 解密;旧版 Master BXENC1 加密归档仍需在 Master 恢复。请备份 Master 配置加密密钥,丢失或更换会导致旧加密备份不可读。Agent 通信应使用 HTTPS。
- 远程目录浏览是同步队列 RPC,默认超时 15 秒。
- Agent 领取后长期不更新的命令会由 Master 超时监控处理。
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
sidebar_position: 5
---

# MySQL 日志增量与指定时间点恢复

MySQL 任务可以开启 **MySQL 日志增量 / 指定时间点恢复(PITR)**。首次和全量间隔到期时创建 `mysqldump` 全实例基线,其间归档**自该基线以来**的原始 binlog。每份日志归档只依赖一份全量,不依赖中间日志备份记录。恢复会自动下载并校验基线与日志,再按顺序应用。

这是逻辑全量加 binlog 的恢复方案,不是 XtraBackup / MariaBackup 物理增量。当前兼容范围为 MySQL 8.0 / 8.4,执行节点需安装 MySQL 8.0.26+ 的 `mysql`、`mysqldump`、`mysqlbinlog` 客户端。MariaDB 不在此模式兼容范围内。

## 配置

1. 新建 MySQL 任务,填写数据库主机、端口和备份账号。
2. 在策略步骤开启 PITR,数据库名称设为 `*`,表示整个实例。为保证跨库事务正确性,PITR 不提供单库筛选。
3. 选择固定执行节点(Master 或 Agent);不支持节点池动态切换。
4. 设置全量间隔,默认 24 小时,可设置 1–720 小时。计划执行频率决定 binlog 的归档间隔。例如每 5 分钟运行一次,正常情况下 RPO 约为 5 分钟加任务耗时。
5. 配置存储目标、压缩、加密和保留策略。远程加密需要同时升级 Master 和 Agent。

数据库必须开启 `log_bin`,使用 InnoDB。全量期间避免 DDL;非事务表无法保证一致性。备份账号除了常规 mysqldump 所需权限,还需查询日志位置、短暂读取锁和复制读取所需权限(通常包括 `REPLICATION CLIENT`、`REPLICATION SLAVE`、`RELOAD`,实际权限按数据库版本配置)。恢复账号必须拥有创建/覆盖全实例对象及设置 `sql_log_bin` 的权限。请给 Agent 到 MySQL 的网络连接配置可信网络或数据库 TLS 隧道。

恢复窗口内请勿执行 RESET MASTER / RESET BINARY LOGS AND GTIDS 或重新初始化实例;这会使旧日志坐标失效。

源端 binlog 保留期必须大于全量间隔、归档间隔和最大故障恢复时间之和;系统不会修改数据库保留参数,也不会自动清除源端 binlog。源日志被清理、日志编号不连续或服务器 UUID 变化时,本次归档失败,不会伪装成可恢复的备份。修复源端保留策略后,可等待下一次周期性全量,或新建 PITR 任务建立基线。

默认 Alpine Docker 镜像安装的是 MariaDB 兼容的 `mysql-client`,不是 Oracle MySQL 客户端。使用 PITR 时,请选择已安装 Oracle MySQL 8.0.26+ 客户端的固定 Agent,或提供包含这些工具的自定义执行镜像。原有全量备份模式不受影响。参见 [Alpine 包定义](https://pkgs.alpinelinux.org/package/v3.24/main/x86/mysql-client)。

## 恢复

在成功的 **MySQL binlog** 记录详情点击恢复,可输入带时区的 RFC3339 时间,例如 `2026-11-14T12:30:00+08:00`。界面展示本条归档的可恢复范围。时间须在基线完成至日志截止之间,不包含截止秒;留空则恢复到记录捕获的日志位置。

时间统一转换为 UTC,重放使用 `mysqlbinlog --stop-datetime`;它应用目标时间之前的事件,目标秒及其后事件不应用。时间分辨率为秒,不承诺亚秒精度。恢复窗口截至**已成功归档**的日志,不包含归档后的源端事务。

恢复覆盖任务当前配置的**整个实例**,包括数据库与账号等系统对象。建议先把任务连接改到隔离的恢复实例,并实际验收业务数据。恢复不是事务性的,失败不自动回滚已导入的数据。当前不支持自动验证演练,应定期在隔离实例恢复并检查业务结果。

API 使用现有 `POST /api/backup/records/:id/restore`,请求体可增加 `restoreTime`:

```json
{"restoreTime":"2026-11-14T12:30:00+08:00"}
```

自动保留和手动删除都会保护仍有日志备份依赖的全量基线,因此实际保留量可能超过最大份数。先删除或过期其日志记录后,基线才能清理。

## 密钥与升级

远程 Agent 仅获取记录专属派生密钥,新远程加密归档采用 `BXENC2`,包含记录身份、分块顺序与完整结束标记的认证。Master 仍可读取旧 `BXENC1` 归档。请保存 Master 配置加密密钥;仅保存备份文件而丢失此密钥无法恢复。该功能不轮换密钥,不自动处理旧主密钥迁移。
1 change: 1 addition & 0 deletions docs-site/sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ const sidebars: SidebarsConfig = {
'features/storage-backends',
'features/sap-hana',
'features/sql-server',
'features/mysql-pitr',
'features/multi-node',
'features/notifications',
],
Expand Down
81 changes: 53 additions & 28 deletions server/internal/agent/client.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package agent

import (
"backupx/server/internal/backup"
"bytes"
"context"
"crypto/tls"
Expand Down Expand Up @@ -150,22 +151,26 @@ func (c *MasterClient) SubmitCommandResult(ctx context.Context, cmdID uint, succ

// TaskSpec 与 service.AgentTaskSpec 对齐
type TaskSpec struct {
TaskID uint `json:"taskId"`
Name string `json:"name"`
Type string `json:"type"`
SourcePath string `json:"sourcePath"`
SourcePaths string `json:"sourcePaths"`
ExcludePatterns string `json:"excludePatterns"`
DBHost string `json:"dbHost"`
DBPort int `json:"dbPort"`
DBUser string `json:"dbUser"`
DBPassword string `json:"dbPassword"`
DBName string `json:"dbName"`
DBPath string `json:"dbPath"`
ExtraConfig string `json:"extraConfig"`
Compression string `json:"compression"`
Encrypt bool `json:"encrypt"`
StorageTargets []StorageTargetConfig `json:"storageTargets"`
MySQLPITR bool `json:"mysqlPitr"`
MySQLBase *backup.MySQLCheckpoint `json:"mysqlBase,omitempty"`
BaseRecordID uint `json:"baseRecordId,omitempty"`
TaskID uint `json:"taskId"`
Name string `json:"name"`
Type string `json:"type"`
SourcePath string `json:"sourcePath"`
SourcePaths string `json:"sourcePaths"`
ExcludePatterns string `json:"excludePatterns"`
DBHost string `json:"dbHost"`
DBPort int `json:"dbPort"`
DBUser string `json:"dbUser"`
DBPassword string `json:"dbPassword"`
DBName string `json:"dbName"`
DBPath string `json:"dbPath"`
ExtraConfig string `json:"extraConfig"`
Compression string `json:"compression"`
EncryptionKey string `json:"encryptionKey,omitempty"`
Encrypt bool `json:"encrypt"`
StorageTargets []StorageTargetConfig `json:"storageTargets"`
}

// StorageTargetConfig 与 service.AgentStorageTargetConfig 对齐
Expand All @@ -178,9 +183,12 @@ type StorageTargetConfig struct {
}

// GetTaskSpec 拉取任务规格
func (c *MasterClient) GetTaskSpec(ctx context.Context, taskID uint) (*TaskSpec, error) {
func (c *MasterClient) GetTaskSpec(ctx context.Context, taskID uint, recordIDs ...uint) (*TaskSpec, error) {
var spec TaskSpec
path := fmt.Sprintf("/api/agent/tasks/%d", taskID)
if len(recordIDs) > 0 {
path += fmt.Sprintf("?recordId=%d", recordIDs[0])
}
if err := c.do(ctx, http.MethodGet, path, nil, &spec); err != nil {
return nil, err
}
Expand All @@ -189,16 +197,18 @@ func (c *MasterClient) GetTaskSpec(ctx context.Context, taskID uint) (*TaskSpec,

// RecordUpdate 与 service.AgentRecordUpdate 对齐
type RecordUpdate struct {
Status string `json:"status,omitempty"`
FileName string `json:"fileName,omitempty"`
FileSize int64 `json:"fileSize,omitempty"`
Checksum string `json:"checksum,omitempty"`
StoragePath string `json:"storagePath,omitempty"`
StorageTargetID uint `json:"storageTargetId,omitempty"`
StorageTransferMode string `json:"storageTransferMode,omitempty"`
StorageUploadResults []StorageResultItem `json:"storageUploadResults,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
LogAppend string `json:"logAppend,omitempty"`
MySQLCheckpoint *backup.MySQLCheckpoint `json:"mysqlCheckpoint,omitempty"`
BaseRecordID uint `json:"baseRecordId,omitempty"`
Status string `json:"status,omitempty"`
FileName string `json:"fileName,omitempty"`
FileSize int64 `json:"fileSize,omitempty"`
Checksum string `json:"checksum,omitempty"`
StoragePath string `json:"storagePath,omitempty"`
StorageTargetID uint `json:"storageTargetId,omitempty"`
StorageTransferMode string `json:"storageTransferMode,omitempty"`
StorageUploadResults []StorageResultItem `json:"storageUploadResults,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
LogAppend string `json:"logAppend,omitempty"`
}

type StorageResultItem struct {
Expand Down Expand Up @@ -251,7 +261,18 @@ func (c *MasterClient) UploadArtifact(ctx context.Context, recordID, targetID ui
}

// RestoreSpec 与 service.AgentRestoreSpec 对齐
type RestoreArtifact struct {
BackupRecordID uint `json:"backupRecordId"`
FileName string `json:"fileName"`
StoragePath string `json:"storagePath"`
Checksum string `json:"checksum"`
EncryptionKey string `json:"encryptionKey,omitempty"`
Storage StorageTargetConfig `json:"storage"`
}

type RestoreSpec struct {
RestoreTime *time.Time `json:"restoreTime,omitempty"`
Artifacts []RestoreArtifact `json:"artifacts,omitempty"`
RestoreRecordID uint `json:"restoreRecordId"`
BackupRecordID uint `json:"backupRecordId"`
TaskID uint `json:"taskId"`
Expand All @@ -267,6 +288,7 @@ type RestoreSpec struct {
DBPath string `json:"dbPath,omitempty"`
ExtraConfig string `json:"extraConfig,omitempty"`
Compression string `json:"compression"`
EncryptionKey string `json:"encryptionKey,omitempty"`
Encrypt bool `json:"encrypt"`
Storage StorageTargetConfig `json:"storage"`
StoragePath string `json:"storagePath"`
Expand All @@ -291,8 +313,11 @@ func (c *MasterClient) GetRestoreSpec(ctx context.Context, restoreRecordID uint)
return &spec, nil
}

func (c *MasterClient) DownloadRestoreArtifact(ctx context.Context, restoreRecordID uint) (io.ReadCloser, error) {
func (c *MasterClient) DownloadRestoreArtifact(ctx context.Context, restoreRecordID uint, recordIDs ...uint) (io.ReadCloser, error) {
path := fmt.Sprintf("/api/agent/restores/%d/artifact", restoreRecordID)
if len(recordIDs) > 0 {
path += fmt.Sprintf("?recordId=%d", recordIDs[0])
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+path, nil)
if err != nil {
return nil, err
Expand Down
Loading
Loading