Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 18 additions & 16 deletions tiflash/tiflash-mintso-scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,24 @@ MPPクエリを処理する際、TiDBはクエリを1つ以上のMPPタスクに
EXPLAIN SELECT count(*) FROM t0 a JOIN t0 b ON a.id = b.id;
```

+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
| id | estRows | task | access object | operator info |
+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
| HashAgg_44 | 1.00 | root | | funcs:count(Column#8)->Column#7 |
| └─TableReader_46 | 1.00 | root | | MppVersion: 2, data:ExchangeSender_45 |
| └─ExchangeSender_45 | 1.00 | mpp[tiflash] | | ExchangeType: PassThrough |
| └─HashAgg_13 | 1.00 | mpp[tiflash] | | funcs:count(1)->Column#8 |
| └─Projection_43 | 12487.50 | mpp[tiflash] | | test.t0.id |
| └─HashJoin_42 | 12487.50 | mpp[tiflash] | | inner join, equal:[eq(test.t0.id, test.t0.id)] |
| ├─ExchangeReceiver_22(Build) | 9990.00 | mpp[tiflash] | | |
| │ └─ExchangeSender_21 | 9990.00 | mpp[tiflash] | | ExchangeType: Broadcast, Compression: FAST |
| │ └─Selection_20 | 9990.00 | mpp[tiflash] | | not(isnull(test.t0.id)) |
| │ └─TableFullScan_19 | 10000.00 | mpp[tiflash] | table:a | pushed down filter:empty, keep order:false, stats:pseudo |
| └─Selection_24(Probe) | 9990.00 | mpp[tiflash] | | not(isnull(test.t0.id)) |
| └─TableFullScan_23 | 10000.00 | mpp[tiflash] | table:b | pushed down filter:empty, keep order:false, stats:pseudo |
+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
```
+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
| id | estRows | task | access object | operator info |
+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
| HashAgg_44 | 1.00 | root | | funcs:count(Column#8)->Column#7 |
| └─TableReader_46 | 1.00 | root | | MppVersion: 2, data:ExchangeSender_45 |
| └─ExchangeSender_45 | 1.00 | mpp[tiflash] | | ExchangeType: PassThrough |
| └─HashAgg_13 | 1.00 | mpp[tiflash] | | funcs:count(1)->Column#8 |
| └─Projection_43 | 12487.50 | mpp[tiflash] | | test.t0.id |
| └─HashJoin_42 | 12487.50 | mpp[tiflash] | | inner join, equal:[eq(test.t0.id, test.t0.id)] |
| ├─ExchangeReceiver_22(Build) | 9990.00 | mpp[tiflash] | | |
| │ └─ExchangeSender_21 | 9990.00 | mpp[tiflash] | | ExchangeType: Broadcast, Compression: FAST |
| │ └─Selection_20 | 9990.00 | mpp[tiflash] | | not(isnull(test.t0.id)) |
| │ └─TableFullScan_19 | 10000.00 | mpp[tiflash] | table:a | pushed down filter:empty, keep order:false, stats:pseudo |
| └─Selection_24(Probe) | 9990.00 | mpp[tiflash] | | not(isnull(test.t0.id)) |
| └─TableFullScan_23 | 10000.00 | mpp[tiflash] | table:b | pushed down filter:empty, keep order:false, stats:pseudo |
+--------------------------------------------+----------+--------------+---------------+----------------------------------------------------------+
```

例えば、上記のクエリは各TiFlashノードに2つのMPPタスクを生成しますが、 `ExchangeSender_45`のエグゼキューターを含むMPPタスクは`ExchangeSender_21`エグゼキューターを含むMPPタスクに依存しています。同時実行性の高いシナリオでは、スケジューラーが各クエリに対して`ExchangeSender_45`のエグゼキューターを含むMPPタスクをスケジュールすると、システムはデッドロック状態になります。

Expand Down
16 changes: 12 additions & 4 deletions tiflash/tiflash-spill-disk.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ TiFlash は、データをディスクに書き出すための 2 つのトリガ

3. TiFlashのログから、クエリは単一のTiFlashノードで 29.55 GiB のメモリを消費する必要があることがわかります。

[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 29.55 GiB."] [source=MemoryTracker] [thread_id=468]
```
[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 29.55 GiB."] [source=MemoryTracker] [thread_id=468]
```

4. 次のステートメントを実行します。このステートメントは、キーが`GROUP BY`ハッシュ集計子のメモリ使用量を10737418240(10 GiB)に制限します。

Expand All @@ -72,7 +74,9 @@ TiFlash は、データをディスクに書き出すための 2 つのトリガ

5. TiFlashのログを見ると、 `tidb_max_bytes_before_tiflash_external_group_by`設定するとTiFlash が中間結果のスピルをトリガーし、クエリで使用されるメモリが大幅に削減されることがわかります。

[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 12.80 GiB."] [source=MemoryTracker] [thread_id=110]
```
[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 12.80 GiB."] [source=MemoryTracker] [thread_id=110]
```

### クエリレベルのスピル {#query-level-spilling}

Expand Down Expand Up @@ -109,7 +113,9 @@ TiFlash v7.4.0以降、クエリレベルでの自動スピルをサポートし

3. TiFlashのログから、クエリが単一のTiFlashノードで 29.55 GiB のメモリを消費していることがわかります。

[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 29.55 GiB."] [source=MemoryTracker] [thread_id=468]
```
[DEBUG] [MemoryTracker.cpp:69] ["Peak memory usage (total): 29.55 GiB."] [source=MemoryTracker] [thread_id=468]
```

4. 以下のステートメントを実行します。これらのステートメントは、 TiFlashノード上のクエリの最大メモリ使用量を5GiBに制限します。

Expand All @@ -130,7 +136,9 @@ TiFlash v7.4.0以降、クエリレベルでの自動スピルをサポートし

5. TiFlashのログから、クエリ レベルのスピルを構成すると、 TiFlash中間結果のスピルがトリガーされ、クエリで使用されるメモリが大幅に削減されることがわかります。

[DEBUG] [MemoryTracker.cpp:101] ["Peak memory usage (for query): 3.94 GiB."] [source=MemoryTracker] [thread_id=1547]
```
[DEBUG] [MemoryTracker.cpp:101] ["Peak memory usage (for query): 3.94 GiB."] [source=MemoryTracker] [thread_id=1547]
```

## 注記 {#notes}

Expand Down
50 changes: 28 additions & 22 deletions tiflash/troubleshoot-tiflash.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,26 +79,28 @@ TiFlashのワークロードが大きすぎてTiFlashデータのレプリケー
curl http://<pd_ip>:<pd_port>/pd/api/v1/config/rules/group/tiflash
```

[
```
[
{
"group_id": "tiflash",
"id": "table-45-r",
"override": true,
"start_key": "7480000000000000FF2D5F720000000000FA",
"end_key": "7480000000000000FF2E00000000000000F8",
"role": "learner",
"count": 1,
"label_constraints": [
{
"group_id": "tiflash",
"id": "table-45-r",
"override": true,
"start_key": "7480000000000000FF2D5F720000000000FA",
"end_key": "7480000000000000FF2E00000000000000F8",
"role": "learner",
"count": 1,
"label_constraints": [
{
"key": "engine",
"op": "in",
"values": [
"tiflash"
]
}
"key": "engine",
"op": "in",
"values": [
"tiflash"
]
}
]
}
]
```

2. TiFlashに関連するすべてのデータ複製ルールを削除します。例えば、 `id`が`table-45-r`であるルールを例に挙げます。以下のコマンドで削除します。

Expand All @@ -121,11 +123,13 @@ show warnings;

この例では、警告メッセージは、TiDB 5.4 以前のバージョンでは`subtime`関数がサポートされていないため、TiDB が MPP モードを選択しないことを示しています。

+---------+------+-----------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------------------------------+
| Warning | 1105 | Scalar function 'subtime'(signature: SubDatetimeAndString, return type: datetime) is not supported to push down to tiflash now. |
+---------+------+-----------------------------------------------------------------------------+
```
+---------+------+-----------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------------------------------+
| Warning | 1105 | Scalar function 'subtime'(signature: SubDatetimeAndString, return type: datetime) is not supported to push down to tiflash now. |
+---------+------+-----------------------------------------------------------------------------+
```

## TiFlashレプリカは常に利用できません {#tiflash-replica-is-always-unavailable}

Expand Down Expand Up @@ -186,7 +190,9 @@ TiDB クラスターを展開した後、 TiFlashレプリカの作成が継続

- 値を`low-space-ratio`に変更すると、PD は新しいしきい値に達するまでTiFlashノードへのリージョンのスケジュールを再開できるようになります。

tiup ctl:nightly pd -u http://${pd-ip}:${pd-port} config set low-space-ratio 0.9
```
tiup ctl:nightly pd -u http://${pd-ip}:${pd-port} config set low-space-ratio 0.9
```

- 新しいTiFlashノードをスケールアウトします。PD はTiFlashノード間でリージョンのバランスを自動的に取り、十分なディスク容量を持つTiFlashノードへのリージョンのスケジュールを再開します。

Expand Down
40 changes: 23 additions & 17 deletions tiflash/use-tidb-to-read-tiflash.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,26 +17,30 @@ TiFlashレプリカを持つテーブルの場合、TiDBオプティマイザー
desc select count(*) from test.t;
```

+--------------------------+---------+--------------+---------------+--------------------------------+
| id | estRows | task | access object | operator info |
+--------------------------+---------+--------------+---------------+--------------------------------+
| StreamAgg_9 | 1.00 | root | | funcs:count(1)->Column#4 |
| └─TableReader_17 | 1.00 | root | | data:TableFullScan_16 |
| └─TableFullScan_16 | 1.00 | cop[tiflash] | table:t | keep order:false, stats:pseudo |
+--------------------------+---------+--------------+---------------+--------------------------------+
3 rows in set (0.00 sec)
```
+--------------------------+---------+--------------+---------------+--------------------------------+
| id | estRows | task | access object | operator info |
+--------------------------+---------+--------------+---------------+--------------------------------+
| StreamAgg_9 | 1.00 | root | | funcs:count(1)->Column#4 |
| └─TableReader_17 | 1.00 | root | | data:TableFullScan_16 |
| └─TableFullScan_16 | 1.00 | cop[tiflash] | table:t | keep order:false, stats:pseudo |
+--------------------------+---------+--------------+---------------+--------------------------------+
3 rows in set (0.00 sec)
```

```sql
explain analyze select count(*) from test.t;
```

+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
| StreamAgg_9 | 1.00 | 1 | root | | time:83.8372ms, loops:2 | funcs:count(1)->Column#4 | 372 Bytes | N/A |
| └─TableReader_17 | 1.00 | 1 | root | | time:83.7776ms, loops:2, rpc num: 1, rpc time:83.5701ms, proc keys:0 | data:TableFullScan_16 | 152 Bytes | N/A |
| └─TableFullScan_16 | 1.00 | 1 | cop[tiflash] | table:t | tiflash_task:{time:43ms, loops:1, threads:1}, tiflash_scan:{...} | keep order:false, stats:pseudo | N/A | N/A |
+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
```
+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
| StreamAgg_9 | 1.00 | 1 | root | | time:83.8372ms, loops:2 | funcs:count(1)->Column#4 | 372 Bytes | N/A |
| └─TableReader_17 | 1.00 | 1 | root | | time:83.7776ms, loops:2, rpc num: 1, rpc time:83.5701ms, proc keys:0 | data:TableFullScan_16 | 152 Bytes | N/A |
| └─TableFullScan_16 | 1.00 | 1 | cop[tiflash] | table:t | tiflash_task:{time:43ms, loops:1, threads:1}, tiflash_scan:{...} | keep order:false, stats:pseudo | N/A | N/A |
+--------------------------+---------+---------+--------------+---------------+----------------------------------------------------------------------+--------------------------------+-----------+------+
```

`cop[tiflash]`は、タスクが処理のためにTiFlashに送信されることを意味します。TiFlash レプリカを選択していない場合は、 `analyze table`ステートメントを使用して統計情報を更新し、 `explain analyze`ステートメントを使用して結果を確認できます。

Expand All @@ -52,8 +56,10 @@ explain analyze select count(*) from test.t;

- TiDBインスタンスレベル、つまりINSTANCEレベル。TiDB設定ファイルに以下の設定項目を追加します。

[isolation-read]
engines = ["tikv", "tidb", "tiflash"]
```
[isolation-read]
engines = ["tikv", "tidb", "tiflash"]
```

**INSTANCE レベルのデフォルト設定は`["tikv", "tidb", "tiflash"]`です。**

Expand Down
12 changes: 7 additions & 5 deletions tiflash/use-tiflash-mpp-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,13 @@ set @@session.tidb_enforce_mpp=1;
> show warnings;
> ```
>
> +---------+------+-----------------------------------------------------------------------------+
> | Level | Code | Message |
> +---------+------+-----------------------------------------------------------------------------+
> | Warning | 1105 | MPP mode may be blocked because there aren't tiflash replicas of table `t`. |
> +---------+------+-----------------------------------------------------------------------------+
> ```
> +---------+------+-----------------------------------------------------------------------------+
> | Level | Code | Message |
> +---------+------+-----------------------------------------------------------------------------+
> | Warning | 1105 | MPP mode may be blocked because there aren't tiflash replicas of table `t`. |
> +---------+------+-----------------------------------------------------------------------------+
> ```

## MPPモードのアルゴリズムサポート {#algorithm-support-for-the-mpp-mode}

Expand Down
Loading
Loading