Skip to content

Commit af72c5d

Browse files
authored
Added missing config parameters and general standardization to Kafka input plugin doc. Fixes #2268. (#2269)
Kafka input plugin doc updates: - Fixed missing configuration Parameters: enable_auto_commit, poll_timeout_ms - Fixed case issue: Changed Buffer_Max_Size → buffer_max_size (lowercase to match source code) - Sorted table alphabetically - Fixed header column case: Changed default → Default in the table header - Fixed default value for format: Changed none → none (matches actual default in source) - Fixed description: Added missing period to end of rdkafka.{property} description - Fixed .conf example capitalization Signed-off-by: Eric D. Schabell <eric@schabell.org>
1 parent 439d699 commit af72c5d

File tree

1 file changed

+23
-21
lines changed

1 file changed

+23
-21
lines changed

pipeline/inputs/kafka.md

Lines changed: 23 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,19 @@ This plugin uses the official [librdkafka C library](https://github.com/edenhill
88

99
## Configuration parameters
1010

11-
| Key | Description | default |
11+
| Key | Description | Default |
1212
|:---------------------|:-----------------------------------------------------------------------------------------------------------------------------------------|:-------------|
1313
| `brokers` | Single or multiple list of Kafka Brokers. For example: `192.168.1.3:9092`, `192.168.1.4:9092`. | _none_ |
14-
| `topics` | Single entry or list of comma-separated topics (`,`) that Fluent Bit will subscribe to. | _none_ |
15-
| `format` | Serialization format of the messages. If set to `json`, the payload will be parsed as JSON. | _none_ |
14+
| `buffer_max_size` | Specify the maximum size of buffer per cycle to poll Kafka messages from subscribed topics. To increase throughput, specify larger size. | `4M` |
1615
| `client_id` | Client id passed to librdkafka. | _none_ |
16+
| `enable_auto_commit` | Rely on Kafka auto-commit and commit messages in batches. | `false` |
17+
| `format` | Serialization format of the messages. If set to `json`, the payload will be parsed as JSON. | `none` |
1718
| `group_id` | Group id passed to librdkafka. | `fluent-bit` |
1819
| `poll_ms` | Kafka brokers polling interval in milliseconds. | `500` |
19-
| `Buffer_Max_Size` | Specify the maximum size of buffer per cycle to poll Kafka messages from subscribed topics. To increase throughput, specify larger size. | `4M` |
20-
| `rdkafka.{property}` | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) | _none_ |
20+
| `poll_timeout_ms` | Timeout in milliseconds for Kafka consumer poll operations. Only effective when `threaded` is enabled. | `1` |
21+
| `rdkafka.{property}` | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md). | _none_ |
2122
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
23+
| `topics` | Single entry or list of comma-separated topics (`,`) that Fluent Bit will subscribe to. | _none_ |
2224

2325
## Get started
2426

@@ -57,14 +59,14 @@ pipeline:
5759
5860
```text
5961
[INPUT]
60-
Name kafka
61-
Brokers 192.168.1.3:9092
62-
Topics some-topic
63-
poll_ms 100
62+
Name kafka
63+
Brokers 192.168.1.3:9092
64+
Topics some-topic
65+
Poll_ms 100
6466

6567
[OUTPUT]
66-
Name stdout
67-
Match *
68+
Name stdout
69+
Match *
6870
```
6971

7072
{% endtab %}
@@ -103,22 +105,22 @@ pipeline:
103105
104106
```text
105107
[INPUT]
106-
Name kafka
107-
brokers kafka-broker:9092
108-
topics fb-source
109-
poll_ms 100
110-
format json
108+
Name kafka
109+
Brokers kafka-broker:9092
110+
Topics fb-source
111+
Poll_ms 100
112+
Format json
111113

112114
[FILTER]
113115
Name lua
114116
Match *
115-
script kafka.lua
116-
call modify_kafka_message
117+
Script kafka.lua
118+
Call modify_kafka_message
117119

118120
[OUTPUT]
119-
Name kafka
120-
brokers kafka-broker:9092
121-
topics fb-sink
121+
Name kafka
122+
Brokers kafka-broker:9092
123+
Topics fb-sink
122124
```
123125

124126
{% endtab %}

0 commit comments

Comments
 (0)