Skip to content

Commit 5f3e78e

Browse files
thiagomdinizthiagomdiniz
authored
Add docker support (#11)
* first commit for docker support * organizing files for use with docker * script changes for docker compatibility * reducing manual actions * final adjustments * update readme
1 parent c5478d6 commit 5f3e78e

File tree

9 files changed

+268
-28
lines changed

9 files changed

+268
-28
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.env
2+
logs/

‎Dockerfile‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
FROM ubuntu:22.04
2+
3+
# repo for x86_64
4+
ARG ZABBIX_REPOSITORY=https://repo.zabbix.com/zabbix/6.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_6.0-4+ubuntu22.04_all.deb
5+
# repo for arm64
6+
#ARG ZABBIX_REPOSITORY=https://repo.zabbix.com/zabbix/6.0/ubuntu-arm64/pool/main/z/zabbix-release/zabbix-release_6.0-5+ubuntu22.04_all.deb
7+
8+
RUN apt-get update \
9+
&& DEBIAN_FRONTEND=noninteractive apt-get install -y libdatetime-perl liblogger-syslog-perl libdbd-mysql-perl libdbi-perl curl tzdata \
10+
&& curl -LO ${ZABBIX_REPOSITORY} \
11+
&& dpkg -i zabbix-release*.deb && rm -f zabbix-release*.deb \
12+
&& apt-get update && apt-get install -y zabbix-sender \
13+
&& rm -rf /var/lib/apt/lists/* \
14+
&& useradd -r -u 999 zabbix
15+
16+
COPY ./docker-entrypoint.sh /usr/local/bin/
17+
COPY ./mysql_zbx_part.pl /usr/local/bin/
18+
COPY ./docker/zabbix_exec_db_partitioning.sh /usr/local/bin/
19+
20+
RUN sed -i 's~my $is_container = 0;~my $is_container = 1;~g' /usr/local/bin/mysql_zbx_part.pl \
21+
&& chmod 755 /usr/local/bin/mysql_zbx_part.pl \
22+
/usr/local/bin/zabbix_exec_db_partitioning.sh \
23+
/usr/local/bin/docker-entrypoint.sh
24+
25+
USER zabbix
26+
27+
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

‎README.md‎

Lines changed: 76 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,26 @@ https://blog.zabbix.com/partitioning-a-zabbix-mysql-database/13531/
2929
Or check out our Zabbix book for a detailed description:
3030
https://www.amazon.com/Zabbix-Infrastructure-Monitoring-Cookbook-maintaining-dp-1801078327/dp/1801078327
3131

32+
MAKE SURE TO UNCOMMENT THE CORRECT LINES FOR THE VERSION YOU NEED. Check the blog post for more information.
33+
```
34+
# MySQL 5.5
35+
# MySQL 5.6 + MariaDB
36+
# MySQL 8.x (NOT MariaDB!)
37+
```
38+
39+
Uncomment the following line for Zabbix 5.4 and OLDER:
40+
```
41+
# $dbh->do("DELETE FROM auditlog_details WHERE NOT EXISTS (SELECT NULL FROM auditlog WHERE auditlog.auditid = auditlog_details.auditid)");
42+
```
43+
44+
Comment the following line for Zabbix 6.4 and OLDER:
45+
```
46+
'history_bin' => { 'period' => 'day', 'keep_history' => '60'},
47+
```
48+
49+
[Run directly on your server](#run-directly-on-your-server) or [run in a Docker container](#run-in-a-docker-container).
50+
51+
### Run directly on your server
3252

3353
Place the script in (create the folder if it doesn't exist):
3454
```
@@ -83,33 +103,77 @@ On a Debian based systems (like Ubuntu) run:
83103
apt-get install libdatetime-perl liblogger-syslog-perl libdbd-mysql-perl
84104
```
85105

86-
MAKE SURE TO UNCOMMENT THE CORRECT LINES FOR THE VERSION YOU NEED. Check the blog post for more information.
106+
That's it! You are now done and you have setup MySQL partitioning. We could execute the script manually with:
87107
```
88-
# MySQL 5.5
89-
# MySQL 5.6 + MariaDB
90-
# MySQL 8.x (NOT MariaDB!)
108+
perl /usr/share/zabbix/mysql_zbx_part.pl
91109
```
92110

93-
Uncomment the following line for Zabbix 5.4 and OLDER:
111+
Then we can check and see if it worked with:
94112
```
95-
# $dbh->do("DELETE FROM auditlog_details WHERE NOT EXISTS (SELECT NULL FROM auditlog WHERE auditlog.auditid = auditlog_details.auditid)");
113+
journalctl -t mysql_zbx_part
96114
```
97115

98-
Comment the following line for Zabbix 6.4 and OLDER:
116+
### Run in a Docker container
117+
118+
#1 Assuming you are in the root directory of this Git repository.
119+
120+
Edit the Dockerfile and uncomment the `ZABBIX_REPOSITORY` argument line according to your Docker host architecture (x86_64 or arm64), then build the Docker image:
121+
99122
```
100-
'history_bin' => { 'period' => 'day', 'keep_history' => '60'},
123+
docker build -t zabbix-db-partitioning .
101124
```
102125

103-
That's it! You are now done and you have setup MySQL partitioning. We could execute the script manually with:
126+
Create log directory. This folder will be mounted as a volume in the container, thus persisting the logs for future reference.
127+
104128
```
105-
perl /usr/lib/zabbix/mysql_zbx_part.pl
129+
mkdir logs
130+
sudo chgrp 999 logs/
131+
chmod 775 logs/
106132
```
107133

108-
Then we can check and see if it worked with:
134+
Create the `.env` file based on the [template](docker/.env.example) and edit it as per your environment.
135+
109136
```
110-
journalctl -t mysql_zbx_part
137+
cp docker/.env.example .env
138+
sudo chown root: .env
139+
sudo chmod 400 .env
140+
```
141+
142+
The command below runs the container to perform the partitioning tasks and, when the perl script finishes executing, the container is automatically stopped and deleted.
143+
144+
```
145+
sudo docker run --rm \
146+
--name zabbix-db-partitioning \
147+
-v ./logs:/logs \
148+
--env-file ./.env \
149+
zabbix-db-partitioning
150+
```
151+
152+
After running the container, you can check the logs:
153+
```
154+
cat logs/mysql_zbx_part.log
155+
```
156+
157+
Edit `root` user crontab:
158+
159+
```
160+
crontab -e
111161
```
112162

163+
Add the line in the crontab, adjusting the schedule. Change `project_dir` to the root directory of this Git repository on your file system.
164+
165+
```
166+
55 22 * * * docker run --rm --name zabbix-db-partitioning -v /project_dir/logs:/logs --env-file /project_dir/.env zabbix-db-partitioning
167+
```
168+
169+
To monitor Perl script execution:
170+
171+
- Import the [zbx_mysql_partitioning_template.yaml](docker/zbx_mysql_partitioning_template.yaml) template into your Zabbix;
172+
- Add the template to an existing host or create a new host;
173+
- Uncomment and configure the `zabbix_sender` variables in the `.env` file;
174+
175+
With that the container will send the result of executing the Perl script to your Zabbix and a trigger will be fired if an error occurs or if the script has not been executed in the last 2 days.
176+
113177
### Partitioning by week
114178
NOTE: See version 2.1 for older (before 20-09-2021) partitioned databases. Otherwise use 3.0+ upwards (recommended to use current).
115179

‎docker-entrypoint.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
set -e
3+
4+
if [[ -n "${LOG_PATH}" ]]; then
5+
/usr/local/bin/zabbix_exec_db_partitioning.sh >> $LOG_PATH 2>&1
6+
else
7+
echo "LOG_PATH environment variable is missing. Exiting..."
8+
exit 1
9+
fi

‎docker/.env.example‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# log file path inside the container
2+
LOG_PATH=/logs/mysql_zbx_part.log
3+
4+
# timezone
5+
TZ=Etc/UTC
6+
7+
# mysql connection settings
8+
DB_HOST=your_db_host
9+
DB_PORT=3306
10+
DB_DATABASE=your_database_name
11+
DB_USER=your_database_user
12+
DB_PASSWORD=your_password
13+
14+
# zabbix_sender settings
15+
#ZABBIX_SERVER=yourserver.address
16+
#ZABBIX_HOST=Your host name
17+
#ZABBIX_ITEM_KEY=zabbix.db.partitioning

‎docker/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Zabbix MySQL partitioning Perl script - Docker
2+
3+
Follow the instructions in the [main README](../README.md).

‎docker/zabbix_exec_db_partitioning.sh‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash
2+
3+
echo -e "\n$(date '+%Y-%m-%d %H:%M') - Starting script execution..."
4+
5+
# Runs the table partitioning script and sends Zabbix the value 1 (success) or 0 (fail).
6+
/usr/local/bin/mysql_zbx_part.pl \
7+
&& ITEM_VALUE=1 \
8+
|| ITEM_VALUE=0
9+
10+
11+
if [[ -n "${ZABBIX_SERVER}" ]]; then
12+
/usr/bin/zabbix_sender -z $ZABBIX_SERVER -s "$ZABBIX_HOST" -k "$ZABBIX_ITEM_KEY" -o $ITEM_VALUE
13+
fi

‎docker/zbx_mysql_partitioning_template.yaml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
zabbix_export:
2+
version: '6.0'
3+
date: '2023-05-20T04:16:07Z'
4+
groups:
5+
-
6+
uuid: f95e6745d71947769f0bc3576f009fcc
7+
name: Templates/Databases/MySQL Partitioning
8+
templates:
9+
-
10+
uuid: 951111adbeab4e91a61b50c7b5fcbca8
11+
template: 'Zabbix database partitioning'
12+
name: 'Zabbix database partitioning'
13+
description: 'Checks if the script that performs the daily maintenance of the partitions of the "history" and "trends" tables in the MySQL database of Zabbix is being executed.'
14+
groups:
15+
-
16+
name: Templates/Databases/MySQL Partitioning
17+
items:
18+
-
19+
uuid: 531df4426c094f11a9c42680113243aa
20+
name: 'Partitioning script execution state'
21+
type: TRAP
22+
key: zabbix.db.partitioning
23+
delay: '0'
24+
valuemap:
25+
name: 'Script execution state'
26+
tags:
27+
-
28+
tag: component
29+
value: database
30+
triggers:
31+
-
32+
uuid: c2af3acc3748476c8dd48447e3c1b996
33+
expression: |
34+
last(/Zabbix database partitioning/zabbix.db.partitioning)<>1
35+
or
36+
nodata(/Zabbix database partitioning/zabbix.db.partitioning,50h,"strict")=1
37+
name: 'Database tables partitioning script was not executed'
38+
priority: AVERAGE
39+
description: 'This means that, if not fixed, new partitions will not be created and Zabbix may stop logging metrics due to lack of partitions. See problem information in the partitioning script log.'
40+
tags:
41+
-
42+
tag: scope
43+
value: capacity
44+
valuemaps:
45+
-
46+
uuid: 8d55537d35014da58ecc3572805f16d0
47+
name: 'Script execution state'
48+
mappings:
49+
-
50+
value: '1'
51+
newvalue: Executed
52+
-
53+
value: '0'
54+
newvalue: 'Not executed'

‎mysql_zbx_part.pl‎

Lines changed: 67 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,56 @@
11
#!/usr/bin/perl
22
use strict;
33
use DBI;
4-
use Sys::Syslog qw(:standard :macros);
54
use DateTime;
65

7-
openlog("mysql_zbx_part", "ndelay,pid", LOG_LOCAL0);
6+
# the Dockerfile will change the value to 1 in the container build process
7+
my $is_container = 0;
8+
# initializing some variables
9+
my $db_schema;
10+
my $db_host;
11+
my $db_port;
12+
my $dsn;
13+
my $db_user_name;
14+
my $db_password;
15+
my $curr_tz;
16+
17+
if ($is_container) {
18+
# check if environment variables exists
19+
if (not defined $ENV{'DB_HOST'}
20+
or not defined $ENV{'DB_PORT'}
21+
or not defined $ENV{'DB_DATABASE'}
22+
or not defined $ENV{'DB_USER'}
23+
or not defined $ENV{'DB_PASSWORD'}
24+
or not defined $ENV{'LOG_PATH'}
25+
or not defined $ENV{'TZ'}
26+
) {
27+
print "Environment variables are missing! Exiting...\n";
28+
exit 1;
29+
}
30+
# open log file
31+
open( OUTPUT, ">>", $ENV{'LOG_PATH'} ) or die $!;
32+
33+
# do not manually modify the next 7 lines, they are only used when the script is run in a container
34+
$db_schema = $ENV{'DB_DATABASE'};
35+
$db_host = $ENV{'DB_HOST'};
36+
$db_port = $ENV{'DB_PORT'};
37+
$dsn = 'DBI:mysql:'.$db_schema.':host='.$db_host.';port='.$db_port;
38+
$db_user_name = $ENV{'DB_USER'};
39+
$db_password = $ENV{'DB_PASSWORD'};
40+
$curr_tz = $ENV{'TZ'};
41+
}
42+
else {
43+
use Sys::Syslog qw(:standard :macros);
44+
openlog("mysql_zbx_part", "ndelay,pid", LOG_LOCAL0);
45+
46+
# edit next 5 lines if the script is run directly in your server
47+
$db_schema = 'zabbix';
48+
$dsn = 'DBI:mysql:'.$db_schema.':mysql_socket=/var/lib/mysql/mysql.sock';
49+
$db_user_name = 'zabbix';
50+
$db_password = 'password';
51+
$curr_tz = 'Etc/UTC';
52+
}
853

9-
my $db_schema = 'zabbix';
10-
my $dsn = 'DBI:mysql:'.$db_schema.':mysql_socket=/var/lib/mysql/mysql.sock';
11-
my $db_user_name = 'zabbix';
12-
my $db_password = 'password';
1354
my $tables = { 'history' => { 'period' => 'day', 'keep_history' => '60'},
1455
'history_log' => { 'period' => 'day', 'keep_history' => '60'},
1556
'history_str' => { 'period' => 'day', 'keep_history' => '60'},
@@ -31,8 +72,6 @@
3172
};
3273
my $amount_partitions = 10;
3374

34-
my $curr_tz = 'Etc/UTC';
35-
3675
# name templates for the different periods
3776
my $partition_name_templates = { 'day' => 'p%Y_%m_%d',
3877
'week' => 'p%Y_w%W',
@@ -45,7 +84,7 @@
4584

4685
unless ( check_have_partition() ) {
4786
print "Your installation of MySQL does not support table partitioning.\n";
48-
syslog(LOG_CRIT, 'Your installation of MySQL does not support table partitioning.');
87+
log_writer('Your installation of MySQL does not support table partitioning.', LOG_CRIT);
4988
exit 1;
5089
}
5190

@@ -65,7 +104,7 @@
65104

66105
foreach my $key (sort keys %{$tables}) {
67106
unless (defined($part_tables->{$key})) {
68-
syslog(LOG_ERR, 'Partitioning for "'.$key.'" is not found! The table might be not partitioned.');
107+
log_writer('Partitioning for "'.$key.'" is not found! The table might be not partitioned.', LOG_ERR);
69108
next;
70109
}
71110

@@ -100,7 +139,7 @@ sub check_have_partition {
100139
# my $sth = $dbh->prepare(qq{select version();});
101140
# $sth->execute();
102141
# my $row = $sth->fetchrow_array();
103-
142+
104143
# $sth->finish();
105144
# return 1 if $row >= 8;
106145
#
@@ -118,13 +157,13 @@ sub create_next_partition {
118157
my $next_name = name_next_part($period, $curr_part);
119158

120159
if (grep { $_ eq $next_name } keys %{$table_part}) {
121-
syslog(LOG_INFO, "Next partition for $table_name table has already been created. It is $next_name");
160+
log_writer("Next partition for $table_name table has already been created. It is $next_name", LOG_INFO);
122161
}
123162
else {
124-
syslog(LOG_INFO, "Creating a partition for $table_name table ($next_name)");
163+
log_writer("Creating a partition for $table_name table ($next_name)", LOG_INFO);
125164
my $query = 'ALTER TABLE '."$db_schema.$table_name".' ADD PARTITION (PARTITION '.$next_name.
126165
' VALUES less than (UNIX_TIMESTAMP("'.date_next_part($period, $curr_part).'") div 1))';
127-
syslog(LOG_DEBUG, $query);
166+
log_writer($query, LOG_DEBUG);
128167
$dbh->do($query);
129168
}
130169
}
@@ -143,11 +182,11 @@ sub remove_old_partitions {
143182

144183
foreach my $partition (sort keys %{$table_part}) {
145184
if ($table_part->{$partition}->{'partition_description'} <= $curr_date->epoch) {
146-
syslog(LOG_INFO, "Removing old $partition partition from $table_name table");
185+
log_writer("Removing old $partition partition from $table_name table", LOG_INFO);
147186

148187
my $query = "ALTER TABLE $db_schema.$table_name DROP PARTITION $partition";
149188

150-
syslog(LOG_DEBUG, $query);
189+
log_writer($query, LOG_DEBUG);
151190
$dbh->do($query);
152191
}
153192
}
@@ -187,3 +226,15 @@ sub delete_old_data {
187226
# Uncomment the following line for Zabbix 5.4 and earlier
188227
# $dbh->do("DELETE FROM auditlog_details WHERE NOT EXISTS (SELECT NULL FROM auditlog WHERE auditlog.auditid = auditlog_details.auditid)");
189228
}
229+
230+
sub log_writer {
231+
my $log_line = shift;
232+
233+
if ($is_container) {
234+
print OUTPUT $log_line . "\n";
235+
}
236+
else {
237+
my $log_priority = shift;
238+
syslog($log_priority, $log_line);
239+
}
240+
}

0 commit comments

Comments
 (0)