Skip to content

Commit 09052d3

Browse files
fcostaoliveirafcostaoliveira
authored
Add UndefinedBehaviorSanitizer (UBSan) support. (#324)
1 parent 045f75b commit 09052d3

File tree

4 files changed

+243
-123
lines changed

4 files changed

+243
-123
lines changed

‎.github/workflows/ubsan.yml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
name: UBSan (UndefinedBehaviorSanitizer)
2+
3+
# Undefined behavior detection using UndefinedBehaviorSanitizer
4+
# This workflow builds memtier_benchmark with UBSan enabled and runs
5+
# the full test suite to detect undefined behavior issues.
6+
7+
on: [push, pull_request]
8+
9+
jobs:
10+
test-with-ubsan:
11+
runs-on: ubuntu-latest
12+
name: Undefined behavior detection (UBSan)
13+
steps:
14+
- name: Checkout code
15+
uses: actions/checkout@v4
16+
17+
- name: Install build dependencies
18+
run: |
19+
sudo apt-get -qq update
20+
sudo apt-get install -y \
21+
build-essential \
22+
autoconf \
23+
automake \
24+
pkg-config \
25+
libevent-dev \
26+
zlib1g-dev \
27+
libssl-dev
28+
29+
- name: Build with UBSan
30+
run: |
31+
autoreconf -ivf
32+
./configure --enable-ubsan
33+
make -j
34+
35+
- name: Setup Python
36+
uses: actions/setup-python@v2
37+
with:
38+
python-version: '3.10'
39+
architecture: x64
40+
41+
- name: Install Python test dependencies
42+
run: pip install -r ./tests/test_requirements.txt
43+
44+
- name: Install Redis
45+
run: |
46+
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
47+
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
48+
sudo apt-get -qq update
49+
sudo apt-get install redis
50+
sudo service redis-server stop
51+
52+
- name: Increase connection limit
53+
run: |
54+
sudo sysctl -w net.ipv4.tcp_fin_timeout=10
55+
sudo sysctl -w net.ipv4.tcp_tw_reuse=1
56+
ulimit -n 40960
57+
58+
- name: Generate TLS test certificates
59+
run: ./tests/gen-test-certs.sh
60+
61+
- name: Test OSS TCP with UBSan
62+
timeout-minutes: 10
63+
run: |
64+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 ./tests/run_tests.sh
65+
66+
- name: Test OSS TCP TLS with UBSan
67+
timeout-minutes: 10
68+
run: |
69+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 TLS=1 ./tests/run_tests.sh
70+
71+
- name: Test OSS TCP TLS v1.2 with UBSan
72+
timeout-minutes: 10
73+
run: |
74+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 TLS_PROTOCOLS='TLSv1.2' TLS=1 ./tests/run_tests.sh
75+
76+
- name: Test OSS TCP TLS v1.3 with UBSan
77+
timeout-minutes: 10
78+
run: |
79+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 TLS_PROTOCOLS='TLSv1.3' TLS=1 ./tests/run_tests.sh
80+
81+
- name: Test OSS-CLUSTER TCP with UBSan
82+
timeout-minutes: 10
83+
run: |
84+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 OSS_STANDALONE=0 OSS_CLUSTER=1 ./tests/run_tests.sh
85+
86+
- name: Test OSS-CLUSTER TCP TLS with UBSan
87+
timeout-minutes: 10
88+
run: |
89+
UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 OSS_STANDALONE=0 OSS_CLUSTER=1 TLS=1 ./tests/run_tests.sh
90+

‎DEVELOPMENT.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Development Guide
2+
3+
This document provides information for developers working on memtier_benchmark.
4+
5+
## Building from Source
6+
7+
### Prerequisites
8+
9+
The following libraries are required for building:
10+
11+
* libevent 2.0.10 or newer.
12+
* OpenSSL (unless TLS support is disabled by `./configure --disable-tls`).
13+
14+
The following tools are required:
15+
* autoconf
16+
* automake
17+
* pkg-config
18+
* GNU make
19+
* GCC C++ compiler
20+
21+
### CentOS/Red Hat Linux 7 or newer
22+
23+
Use the following to install prerequisites:
24+
```
25+
$ sudo yum install autoconf automake make gcc-c++ \
26+
zlib-devel libmemcached-devel libevent-devel openssl-devel
27+
```
28+
29+
### Ubuntu/Debian
30+
31+
Use the following to install prerequisites:
32+
33+
```
34+
$ sudo apt-get install build-essential autoconf automake \
35+
libevent-dev pkg-config zlib1g-dev libssl-dev
36+
```
37+
38+
### macOS
39+
40+
To build natively on macOS, use Homebrew to install the required dependencies:
41+
42+
```
43+
$ brew install autoconf automake libtool libevent pkg-config openssl@3.0
44+
```
45+
46+
When running `./configure`, if it fails to find libssl it may be necessary to
47+
tweak the `PKG_CONFIG_PATH` environment variable:
48+
49+
```
50+
PKG_CONFIG_PATH=`brew --prefix openssl@3.0`/lib/pkgconfig ./configure
51+
```
52+
53+
### Building and Installing
54+
55+
After downloading the source tree, use standard autoconf/automake commands:
56+
57+
```
58+
$ autoreconf -ivf
59+
$ ./configure
60+
$ make
61+
$ sudo make install
62+
```
63+
64+
## Testing
65+
66+
The project includes a basic set of integration tests.
67+
68+
### Integration Tests
69+
70+
Integration tests are based on [RLTest](https://github.com/RedisLabsModules/RLTest), and specific setup parameters can be provided
71+
to configure tests and topologies (OSS standalone and OSS cluster). By default the tests will be ran for all common commands, and with OSS standalone setup.
72+
73+
To run all integration tests in a Python virtualenv, follow these steps:
74+
75+
$ mkdir -p .env
76+
$ virtualenv .env
77+
$ source .env/bin/activate
78+
$ pip install -r tests/test_requirements.txt
79+
$ ./tests/run_tests.sh
80+
81+
To understand what test options are available simply run:
82+
83+
$ ./tests/run_tests.sh --help
84+
85+
### Memory Leak Detection with Sanitizers
86+
87+
memtier_benchmark supports building with AddressSanitizer (ASAN) and LeakSanitizer (LSAN) to detect memory errors and leaks during testing.
88+
89+
To build with sanitizers enabled:
90+
91+
$ ./configure --enable-sanitizers
92+
$ make
93+
94+
To run tests with leak detection:
95+
96+
$ ASAN_OPTIONS=detect_leaks=1 ./tests/run_tests.sh
97+
98+
If memory leaks or errors are detected, tests will fail with detailed error messages showing the location of the issue.
99+
100+
To verify ASAN is enabled:
101+
102+
$ ldd ./memtier_benchmark | grep asan
103+
104+
### Undefined Behavior Detection with UBSan
105+
106+
memtier_benchmark supports building with UndefinedBehaviorSanitizer (UBSan) to detect undefined behavior such as integer overflows, null pointer dereferences, and alignment issues.
107+
108+
To build with UBSan enabled:
109+
110+
$ ./configure --enable-ubsan
111+
$ make
112+
113+
To run tests with undefined behavior detection:
114+
115+
$ UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 ./tests/run_tests.sh
116+
117+
UBSan can be combined with ASAN for comprehensive testing:
118+
119+
$ ./configure --enable-sanitizers --enable-ubsan
120+
$ make
121+
122+
**Note:** UBSan can be used together with ASAN/LSAN, but not with ThreadSanitizer (TSAN).
123+
124+
### Data Race Detection with Thread Sanitizer
125+
126+
memtier_benchmark supports building with ThreadSanitizer (TSAN) to detect data races and threading issues.
127+
128+
To build with Thread Sanitizer enabled:
129+
130+
$ ./configure --enable-thread-sanitizer
131+
$ make
132+
133+
To run tests with race detection (requires disabling ASLR on kernel 6.6+):
134+
135+
$ TSAN_OPTIONS="suppressions=$(pwd)/tsan_suppressions.txt" setarch `uname -m` -R ./tests/run_tests.sh
136+
137+
To verify TSAN is enabled:
138+
139+
$ ldd ./memtier_benchmark | grep tsan
140+
141+
**Note:** TSAN and ASAN are mutually exclusive and cannot be used together. A suppression file (`tsan_suppressions.txt`) is provided to ignore known benign data races that do not affect correctness.
142+

‎README.md‎

Lines changed: 1 addition & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -55,129 +55,7 @@ brew install memtier_benchmark
5555

5656
### Installing from source
5757

58-
#### Prerequisites
59-
60-
The following libraries are required for building:
61-
62-
* libevent 2.0.10 or newer.
63-
* OpenSSL (unless TLS support is disabled by `./configure --disable-tls`).
64-
65-
The following tools are required
66-
* autoconf
67-
* automake
68-
* pkg-config
69-
* GNU make
70-
* GCC C++ compiler
71-
72-
#### CentOS/Red Hat Linux 7 or newer
73-
74-
Use the following to install prerequisites:
75-
```
76-
$ sudo yum install autoconf automake make gcc-c++ \
77-
zlib-devel libmemcached-devel libevent-devel openssl-devel
78-
```
79-
80-
#### Ubuntu/Debian
81-
82-
Use the following to install prerequisites:
83-
84-
```
85-
$ sudo apt-get install build-essential autoconf automake \
86-
libevent-dev pkg-config zlib1g-dev libssl-dev
87-
```
88-
89-
#### macOS
90-
91-
To build natively on macOS, use Homebrew to install the required dependencies:
92-
93-
```
94-
$ brew install autoconf automake libtool libevent pkg-config openssl@3.0
95-
```
96-
97-
When running `./configure`, if it fails to find libssl it may be necessary to
98-
tweak the `PKG_CONFIG_PATH` environment variable:
99-
100-
```
101-
PKG_CONFIG_PATH=`brew --prefix openssl@3.0`/lib/pkgconfig ./configure
102-
```
103-
104-
#### Building and installing
105-
106-
After downloading the source tree, use standard autoconf/automake commands:
107-
108-
```
109-
$ autoreconf -ivf
110-
$ ./configure
111-
$ make
112-
$ sudo make install
113-
```
114-
115-
#### Testing
116-
117-
The project includes a basic set of integration tests.
118-
119-
120-
**Integration tests**
121-
122-
123-
Integration tests are based on [RLTest](https://github.com/RedisLabsModules/RLTest), and specific setup parameters can be provided
124-
to configure tests and topologies (OSS standalone and OSS cluster). By default the tests will be ran for all common commands, and with OSS standalone setup.
125-
126-
127-
To run all integration tests in a Python virtualenv, follow these steps:
128-
129-
$ mkdir -p .env
130-
$ virtualenv .env
131-
$ source .env/bin/activate
132-
$ pip install -r tests/test_requirements.txt
133-
$ ./tests/run_tests.sh
134-
135-
To understand what test options are available simply run:
136-
137-
$ ./tests/run_tests.sh --help
138-
139-
140-
**Memory leak detection with sanitizers**
141-
142-
143-
memtier_benchmark supports building with AddressSanitizer (ASAN) and LeakSanitizer (LSAN) to detect memory errors and leaks during testing.
144-
145-
To build with sanitizers enabled:
146-
147-
$ ./configure --enable-sanitizers
148-
$ make
149-
150-
To run tests with leak detection:
151-
152-
$ ASAN_OPTIONS=detect_leaks=1 ./tests/run_tests.sh
153-
154-
If memory leaks or errors are detected, tests will fail with detailed error messages showing the location of the issue.
155-
156-
To verify ASAN is enabled:
157-
158-
$ ldd ./memtier_benchmark | grep asan
159-
160-
161-
**Data race detection with Thread Sanitizer**
162-
163-
164-
memtier_benchmark supports building with ThreadSanitizer (TSAN) to detect data races and threading issues.
165-
166-
To build with Thread Sanitizer enabled:
167-
168-
$ ./configure --enable-thread-sanitizer
169-
$ make
170-
171-
To run tests with race detection (requires disabling ASLR on kernel 6.6+):
172-
173-
$ TSAN_OPTIONS="suppressions=$(pwd)/tsan_suppressions.txt" setarch `uname -m` -R ./tests/run_tests.sh
174-
175-
To verify TSAN is enabled:
176-
177-
$ ldd ./memtier_benchmark | grep tsan
178-
179-
**Note:** TSAN and ASAN are mutually exclusive and cannot be used together. A suppression file (`tsan_suppressions.txt`) is provided to ignore known benign data races that do not affect correctness.
180-
58+
For detailed instructions on building from source, running tests, and using sanitizers for development, see [DEVELOPMENT.md](DEVELOPMENT.md).
18159

18260
## Using Docker
18361

‎configure.ac‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,16 @@ AS_IF([test "x$enable_sanitizers" = "xyes"], [
7979
LDFLAGS="$LDFLAGS -fsanitize=address -fsanitize=leak"
8080
], [])
8181

82+
# UndefinedBehaviorSanitizer (UBSan) is optional and can be combined with ASAN.
83+
AC_ARG_ENABLE([ubsan],
84+
[AS_HELP_STRING([--enable-ubsan],
85+
[Enable UndefinedBehaviorSanitizer for undefined behavior detection])])
86+
AS_IF([test "x$enable_ubsan" = "xyes"], [
87+
AC_MSG_NOTICE([Enabling UndefinedBehaviorSanitizer])
88+
CXXFLAGS="$CXXFLAGS -fsanitize=undefined -fno-omit-frame-pointer"
89+
LDFLAGS="$LDFLAGS -fsanitize=undefined"
90+
], [])
91+
8292
# Thread Sanitizer (TSAN) is optional and mutually exclusive with ASAN.
8393
AC_ARG_ENABLE([thread-sanitizer],
8494
[AS_HELP_STRING([--enable-thread-sanitizer],

0 commit comments

Comments
 (0)