Skip to content

fix(redis): poll FT.INFO to avoid flaky test_redis_query race condition#4767

Open
rajdhruvsingh wants to merge 2 commits into
open-telemetry:mainfrom
rajdhruvsingh:fix/redisearch-async-indexing-race-condition
Open

fix(redis): poll FT.INFO to avoid flaky test_redis_query race condition#4767
rajdhruvsingh wants to merge 2 commits into
open-telemetry:mainfrom
rajdhruvsingh:fix/redisearch-async-indexing-race-condition

Conversation

@rajdhruvsingh

Copy link
Copy Markdown
Contributor

Description

Fixes #4589

FT.CREATE returns OK once the index is registered, but the backfill
scan of pre-existing keys happens asynchronously in the background.
In TestRedisearchInstrument.setUp, prepare_data() writes a JSON
document (test:001) and then create_index() immediately creates an
index over the test: prefix. If test_redis_query issues FT.SEARCH
before the backfill completes, RediSearch returns total=0 instead of
1, causing an intermittent assertion failure in CI.

Fix by polling FT.INFO after create_index() returns until
indexing=0 and percent_indexed=1, with a 5s timeout and 50ms
interval. On fast machines the poll exits on the first iteration with
zero overhead; on slower CI machines under load it waits as needed.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Ran TestRedisearchInstrument::test_redis_query in a 30-iteration
loop locally against the redis/redis-stack:7.2.0-v12 Docker image
(same image used in CI via docker-compose.yml). All 30 runs passed.

Also verified the poll logic by temporarily adding debug prints showing
indexing and percent_indexed values from FT.INFO — confirmed the
fields are read correctly and the loop exits as expected.

To reproduce locally:

  1. docker compose up -d otredis (from tests/opentelemetry-docker-tests/tests/)
  2. pip install -e instrumentation/opentelemetry-instrumentation-redis/
  3. for i in $(seq 1 30); do python -m pytest tests/opentelemetry-docker-tests/tests/redis/test_redis_functional.py::TestRedisearchInstrument::test_redis_query -x -q; done

Does This PR Require a Core Repo Change?

  • No.

Checklist

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@rajdhruvsingh rajdhruvsingh requested a review from a team as a code owner July 1, 2026 08:26
@rajdhruvsingh rajdhruvsingh force-pushed the fix/redisearch-async-indexing-race-condition branch from 9a47a6a to 98b5cba Compare July 1, 2026 08:32
FT.CREATE returns OK once the index is registered, but the backfill
scan of pre-existing keys happens asynchronously. If FT.SEARCH is
issued before the backfill completes, it intermittently returns
total=0 instead of 1, causing test_redis_query to flake.

Fix by polling FT.INFO after create_index() until indexing=0 and
percent_indexed=1, with a 5s timeout.

Fixes open-telemetry#4589
@rajdhruvsingh rajdhruvsingh force-pushed the fix/redisearch-async-indexing-race-condition branch from 98b5cba to 4e1453c Compare July 1, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Flaky redis docker test

1 participant