Skip to content

feat: Addresses performance issues in the Redis online store#6337

Open
ntkathole wants to merge 1 commit intofeast-dev:masterfrom
ntkathole:redis_perf_improve
Open

feat: Addresses performance issues in the Redis online store#6337
ntkathole wants to merge 1 commit intofeast-dev:masterfrom
ntkathole:redis_perf_improve

Conversation

@ntkathole
Copy link
Copy Markdown
Member

@ntkathole ntkathole commented Apr 26, 2026

Summary

Addresses several correctness and performance issues in the Redis online store
identified in #3596.

Changes

1. Batched get_online_features pipeline (read latency: O(N_fv) → O(1))

The base class get_online_features calls online_read once per feature view,
each opening and executing its own Redis pipeline. Since all feature views for
the same entity already share one Redis hash key, all HMGET commands across
every feature view can be batched into a single pipeline execution.

RedisOnlineStore now overrides get_online_features to pre-compute all
entity keys and hash field keys, issue one pipeline.execute() for everything,
then demultiplex results back per feature view. For a request spanning N feature
views this cuts Redis round trips from N to 1.

2. Fixed delete_table N+1 query pattern

The previous implementation called client.hgetall(key) synchronously inside
the scan_iter loop — one blocking round trip per entity key. Replaced with a
three-phase approach:

  • Phase 1: collect all matching keys from SCAN (no per-key round trips)
  • Phase 2: pipeline HKEYS for all collected keys (1 round trip)
  • Phase 3: pipeline all HDEL/DEL commands (1 round trip)

3. skip_dedup config option for write throughput

Added skip_dedup: bool = False to RedisOnlineStoreConfig. When enabled,
online_write_batch skips the first timestamp-read pipeline and writes all
rows in a single pass, halving round trips. Intended for initial bulk loads or
append-only pipelines where concurrent out-of-order writes are not a concern.

4. online_write_batch_async implemented

Previously raised NotImplementedError. Now fully implemented using the
existing _get_client_async infrastructure, with skip_dedup support.

Files Changed

  • sdk/python/feast/infra/online_stores/redis.py — only file modified; base
    class OnlineStore and all other store implementations are untouched.

Performance Impact

Scenario Before After
get_online_features with N feature views N Redis round trips 1 Redis round trip
online_write_batch with skip_dedup=True 2 round trips 1 round trip
delete_table with K entity keys K+1 round trips 3 round trips
Async batch writes NotImplementedError Supported

Related


Open in Devin Review

@ntkathole ntkathole self-assigned this Apr 26, 2026
@ntkathole ntkathole requested a review from a team as a code owner April 26, 2026 13:07
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional findings.

Open in Devin Review

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole ntkathole force-pushed the redis_perf_improve branch from bde2067 to 59b6694 Compare April 26, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redis Online Store adapter is not production-ready

1 participant