Skip to content

Fuzzy job class search and queue filter - #19

Open
ryanmrodriguez wants to merge 1 commit into
cmer:mainfrom
ryanmrodriguez:20261104-fuzzy-job-search
Open

ryanmrodriguez wants to merge 1 commit into
cmer:mainfrom
ryanmrodriguez:20261104-fuzzy-job-search

Conversation

@ryanmrodriguez

Copy link
Copy Markdown

Problem

On the Jobs page, class search was a case-sensitive prefix match and the queue filter needed the exact queue name. Typing Docu found nothing, even with dozens of failed CoreModels::Screenshots::UserDocumentScreenshotJobs on screen.

Change

Both fields are now a case-insensitive substring match: charge finds Billing::ChargeSubscriptionJob, HOOK finds webhooks.

The obvious fix, LIKE '%frag%', can't use an index, so every list, all seven tab counts and bulk "all matching" would scan the jobs table. Instead, Flightdeck::KnownNames matches the fragment in Ruby against the distinct class and queue names (SELECT DISTINCT over the existing class_name / queue_name indexes), and JobsQuery filters with an indexed IN (...). A side benefit is that case folding is the same on every adapter (SQLite's LIKE folds ASCII only, MySQL's depends on collation).

Freshness and cost

  • The name lists are memoized per process for one poll interval, independent of Rails.cache, so a filtered page costs a couple of DISTINCTs per interval even with no cache store.
  • A search that matches nothing re-reads first (at most once a second), so a class that starts failing right after a deploy is findable immediately.
  • Retry/discard run inside KnownNames.fresh { }, so "all matching" never skips a job with a brand-new class or queue name.
  • Concurrent refreshes share one read (per-column mutex, connection checked out before the lock to avoid a pool deadlock), and the read bypasses the query cache.

The exact class_name param filter is unchanged.

Known limitation: on accent-insensitive MySQL collations, IN can match collation-equivalent names (e.g. email vs émail). The old exact queue filter behaved the same way.

Testing

  • New test/known_names_test.rb: query count with no cache store, TTL expiry, no-match re-read and its rate limit, slow reads, contended threads sharing one read, query cache bypass, nested/raising fresh.
  • Bulk retry/discard regression tests for brand-new class and queue names (pinned clock). Each new guard test was confirmed to fail without its fix.
  • bundle exec rake test and FLIGHTDECK_TEST_API_ONLY=1 bundle exec rake test: 298 runs, 0 failures each (SQLite locally; PostgreSQL/MySQL via CI). Rubocop clean.
  • Checked by hand in bin/demo.

Class search was a case-sensitive prefix match and the queue filter an exact
match, so "Docu" never found UserDocumentScreenshotJob. Both are now a
case-insensitive substring match.

Fragments are resolved in Ruby against the distinct names Solid Queue holds
(Flightdeck::KnownNames), leaving SQL an indexed IN list instead of a
leading-wildcard LIKE that would scan the jobs table for every list, tab
count and bulk action. Case folding is then the same on every adapter.

The names are memoized per process for one poll interval, re-read before
answering "no matches", and read fresh inside retry/discard so "all
matching" never skips a job with a brand-new class or queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant