Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- An Owners page mapping failed jobs to their CODEOWNERS teams, with search, turned on with `config.codeowners_path`. Its class names and counts open the Failed list, filtered by class and time window.

## [1.2.0] - 2026-09-04

### Added
Expand Down
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@ the engine answers 401 behind your constraint.
| `display_timezone` | `"UTC"` | Timezone for displayed timestamps. Queries stay in UTC. |
| `backtrace_lines` | `50` | Backtrace frames shown on job detail. |
| `ui_font` | `"public-sans"` | Default interface typeface. Each user can pick another one in the sidebar. |
| `codeowners_path` | `nil` | CODEOWNERS file, relative to `Rails.root` or absolute. Turns on the Owners page. |

## Interface font

Expand All @@ -157,6 +158,29 @@ rather than inlined, so a browser only downloads the one face it is rendering
in. The monospace face (IBM Plex Mono) is fixed: the job tables depend on its
tabular figures.

## Job owners

The Owners page lists failed jobs per class, with the owners of each class and
counts for the last 1h, 3h, 24h, 3d and 7d. It reads your CODEOWNERS file:

```ruby
Flightdeck.configure do |config|
config.codeowners_path = ".github/CODEOWNERS"
end
```

The file has to be present wherever the dashboard runs, so check that your
deploy doesn't leave `.github` out. Each class is matched by the file that
defines it, which means loading the class. A class that can't be loaded, or is
defined outside the repo (`ActionMailer::MailDeliveryJob`), shows as Unowned.

Filter by owner with the chips, and search by any part of the class name. Class
names and counts open the Failed list, filtered to that class and time window;
"Retry all matching" and "Discard all matching" there stay inside that window.

Counts cover jobs that are still failed: Solid Queue deletes a failed execution
when its job is retried or discarded, so this isn't a history of failures.

## Chart history and job retention

Flightdeck adds no tables and changes no retention: charts read Solid Queue's
Expand Down

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions app/assets/flightdeck/manifest.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"flightdeck.css": {
"file": "flightdeck-38ba36531cee.css",
"digest": "38ba36531cee",
"sha256": "38ba36531cee83452b312ce04f858f83eeb262b498b6614c775e1947ba2303da",
"file": "flightdeck-a9144fe69881.css",
"digest": "a9144fe69881",
"sha256": "a9144fe6988181da79c000d89c826bfbaa02ce2a932d672b69f858698a1e62d0",
"content_type": "text/css; charset=utf-8",
"size": 31759
"size": 31867
},
"flightdeck.js": {
"file": "flightdeck-30004819780c.js",
Expand Down
3 changes: 2 additions & 1 deletion app/controllers/flightdeck/application_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ def list_filters
@list_filters ||= {
class_name: params[:class_name].presence,
queue_name: params[:queue_name].presence,
q: params[:q].presence
q: params[:q].presence,
failed_within: params[:failed_within].presence_in(JobsQuery::FAILED_WINDOWS.keys)
}.compact
end

Expand Down
15 changes: 15 additions & 0 deletions app/controllers/flightdeck/owners_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# frozen_string_literal: true

module Flightdeck
class OwnersController < ApplicationController
def index
@codeowners = Codeowners.current
@failures = OwnerFailures.new(@codeowners, **owner_filters) if @codeowners
end

private
def owner_filters
{ owner: params[:owner].to_s.presence, q: params[:q].to_s.strip.presence }
end
end
end
11 changes: 11 additions & 0 deletions app/helpers/flightdeck/owners_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# frozen_string_literal: true

module Flightdeck
module OwnersHelper
# A count that opens the failed jobs behind it. Zero has nothing to open.
def fd_failed_count_link(count, **filters)
label = number_with_delimiter(count)
count.zero? ? label : link_to(label, jobs_path(state: :failed, **filters))
end
end
end
103 changes: 103 additions & 0 deletions app/models/flightdeck/codeowners.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# frozen_string_literal: true

module Flightdeck
# Answers "who owns this job class?" from the host's CODEOWNERS file.
#
# Rules follow GitHub's: the last matching line wins, and a matching line with
# no owners leaves the path unowned. Patterns are matched against the file
# that defines the class, so components and engines resolve like any other
# path. A class that can't be loaded or lives outside the repo has no owners.
class Codeowners
Rule = Struct.new(:regexp, :owners)

# Directories CODEOWNERS may sit in; its patterns are relative to their parent.
NESTED_IN = %w[.github .gitlab docs].freeze

class << self
# Read per request, not at boot, and re-parsed only when the file changes.
def current
path = Flightdeck.config.codeowners_path.presence
return unless path

path = File.realpath(path.to_s, Rails.root)
mtime = File.mtime(path)
return @current if @current && @current.path == path && @current.mtime == mtime

@current = new(File.read(path, encoding: "UTF-8").scrub, path: path, mtime: mtime)
rescue SystemCallError
nil
end
end

attr_reader :path, :mtime

def initialize(source, path: nil, mtime: nil)
@path = path
@mtime = mtime
@rules = source.each_line.filter_map { |line| parse(line) }
@class_owners = Concurrent::Map.new
end

def owners_for_class(class_name)
@class_owners.compute_if_absent(class_name) do
file = relative_source_file(class_name)
file ? owners_for_path(file) : []
end
end

def owners_for_path(relative_path)
@rules.reverse_each.find { |rule| rule.regexp.match?(relative_path) }&.owners || []
end

private
def root
directory = File.dirname(path)
NESTED_IN.include?(File.basename(directory)) ? File.dirname(directory) : directory
end

# One broken job class must not take the page down with it, hence
# ScriptError: a file that fails to load raises SyntaxError or LoadError.
def relative_source_file(class_name)
return unless class_name.safe_constantize

file = File.realpath(Object.const_source_location(class_name).first)
file.delete_prefix("#{root}/") if file.start_with?("#{root}/")
rescue StandardError, ScriptError
nil
end

def parse(line)
pattern, *rest = line.scan(/(?:\\.|\S)+/)
return if pattern.nil? || pattern.start_with?("#", "\\#")

owners = rest.take_while { |token| !token.start_with?("#") }.map { |owner| display_name(owner) }
Rule.new(regexp_for(pattern), owners)
end

# @acme/billing and @alice read as billing and alice; emails stay whole.
def display_name(owner) = owner.delete_prefix("@").split("/").last

def regexp_for(pattern)
anchored = pattern.chomp("/").include?("/")
# A trailing `/**` says what the descendants clause below already does.
glob = pattern.delete_prefix("/").chomp("/").chomp("/**")
# `dir/*` is direct children only; anything else also covers what's below.
descendants = glob.end_with?("/*") ? "" : "(?:/.*)?"

Regexp.new("\\A#{"(?:.*/)?" unless anchored}#{translate(glob)}#{descendants}\\z")
end

# `**` crosses directories only as a whole segment; elsewhere it is a `*`.
def translate(glob)
glob.gsub(%r{\\.?|(?<![^/])\*\*/|\*+|\?|[^*?\\]+}) do |token|
case token
when "**/" then "(?:.*/)?"
when /\A\*/ then "[^/]*"
when "?" then "[^/]"
when /\A\\/ then Regexp.escape(token[1] || "\\")
else Regexp.escape(token)
end
end
end
end
end
33 changes: 28 additions & 5 deletions app/models/flightdeck/jobs_query.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,26 @@ class JobsQuery
ARGUMENTS_PREVIEW_BYTES = 800
ERROR_PREVIEW_BYTES = 1_000

# Lengths of the "failed within" filter, also the Owners page columns.
FAILED_WINDOWS = {
"1h" => 1.hour,
"3h" => 3.hours,
"24h" => 24.hours,
"3d" => 3.days,
"7d" => 7.days
}.freeze

class InvalidState < ArgumentError; end

attr_reader :state, :class_name, :queue_name, :q, :before_id, :limit, :count_cap
attr_reader :state, :class_name, :queue_name, :q, :failed_within, :before_id, :limit, :count_cap

def initialize(state: :all, class_name: nil, queue_name: nil, q: nil, before_id: nil,
limit: nil, count_cap: nil)
def initialize(state: :all, class_name: nil, queue_name: nil, q: nil, failed_within: nil,
before_id: nil, limit: nil, count_cap: nil)
@state = normalize_state(state)
@class_name = class_name.presence
@queue_name = queue_name.presence
@q = q.presence
@failed_within = failed_within.presence_in(FAILED_WINDOWS.keys)
@before_id = before_id.presence && Integer(before_id, exception: false)
@limit = (limit || Flightdeck.config.per_page).to_i.clamp(1, 200)
@count_cap = (count_cap || Flightdeck.config.count_cap).to_i
Expand Down Expand Up @@ -66,7 +76,7 @@ def count
def count_capped? = count >= count_cap

def filters?
class_name.present? || queue_name.present? || q.present?
class_name.present? || queue_name.present? || q.present? || failed_within.present?
end

# The filtered relation with no ordering, limit or cursor — what bulk
Expand Down Expand Up @@ -97,7 +107,7 @@ def self.state_counts(**filters)
# Only the filters change what a count means — the cursor and page size do
# not, so they are deliberately absent from the key.
def cache_filters
{ class_name: class_name, queue_name: queue_name, q: q }
{ class_name: class_name, queue_name: queue_name, q: q, failed_within: failed_within }
end

def normalize_state(value)
Expand Down Expand Up @@ -206,10 +216,23 @@ def apply_filters(relation)
relation = relation.where(jobs_table[:class_name].eq(class_name)) if class_name
relation = relation.where(queue_table[:queue_name].eq(queue_name)) if queue_name
relation = relation.where(jobs_table[:class_name].matches("#{escape_like(q)}%", nil, true)) if q
relation = apply_failed_within(relation) if failed_within

relation
end

# Only failed jobs can have failed recently, so every other state is empty
# and the tabs add up.
def apply_failed_within(relation)
recent = SolidQueue::FailedExecution.where(created_at: (Time.current - FAILED_WINDOWS.fetch(failed_within))..)

case state
when :failed then relation.merge(recent)
when :all then relation.where(id: recent.select(:job_id))
else relation.none
end
end

def join_jobs?
return false unless execution_backed?

Expand Down
83 changes: 83 additions & 0 deletions app/models/flightdeck/owner_failures.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# frozen_string_literal: true

module Flightdeck
# Failed jobs per class, with the owners of each class.
#
# Solid Queue deletes a failed execution when its job is retried or discarded,
# so a window counts jobs that failed in it and are still failed. It is not a
# history of failures.
class OwnerFailures
WINDOWS = JobsQuery::FAILED_WINDOWS
UNOWNED = "Unowned"

Row = Struct.new(:class_name, :owners, :total, :windows, keyword_init: true)

attr_reader :owner, :q

def initialize(codeowners, owner: nil, q: nil)
@codeowners = codeowners
@owner = owner
@q = q
end

def filtered? = owner.present? || q.present?

def rows
searched_rows.select { |row| owner.nil? || labels(row).include?(owner) }
end

def total = rows.sum(&:total)

def window_totals
WINDOWS.keys.each_index.map { |index| rows.sum { |row| row.windows[index] } }
end

# Largest first, Unowned last. A class with two owners counts for both. The
# selected owner stays listed at zero, or it could not be seen or changed.
def owner_totals
totals = Hash.new(0)
searched_rows.each { |row| labels(row).each { |label| totals[label] += row.total } }
totals[owner] += 0 if owner
unowned = totals.delete(UNOWNED)
sorted = totals.sort_by { |label, count| [ -count, label ] }.to_h
unowned ? sorted.merge(UNOWNED => unowned) : sorted
end

private
def labels(row) = row.owners.presence || [ UNOWNED ]

# Case-insensitive substring. Every class is already in memory here, so
# this costs no extra query.
def searched_rows
return all_rows unless q

needle = q.downcase
all_rows.select { |row| row.class_name.downcase.include?(needle) }
end

def all_rows
@all_rows ||= counts
.map do |class_name, total, *windows|
Row.new(class_name: class_name, owners: @codeowners.owners_for_class(class_name),
total: total.to_i, windows: windows.map(&:to_i))
end
.sort_by { |row| [ -row.total, row.class_name ] }
end

# Owners stay out of the cached value, so a CODEOWNERS change shows at once.
def counts
Cache.fetch("owner_failures", expires_in: Flightdeck.config.chart_cache_ttl) { count_by_class }
end

def count_by_class
failed = SolidQueue::FailedExecution.arel_table
class_name = SolidQueue::Job.arel_table[:class_name]
now = Time.current
in_window = WINDOWS.values.map do |length|
Arel::Nodes::Case.new.when(failed[:created_at].gteq(now - length)).then(1).else(0).sum
end

SolidQueue::FailedExecution.joins(:job).group(class_name).pluck(class_name, failed[:id].count, *in_window)
end
end
end
5 changes: 5 additions & 0 deletions app/views/flightdeck/jobs/_filter_bar.html.erb
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
<%= form_with url: jobs_path, method: :get, class: "fd-filterbar", data: { turbo_action: "advance" } do |form| %>
<%= form.hidden_field :state, value: params[:state], id: "fd-filter-state" %>
<%= form.hidden_field :class_name, value: query.class_name, id: nil if query.class_name %>
<%= form.hidden_field :failed_within, value: query.failed_within, id: nil if query.failed_within %>
<%= form.search_field :q, value: params[:q],
placeholder: "Search job class…",
"aria-label": "Search jobs by class name" %>
<%= form.text_field :queue_name, value: params[:queue_name],
placeholder: "Queue", "aria-label": "Filter by queue" %>
<%= form.submit "Filter", class: "fd-btn" %>

<% if query.class_name %><span class="fd-chip off"><%= query.class_name %></span><% end %>
<% if query.failed_within %><span class="fd-chip off">failed in last <%= query.failed_within %></span><% end %>

<% if query.filters? %>
<%= link_to jobs_path(state: params[:state].presence), class: "fd-chip" do %>
clear filters <u>&#10005;</u>
Expand Down
Loading