Skip to content

perf(desktop-gpui): IDCT-scale JPEG library thumbnails - #2721

Open
kyzer1023 wants to merge 1 commit into
CapSoftware:mainfrom
kyzer1023:cursor/jpeg-thumbnail-idct-f70c
Open

kyzer1023 wants to merge 1 commit into
CapSoftware:mainfrom
kyzer1023:cursor/jpeg-thumbnail-idct-f70c

Conversation

@kyzer1023

@kyzer1023 kyzer1023 commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Library card thumbnails (thumbnail_rgba in apps/desktop-gpui/src/library.rs) decoded JPEGs at full resolution and then resampled to the 392×224 cover size. JPEGs now use jpeg_decoder::Decoder::scale to IDCT-scale toward the cover size first, then resize to the exact size. A 1920×1080 or 3840×2160 frame decodes at 480×270.

The sidebar's existing IDCT decode moved into a shared library/jpeg_thumbnail.rs, so decode_jpeg_thumbnail is the only IDCT decoder. The sidebar still asks for a square max; library cards pass the cover size. PNG is unchanged, and CMYK / 16-bit gray JPEGs still fall back to a full decode.

The pixels differ slightly from the old full-resolution resample, so the cache prefix now ends in -idct. Old cache files are left on disk.

One commit, 4 files, +153/-30. Based on main at 85219015.

Correctness

jpeg_thumbnail tests pass (square downsample, historical cover sizes, 1080p to 480×270, CMYK returns None), run through a standalone crate that path-includes the module. The full desktop-gpui app and the cache-prefix tests in library.rs were not built here because gpui is patched to a local zed-cap checkout.

Benchmarks

Release, Linux VM, median of 7 samples after one warmup. Before is the old image full decode plus cover resample.

Source Before After Speedup Mean channel diff
Synthetic 1920×1080 7.12 ms 5.00 ms 1.42× 4.73
Synthetic 3840×2160 23.08 ms 12.73 ms 1.81× 9.96
sequoia-dark.jpg 4000×2500 36.07 ms 20.05 ms 1.80× 1.40
tahoe-dark.jpg 3840×2160 68.03 ms 54.30 ms 1.25× 1.58

A second run put synthetic 1080p at 1.67× and 4K at 1.87×.

Overlap

Does not touch clip thumbnails (#2707) or recording recovery in library.rs (#2630).

RetriggerView in GreptileConfidence Score: 4/5

No blocking behavior defect was found, but the explicit comments policy must be satisfied before merging.

Findings

  1. P2 CMYK test misses its branch ▶
  2. P2 Unsupported JPEGs decode twice ▶
  3. P2 Comment breaks the guide ▶
Fix with agent prompt
### Issue 1
apps/desktop-gpui/src/jpeg_thumbnail.rs:101-105
`cmyk_frame_returns_none` uses a truncated JPEG header with no scan data. The test passes when decoding fails, even if CMYK rejection is removed. Use a complete CMYK JPEG so the test guards the unsupported-format branch.

### Issue 2
apps/desktop-gpui/src/jpeg_thumbnail.rs:17-18
The shared helper decodes pixels before rejecting unsupported formats. Library thumbnails now pay for that discarded decode and then decode again through the fallback. Check `info.pixel_format` before calling `decoder.decode()` so CMYK and 16-bit gray files go straight to the fallback.

```suggestion
    if !matches!(
        info.pixel_format,
        jpeg_decoder::PixelFormat::RGB24 | jpeg_decoder::PixelFormat::L8
    ) {
        return None;
    }
    let pixels = decoder.decode().ok()?;
    let decoded = match info.pixel_format {
```

### Issue 3
apps/desktop-gpui/src/jpeg_thumbnail.rs:34
The new `cover_target` comment only describes what the function does. The repository guide bans comments that narrate code and requires non-obvious context instead. Remove this comment or express the guarantee through a test. This repository requirement must be satisfied before merging.

```suggestion

```

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Library JPEG thumbnails now decode at a smaller size before the final resize.

  • Library cards decode JPEG thumbnails at a smaller size first.

Reviews (1) · Last reviewed commit: "perf(desktop-gpui): IDCT-scale JPEG libr..." · Reviewed by Greptile

Co-authored-by: Kyzer  <kyzer1023@users.noreply.github.com>
@vercel

vercel Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@kyzer1023 is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

Comment on lines +101 to +105
let bytes = [
0xFF, 0xD8, 0xFF, 0xC0, 0x00, 0x14, 0x08, 0x00, 0x08, 0x00, 0x08, 0x04, 0x01, 0x11,
0x00, 0x02, 0x11, 0x00, 0x03, 0x11, 0x00, 0x04, 0x11, 0x00,
];
assert!(decode_jpeg_thumbnail(&bytes, |w, h| (w, h)).is_none());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 CMYK test misses its branch

cmyk_frame_returns_none uses a truncated JPEG header with no scan data. The test passes when decoding fails, even if CMYK rejection is removed. Use a complete CMYK JPEG so the test guards the unsupported-format branch.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/src/jpeg_thumbnail.rs
Line: 101-105

Comment:
**CMYK test misses its branch**

`cmyk_frame_returns_none` uses a truncated JPEG header with no scan data. The test passes when decoding fails, even if CMYK rejection is removed. Use a complete CMYK JPEG so the test guards the unsupported-format branch.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +17 to +18
let pixels = decoder.decode().ok()?;
let decoded = match info.pixel_format {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unsupported JPEGs decode twice

The shared helper decodes pixels before rejecting unsupported formats. Library thumbnails now pay for that discarded decode and then decode again through the fallback. Check info.pixel_format before calling decoder.decode() so CMYK and 16-bit gray files go straight to the fallback.

Suggested change
let pixels = decoder.decode().ok()?;
let decoded = match info.pixel_format {
if !matches!(
info.pixel_format,
jpeg_decoder::PixelFormat::RGB24 | jpeg_decoder::PixelFormat::L8
) {
return None;
}
let pixels = decoder.decode().ok()?;
let decoded = match info.pixel_format {
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/src/jpeg_thumbnail.rs
Line: 17-18

Comment:
**Unsupported JPEGs decode twice**

The shared helper decodes pixels before rejecting unsupported formats. Library thumbnails now pay for that discarded decode and then decode again through the fallback. Check `info.pixel_format` before calling `decoder.decode()` so CMYK and 16-bit gray files go straight to the fallback.

```suggestion
    if !matches!(
        info.pixel_format,
        jpeg_decoder::PixelFormat::RGB24 | jpeg_decoder::PixelFormat::L8
    ) {
        return None;
    }
    let pixels = decoder.decode().ok()?;
    let decoded = match info.pixel_format {
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Some((decoded, u32::from(info.width), u32::from(info.height)))
}

/// Cover size for a card. Never upscales.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Comment breaks the guide

The new cover_target comment only describes what the function does. The repository guide bans comments that narrate code and requires non-obvious context instead. Remove this comment or express the guarantee through a test. This repository requirement must be satisfied before merging.

Suggested change
/// Cover size for a card. Never upscales.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/src/jpeg_thumbnail.rs
Line: 34

Comment:
**Comment breaks the guide**

The new `cover_target` comment only describes what the function does. The repository guide bans comments that narrate code and requires non-obvious context instead. Remove this comment or express the guarantee through a test. This repository requirement must be satisfied before merging.

```suggestion

```

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant