Skip to content

New Test Ideas #292

Open
Open

Description

@AlexanderKaran
  • Pre-rendered or static content runtimes tests
  • SPA Mode. Frameworks that support a full SPA mode not just a route
  • caching repeated navigations is also a big performance thing, it might be nice for the benchmark to include a repeated navigation somewhere to demonstrate and measure the reuse of cached data (from TanStack Team)
  • Add a navigation-after-hover benchmark: hover for 100 ms, click, and measure time from click to destination content painted across all frameworks. (from TanStack Team)

Activity

  1. delucis commented on Jul 14, 2026

    @delucis
    Contributor

    It would be interesting to test hydrated components with server rendered markup in addition to the current tests for client rendering. Client rendering in React for example should be largely equivalent across different frameworks that support React, so the tests are likely mostly measuring React rather than the framework. The bigger differences may be where the frameworks provide unique takes on the server–client boundary to optimize performance (islands vs React Server Components etc.)

  2. dreyfus92 commented on Sep 28, 2026

    @dreyfus92
    Member

    nice list, two things i would add are:

    • a cpu throttled variant, everything runs unthrottled desktop right now which hides js cost. something like 4x would show where hydration actually hurts
    • hmr latency next to dev server startup, the time from a file edit until the browser shows it. startup is measured but the edit loop is what devs feel all day
  3. Sheraff commented on Oct 5, 2026

    @Sheraff

    My biggest request for that list would be this: measure an actual navigation. I think this is probably one of the most important metrics for a "meta-framework benchmark". Basically: the time from a click on a link until the destination of that link is rendered.

    Currently the only measure close to this is the INP (called "Interaction Breakdown" in https://frameworks.e18e.dev/all-frameworks) but this stops before the actual navigation, it's basically just the click handling itself, and since all frameworks included in this benchmark have async operations between the click and the render of the next page, it doesn't actually measure the navigation.

  4. delucis commented on Oct 5, 2026

    @delucis
    Contributor

    This might be relevant here as presumably @Sheraff means measuring SPA navigation: https://developer.chrome.com/docs/web-platform/soft-navigations

  5. Sheraff commented on Oct 22, 2026

    @Sheraff

    I just realized that this benchmark does not measure the amount of bytes loaded by the client. There is "body size" which is the HTML document in bytes, but another good one would be "total bytes on initial load" (i.e. HTML + JS bundles since i don't think these apps ship any CSS)

  6. AlexanderKaran commented on Nov 11, 2026

    @AlexanderKaran
    CollaboratorAuthor

    Most of the meta-frameworks have a built in Image component or an official lib for it. Taking a look at these across a few tests would be good

  7. AlexanderKaran commented on Nov 14, 2026

    @AlexanderKaran
    CollaboratorAuthor

    #847

    With regards to framework dependency count: https://frameworks.e18e.dev/dev-time/#dependency-counts

    Sometimes a package will include dependencies as part of their distribution, without including it in package.json#dependencies. For example, such packages may not appear via npm ls, and using npm audit won't reveal vulnerabilities of vendored packages such as this vercel/next.js#83459.

    From what I gather, vendored packages for Next.js can be found at: packages/next/src/compiled.

    It would be great to have a deterministic way to reveal the number of vendored dependencies each framework has as part of the benchmark, and to represent them in some way as part of the total dependency count

    From @theetrain

  8. motsell commented on Nov 20, 2026

    @motsell

    One DevTime test idea: record build memory high-water and constrained-memory pass/fail alongside build time. On Linux, a dedicated cgroup's memory.peak and OOM events would be more useful than a single process's max RSS when a build creates multiple workers. Fix CPU/RAM and cache conditions across frameworks, distinguish an empty build directory from a repeat build, and alternate repeat blocks to expose drift.

    We ran a small synthetic Next.js 16.4.6 application in one local ARM64 VM, stopped/resized in A/B/A order (2 vCPU/4 GiB, 4 vCPU/8 GiB, 2 vCPU/4 GiB). Three clean-directory runs per block gave medians 10.422 → 3.223 → 3.928 seconds; repeat builds with .next retained gave 4.050 → 1.313 → 1.412 seconds. All 18 builds completed with no guest OOM kills. Our memory observation was only sampled guest-wide MemAvailable, not process-tree or cgroup peak memory. We could not isolate CPU vs RAM, cache or sequential host effects, and the second A block substantially changed the interpretation. This is one fixture, not a framework ranking. A standardized memory metric would make the result more useful than build time alone.

    Disclosure: MOTSELL Engineering (VPS provider) ran this local test; AI assisted with automation and drafting. No commercial benchmark or sales link.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions