feat(shader): screen_texture / screen_uv / noise_uv builtins + {vertex, fragment} shader assets (+ Water Overworld example) by obiot · Pull Request #1548 · melonjs/melonJS

obiot · 2026-07-11T08:41:45Z

Summary

Screen/noise shader ergonomics inside ShaderEffect fragment bodies — design contributed by the melonJS editor team (ported from their customized v19 build), implemented on top of the #1543/#1544 toFrameTexture infrastructure. Users write three magic names and never touch JS plumbing or UV math:

uniform sampler2D screenTex : screen_texture;   // ← engine keeps it filled with the screen so far
vec4 apply(vec4 color, vec2 uv) {
    vec2 flow = texture2D(uNoise, noise_uv + uTime * 0.25).rg;          // ← frame-local 0..1
    vec4 refracted = texture2D(screenTex, screen_uv + flow * 0.005);    // ← screen-space 0..1
    return refracted * texture2D(uSampler, uv + flow * 0.005);
}

How it works

: screen_texture annotation: parsed + stripped at construction; the sampler is auto-wired to the renderer's shared frame capture through the existing setTexture live-entry path. The engine refreshes the capture right before the effect draws — drawImage's customShader block (scene behind the sprite) and endPostEffect (camera FBO for camera chains, parent target for renderable chains) — true back-buffer-copy semantics on every path, one screen copy per draw, only for effects that use it. Optional (repeat) wrap under WebGL 2.
screen_uv / noise_uv varyings: computed in a deterministically-templated vertex variant (user source is never regex-rewritten). noise_uv undoes atlas packing (fed per draw via an internal _setNoiseUVRect hook in the quad batchers), so seamless noise tiles correctly regardless of where the sprite's frame sits in its atlas.
New renderer helper getSharedFrameTexture() gives the shared capture slot stable identity from effect-construction time.

Backward compatibility

No JS API changes. The annotation was never valid GLSL (no existing shader can contain it); the varyings activate only when referenced and never when the body declares them itself; a body using none of the builtins compiles byte-identical to before. Manual setTexture(name, renderer.toFrameTexture()) keeps working.

Divergence from the contributed build (intentional)

Their sprite-path implementation aliased the screen sampler to uSampler's unit (the sprite's own texture, not the screen — their true-capture helper was orphaned after rebasing onto toFrameTexture). This port wires every path to the real screen capture — matching the intended “everything drawn so far” semantics, which is the point of the feature.

Also in this PR — `{vertex, fragment}` shader-asset pairs

A "shader" asset can now be a complete program pair — src: {vertex, fragment} URLs (fetched in parallel) or inline via data: — compiled at load time into a shared raw GLShader, returned by the same loader.getShader(). The honest type for full programs (Mesh custom shaders, renderer.customShader, custom batchers): no apply() wrapping, no builtins injection — the user owns both stages. Same shared/clone()/unload lifecycle; WebGL-only (null under Canvas, unload-safe). 6 adversarial tests, mutation-verified (dropped shared flag / swapped stages / removed already-loaded guard each kill exactly their matching test).

This covers the useful half of the contributed build's "full shader" surface; the remaining half (void main bodies inside ShaderEffect, with vertex-source rewriting) stays deferred until someone needs it.

Example — Water Overworld

Their demo scene ported 1:1 (GandalfHardcore free 32×32 pack, itch.io): TMX overworld, animated props, drifting clouds, playable character (A/D/W/Shift), and the water shader verbatim — a pond that mirrors and refracts the entire scene behind it. Verified side-by-side against their build served statically.

Verification

8 new tests (shadereffect-builtins.spec.js): parse/wiring introspection, true screen sampling + orientation (two-tone backdrop + swizzle readback), frame-local noise_uv on an atlas sub-frame (both axes), no-capture-without-annotation (spy), clone, self-declared-name back-compat
Full suite 4810 passed | 15 skipped | 0 failed; lint + biome + build green
CodeRabbit review: 1 finding (unhandled video.init false return in the example) — fixed, re-review clean (0 findings)
Headless visual: example screenshotted against the reference build — scene, water line, reflection and cloud drift match

🤖 Generated with Claude Code

https://claude.ai/code/session_019t8kP8vp58ZrvaD2FqJU6A

@ignore

…se_uv Inside a ShaderEffect fragment body (design contributed by the melonJS editor team, mirroring Godot's hint_screen_texture / SCREEN_UV): - `uniform sampler2D name : screen_texture;` — annotation parsed and stripped; the sampler is auto-wired to the renderer's shared toFrameTexture capture (live GPU-resident entry) and the engine refreshes the capture right before the effect draws: in drawImage's customShader block (scene behind the sprite) and in endPostEffect (camera FBO = the scene for camera chains, parent target for renderable chains). True BackBufferCopy semantics on every path. Optional `(repeat)` wrap honored under WebGL 2 (captures are NPOT). - `screen_uv` — free varying, clip→0..1 screen position, computed in a deterministically-templated vertex variant (user source never regex- rewritten). - `noise_uv` — free varying, frame-local 0..1 across the drawn object (undoes atlas packing, scaled to object pixels); fed per draw through the new @ignore ShaderEffect._setNoiseUVRect from the quad batchers' single-texture path and both blitTextures. Back-compat: annotation was never valid GLSL; varyings activate only when referenced and never when the body declares them itself; a body using no builtins compiles byte-identical to before. New renderer helper getSharedFrameTexture() gives the shared capture slot a stable identity from effect-construction time. 8 new tests (shadereffect-builtins.spec.js): parse/wiring/introspection, true screen sampling + orientation via two-tone backdrop + swizzle readback, frame-local noise_uv on an atlas sub-frame (both axes), no-capture-without-annotation (spy), clone, self-declared back-compat. Full suite 4810 passed | 15 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019t8kP8vp58ZrvaD2FqJU6A

…d 1:1 Side-scroller overworld (GandalfHardcore free 32x32 pack) with the water-refraction shader from the contributed build, verbatim: two cellular NoiseTexture2d flow maps + screenTex : screen_texture + screen_uv mirror + noise_uv flow — the canonical showcase of the new shader builtins. TMX level, atlas strip-sliced animated props (campfire, portal, cooking area), drifting clouds, playable character (A/D walk, W jump, Shift run — their controller plugin replaced by ~25 lines of plain melonJS input/body code with the same tuning). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019t8kP8vp58ZrvaD2FqJU6A

- water sprite uses the two regions verbatim (their settings semantics) instead of getAnimationSettings, whose bottom-align trim mis-placed the frame and bled the adjacent cloud5 atlas pixels (the white artifact on the left edge); the 2dWater art carries its own transparent sky band, matching the original placement - clouds drift right and respawn off the left edge (ported from the demo's update user-code: speed 1+random, wrap at x 1040) - handle video.init() returning false (CodeRabbit finding), alongside the existing thrown-error path Verified side-by-side against the zip's build served statically — scene, water surface line, reflection and drift now match; CodeRabbit re-review clean (0 findings). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019t8kP8vp58ZrvaD2FqJU6A

Copilot