Skip to content

test: Add StandardShaderBall test scene and common models directory - #2149

Open
jmp-22 wants to merge 5 commits into
AcademySoftwareFoundation:mainfrom
jmp-22:test/add-shaderball-testrender-scene
Open

test: Add StandardShaderBall test scene and common models directory#2149
jmp-22 wants to merge 5 commits into
AcademySoftwareFoundation:mainfrom
jmp-22:test/add-shaderball-testrender-scene

Conversation

@jmp-22

@jmp-22 jmp-22 commented Aug 22, 2026

Copy link
Copy Markdown

Description

Closes #1976

Adds a render-shaderball testsuite entry that renders the USD Working Group's StandardShaderBall asset in testrender, and introduces testsuite/common/models/ as a home for geometry that more than one test can share.

The shader ball is a widely used reference scene for evaluating material response (reflection, transmission, roughness, Fresnel, caustics, subsurface) so having it in the testsuite provides a somewhat realistic comparison against other renderers.

The USD Working Group asset ships as USD with MaterialX shading networks. testrender only reads Wavefront OBJ, so the scene was imported into Blender and re-exported as OBJ + MTL. testrender already reads object names and material names out of the .obj/.mtl, which makes it straightforward to rebind the material assignments to OSL shader groups in the scene .xml description. All 12 material names map 1:1 onto shader groups in scene.xml.

Assisted-by: Claude Code / Claude Opus 5

Used Claude Code to draft the two attribution READMEs and the notices embedded in the OBJ/MTL files.
I verified the licensing facts against the upstream repository and edited the modification statement to correctly describe my changes to the model.

testrender vs USD reference

Reference images were rendered in Houdini's Karma renderer unless otherwise noted. Images shown here have been converted from EXR to PNG.

Basic test scene

testrender Reference
shaderball_default shaderball-none-1080p

Shader ball with correct UVs

testrender Reference (from USDView)
shaderball_uvgrid shaderball-uv-reference

Shader ball with subsurface geometry

testrender Reference
shaderball_glass shaderball-glass-1080p

Other testrender outputs

Shader ball with various materials

Plastic Metal
shaderball_plastic shaderball_metal

Scene render passes

Normal UV
shaderball_normals shaderball_uvs

Added

New test: testsuite/render-shaderball — 160x120, -aa 8, using the same set of comparison settings as render-uv and render-microfacet, with failthresh=0.01 and allowfailures=3.

Shared model directory: testsuite/common/models/. testsuite/common is already copied into the build tree at configure time by src/cmake/testing.cmake, so tests reference models the same way they already reference
../common/textures/.... render-bunny's bunny.obj moves here too.

Shaders: the ball itself uses a plain grey glossy (diffuse + GGX dielectric layer), with diffuse_textured on the enclosing box walls and emitter on the area-light geometry. matte, emitter, glossy and metal follow the existing
per-test-directory convention and are unchanged copies of what other render-* tests already carry.

glass.osl, metal.osl and plastic.osl are included but not instantiated in scene.xml. I used these to produce the preview renders above, and chose to include them in the render-shaderball test directory as a useful starting points for follow-up tests or experimentation. I'm happy to exclude them if reviewers would rather the directory only carry what the test uses.

Licensing

I'm not very experienced with asset licensing, so flagging a few things for review. I used Claude Opus 5 to help generate attribution information.

The USD shaderball scene assets is licensed CC BY 4.0, which differs from OSL's BSD-3-Clause. See the USD Working Group's LICENCE for the scene.

To ensure attribution, I added:

  • testsuite/common/models/shaderball/README.md: Carries the full attribution to authors (Chris Rydalch, André Mazzone, Thomas Anagnostou), the license and its URL, a link to the upstream LICENCE, and a description of the modifications made.
  • testsuite/render-shaderball/maps/README.md: Records that the textures are redistributed unmodified.
  • A notice embedded as comments at the top of shaderball.obj and shaderball.mtl, so attribution travels with the files if they're ever copied out of the repo.

CC BY 4.0 imposes no share-alike obligation, so nothing here affects the licensing of OSL's own code. I've left the OSL repo's THIRD-PARTY.md alone for now, but I'm happy to add an entry there if maintainers would prefer THIRD-PARTY.md to list it.

The only change to the modeled geometry is the addition of light-emitting planes in place of the upstream USD area lights, which have no OBJ representation and which testrender couldn't use anyway. This is all documented in the asset README.

Repository Size

The assets add ~7.2 MB: ~6.0 MB of textures, ~1.2 MB of geometry, and a 28 KB reference image. One texture map, ground.ACEScg.exr, alone is 4.2 MB.

The textures are the USD WG files as-published. If this size increase is deemed to be too large for the repo, we could apply compression on the ground texture or remove the three emitter_* maps and uvgrid.exr (which currently only exist for the previews).

Testing

Both tests pass on the CI-OSL reference image aswf/ci-osl:latest:

$ ctest --test-dir build -R 'render-(bunny|shaderball)'
    Start 584: render-bunny.opt
1/2 Test #584: render-bunny.opt .................   Passed    4.14 sec
    Start 604: render-shaderball.opt
2/2 Test #604: render-shaderball.opt ............   Passed   15.13 sec

100% tests passed, 0 tests failed out of 2

Note that render-bunny needed re-verifying, since its model moved to testsuite/common/models.

Known limitations and follow-ups for review

  • CPU only. There's no OPTIX marker file, so this doesn't currently run under OptiX. I don't have an OPTIX-capable GPU to validate against. Glad to add the marker file if someone with a GPU or CI can confirm it matches, or to leave it as a follow-up. Other platforms may need an alternate reference, as render-microfacet does for Mac ARM.
  • Camera position/angle in testrender is difficult to translate exactly from the USD scene. The angle used here is not 100% accurate to the USD camera, but produces a fairly similar image.
  • The ball uses a placeholder grey shader. Adding the glass and metal variants as their own tests seems like it would be the next step?

Checklist:

  • I have read the guidelines on contributions and code review procedures.
  • I have read the Policy on AI Coding Assistants
    and if I used AI coding assistants, I have an Assisted-by: TOOL / MODEL
    line in the pull request description above.
  • I have updated the documentation if my PR adds features or changes
    behavior.
  • I am sure that this PR's changes are tested in the testsuite.
  • I have run and passed the testsuite in CI before submitting the
    PR, by pushing the changes to my fork and seeing that the automated CI
    passed there. (Exceptions: If most tests pass and you can't figure out why
    the remaining ones fail, it's ok to submit the PR and ask for help. Or if
    any failures seem entirely unrelated to your change; sometimes things break
    on the GitHub runners.)
  • My code follows the prevailing code style of this project and I
    fixed any problems reported by the clang-format CI test.

jmp-22 added 4 commits August 22, 2026 10:55
Add a new testrender-based test scene featuring the ASWF USD Working
Group StandardShaderBall asset and scene, rendered using testrender.

Scene geometry was converted from USDA format to OBJ format in Blender.
The scene uses testrender's support for ShaderGroup/MTL material group
name matching to assign shaders to the scene. Lighting is created from scene
geometry, with an emission shader applied to planes to mimic the area lights
of the USD scene. Camera angle, textures, and lighting have been tweaked to
provide a close, but not exact, replica of the USD scene.

The new render-shaderball test has been integrated into the testsuite,
referencing an image generated at 320x240 px, using setting -aa 4 on the CPU.

Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
Add a new directory to store 3D models to be reused amongst tests.
Migrated the existing bunny.obj and new shaderball.obj models and
updated their respective test scenes to reference the new common/models
location.

The render-shaderball test's output dimensions were halved to reduce render time.

Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
The common/models paths introduced for the bunny and shaderball scenes
were written as "data/../common/models/...", which depends on a test's
"data" entry being a symlink to its source directory. That holds on Unix,
but runtest.py copies the source directory to "data" on Windows, so
"data/.." resolves to the test's temporary build directory and the model
is never found.

Reference the models as "../common/models/..." instead. This also matches
how existing tests reference shared assets such as ../common/textures/grid.tx.

Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
The StandardShaderBall geometry and textures are licensed under CC BY
4.0, which requires retaining the copyright notice and license
reference, and indicating where the material has been modified.

Add a README alongside the model recording the upstream authors, the
license and its URL, and the changes made in converting the USD scene to
OBJ. Add a shorter README alongside the textures noting that they are
redistributed unmodified. Embed the same notice as comments at the head
of shaderball.obj and shaderball.mtl.

Also add the missing copyright and SPDX headers to diffuse_textured.osl
and plastic.osl.

Assisted-by: Claude Code / Claude Opus 5

Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
@lgritz

lgritz commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Hi, just as an aside, your PR description contains this text:

> Closes issue #1976

I think the use of a > to make it a quoted paragraph and the nonstandard phrasing "Closes issue" -- or possibly either of those in isolation -- causes this PR to merely be referenced in the original issue. But if instead you had written it with the "magic words"

Closes #1976

Then GitHub would fully understand not just the link, but also that this PR, upon acceptance, should cause automatic closure of the original issue. I think "Fixes #xyzw" also works. But many other phrasings that humans would understand perfectly well, are nonetheless not recognized as being equivalent and will not cause the issue to be closed when this is merged.

I think that if you edit the description to say it correctly, it still might establish the link?

Comment on lines +42 to +43
Note that this is a different license from the BSD-3-Clause license that
covers OSL's own source code. See the repository's `THIRD-PARTY.md`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OSL project uses 2 standard licenses: BSD-3-Clause for code, and CC-BY-4.0 for docs and other non-code things. We mention this and link to the CC pages in the main README.md of this project. So you aren't adding any new or different kinds of licenses here, and I think you can just minimally note that the original data the shaderball file is derived from was also CC-BY-4.0 and give a link to the usd-wg repo where it lives.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the licensing clarifications. I have kept both READMEs for the model and textures, but reduced the length of the file to minimally state the CC-BY-4.0 license of the original asset and include a link to the usd-wg repo. I decided to keep the credits to the authors too.

Additionally, in testsuite/common/models/shaderball/README.md, I shortened the modifications section while still explaining the edits I made to the scene in order to match lighting as closely as possible.

Please let me know if you'd like me to remove any further attribution- or license-related info. Edits made in 8d2b7d6.

@lgritz

lgritz commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

This is great! (And I'm especially happy to see this land clean, well organized, and fully passing CI on the first shot.)

I have a feeling that we will use the shaderball for many more tests over time, so I wonder if, just like the shaderball geometry, the shaderball textures should be moved to testsuite/common/textures/shaderball?

It seems that the render-shaderball test still contains a lot of .osl files that aren't used by the test. Presumably those were used to generate the examples you posted above in this PR. But probably the final form of this PR should remove the ones that we don't intend to directly use.

Really, those two items are about all I can think of suggesting... this is everything I'd hoped for.

I do have another question, which may be more revealing of my unfamiliarity with the USD test shaderball than it is about anything you have done:

In the first pair of rendered examples above, the testrender image on the left shows polygon faceting of the silhouette -- everywhere, but it's most easily visible in the "corners" of the triangular-ish depression of the outer shaderball surface. (Are there agreed upon names for the different geometric/geographic features of the shader ball?) But in the reference image on the right from Houdini, it has been fully subdivided so that no tessellation artifacts are visible.

All the other examples above, which I believe are testrender vs USDView, appear to be about the same level of tessellation as the OSL version. So my questions are:

  • Is the USDView rough tessellation what users expect from a shader ball image? Is that adequate?
  • How are people used to it appearing in MaterialX tests, in terms of tessellation, but also camera parameters, lighting, etc.? Is there any difference between typical MatX and USD use of this test asset, and if so, is there advantage to us matching one versus the other?
  • Is there any advantage to using a higher quality conversion for the sake of looking better and/or more closely matching MatX, USD, or how it would look in a good renderer like Karma?

I think what I'm really getting at is, is this the best tessellation rate that will serve us well for matching MatX and/or USD and testing material behaviors with OSL, versus (obviously) the tradeoff of not wanting a truly ginormous obj file checked into our repo if the increase in quality will never be needed.

@fpsunflower

Copy link
Copy Markdown
Contributor

Nice work - this is exciting to see :)

The only other thing I'd add to what Larry said, is we might want to stick to a square aspect ratio for these tests? I don't think the 4:3 aspect adds much here (although again, matching what other folks are doing to present the shaderball is properly best).

It should be possible to get a better match for the glass case, but might need a bit more work on the glass shader to pass in the medium properties through. Hopefully there are enough other examples in our current testsuite to figure this out, but feel free to ask here if you get stuck or if something doesn't work.

If you are able to refactor the layout so that the shaderball geo and textures are truly shared, I would encourage you to make one or two more tests showing off the other examples you posted above. In particular the uv grid (to show the texture layout is the right way around) and the glass case (to show off the internal geometry bars).

@lgritz

lgritz commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

@fpsunflower The glass image example is just part of the exposition of the PR, and is not one of the added tests, so I don't think we need to address that now.

Aspect ratio -- pick whatever is commonly used by others for shaderball test images.

Believe it or not, there's another PR in flight that explains that testrender's handling of fov is wrong, so once that is accepted and merged, these images will change appearance and need slight change (if not square!) and need the ref images updated because it will be very hard to be exact. So maybe I'll make sure that one is merged so the fix can be wrapped up in this one without having to revisit afterwards.

Move the shaderball textures alongside the geometry in testsuite/common
so that future tests can share them, and update the scene and
diffuse_textured default to the new paths.

Remove glass.osl, metal.osl and plastic.osl, which were used to generate
the example renders for the PR but are not instantiated by the test.

Also, trim attribution down to note that the data derives from the
usd-wg StandardShaderBall and is CC-BY-4.0, which is already one of the
project's licenses.

Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
@jmp-22

jmp-22 commented Aug 30, 2026

Copy link
Copy Markdown
Author

Hey everyone, I appreciate all your feedback. I'll try to address your questions/review notes as best as possible below.

Firstly, thanks for pointing out the non-standard language I used to mention the GitHub issue. I edited the PR description so hopefully that links this PR to the issue appropriately.

Shaderball scene-specific textures have been moved to a shared location at testsuite/common/textures/shaderball, and the extra .osl shaders that I used to generate example renders have been removed in 8d2b7d6. I've included the uv-grid.exr texture in the common directory so that future shaderball tests can utilize it.


@lgritz: Here's what I've found to answer in relation to the comparison to MaterialX and the asset's tessellation rate:

MaterialX vs USD

I'm not familiar with MaterialX and their shaderball testing standards, but I used Claude to help compare the USD scene and the MaterialX test scene, and it seems they use different assets with different lighting. This is what we were able to find:

MaterialX USD StandardShaderBall
geometry it's own resources/Geometry/shaderball.glb this asset
lighting HDR environment enclosing box + emissive area lights
camera auto-framed to a bounding sphere authored in camera.usda

It seems that MaterialX's automated render tests don't use a shaderball at all: _options.mtlx renders sphere.obj at 512x512 with enableDirectLighting=false and HDRI lighting only (san_giuseppe_bridge.hdr). The shaderball people actually recognize is the MaterialX Viewer default: Main.cpp loads shaderball.glb with san_giuseppe_bridge_split.hdr, and Viewer.cpp positions the camera above the model at (0,0,5) and looks down at a 45 degree view angle, auto-framed to the mesh.

I take this to mean that matching MaterialX would mean a different model and different lighting. Looking through other testsuite tests, I think OSL already covers this sphere + HDRI setup: render-mx-dielectric, render-mx-conductor, render-mx-sheen and render-mx-layer all render spheres against an environment shader. This PR adds the USD-matching coverage that we previously didn't have.

I hope this helps answer your question, but If we want an authoritative answer on MaterialX convention that's probably a question for the maintainers considering my lack of experience with MaterialX.

Shaderball Tessellation

In regards to your question about the shaderball's triangulated vs. subdivision geometry, here's what I gather from the asset's README.md and other documentation.

The USD scene comes with a surface_geometry VariantSet, used to show different surface representations of the shaderball model. From the README, we can see that there are two different variants:

The surface_geometry VariantSet allows the above elements to be defined by either subdivision or triangulated polygon surfaces. Both variants are intended to provide equivalent geometry and dimensions.

  • subdiv

    • Quad-mesh, which is also marked as a catmullClark subdivision surface.

    • Care has been taken to create a UV layout that can be used both to exercise spatially varying materials (e.g. surface textures) and also provide a coordinate frame for anisotropic specular roughness.

    • Air bubbles are represented as inverted faces inside of the mesh.

  • triangulated

    • A tessellated version of subdiv, which has Loop-subdivision applied to produce the triangulated topology.

    • The UV layout should be equivalent to subdiv.

    • Includes authored per-vertex normals attribute to address polygonal facetting.

From my understanding, OSL's testrender engine doesn't natively support subdivision surfaces. Therefore, I think it would be more accurate to use the "official" triangulated variant of the model instead of creating our own subdivided OBJ.

On a side note, the example USDView image I used in the PR description appears to have been a screenshot of the model at a lower quality level, since USDView can actually render nice subdivision surfaces.

However, looking closer at an example comparison of the Catmull-Clark surfaces vs. their tessellated versions, it appears that my exported OBJ is lacking in quality compared to the triangulated variant.

image

Subdivision on the left, triangulation on the right. Link to original
Credit: https://github.com/usd-wg/assets/blob/main/full_assets/StandardShaderBall/

Thinking about my export process, what I believe happened is that the USD scene imported into Blender in the subdiv variant, but any subdivision modifiers (as they are called in Blender) were not applied to produce the triangulated topology. So what I've actually exported is a triangulated version of the Catmull-Clark quad mesh.

To fix this, I'll export the triangulated variant of the USD shaderball from Blender, which should have exactly the same topology as the "official" USD tessellated version seen above. Based on my tests, it should increase the size of the OBJ scene model by about 4 MB. I'll post the updated example renders once I have exported the new OBJ and got the test passing.


@fpsunflower: Thanks very much for taking a look. I'm happy to look at creating additional tests for the UV-grid texture and glass shader after we've got the scene geometry sorted out.

In terms of aspect ratio, I'm happy for the tests to output 1:1 renders in 128x128. This matches the aspect ratio used by camera.usda in the USD scene, matches the ratio of shaderball renders included in the USD WG asset repo, and those dimensions match the render-bunny test and others in the testsuite.


For my next steps, I'll be working on another commit to correct the tessellation rate and change the aspect ratio to 1:1. Please let me know if you'd like me to take a different direction on any of this, or if you have any questions. Thanks!

@lgritz

lgritz commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

For my next steps, I'll be working on another commit to correct the tessellation rate and change the aspect ratio to 1:1. Please let me know if you'd like me to take a different direction on any of this, or if you have any questions. Thanks!

Sounds good to me. Do those two minor changes (and update ref images as needed), and let's merge this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a test scene based on the USD standard shaderball

3 participants