Skip to content

Fix OVRTX GPU transform default#6626

Closed
ruziniuuuuu wants to merge 1 commit into
isaac-sim:developfrom
ruziniuuuuu:codex/fix-newton-ovrtx-output-sync
Closed

Fix OVRTX GPU transform default#6626
ruziniuuuuu wants to merge 1 commit into
isaac-sim:developfrom
ruziniuuuuu:codex/fix-newton-ovrtx-output-sync

Conversation

@ruziniuuuuu

@ruziniuuuuu ruziniuuuuu commented Jul 20, 2026

Copy link
Copy Markdown

Description

Default OVRTX's environment-controlled read_gpu_transforms setting to the safe CPU-read path. OVRTX warns that enabling Fabric transform reads together with geometry streaming can prevent dynamic geometry from streaming correctly; in Newton workloads this intermittently appears as missing robot links in OVRTX camera output.

The existing ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=1 override remains available as an explicit opt-in for workloads that have validated the previous GPU path. No dependency changes are required.

A standalone four-environment differential reproducer found:

Configuration Missing-link reproductions
OVRTX 0.3, current GPU-transform default 2 / 3
OVRTX 0.3, GPU transform reads disabled 0 / 10
OVRTX 0.4, current GPU-transform default 1 / 5
OVRTX 0.4, GPU transform reads disabled 0 / 4 completed runs

The Newton viewer-disabled control reproduced 0 / 3 times. USD visibility stayed unchanged and Newton body/joint states stayed finite and synchronized during captured failures.

Fixes #6625

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Not applicable to the adapter default itself. Issue #6625 describes the captured missing-link evidence and standalone reproducer.

Validation

  • Regression test was added first and failed against the previous default.
  • pytest source/isaaclab_ov/test/test_ovrtx_renderer_contract.py -q: 14 passed.
  • ./isaaclab.sh -p tools/changelog/cli.py check develop: passed.
  • ./isaaclab.sh -f: passed before commit and again before push.
  • Standalone MRE: OVRTX 0.3 safe path completed 10 / 10 runs without the symptom; OVRTX 0.4 safe path completed 4 / 4 runs without it.

Current-develop end-to-end validation used its exact Newton pin (newton 1.4.0.dev0, warp-lang 1.15.0.dev20260626) with OVRTX 0.3: the standalone Newton-viewer MRE completed 1,000 steps across four environments with reproduced=False, finite/synchronized state, and no unsafe transform/geometry-streaming warning. A separate RTX/MDL shutdown segfault occurred only after the result and final capture were written.

Risk

The CPU transform-read path may have different renderer performance characteristics. Explicit opt-in to the previous behavior is preserved with ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=1.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation (changelog fragment)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Default OVRTX to the safe CPU transform-read path while preserving an explicit environment-variable opt-in for validated GPU workloads.

Refs isaac-sim#6625
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Jul 20, 2026
@ruziniuuuuu

ruziniuuuuu commented Jul 20, 2026

Copy link
Copy Markdown
Author

Standalone reproducer and captured output

The complete reproducer is published at an immutable commit:

It can be run directly from that folder:

python repro.py --num-envs 4 --steps 5000 --seed 17

# Automated fresh-process trial
python repro.py --headless --steps 1000 --stop-on-failure

The repository contains the robot asset and does not import GalbotLab. Four environments receive identical robot and camera states; the detector compares persistent denoised silhouettes and only reports a rendering failure while Newton state is finite and synchronized. The entry point supports both the stable and current develop launcher locations.

Reference OVRTX frame (env_0) Missing-link OVRTX frame (env_1)
Reference OVRTX frame OVRTX frame with missing robot link

The captured failure affected environment 1 with 399 differing silhouette pixels (352 persistent across the three-frame confirmation window, maximum RGB channel error 211). USD visibility remained unchanged; the exact camera poses, body/joint state, package versions, and detection values are included in evidence/.

Newton visualizer framebuffer from the same capture:

Newton visualizer framebuffer

Fix validation on current develop

I overlaid the PR head (42ba436fec2f25cd7ccf47889f1ee555cce28f52) onto current Isaac Lab develop and used its exact Newton dependency (c7ae7c7648cd0717df39e5c94b95d5a02c997320, newton 1.4.0.dev0, warp-lang 1.15.0.dev20260626) with OVRTX 0.3.0.312915.

The Newton-viewer/OVRTX run completed 1,000 simulation and render steps across four environments with reproduced=False; all compared body/joint states stayed finite and synchronized. The unsafe readTransformsFromFabricInRenderDelegate plus geometry-streaming warning was absent. After the result and final capture were written, Isaac Sim hit a separate RTX/MDL shutdown segfault; it did not occur during the test loop and is not counted as validation of that cleanup path.

@ruziniuuuuu
ruziniuuuuu marked this pull request as ready for review July 20, 2026 10:28
@ruziniuuuuu
ruziniuuuuu requested a review from a team July 20, 2026 10:28
@greptile-apps

greptile-apps Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR changes the default for OVRTX's read_gpu_transforms setting from enabled ("1") to disabled ("0"), fixing intermittent missing robot links in Newton/OVRTX camera output caused by a renderer-side incompatibility between Fabric transform reads and RTX geometry streaming. The previous GPU-read path remains available as an explicit opt-in via ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=1.

  • ovrtx_renderer.py: One-line default change in _read_gpu_transforms_enabled() with an explanatory comment; the function is consumed at OVRTXRenderer.__init__ time when constructing RendererConfig.
  • test_ovrtx_renderer_contract.py: Two new contract tests verify the disabled default and the explicit opt-in path using monkeypatch against the env var.
  • changelog.d/ and CONTRIBUTORS.md: Standard housekeeping fragments added.

Confidence Score: 5/5

Safe to merge — the change is a one-line default flip with a well-tested escape hatch preserved via env var.

The fix is narrowly scoped to a single default value in a helper function. The CPU read path was already a supported, tested code branch; only the default changes. Two new contract tests directly cover the new default and the opt-in path, and the existing test suite still passes. The env-var override ensures no workloads are permanently broken by the new default.

No files require special attention.

Important Files Changed

Filename Overview
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py Single-line default change from "1" to "0" in _read_gpu_transforms_enabled(); well-commented, no structural changes.
source/isaaclab_ov/test/test_ovrtx_renderer_contract.py Imports _read_gpu_transforms_enabled and adds two focused monkeypatch tests covering the new default and opt-in; properly guarded by the module-level skipif.
source/isaaclab_ov/changelog.d/codex-fix-newton-ovrtx-transforms.rst New changelog fragment documenting the fix and the opt-in env var; content is accurate.
CONTRIBUTORS.md Adds "Chenyu Cao" in alphabetical order; no issues.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Env as Environment Variable ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS
    participant Fn as _read_gpu_transforms_enabled()
    participant Init as OVRTXRenderer.__init__
    participant RC as RendererConfig
    participant R as ovrtx.Renderer

    Init->>Fn: call at construction time
    Fn->>Env: os.environ.get(key, "0")
    alt env var unset (default)
        Env-->>Fn: "0"
        Fn-->>Init: False (CPU path)
    else "env var = "1" (opt-in)"
        Env-->>Fn: "1"
        Fn-->>Init: True (GPU path)
    else env var invalid
        Fn-->>Init: raises ValueError
    end
    Init->>RC: "RendererConfig(read_gpu_transforms=False/True)"
    RC->>R: Renderer(config)
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Env as Environment Variable ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS
    participant Fn as _read_gpu_transforms_enabled()
    participant Init as OVRTXRenderer.__init__
    participant RC as RendererConfig
    participant R as ovrtx.Renderer

    Init->>Fn: call at construction time
    Fn->>Env: os.environ.get(key, "0")
    alt env var unset (default)
        Env-->>Fn: "0"
        Fn-->>Init: False (CPU path)
    else "env var = "1" (opt-in)"
        Env-->>Fn: "1"
        Fn-->>Init: True (GPU path)
    else env var invalid
        Fn-->>Init: raises ValueError
    end
    Init->>RC: "RendererConfig(read_gpu_transforms=False/True)"
    RC->>R: Renderer(config)
Loading

Reviews (1): Last reviewed commit: "Fix OVRTX GPU transform default" | Re-trigger Greptile

@pbarejko
pbarejko requested a review from mataylor-nvidia July 21, 2026 04:49

@pbarejko pbarejko left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thank you for the PR.

I wouldn't merge this change. Unfortunately, I am sad to say, this doesn't fix anything. The problem is upstream - Fabric. This PR hides the symptom and puts Isaac Lab on very slow code path - it would have huge performance implications. My best recommendation is to use ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=0 in your local workflow.

This is a known issue that has been reported upstream and we should have fix for this.

@mataylor-nvidia can you check:

  1. Rayyan's fix in Fabric is available in ovrtx-0.4 release? If it is this would confirm if there another hiding bug somewhere and we would have to file another nvbug with provided repro steps.
  2. If it's not, I know you have built Rayyan's branch, can you use it to validate that it fixes @ruziniuuuuu problem without flipping read_gpu_transforms?

@ruziniuuuuu

ruziniuuuuu commented Jul 21, 2026

Copy link
Copy Markdown
Author

I retested ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=0 on Isaac Lab v3.0.0-beta2.patch1. Setting it alone does not work because this version does not read the variable and still passes read_gpu_transforms=True to OVRTX.

Environment variable only: links still disappear

env_0 env_2
Missing geometry in env 0 Missing arm link in env 2

I then added a small local backport so that 0 actually reaches RendererConfig(read_gpu_transforms=False).

Environment variable with backport: links remain complete

env_0 env_2
Complete robot in env 0 Complete robot in env 2

With the backport, 3 independent 1,000-step runs completed with reproduced=False, and the Fabric/geometry-streaming warning disappeared.

Updated standalone reproducer and evidence

@pbarejko

Copy link
Copy Markdown
Collaborator

I retested ISAAC_LAB_OVRTX_READ_GPU_TRANSFORMS=0 on Isaac Lab v3.0.0-beta2.patch1. Setting it alone does not work because this version does not read the variable and still passes read_gpu_transforms=True to OVRTX.

Environment variable only: links still disappear

env_0 env_2
Missing geometry in env 0 Missing arm link in env 2
I then added a small local backport so that 0 actually reaches RendererConfig(read_gpu_transforms=False).

Environment variable with backport: links remain complete

env_0 env_2
Complete robot in env 0 Complete robot in env 2
With the backport, 3 independent 1,000-step runs completed with reproduced=False, and the Fabric/geometry-streaming warning disappeared.

Updated standalone reproducer and evidence

Good you have a workaround! In long term we will fix the problem upstream in OvRTX and you wouldn't have to manipulate read-gpu-transforms variable. This problem should be addressed in next release of ovrtx-0.5.

@pbarejko

Copy link
Copy Markdown
Collaborator

Hi @ruziniuuuuu, we have verified upstream fix for this issue: #6625 (comment)

Therefore I am taking the liberty and closing this PR since this is not applicable solution.

@pbarejko pbarejko closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants