Skip to content

[ET-VK] Conv2d with padding="valid" asserts in make_ivec2 (breaks every SigLIP/CLIP patch embed) #22774

Description

@giuliocorradi

🐛 Describe the bug

Summary

nn.Conv2d(..., padding="valid") exports a one-element padding list, and the
Vulkan convolution reads padding as an ivec2, so it aborts at the first
inference:

make_ivec2 at backends/vulkan/runtime/utils/VecUtils.h:367:
  (ints.size() == 2) is false!

The identical convolution written with padding=0 lowers and runs correctly.

Reproduction

No model needed. The two modules differ only in how padding is spelled.

import torch
from executorch.exir import to_edge
from executorch.backends.vulkan.partitioner.vulkan_partitioner import VulkanPartitioner
from executorch.extension.pybindings.portable_lib import _load_for_executorch

class C(torch.nn.Module):
    def __init__(self, pad):
        super().__init__()
        self.c = torch.nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=pad)
    def forward(self, x):
        return self.c(x)

def run(pad, tag):
    x = torch.randn(1, 3, 224, 224)
    mod = C(pad)
    low = to_edge(torch.export.export(mod, (x,))).to_backend(VulkanPartitioner())
    path = f"/tmp/{tag}.pte"
    with open(path, "wb") as f:
        low.to_executorch().write_to_file(f)
    out = _load_for_executorch(path).forward((x,))[0]
    print(tag, "ok, max|d| =", (out - mod(x)).abs().max().item())

run(0, "pad_zero")            # runs, max|d| 7.2e-07
run("valid", "pad_valid")     # asserts in make_ivec2

Result

padding outcome
0 runs, max|d| 7.2e-07
"valid" asserts, make_ivec2 ... (ints.size() == 2) is false

Reproduced at kernel 3 stride 1 and at kernel 14 stride 14; the kernel geometry
is irrelevant, only the spelling of the padding.

Expected behaviour

padding="valid" means no padding, so it should lower identically to
padding=0.

Suggested fix

Normalise the padding list to the spatial rank where the Vulkan convolution
reads it — broadcast a length-1 list, as PyTorch does elsewhere — rather than
requiring exactly two elements. A length-1 padding list is a legal
representation of symmetric padding.

Impact

padding="valid" is how HuggingFace writes patch-embedding convolutions. In
particular SiglipVisionEmbeddings is:

self.patch_embedding = nn.Conv2d(
    in_channels=config.num_channels,
    out_channels=self.embed_dim,
    kernel_size=self.patch_size,
    stride=self.patch_size,
    padding="valid",
)

so every SigLIP vision tower hits this, and with it PaliGemma, π₀.₅, and any
model using a SigLIP or similarly-written CLIP/ViT patch embed. The workaround
is trivial once known — rewrite the module's padding attribute to (0, 0)
before export — but the failure mode is an assert deep in the runtime with no
indication that padding spelling is the cause.

Versions

collect_env.py output
Collecting environment information...
PyTorch version: 2.12.1+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCm SDK used to build PyTorch: N/A
HIP used to build PyTorch: N/A

OS: Ubuntu 24.04.4 LTS (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.39

Python version: 3.12.3 (main, Jul 15 2026, 23:46:41) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-7.0.0-30-generic-x86_64-with-glibc2.39
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A
CPU: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, 16 cores / 32 threads (full lscpu elided)

Versions of relevant libraries:
[pip3] executorch==1.4.0a0+b20f16a
[pip3] numpy==2.4.6
[pip3] pytorch_tokenizers==1.4.1
[pip3] torch==2.12.1+cpu
[pip3] torchao==0.18.0.dev20260715+cpu
[conda] Could not collect

cc @SS-JIA @manuelcandales @digantdesai @cbilgin

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugmodule: vulkanIssues related to the Vulkan delegate and code under backends/vulkan/triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions