Skip to content

Fix left shift on signed int channel setting sign bit and causing UB#28

Merged
MarkCallow merged 2 commits into
KhronosGroup:mainfrom
walcht:fix_left_shift_on_signed_int_setting_sign_bit
Jul 15, 2026
Merged

Fix left shift on signed int channel setting sign bit and causing UB#28
MarkCallow merged 2 commits into
KhronosGroup:mainfrom
walcht:fix_left_shift_on_signed_int_setting_sign_bit

Conversation

@walcht

@walcht walcht commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

I don't whether this was causing any issues, but:

  • When the signed integer (type: int) is 192, the following undefined behavior occurs: 11000000 << 24 => is undefined behavior because setting sign bit of an integer via a left shift operation is UB. Simply cast channel to uint32_t before shifting.

  • To replicate: build with -fsanitize="undefined" and navigate to KTX-Software/tests/cts/clitests then:
    ktx transcode --testrun input/ktx2/valid_R16G16_SFLOAT_CUBE_ARRAY.ktx2 / output/transcode/transcode_error_not_transcodable/output_R16G16_SFLOAT_CUBE_ARRAY.ktx2

You will get the following:

KTX-Software/external/dfdutils/createdfd.c:123:18: runtime error: left shift of 192 by 24 places cannot be represented in type 'int'

* When the signed integer (type: int) is 192, the following undefined behavior
occurs: 11000000 << 24 => is undefined behavior because setting sign bit of an
integer via a left shift operation is UB. Simply cast channel to
uint32_t before shifting.

* To replicate: build with -fsanitize="undefined" and navigate to
KTX-Software/tests/cts/clitests then:
ktx transcode --testrun input/ktx2/valid_R16G16_SFLOAT_CUBE_ARRAY.ktx2 /
output/transcode/transcode_error_not_transcodable/output_R16G16_SFLOAT_CUBE_ARRAY.ktx2

You will get the following:

KTX-Software/external/dfdutils/createdfd.c:123:18:
runtime error: left shift of 192 by 24 places cannot be represented in type 'int'

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@MarkCallow

Copy link
Copy Markdown
Collaborator

The sampleNo and channel parameters should really be unsigned but I have not looked to see what the knock-on effects of that change would be. There is no reason for those values to ever be signed.

@walcht

walcht commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Agree. That cast from int->uint32_t is safe (channel at most sets the 32-24=8th bit). I didn't want to introduce any other changes (e.g., setting channel as uint32_t in parameter list).

Comment thread createdfd.c
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@MarkCallow MarkCallow merged commit 875e493 into KhronosGroup:main Jul 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants