use size_t when sizing uncompressed image write buffers#644
Merged
solidpixel merged 7 commits intoJun 13, 2026
Conversation
Contributor
|
Thanks. When reviewing this one I found a few more of a similar pattern, so added those to this PR too. |
Contributor
|
Updated PR should avoid any mixed int/uint/size_t usage on this code path, unless there are explicit casts. The casts should be temporary and vanish in a future major release when we can change the definition of astcenc_image to accept size_t for image sizes. (Note that practically the size will still be limited to uint32_t limits per axis, because so many image formats require that, the change is just to clean up the interface semantics). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following the size_t work that went into the loaders and alloc_image, I went back over the matching write paths and the uncompressed KTX and DDS writers still size their buffers with 32-bit maths. store_ktx_uncompressed_image and store_dds_uncompressed_image allocate the interleaved pixel buffer, and derive the per-plane and per-row pointers, from dim_x * dim_y * dim_z * image_components as unsigned int; floatx4_array_from_astc_img does the same for the float buffer behind the HDR and EXR writers. On a -dl decode those dimensions come from the .astc header, so a 32768x32768 RGBA image makes that product exactly 2^32, which truncates to zero. The pixel buffer is then allocated at a few bytes while the scanline loop still copies the full surface, so it writes well past the allocation; under ASan this shows up as a heap-buffer-overflow on the first row written.
Widening the dimensions to size_t pushes the buffer size, the plane and row strides, and the source row index into 64-bit, so the buffer is sized for what the loop actually writes. Doing it in the writer keeps the decode-to-file path safe whatever produced the image, and follows the loader and alloc_image changes already merged.