Skip to content

testbench1.cpp: undefined behavior in signed saturation reference code for testcases 610 and 612 #1

Description

@oltolm

The reference implementation for compress_saturated in testbench1.cpp is incorrect for signed destination types in testcases 610 and 612.

It computes the saturated result with:

(RT(1) << (sizeof(RT) * 8 - 1)) - (elements1[i] > 0)

This has undefined behavior for signed RT because it left-shifts into the sign bit, and for positive overflow cases it can overflow again when subtracting 1.

As a result, the testbench reference code can produce incorrect expected results even when compress_saturated itself is correct.

Expected behavior

For signed narrowing saturation, the reference code should clamp to std::numeric_limits<RT>::max() for positive overflow and std::numeric_limits<RT>::min() for negative overflow.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions