Normalize signed integer values to range [0.0..1.0]

Question

We read files that can contain all kinds of data types, which we normalize before further processing.

This also works with MSVC, GCC, and Clang on my PC. However, GCC does strange things on our CI.

The code reduced to the bare essentials:

#include <cstdint>
#include <iostream>
#include <type_traits>

template <typename SourceType, typename TargetType>
TargetType convert(SourceType value)
{
    if constexpr (std::is_integral_v<SourceType> && std::is_floating_point_v<TargetType>)
    {
        // Double only has a precision of (53 bits mantissa), some platforms have long double, so we can represent all values accurately.
        using DoubleType = std::conditional_t<(std::numeric_limits<SourceType>::digits > std::numeric_limits<double>::digits),
                                              long double,
                                              double>;

        // Conversion from integer to double/float: Fit all values into the range [0;1].
        DoubleType result = 0.0;
        if constexpr (std::is_signed_v<SourceType>)
        {
            // Solutions looks more complex, because the behavior differs between compilers: MSVC may rely on
            // implementation-defined behavior for signed overflows, while Clang/GCC applies wraparound semantics
            // when converting negative values to unsigned types.

            using Unsigned = std::make_unsigned_t<SourceType>;

            constexpr auto min = std::numeric_limits<SourceType>::min();
            constexpr auto max = std::numeric_limits<Unsigned>::max();

            const Unsigned shifted = static_cast<Unsigned>(static_cast<std::int64_t>(value) - static_cast<std::int64_t>(min));

            result = static_cast<DoubleType>(shifted) / static_cast<DoubleType>(max);
            std::cout << "min: " << min
                      << " shifted: " << shifted
                      << " shifted casted: " << static_cast<DoubleType>(shifted)
                      << " range: " << max
                      << " range casted: " << static_cast<DoubleType>(max)
                      << " digits: " << std::numeric_limits<DoubleType>::digits
                      << " size: " << sizeof(DoubleType)
                      << " result: " << result << '\n';
        }
        else
        {
            constexpr auto range = std::numeric_limits<SourceType>::max();
            result = static_cast<DoubleType>(value) / static_cast<DoubleType>(range);
        }
        // Avoid redundant cast warning
        if constexpr (std::is_same_v<TargetType, DoubleType>)
        {
            return result;
        }
        return static_cast<TargetType>(result);
    }
}

Test code:

#include <gtest/gtest.h>

const auto checkValueRangeBounds = []<typename T>(std::type_identity<T>) {
    EXPECT_EQ((convert<T, float>(std::numeric_limits<T>::lowest())), 0.0F);
    EXPECT_EQ((convert<T, float>(std::numeric_limits<T>::max())), 1.0F);

    EXPECT_EQ((convert<T, double>(std::numeric_limits<T>::lowest())), 0.0);
    EXPECT_EQ((convert<T, double>(std::numeric_limits<T>::max())), 1.0);
};

TEST(convert, bounds)
{
    checkValueRangeBounds(std::type_identity<std::int8_t>{});
    checkValueRangeBounds(std::type_identity<std::int16_t>{});
    checkValueRangeBounds(std::type_identity<std::int32_t>{});
    checkValueRangeBounds(std::type_identity<std::int64_t>{});
    checkValueRangeBounds(std::type_identity<std::uint8_t>{});
    checkValueRangeBounds(std::type_identity<std::uint16_t>{});
    checkValueRangeBounds(std::type_identity<std::uint32_t>{});
    checkValueRangeBounds(std::type_identity<std::uint64_t>{});
}

This works for all cases, except std::int64_t on GCC.

The output for this case:

min: -9223372036854775808 shifted: 0 shifted casted: 0 range: 18446744073709551615 range casted: 1.84467e+19 digits: 64 size: 16 result: 0
min: -9223372036854775808 shifted: 18446744073709551615 shifted casted: -1 range: 18446744073709551615 range casted: 1.84467e+19 digits: 64 size: 16 result: -5.42101e-20

It seems he can correctly convert UINT64_MAX to long double in the constexpr case, but not during the runtime, as the expected result is 1.0 and not -5.42101e-20 (nearly 0.0).

Since I can't reproduce it locally with my GCC 11: Does anyone have any idea what causes the GCC to behave differently on our CI (GCC 10 & 14)?

Please provide minimal reproducible example. Best version is in form of unit test. For example by using gtest or catch2. Note this site godbolt.org is great to find differences between compilers. — Marek R
– Marek R, Commented Sep 16 at 14:46
your code example is excessive. other than that, if he,she,you, operate on values close to UINT64_MAX range, with that, you do "normalization", i know you are in danger zone. — jorge is not ai
– jorge is not ai, Commented Sep 16 at 14:48
Offtopic: Are you sure you provided correct version of the code? Looks like template parameter order is wrong, If TargetType would be first then SourceType could be deduced. — Marek R
– Marek R, Commented Sep 16 at 14:53
Unsigned shifted = static_cast<Unsigned>(static_cast<std::int64_t>(value/2) - static_cast<std::int64_t>(min/2)); shifted *= 2; if (value > 0 && (value & 1)) shifted += 1; if (value < 0 && (value & 1)) shifted -= 1; if (min & 1) shifted -= 1; — Eljay
– Eljay, Commented Sep 16 at 17:20
@Marek R Good point. The entire method, where I have now only fixed one broken case, comes from a former colleague, so I haven't given it any further thought until now. — SunBlack
– SunBlack, Commented Sep 16 at 21:45

Toby Speight · Accepted Answer · 2025-09-20 13:34:08Z

8

Sanitizers are your friends.

Here are Catch2 tests which reproduce your issue: https://godbolt.org/z/6jKE7Pqqv

Now after enabling Undefined Behavior sanitizers, Clang and GCC find integer overflow: https://godbolt.org/z/jWaxEh95K

Program stderr

/app/example.cpp:28:88: runtime error: signed integer overflow: 9223372036854775807 - -9223372036854775808 cannot be represented in type 'long int'

Program stderr

/app/example.cpp:28:88: runtime error: signed integer overflow: 9223372036854775807 - -9223372036854775808 cannot be represented in type 'int64_t' (aka 'long')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /app/example.cpp:28:88 in 
/app/example.cpp:28:88: runtime error: signed integer overflow: 9223372036854775807 - -9223372036854775808 cannot be represented in type 'int64_t' (aka 'long')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /app/example.cpp:28:88 in 
/app/example.cpp:28:88: runtime error: signed integer overflow: 9223372036854775807 - -9223372036854775808 cannot be represented in type 'int64_t' (aka 'long')
SUMMARY

So you have integer overflow here:

const Unsigned shifted = static_cast<Unsigned>(static_cast<std::int64_t>(value) - static_cast<std::int64_t>(min));

and this makes sense. If you are subtracting minimal value of integer type from maximum integer type which is signed, result is outside the representable range. You have to correct your code to prevent this integer overflow.

Anyway you made this a bit too complex; simple conversion to respective unsigned type just works. Here is my take:

template <std::floating_point TargetType, std::integral SourceType>
TargetType convert(SourceType value)
{
    static constexpr auto low = std::numeric_limits<SourceType>::min();
    std::make_unsigned_t<SourceType> tmp = value;
    tmp -= low;
    static constexpr auto tmpMax = std::numeric_limits<std::make_unsigned_t<SourceType>>::max();
    return static_cast<TargetType>(tmp) / static_cast<TargetType>(tmpMax);
}

https://godbolt.org/z/Y14Td3W33

edited Sep 20 at 13:34

Toby Speight

32.1k58 gold badges82 silver badges117 bronze badges

answered Sep 16 at 15:07

Marek R

40.3k6 gold badges70 silver badges165 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

SunBlack Sep 16 at 21:21

Thank you! It seems that I got a little blind while testing at some point, because for some reason I didn't want to take advantage of the integer conversion rules. Somehow, I had it in my head that I should avoid assigning negative numbers to an unsigned type (i.e., that what -Wsign-compare usually warns about can be utilized with a explicit cast here as wanted behavior). Seems our CI is now fine with all test cases :-).

Collectives™ on Stack Overflow

Normalize signed integer values to range [0.0..1.0]

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related