Why doesn't my cast from enum to int work in release builds?

I have discovered that the following code works correctly in a debug build, but incorrectly in release:

enum FileFormatDetection
{
    Binary,
    ASCII,
    Auto
};

FileFormatDetection detection_mode = (FileFormatDetection)ReadIntKey(NULL, KEY_ALL, "FIL_ASCIIORBINARY", -1); // Assigns -1, as confirmed by debug output
if (detection_mode < 0 || detection_mode > 2)
{
    // Not a valid value, so set to default:
    detection_mode = Auto;
}

In each build, debugging output confirmed that the value was -1. In a debug build, the value -1 caused the if-branch to be entered; in release, it didn't.

I have tried casting detection_mode to an int as follows:

if ((int)detection_mode < 0 || (int)detection_mode > 2)

and:

if (int(detection_mode) < 0 || int(detection_mode) > 2)

but neither makes any difference.

The only way I could make this work was to cast the enum variable to an integer stack-variable and test that:

int detection_int = (int)detection_mode;
if (detection_int < 0 || detection_int > 2)
{
    ...

Now the if-branch is entered, as expected.

I can't understand why this was necessary - I still think the original code (or at least testing the cast temporaries) should have worked. Can anyone explain why it didn't?

Answers


The problem is that the cast is causing undefined behavior as you are assigning a value that is not one of the enumerators to the enumeration.

When you compile in release mode, the optimizer is seeing the expression:

if (detection_mode < 0 || detection_mode > 2)

and knows that the type of detection_mode is FileFormatDetection, it knows that all valid values of that particular enumeration are in range, so the if statement can never be true (at least in a well defined program) so it removes the if altogether.

Casting to int in the if will not help, because the same line of reasoning still applies: the value before the cast (again in a well defined program) is in the range [0..2] so the optimizer removes the if. It is important to note that whether the cast is explicit or not does not really matter, as the enum will be converted to int for the comparison in both cases.

If you store the value in an int then everything changes. There are quite a few int values that fall outside of that range, so the comparison must be performed. Also note that with this change, the program becomes well defined, since -1 is a valid int. This is actually what you should be doing: get the value as an int, and only if it can be converted to the enum, perform the conversion.


The variable detection_mode has type FileFormatDetection, so it can only (legally) contain values in the range 0..2. Anything else is undefined behavior. In your case, apparently, the optmizer is recognizing this, and suppressing the test for less than zero.

What you should be doing is reading an int, and only converting to FileFormatDetection after the range check:

int raw_detection_mode = ReadIntKey(...);
if ( raw_detection_mode <= 0 || raw_detection_mode > 2 ) {
    raw_detection_mode = Auto;
}
FileFormatDetection detection_mode =
    static_cast<FileFormatDetection>( raw_detection_mode );

Need Your Help

Exif-read-data, looping thru arrays/section - getting result

php function foreach exif

I'm trying to extract the exif information from an image with this foreach-loop and printing out for example the camera maker:

Can I ignore or suppress warnings in JDBC for MySQL?

mysql jdbc mysql-error-1364

I have an INSERT INTO ... ON DUPLICATE KEY UPDATE ... statement that executes fine (but with warnings) in the mysql&gt; prompt: