Integer overflow while malloc memory

Question

We have struct image pointer:

struct image {
uint16_t size_x;
uint16_t size_y;
struct pixel *px;
};

and:

  img->px = malloc(sizeof(struct pixel) * height * width);

where pixel:

struct pixel {
   uint8_t red;
   uint8_t green;
   uint8_t blue;
   uint8_t alpha;
   }

where width & height:

  long height = strtol(height_arg, &end_ptr, 10);
  long width = strtol(width_arg, &end_ptr, 10);

So, since we use malloc() to allocate memory and it uses size_t for allocating. Since we multiplying height and width which are long typed to allocate memory, is there integer overflow expected? If, yes then how to handle it?

Later on we iterate over picture and color it:

  for (int i = 0; i < img->size_y; i++) {
      for (int j = 0; j < img->size_x; j++) {
         image_data[i][j].red = palette[0].red;
         image_data[i][j].green = palette[0].green;
         image_data[i][j].blue = palette[0].blue;
         image_data[i][j].alpha = 0xff;
      }
  }

where

 img->size_x = width;
 img->size_y = height;

The sizeof operator returns a size_t type; if that is unsigned long long (it may be), then the multiplication will be performed as long long. — Adrian Mole
– Adrian Mole, Commented Apr 12, 2021 at 13:39
What possible values can you get for width and height? What are your requirements and limitations for the dimensions? — Some programmer dude
– Some programmer dude, Commented Apr 12, 2021 at 13:39
@Someprogrammerdude: Re “On computers all integer types are limited”: Not on my Turing machine. It just prints a message on the console asking the operator to insert another tape. — Eric Postpischil
– Eric Postpischil, Commented Apr 12, 2021 at 13:56

Adalcar · Accepted Answer · 2021-04-12 13:55:18Z

2

Integer overflow may happen at every point of this program:

strtol("9999999999", &end_ptr, 10) -> will overflow and return LONG_MAX
height * width -> can overflow and return a negative value (since those are signed longs)
sizeof(struct pixel) * height * width 
-> can return a bigger value than UNSIGNED_MAX, wrapping around to a smaller value than expected

Malloc can allocate some very large data segments, and may not fail immediately, so you should seriously consider verifying your results before allocating.

edited Apr 12, 2021 at 13:55

answered Apr 12, 2021 at 13:47

Adalcar

1,46812 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Eric Postpischil Over a year ago

Unless size_t is narrower than the size of height and width, sizeof(struct pixel) * height * width cannot overflow in the same way as height * width as sizeof is unsigned, so it differs in that the operation is defined to wrap (not overflow) and cannot return a negative value (since none is representable).

Adalcar Over a year ago

@EricPostpischil Which is why I mentioned "returning a smaller value than expected". Wrapping around is still considered an overflow, no? Sorry if it was unclear

Eric Postpischil Over a year ago

The C standard specifically uses “overflow” to mean the ideal result of an operation is not representable in the type, and it defines unsigned arithmetic to wrap, meaning the ideal result is the wrapped result. C 2018 6.2.5 9 explicitly states “A computation involving unsigned operands can never overflow,…”

Adalcar Over a year ago

Ah, my bad, fixed the answer, thanks for the lesson

klutt · Accepted Answer · 2021-04-12 14:11:12Z

2

Yes, there is a risk.

You can check it like this:

struct pixel *px;    

if (height > SIZE_MAX / width || sizeof *px  > SIZE_MAX / (width * height)) {
    // Handle overflow
} 

px = malloc(sizeof *px * width * height);

if(!px) {
    // Handle allocation error
}

img->px = px;

edited Apr 12, 2021 at 14:11

answered Apr 12, 2021 at 13:52

klutt

31.7k19 gold badges65 silver badges114 bronze badges

2 Comments

Emanuel P Over a year ago

You also need to factor in the sizeof(struct pointer)

Emanuel P Over a year ago

@MarkEzberg You should also consider that width and height can also be 0 or negative. Also, you won't want to allocate SIZE_MAX amount of bytes. The best solution is to just predefine and check for a reasonable limit to the size of an image, and this should be less than any potential overflow (which is technically a wrapping around of the value, not an overflow, if you want to be pedantic)

Collectives™ on Stack Overflow

Integer overflow while malloc memory

2 Answers 2

4 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related