Cast int before potentially exceeding INT_MAX by radarhere · Pull Request #8402 · python-pillow/Pillow
Resolves #8401
tablen is an integer
and before we set it to im->bands * im->ysize, we check that the value will be less than INT_MAX
| if (im->xsize > INT_MAX / im->bands || im->ysize > INT_MAX / im->bands) { | |
| state->errcode = IMAGING_CODEC_MEMORY; | |
| return -1; |
but we then temporarily multiply it by 8, which might exceed INT_MAX.
| c->tablen = im->bands * im->ysize; | |
| /* below, we populate the starttab and lentab into the bufsize, | |
| each with 4 bytes per element of tablen | |
| Check here before we allocate any memory | |
| */ | |
| if (c->bufsize < 8 * c->tablen) { |
So this PR casts it to int64_t before applying the multiplication.