Home Original page

[Python-Dev] [Python-checkins] cpython: Issue #14716: Change integer overflow check in unicode_writer_prepare()

Mark Dickinson dickinsm at gmail.com
Mon May 7 13:35:27 CEST 2012 
On Mon, May 7, 2012 at 12:08 PM, victor.stinner
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/ab500b297900
> changeset:   76821:ab500b297900
> user:        Victor Stinner <victor.stinner at gmail.com>
> date:        Mon May 07 13:02:44 2012 +0200
> summary:
>  Issue #14716: Change integer overflow check in unicode_writer_prepare()
> to compute the limit at compile time instead of runtime. Patch writen by Serhiy
> Storchaka.

>     if (newlen > PyUnicode_GET_LENGTH(writer->buffer)) {
> -        /* overallocate 25% to limit the number of resize */
> -        if (newlen <= (PY_SSIZE_T_MAX - newlen / 4))
> +        /* Overallocate 25% to limit the number of resize.
> +           Check for integer overflow:
> +           (newlen + newlen / 4) <= PY_SSIZE_T_MAX */
> +        if (newlen <= (PY_SSIZE_T_MAX - PY_SSIZE_T_MAX / 5))
>             newlen += newlen / 4;

Hmm.  Very clever, but it's not obvious that that overflow check is
mathematically sound.  As it turns out, the maths works provided that
PY_SSIZE_T_MAX isn't congruent to 4 modulo 5;  since PY_SSIZE_T_MAX
will almost always be one less than a power of 2 and powers of 2 are
always congruent to 1, 2 or 4 modulo 5, we're safe.

Is the gain from this kind of micro-optimization really worth the cost
of replacing obviously correct code with code whose correctness needs
several minutes of thought?

Mark
More information about the Python-Dev mailing list