The strxfrm function in the locale module can potentially raise a
MemoryError.

The failing malloc is in Modules/_localemodule.c, line 291.

This is because the variable n0 of type Py_ssize_t is passed to
PyArg_ParseTuple, which expects an int when PY_SSIZE_T_CLEAN is not defined.

Patch attached, which also fixes an unrelated memory leak.

Could someone triage this?  And review my simple fix?  I think it should
be a release blocker.

It seems to me to be a problem on all 64-bit non-Windows platforms, or
any platform if using UCS-4.  The 4 high-order bytes of an 8-byte
Py_ssize_t variable are not initialized.

Thanks.

Nice catch. I wonder why not all of our modules are PY_SSIZE_T_CLEAN in
py3k.
This function is not actually tested, could you add a test to the
regression suite?

All right.

The function strcoll also isn't tested (except for issue #3303).  I'll
look into that one, too.

I've added the tests.

I found that on Windows, strxfrm has been unavailable in py3k since
r61306, because of an omission in PC/pyconfig.h.  (cf. patch)

In determining whether I should first test for the existence of strcoll
(or strxfrm) in the locale module, as was done in TestMiscellaneous, I
found this is no longer needed in py3k since r61339.  (cf. patch)  (In
python2 such checks are still necessary.)

The tests in TestEnUSCollation I added don't work for all encodings
(e.g. Asian ones, or ASCII).  Now checked for encodings which are known
to work.

Found and fixed a bug (result not returned) in getpreferredencoding in
Lib/locale.py.

This test is skipped on Mac, since r30377.  I've tested it on 10.3
(PPC), and it passed, so I enabled it again.

I did, however, find that FreeBSD and Mac OS X have known bugs in
wcscoll and wcsxfrm: http://www.freebsd.org/cgi/man.cgi?query=wcsxfrm
(cf. patch)

Nice work, thanks!
One detail: when giving you credit, who should I mention? just "egreen"?

I've entered my real name.

Ok, thanks.

The patch is now committed to py3k (r75539) and 3.1 (r75541), closing.

stage: resolved

messages: + msg94242

messages: + msg94186