build fails with:
./python -E ../../Python-3.7.3/setup.py  build
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ gmake
xlc_r -c   -DNDEBUG -O -q64 -qlanglvl=extc99     -IObjects -IInclude -IPython -I. -I../../Python-3.7.3/Include -I/raid/checkouts-raid/robb/Python-2.7.15/Modules/zlib   -DPy_BUILD_CORE -o Modules/_math.o ../../Python-3.7.3/Modules/_math.c
LIBPATH=/raid/checkouts-raid/robb/nepal/build-py37 CC='xlc_r' LDSHARED='Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/raid/checkouts-raid/robb/zlib-dl/Release/rs6000aix_64/lib  -Wl,-blibpath:/opt/IBM/xlmass/8.1.3/lib/aix61:/opt/IBM/xlc/13.1.3/lib:/usr/lib:/lib:/opt/Python-3.7/lib -q64   ' OPT='-DNDEBUG -O'         _TCLTK_INCLUDES='' _TCLTK_LIBS=''       ./python -E ../../Python-3.7.3/setup.py  build
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

Makefile:626: recipe for target 'sharedmods' failed
gmake: *** [sharedmods] Error 1

Could you please use a debugger and step through _Py_HashRandomization_Init and pyurandom to see, where the initialization of the RNG is failing?

Try to compress config.log to attach it. Or at least attach the output of "./configure" as a file.

I'm looking for HAVE_GETRANDOM, HAVE_GETRANDOM_SYSCALL, HAVE_GETENTROPY defines that you can find in pyconfig.h.

About /dev/urandom: does this device exist? Is your user allowed to read from it? For example, run "dd if=/dev/urandom of=random bs=1 count=1" command: does it fail?

from pyconfig.h:


/* Define to 1 if the getrandom() function is available */
/* #undef HAVE_GETRANDOM */

/* Define to 1 if the Linux getrandom() syscall is available */
/* #undef HAVE_GETRANDOM_SYSCALL */

/* Define to 1 if you have the <linux/random.h> header file. */
/* #undef HAVE_LINUX_RANDOM_H */

/* Define to 1 if you have the `getentropy' function. */
/* #undef HAVE_GETENTROPY */

Ok, so Python uses /dev/urandom. Can you try to read a few bytes from it? Like 256 bytes. You can try my dd command.

Opening /dev/urandom seems to return -1

(dbx) print buffer
0x09001000a5f49380 
(dbx) print size
24 
(dbx) print raise
0 
(dbx) step
stopped in dev_urandom at line 311 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  311       if (raise) {
(dbx) step
stopped in dev_urandom at line 378 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  378           fd = _Py_open_noraise("/dev/urandom", O_RDONLY);
(dbx) next
stopped in dev_urandom at line 379 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  379           if (fd < 0) {
(dbx) print fd
-1 
(dbx) step
stopped in dev_urandom at line 380 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  380               return -1;
(dbx) step
stopped in dev_urandom at line 401 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  401   }
(dbx) step
stopped in pyurandom at line 519 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  519   }
(dbx) step
stopped in unnamed block in _Py_HashRandomization_Init at line 611 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  611           if (res < 0) {
(dbx) step
stopped in unnamed block in _Py_HashRandomization_Init at line 612 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
  612               return _Py_INIT_USER_ERR("failed to get random numbers "
(dbx) continue
continue
^ unrecognized command
(dbx) cont
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

The call to open("/dev/urandom", flags) is returning -1, and errno is set to 22, EINVAL - Invalid argument.  could the flags be set incorrectly?

Reading a few bytes from /dev/urandom via dd:


robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1
??S?(#L???????????~]?B?^??8?f&?_|Vi??@??[joG>St??;?$?1?*??24???RD?"4`??aҼ#???f???
?}                                                                               ??^z????????C?@?"?.^?gn??p?t?R*=Ӫ?8*?L??   8??bu?߂???8<d?/yd٠u0$9L_?g?e?P2s?Թ??4<?qF??0'????!
  ?c????%Z?;Pɟo=CIb?1+0 records in.
1+0 records out.
robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1
)|???Cnc9UR???r?(?@w???}?1??I~b?b??CmN?1(??6N?U?(??n???:I@;??@/
N??3w?˫`u??r>?Ŵ?`??y??٫K?.?~陵#5?t??h?e!?G@)B?R???(9kL????X??BUH5=?||??El???a????+????n@??V?c?N?^ͯ?y?n??]??$݇?Mp?,#???K?i8?`?????v??????~`L?1+0 records in.
1+0 records out.
robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1
_d?(?n???1bRԺ{??I??"Ѱ"|???E
                           p??^??q#??
~yZ$u??e}?3?????????qw#?????{*T1+0 records in.nR)?-?B???x??{~hϼ????+?7۪شV????W??"?d?n??V??t??R(\YC?3=?p?-ӽ?m??P???A?M??68??????9Y?](??xn+?a?ܯ?????????W
1+0 records out.
robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$

Robert Boehne: pyurandom() uses _Py_open_noraise("/dev/urandom", O_RDONLY) which uses O_CLOEXEC if available. If this flag available? Does it work?

Please try to build attached urandom.c.

Example on my Fedora 29:

open O_RDONLY succeeded
read(16) -> 16
open O_RDONLY | O_CLOEXEC succeeded
read(16) -> 16

It doesn't look good:

robb@nepal:/raid/checkouts-raid/robb/nepal$ xlc_r -q64 -O0 -g
-qlanglvl=extc1x -o urandom urandom.c

robb@nepal:/raid/checkouts-raid/robb/nepal$ ./urandom

open O_RDONLY failed

open O_RDONLY | O_CLOEXEC failed

robb@nepal:/raid/checkouts-raid/robb/nepal$ uname -a

AIX nepal 1 7 00FA7FB84C00

robb@nepal:/raid/checkouts-raid/robb/nepal$

On Thu, May 9, 2019 at 6:21 PM STINNER Victor <report@bugs.python.org>
wrote:

>
> STINNER Victor <vstinner@redhat.com> added the comment:
>
> Robert Boehne: pyurandom() uses _Py_open_noraise("/dev/urandom", O_RDONLY)
> which uses O_CLOEXEC if available. If this flag available? Does it work?
>
> Please try to build attached urandom.c.
>
> Example on my Fedora 29:
>
> open O_RDONLY succeeded
> read(16) -> 16
> open O_RDONLY | O_CLOEXEC succeeded
> read(16) -> 16
>
> ----------
> Added file: https://bugs.python.org/file48322/urandom.c
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue36843>
> _______________________________________
>

I wonder if there's anyone with AIX 7 who can attempt to reproduce this.  We have another AIX machine, but it is down for the moment.  I would like to eliminate a problem on this machine as the cause.

> open O_RDONLY failed

Ah. That sounds like an issue on your machine or specific to AIX. I don't see what Python can do to support a platform where /dev/urandom doesn't work. Python really needs /dev/urandom at startup to initialize its "hash secret" to reduce the risk of DoS attack attack against dict.

https://python-security.readthedocs.io/vuln/cve-2012-1150_hash_dos.html

Maybe it's a permission issue. Maybe a libc issue. I don't know. But I suggest to close the issue and try to find help from AIX instead.

I close the issue. Maybe contact Michael Felt to get help to debug your issue.

FYI: I was contacted this week by someone with this problem. 

The problem was resolved after they updated AIX (was 7100-04-00-0000).

Please note: any oslevel -s reporting six zeros at the end needs the SP that is released in parallel with the base.

stage: resolved

messages: + msg342009