Attached bug.py does crash *randomly* on AArch64. The code is extract from ctypes.test.test_as_parameter.AsParamPropertyWrapperTestCase.test_callbacks test.

Example with Python 2.7.15 and Python 3.6.8 on RHEL8:

# python2 bug.py
Illegal instruction (core dumped)
[root@cav-thunderx2s-cn88xx-01 ~]# python3 bug.py
...
OK
[root@cav-thunderx2s-cn88xx-01 ~]# python3 bug.py
Illegal instruction (core dumped)

I can reproduce the crash on Python 2.7.16rc compiled manually:
./configure --enable-unicode=ucs4 --with-system-ffi && make

RHEL8 currently uses libffi-3.1-18.el8.aarch64.

(I tried optimization levels -O0, -O1, -O2, -O3: I am always able to *randomly* trigger the crash.)

Original bug report, Python 2 crash on RHEL8:
https://bugzilla.redhat.com/show_bug.cgi?id=1652930

--

I don't know if it's related but I also saw the following error which has been reported in bpo-30991.

FAIL: test_pass_by_value (ctypes.test.test_structures.StructureTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/src/python-3.6.2/Lib/ctypes/test/test_structures.py", line 416, in test_pass_by_value
    self.assertEqual(s.first, 0xdeadbeef)
AssertionError: 195948557 != 3735928559

> I can reproduce the crash on Python 2.7.16rc compiled manually:
> ./configure --enable-unicode=ucs4 --with-system-ffi && make

I can also reproduce the crash on Python 3.7.2 compiled manually with:
./configure --with-pydebug && make
(_ctypes uses system libffi)

bug2.py: example simplified a little bit more. Changes:

* unittest is no longer used
* move local variables to globals, to make sure that memory is not released in any way during function calls

Sadly, even with debug symbols on RHEL8 and gcc -g -O0 -fno-omit-frame-pointer, gdb is unable to retrieve the backtrace when the crash occurs :-(

Program received signal SIGILL, Illegal instruction.
0x0000ffffbe7a0058 in ?? ()
(gdb) where
#0  0x0000ffffbe7a0058 in ?? ()
#1  0x0000ffffffffe8a0 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

# gdb -args ./python ../bug2.py 
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-5.el8

(gdb) run
...

Program received signal SIGILL, Illegal instruction.
0x0000ffffbded0058 in ?? ()

(gdb) disassemble $pc,$pc+10
Dump of assembler code from 0xffffbded0058 to 0xffffbded0062:
=> 0x0000ffffbded0058:  ldr     x16, 0xffffbded0064
   0x0000ffffbded005c:  adr     x17, 0xffffbded006c
   0x0000ffffbded0060:  br      x16
End of assembler dump.

(gdb) print *(void**)0xffffbded0064
$1 = (void *) 0xffffbdea4f64 <ffi_closure_SYSV>


According to /proc/95475/maps, 0xffffbded0064 lives in this memory mapping:

ffffbded0000-ffffbdee0000 r-xs 00000000 fd:00 34943874   /tmp/ffiQhRWVt (deleted)

We believe this is a libffi bug.  On certain aarch64 implementations, it is necessary to flush both the writable mapping and the executable mapping.  I have a patch which I confirmed to work.

Filed as an libffi issue: https://github.com/libffi/libffi/issues/470

Florian Weimer wrote a fix for libffi, so the bug is in libffi and not Python/ctypes. I close the issue. There are enough open issues to track this bug :-)
https://github.com/libffi/libffi/issues/470

resolution: third party
stage: resolved

messages: + msg335851