test_io fails on an android emulator running an x86 system image at API level 21.

======================================================================
ERROR: test_large_file_ops (test.test_io.CIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 537, in test_large_file_ops
    self.large_file_ops(f)
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 330, in large_file_ops
    self.assertEqual(f.seek(self.LARGE), self.LARGE)
OverflowError: Python int too large to convert to C long

======================================================================
ERROR: test_large_file_ops (test.test_io.PyIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 537, in test_large_file_ops
    self.large_file_ops(f)
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_io.py", line 330, in large_file_ops
    self.assertEqual(f.seek(self.LARGE), self.LARGE)
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/_pyio.py", line 1632, in seek
    return os.lseek(self._fd, pos, whence)
OverflowError: Python int too large to convert to C long

----------------------------------------------------------------------
Ran 541 tests in 32.829s

FAILED (errors=2, skipped=8)
test test_io failed
test_io took 33 sec
1 test failed:
    test_io
Total duration: 0:00:33

This looks similar to issue11184.

The relevant part of the output of configure on x86:

checking size of int... 4
checking size of long... 4
checking size of void *... 4
checking size of short... 2
checking size of float... 4
checking size of double... 8
checking size of fpos_t... 4
checking size of size_t... 4
checking size of pid_t... 4
checking for long long support... yes
checking size of long long... 8
checking for long double support... yes
checking size of long double... 8
checking for _Bool support... yes
checking size of _Bool... 1
checking for uintptr_t... yes
checking size of uintptr_t... 4
checking size of off_t... 4
checking whether to enable large file support... no

And as a consequence in pyconfig.h:
/* Defined to enable large file support when an off_t is bigger than a long
   and long long is available and at least as big as an off_t. You may need to
   add some flags for configuration and compilation to enable this mode. (For
   Solaris and Linux, the necessary defines are already defined.) */
/* #undef HAVE_LARGEFILE_SUPPORT */

On android x86:

>>> import resource
>>> resource.setrlimit(resource.RLIMIT_FSIZE, (2**31, 2**31))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long
>>> resource.setrlimit(resource.RLIMIT_FSIZE, (2**31-1, 2**31-1))
>>> resource.getrlimit(resource.RLIMIT_FSIZE)
(2147483647, 2147483647)
>>> 


The test_io tests that fail may be skipped by trying first to set this resource limit (if existing) to the max file size tested, in a spawned interpreter.

Here is a patch that uses resource.setrlimit() to check whether large file may not be supported.

The resource module is not always available. And resource.setrlimit() itself can fail.

May be check HAVE_LARGEFILE_SUPPORT directly using sysconfig.get_config_vars()?

test_mmap needs similar fix.

Skipping test is the one way.

Other way is to enable large files support on Android (like it was done in issue11184 for AIX). It is hard to believe that Android's libc doesn't support files larger 2GiB. Should be an option to enable this support in a program.

Using the glibc compilation flags recommended for large file support '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1' does not enable large file support. See Suse's 'Large File Support in Linux' document [1].

In the android issue 64613 [2] one of the last comment said, only few days ago,  "fixed in the platform. (partially in L, mostly in M, completely in N.)", L standing for Lollipop (API 21 or 22) and M for Marshmallow (API 23). So it seems these tests should be skipped until platform N is available.

[1] http://users.suse.com/~aj/linux_lfs.html
[2] https://code.google.com/p/android/issues/detail?id=64613

Size of off_t is also 4 when building on API 23 with or without
'-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1'
added to the CC envt variable.

This patch skips the large file support test.

The tests are run on an ext4 file system.
test_large_file_ops does not fail on the x86_64 and arm64 (aka aarch64) Android 64 bits platforms.
test_large_file_ops still fails on x86 with the latest Android API level 24 (i.e. the latest released libc).

FWIW, the second item in https://android.googlesource.com/platform/bionic.git/#32_bit-ABI-bugs may explain why there is still no large file support on Android 32 bits platforms.
HAVE_LARGEFILE_SUPPORT cannot be used to skip the test since it is also undefined on the Android 64 bits platforms and on linux x86_64.
This new patch is not specific to Android and uses the same method as the one used in test_mmap to skip the test on platforms that do not support large files.

New changeset d616304b82aa by Xavier de Gaye in branch '3.6':
Issue #26926: Skip some test_io tests on platforms without large file support
https://hg.python.org/cpython/rev/d616304b82aa

New changeset 878f91b4ad19 by Xavier de Gaye in branch 'default':
Issue #26926: Merge 3.6
https://hg.python.org/cpython/rev/878f91b4ad19

dependencies: - add the 'is_android' attribute to test.support
assignee: xdegaye
components: + Tests, - Library (Lib), Cross-Build
title: Large files are not supported on Android -> test_io large file test failure on 32 bits Android platforms
versions: + Python 3.7
messages: + msg280053
stage: patch review

dependencies: + add the 'is_android' attribute to test.support
messages: + msg266064