Home Original page

[Python-Dev] The untuned tunable parameter ARENA_SIZE

Victor Stinner victor.stinner at gmail.com
Thu Jun 1 04:37:01 EDT 2017 
2017-06-01 10:23 GMT+02:00 INADA Naoki <songofacandy at gmail.com>:
> AFAIK, allocating arena doesn't eat real (physical) memory.
>
> * On Windows, VirtualAlloc is used for arena.  Real memory page is assigned
>   when the page is used first time.
> * On Linux and some other *nix, anonymous mmap is used.  Real page is
>   assigned when first touch, like Windows.

Memory fragmentation is also a real problem in pymalloc. I don't think
that pymalloc is designed to reduce the memory fragmentation.

I know one worst case: the Python parser which allocates small objects
which will be freed when the parser completes, while other objects
living longer are created.
https://github.com/haypo/misc/blob/master/memory/python_memleak.py

In a perfect world, the parser should use a different memory allocator
for that. But currently, the Python API doesn't offer this level of
granularity.

Victor
More information about the Python-Dev mailing list