Home Original page

[Python-Dev] The untuned tunable parameter ARENA_SIZE

Victor Stinner victor.stinner at gmail.com
Thu Jun 1 04:34:23 EDT 2017 
2017-06-01 10:19 GMT+02:00 Antoine Pitrou <solipsis at pitrou.net>:
> Yes, this is the same kind of reason the default page size is still 4KB
> on many platforms today, despite typical memory size having grown by a
> huge amount.  Apart from the cost of fragmentation as you mentioned,
> another issue is when many small Python processes are running on a
> machine: a 2MB overhead per process can compound to large numbers if
> you have many (e.g. hundreds) such processes.
>
> I would suggest we exert caution here.  Small benchmarks generally have
> a nice memory behaviour: not only they do not allocate a lot of memory a,
> but often they will release it all at once after a single run.  Perhaps
> some of those benchmarks would even be better off if we allocated 64MB
> up front and never released it :-)

By the way, the benchmark suite performance supports different ways to
trace memory usage:

* using tracemalloc
* using /proc/pid/smaps
* using VmPeak of /proc/pid/status (max RSS memory)

I wrote the code but I didn't try it yet :-) Maybe we should check the
memory usage before deciding to change the arena size?

Victor
More information about the Python-Dev mailing list