Home Original page

[Python-Dev] Investigating Python memory footprint of one real Web application

Victor Stinner victor.stinner at gmail.com
Fri Jan 20 08:08:38 EST 2017 
Larry Hastings' Gilectomy also moved the reference counter into a
separated memory block, no? (grouping all refcounts into large memory
blocks if I understood correctly.)

https://github.com/larryhastings/gilectomy

Victor

2017-01-20 13:40 GMT+01:00 Christian Heimes <christian at python.org>:
> On 2017-01-20 13:15, INADA Naoki wrote:
>>>
>>> "this script counts static memory usage. It doesn’t care about dynamic
>>> memory usage of processing real request"
>>>
>>> You may be trying to optimize something which is only a very small
>>> fraction of your actual memory footprint.  That said, the marshal
>>> module could certainly try to intern some tuples and other immutable
>>> structures.
>>>
>>
>> Yes.  I hadn't think static memory footprint is so important.
>>
>> But Instagram tried to increase CoW efficiency of prefork application,
>> and got some success about memory usage and CPU throughput.
>> I surprised about it because prefork only shares static memory footprint.
>>
>> Maybe, sharing some tuples which code object has may increase cache efficiency.
>> I'll try run pyperformance with the marshal patch.
>
> IIRC Thomas Wouters (?) has been working on a patch to move the ref
> counter out of the PyObject struct and into a dedicated memory area. He
> proposed the idea to improve cache affinity, reduce cache evictions and
> to make CoW more efficient. Especially modern ccNUMA machines with
> multiple processors could benefit from the improvement, but also single
> processor/multi core machines.
>
> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
More information about the Python-Dev mailing list