I'm using 1/10 version of dump.py
I removed total_size() because it creates some garbages.
sys._debugmallocstats() after load.py:
master:
# arenas allocated total = 1,223
# arenas reclaimed = 5
# arenas highwater mark = 1,218
# arenas allocated current = 1,218
1218 arenas * 262144 bytes/arena = 319,291,392
# bytes in allocated blocks = 218,026,128
# bytes in available blocks = 149,024
23835 unused pools * 4096 bytes = 97,628,160
PR 13036:
# arenas allocated total = 849
# arenas reclaimed = 3
# arenas highwater mark = 846
# arenas allocated current = 846
846 arenas * 262144 bytes/arena = 221,773,824
# bytes in allocated blocks = 217,897,968
# bytes in available blocks = 140,096
61 unused pools * 4096 bytes = 249,856
Now "arena allocated current" is same to after dump.py:
# arenas allocated total = 847
# arenas reclaimed = 1
# arenas highwater mark = 846
# arenas allocated current = 846
846 arenas * 262144 bytes/arena = 221,773,824
# bytes in allocated blocks = 217,998,792
# bytes in available blocks = 131,112
38 unused pools * 4096 bytes = 155,648
It looks nice. Additionally, both of "time python dump.py"
and "time python load.py" become slightly faster.
master dump:
dump (Note that this time includes not only dump, but also constructing data)
real 0m3.539s
user 0m3.266s
sys 0m0.196s
master load:
real 0m1.408s
user 0m1.292s
sys 0m0.116s
PR-13036 dump:
real 0m2.758s
user 0m2.598s
sys 0m0.088s
PR-13036 load:
real 0m1.239s
user 0m1.183s
sys 0m0.056s
Would pickle experts review the PR? |