Message 259546 - Python tracker

Message259546

Author	skrah
Recipients	brett.cannon, florin.papa, pitrou, serhiy.storchaka, skrah, vstinner, yselivanov, zbyrne
Date	2016-02-04.09:24:03
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1454577843.54.0.992494292733.issue26275@psf.upfronthosting.co.za>
In-reply-to

Content
In my experience it is very hard to get stable benchmark results with Python. Even long running benchmarks on an empty machine vary: wget http://www.bytereef.org/software/mpdecimal/benchmarks/telco.py wget http://speleotrove.com/decimal/expon180-1e6b.zip unzip expon180-1e6b.zip taskset -c 0 ./python telco.py full $ taskset -c 0 ./python telco.py full Control totals: Actual ['1004737.58', '57628.30', '25042.17'] Expected ['1004737.58', '57628.30', '25042.17'] Elapsed time: 7.16255 $ taskset -c 0 ./python telco.py full Control totals: Actual ['1004737.58', '57628.30', '25042.17'] Expected ['1004737.58', '57628.30', '25042.17'] Elapsed time: 6.982884 $ taskset -c 0 ./python telco.py full Control totals: Actual ['1004737.58', '57628.30', '25042.17'] Expected ['1004737.58', '57628.30', '25042.17'] Elapsed time: 7.095349000000001 $ taskset -c 0 ./python telco.py full Control totals: Actual ['1004737.58', '57628.30', '25042.17'] Expected ['1004737.58', '57628.30', '25042.17']

Content

In my experience it is very hard to get stable benchmark results with Python.  Even long running benchmarks on an empty machine vary:

wget http://www.bytereef.org/software/mpdecimal/benchmarks/telco.py
wget http://speleotrove.com/decimal/expon180-1e6b.zip
unzip expon180-1e6b.zip

taskset -c 0 ./python telco.py full


$ taskset -c 0 ./python telco.py full

Control totals:
Actual   ['1004737.58', '57628.30', '25042.17']
Expected ['1004737.58', '57628.30', '25042.17']
Elapsed time: 7.16255
$ taskset -c 0 ./python telco.py full

Control totals:
Actual   ['1004737.58', '57628.30', '25042.17']
Expected ['1004737.58', '57628.30', '25042.17']
Elapsed time: 6.982884
$ taskset -c 0 ./python telco.py full

Control totals:
Actual   ['1004737.58', '57628.30', '25042.17']
Expected ['1004737.58', '57628.30', '25042.17']
Elapsed time: 7.095349000000001
$ taskset -c 0 ./python telco.py full

Control totals:
Actual   ['1004737.58', '57628.30', '25042.17']
Expected ['1004737.58', '57628.30', '25042.17']

History
Date	User	Action	Args
2016-02-04 09:24:03	skrah	set	recipients: + skrah, brett.cannon, pitrou, vstinner, serhiy.storchaka, yselivanov, zbyrne, florin.papa
2016-02-04 09:24:03	skrah	set	messageid: <1454577843.54.0.992494292733.issue26275@psf.upfronthosting.co.za>
2016-02-04 09:24:03	skrah	link	issue26275 messages
2016-02-04 09:24:03	skrah	create