Home Original page

[Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

Stefan Drees stefan at drees.name
Mon May 13 14:47:21 CEST 2013 
Hi Ben,

Am 13.05.13 14:25, schrieb Ben Hoyt:
> ...It's not yet production-ready, and is basically still in API and
> performance testing stage. ...
>
> In any case, I really like the API (thanks mostly to Nick Coghlan),
> and performance is great, even with DirEntry being written in Python.
>
> PERFORMANCE: On Windows I'm seeing that scandir.walk() on a large test
> tree (see benchmark.py) is 8-9 times faster than os.walk(), and on
> Linux it's 3-4 times faster. Yes, it is that much faster, and yes,
> those numbers are real. :-)
>
> Please critique away. At this stage it'd be most helpful to critique
> any API or performance-related issues ...

you asked for critique, but the performance seems to be also 2-3 times 
speedup (as stated by benchmark.py) on mac osx 10.8.3 (on MacBook Pro 13 
inch, start of 2011, solid state disk) with python 2.7.4 (the homebrew one):

$> git clone git://github.com/benhoyt/scandir.git
$> cd scandir && python setup.py install
$> python benchmark.py
USING FAST C version
Creating tree at benchtree: depth=4, num_dirs=5, num_files=50
Priming the system's cache...
Benchmarking walks on benchtree, repeat 1/3...
Benchmarking walks on benchtree, repeat 2/3...
Benchmarking walks on benchtree, repeat 3/3...
os.walk took 0.104s, scandir.walk took 0.031s -- 3.3x as fast

$> python benchmark.py -s
USING FAST C version
Priming the system's cache...
Benchmarking walks on benchtree, repeat 1/3...
Benchmarking walks on benchtree, repeat 2/3...
Benchmarking walks on benchtree, repeat 3/3...
os.walk size 226395000, scandir.walk size 226395000 -- equal
os.walk took 0.246s, scandir.walk took 0.125s -- 2.0x as fast

So for now, all well and thank you.

All the best,

Stefan.
More information about the Python-Dev mailing list