Home Original page

[Python-Dev] os.walk() is going to be *fast* with scandir

Akira Li 4kir4.1i at gmail.com
Mon Aug 11 17:26:29 CEST 2014 
Armin Rigo <arigo at tunes.org> writes:

> On 10 August 2014 08:11, Larry Hastings <larry at hastings.org> wrote:
>>> A small tip from my bzr days - cd into the directory before scanning it
>>
>> I doubt that's permissible for a library function like os.scandir().
>
> Indeed, chdir() is notably not compatible with multithreading.  There
> would be a non-portable but clean way to do that: the functions
> openat() and fstatat().  They only exist on relatively modern Linuxes,
> though.

There is os.fwalk() that could be both safer and faster than
os.walk(). It yields rootdir fd that can be used by functions that
support dir_fd parameter, see os.supports_dir_fd set. They use *at()
functions under the hood.

os.fwalk() could be implemented in terms of os.scandir() if the latter
would support fd parameter like os.listdir() does (be in os.supports_fd
set (note: it is different from os.supports_dir_fd)).

Victor Stinner suggested [1] to allow scandir(fd) but I don't see it
being mentioned in the pep 471 [2]: it neither supports nor rejects the
idea.

[1] https://mail.python.org/pipermail/python-dev/2014-July/135283.html
[2] http://legacy.python.org/dev/peps/pep-0471/


--
Akira
More information about the Python-Dev mailing list