> IIRC Nick Coghlan had put a bit of work into this a few months ago as an
> external module with a view to seeing if it got traction before putting
> anything into the stdlib. Might be worth pinging him, or looking to see
> what he'd done. Can't remember the keywords to search for, I'm afraid.
> Something like "directory walker"

Nick's walkdir is "just" an improved walk - with a lot of added
functionality, like filtering etc.
But it's still based on os.walk(), which in turn uses listdir(), since
it's currently the only way to list the content of a directory. So in
short, it won't help for this issue.

To limit the memory usage, on would have to have to use an iterator
based on readdir().