When using rglob() to iterate over the files of a directory containing a broken symlink (a link pointing to itself) rglob breaks with "[Errno 40] Too many levels of symbolic links" (OS: Linux).

Steps to reproduce:

mkdir tmp
touch foo
ls -s foo tmp/foo
cd tmp
file foo
foo: broken symbolic link to foo

python3
>>> from pathlib import Path
>>> for f in Path().rglob("*"):
        print(x)

foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/pathlib.py", line 1105, in rglob
    for p in selector.select_from(self):
  File "/usr/local/lib/python3.8/pathlib.py", line 552, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/usr/local/lib/python3.8/pathlib.py", line 536, in _iterate_directories
    entry_is_dir = entry.is_dir()
OSError: [Errno 40] Too many levels of symbolic links: './foo'


What is more, stat(), is_dir(), is_file() and exists() also do not like those broken links:
>>> Path("foo").is_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/pathlib.py", line 1361, in is_file
    return S_ISREG(self.stat().st_mode)
  File "/usr/local/lib/python3.8/pathlib.py", line 1151, in stat
    return self._accessor.stat(self)
OSError: [Errno 40] Too many levels of symbolic links: 'foo'


Is this intended behaviour or is this a bug? I guess it's not intended, since it makes it impossible to iterate over such a directory with rglob(). I could not find anything similar in the bug tracker, but https://bugs.python.org/issue26012 seems to be related.

Tested with Python 3.8.0a1, 3.6.7 and 3.5.2 (OS: Linux Mint 19)

I confirm this issue with python 3.7

but your script is wrong (you declare f and use x in your script)

/tmp$ mkdir demo
/tmp$ cd demo/
/t/demo$ mkdir tmp                                                                                                                                               /t/demo$ touch foo                                                                                                                                               /t/demo$ ln -s foo tmp/foo                                                                                                                                       /t/demo$ cd tmp/                                                                                                                                                 /t/d/tmp$ file foo                                                                                                                                               
foo: broken symbolic link to foo

/t/d/tmp$ python3                                                                                                                                                mar 19 fév 2019 13:20:15 CET
Python 3.7.2 (default, Jan 16 2019, 19:49:22) 
[GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> for p in Path().rglob('*'):
...     print(p)
... 
foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.7/pathlib.py", line 1105, in rglob
    for p in selector.select_from(self):
  File "/usr/lib64/python3.7/pathlib.py", line 552, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/usr/lib64/python3.7/pathlib.py", line 536, in _iterate_directories
    entry_is_dir = entry.is_dir()
OSError: [Errno 40] Too many levels of symbolic links: './foo'
>>>

A possible solution for python 3.7+ could be to add "ELOOP" to _IGNORED_ERROS in pathlib but I'm not exactly sure of the side effects.

3.5 is in security mode, we can remove 3.5 from the list for this issue.

I consider this an enhancement since you do have a loop in your symlinks and so having it not exactly work isn't totally shocking. But that doesn't mean that if someone can come up with a reasonable solution to fixing this annoyance it woudn't be accepted or appreciated!

And I don't know what a good solution would be. :) I.e. should some other exception be raised? Should it be ignored? I just don't know personally.

In Windows, the error for a path reparse (e.g. symlink or junction) that can't be resolved is ERROR_CANT_RESOLVE_FILENAME. Another common error is ERROR_INVALID_REPARSE_DATA. This can occur if the reparse data is malformed or if the target device is invalid for the reparse type. For example, a junction is restricted to [bind-]mounting local volumes, so it's invalid if it targets a remote device.

Windows errors to ignore should be added to _IGNORED_WINERRORS in Lib/pathlib.py. For example:

    _IGNORED_WINERRORS = (
        # ...
        1921, # ERROR_CANT_RESOLVE_FILENAME - similar to POSIX ELOOP
        4392, # ERROR_INVALID_REPARSE_DATA  - also for disallowed device targets
    )

Hello! 

I make the PR: https://github.com/python/cpython/pull/11964 

But I need help to test it :-(

Any could help me please?

I tried to add a test file in https://github.com/python/cpython/pull/11988 
To fix all now broken tests I had to add a try except block to the _WildcardSelector as well (analogous to the _RecursiveWildcardSelector).

I could only check on Linux and I have no idea how it behaves on any other OS.

As expected the Windows CI build failed.
All test fails were caused by:
OSError: [WinError 1921] The name of the file cannot be resolved by the system: 'C:\\projects\\cpython\\build\\test_python_936\\@test_936_tmp\\brokenLinkLoop'

Therefore, I added WinError 1921 to _IGNORED_WINERRORS as suggested by Eryk Sun.

New changeset d5c120f7eb6f2a9cdab282a5d588afed307a23df by Antoine Pitrou (Jörg Stucke) in branch 'master':
bpo-36035: fix Path.rglob for broken links (GH-11988)
https://github.com/python/cpython/commit/d5c120f7eb6f2a9cdab282a5d588afed307a23df

New changeset aea49b18752880e5d0260f16ca7ff2c6dce78515 by Miss Islington (bot) in branch '3.7':
[3.7] bpo-36035: fix Path.rglob for broken links (GH-11988) (GH-13469)
https://github.com/python/cpython/commit/aea49b18752880e5d0260f16ca7ff2c6dce78515

messages: + msg336150
nosy: + brett.cannon

messages: + msg335940
versions: + Python 3.7