Home Original page

[Python-Dev] Weak dict iterators are fragile

Daniel Stutzbach daniel at stutzbachenterprises.com
Sun Oct 11 18:22:41 CEST 2009 
On Sun, Oct 11, 2009 at 10:50 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> In py3k, the weak dict methods keys(), values() and items() have been
> changed to return iterators (they returned lists in 2.x).
> However, it turns out that it makes these methods quite fragile, because
> a GC collection can occur whenever during iterating, destroy one of the
> weakref'ed objects, and trigger a resizing of the underlying dict, which
> in turn raises an exception ("RuntimeError: dictionary changed size
> during iteration").
>

Ouch!

The iterator from __iter__ is also affected.

1. Add the safe methods listkeys(), listitems(), listvalues() which would
> behave as the keys(), etc. methods from 2.x
>
> 2. Make it so that keys(), items(), values() atomically build a list of
> items internally, which makes them more costly for large weak dicts, but
> robust.
>

-1 on 1.
+0 on 2.

It'd be nice if we could postpone the resize if there are active iterators,
but I don't think there's a clean way to track the iterators.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091011/d674ae2a/attachment.htm>
More information about the Python-Dev mailing list