Home Original page

itertools.flatten()? and copying generators/iterators.

Francis Avila francisgavila at yahoo.com
Wed Oct 29 20:47:54 EST 2003 
"Peter Otten" <__peter__ at web.de> wrote in message
news:bnonsi$l5a$05$1 at news.t-online.com...
>
> def flatten_dict(s, isScalar):
>     try:
>         scalar = isScalar[s.__class__]
>     except KeyError:
...
>         t = s.__class__

Why doesn't the above fail (AttributeError) for classic classes?

...
>             scalar = isScalar[t] = True
>         else:
>             scalar = isScalar[t] = False

This part I never would have thought to do.  Clever!

...
> for flatten_dict(). So you get some speed up at the cost of uglier though
> still quite readable code and under the assumption that either all or no
> instances of a class are iterable.

That's not always a good assumption to make, if we want to decide whether to
iterate based upon some property of the object (which is actually how I
originally discovered I needed a "flatten").

Perhaps we can combine both conditions and caching optimizations?


def flatten_fastcond(s, isScalar, itercond=lambda o: None):
    try:
        scalar = isScalar[s.__class__]
    except KeyError:

        # If itercond has something to say about s, use *it*, and don't
        # add the lookup to isScalar.

        iterate = itercond(s)
        if not iterate is None:
            scalar = iterate
            if not scalar:
                s = iter(s)

        else:
            t = s.__class__
            try:
                s = iter(s)
            except TypeError:
                scalar = isScalar[t] = True
            else:
                scalar = isScalar[t] = False

    if scalar:
        yield s
    else:
        for elem in s:
            for subelem in flatten_dict(elem, isScalar):
                yield subelem


It's usually true that all instances of a given class are iterable or not,
and it's less inconvenient to add an entry to a dictionary than to write a
conditional test (also faster).  If we move the itercond block above 'try',
we'll slow down this more common case, but speed things up if s is a
homogenous structure where itercond does all the work of deciding.  There
are probably other things I could say against it if I thought about it, and
we're fast enough anyway.

As expected, given the same test tree (which doesn't exercise itercond), the
speed is about the same.

>python timeit.py -c -s"import flatten" "flatten.pro_fastcond()"
100 loops, best of 3: 6.29e+003 usec per loop

>python timeit.py -c -s"import flatten" "flatten.pro_dict()"
100 loops, best of 3: 6.29e+003 usec per loop

I'll make up some fancy tree later to test out itercond.
--
Francis Avila
More information about the Python-list mailing list