Home Original page

[Python-ideas] Are there asynchronous generators?

Paul Sokolovsky pmiscml at gmail.com
Mon Jun 29 08:44:39 CEST 2015 
Hello,

On Sun, 28 Jun 2015 18:14:20 -0400
Yury Selivanov <yselivanov.ml at gmail.com> wrote:

> 
> On 2015-06-28 11:52 AM, Paul Sokolovsky wrote:
> >> There is also the problem that one cannot easily feed a queue,
> >> >asynchronous generator, or any asynchronous iterator to a simple
> >> >synchronous consumer like sum() or list() or "".join(). It would
> >> >be nice if there was a way to wrap them to asynchronous ones when
> >> >needed – something like (async
> >> >sum)(asynchronously_produced_numbers()).
> > All that is easily achievable with classical Python coroutines, not
> > with asyncio garden variety of coroutines, which lately were casted
> > into a language level with async/await disablers:
> >
> > def coro1():
> >      yield 1
> >      yield 2
> >      yield 3
> >
> > def coro2():
> >      yield from coro1()
> >      yield 4
> >      yield 5
> >
> > print(sum(coro2()))
> 
> 
> You have easily achieved combining two generators with 'yield from'
> and feeding that to 'sum' builtin.

Right, the point here was that PEP492, banning usage of "yield" in
coroutines, doesn't help with such simple and basic usage of them. And
then I again can say what I said during initial discussion of PEP492:
I have dual feeling about it: promise of making coroutines easier and
more user friendly is worth all support, but step of limiting basic
language usage in them doesn't seem good. What me and other people can
do then is just trust that you guys know what you do and PEP492 will
be just first step. But bottom line is that I personally don't find
async/await worthy to use for now - it's better to stick to old good
yield from, until the promise of truly better coroutines is delivered.  

> There is no way to combine synchronous loops with asynchronous 
> coroutines; by definition, the entire process will block while you
> are iterating trough them.

Indeed, to solve this issue, it requires to use "inversion of inversion
of control" pattern. Typical real-world example is that someone has got
their (unwise) main loop and wants us to do callback mess programming
with it, but we don't want them to call us, we want to call them, at
controlled intervals, to do controlled amount of work.

The solution would be to pass a callback which looks like a
normal function, but which is actually a coroutine. Then foreign main
loop, calling it, would suspend it and pass control to "us", and us can
let another iteration of foreign main loop by resuming that coroutine.

The essence of this approach lies in having a coroutine "look like" a
usual function, or more exactly, in being able to resume a coroutine
from a context of normal function. And that's explicitly not what
Python coroutines are - they require lexical marking of each site where
coroutine suspension may happen (for good reasons which were described
here on the list many times).

During previous phase of discussion, I gave classification of
different types of coroutines to graps/structure all this stuff better:
http://code.activestate.com/lists/python-dev/136046/ 


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com
More information about the Python-ideas mailing list