Add @yields_batches and @yields_elements by TheNeuralBit · Pull Request #19268 · apache/beam
This PR adds two new decorators, @yields_batches and @yields_elements, which override the default interpretation for the output of DoFn.process and DoFn.process_batch. These decorators are handled via changes to OutputProcessor (now renamed to OutputHandler for clarity), which branches depending on whether batches or elements are expected in the results iterable.
Where possible, common logic in OutputHandler was extracted into private helper methods that are re-used. The .pxd definition for these methods indicates they should be inlined in cython-generated code so they don't affect performance.
Fixes #21656
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.