Rewrite of Jpeg2k data handling by wiredfool · Pull Request #1934 · python-pillow/Pillow
Fixes the intermittent test failures and occasional dos/thread deadlock in the J2k stack by rewriting the data handling to not use threading.
Changes proposed in this pull request:
- Adds a new interface/mode to the ImageFile where the C layer can write to the python file like object.
- Rips out the threading in Incremental.c and converts J2k to use the new layer.
Rationale:
It is a relatively common pattern in image libraries to provide a set of read/seek/write functions to the underlying library so that the library can handle pulling/pushing data to a file like object. OpenJpeg does this for the J2k file format, and I worked up a way to do this in the tiff support as well. This additional interface in the C layer allows for a consistent implementation of this core bit of code.
The de/encoder needs to set pushes/pulls_fd =1 from within its initialization routine. If that flag is set, ImageFile.py will call setfd(self.fd) and then call the de/encoder routine. At that point, the de/encoder can point (or wrap) any read/seek/write functions at the _imaging_*_pyfd family of functions which call out to the python file like object. There will be one call to the de/encoder per tile, and the coder will handle any buffer sizes or shuffling within that call. This reduces the complexity of the decoder function, as there is no longer a requirement for multiple calls with additional data to be handled sequentially.
ToDo:
- There's still cleanup of warnings and such.
- Make sure we like the interface and that we're returning the right results from the write/seek/read functions.