Add a zarr writer that (in that order):

• Write the full resolution level by looking at the input image.
• Write the second level by looking at the first level pixels that were just written (the input image is not considered anymore).
• Write the third level by looking at the second level pixels that were just written.

This provides less flexibility than the existing OMEZarrWriter but is less likely to throw out of memory errors.

This PR only adds this new writer (PyramidalOMEZarrWriter). Other changes to existing classes were made to reuse as much code as possible between the two zarr writers.

I create this PR now to discuss about the public API (is only one writeImage() function enough?), but it's a draft because tests need to be added.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for me, but I find it a bit confusing that the API is quite different for OMEZarrWriter and PyramidalOMEZarrWriter, even when it seems like they should be doing something similar.

For example,

• OMEZarrWriter.Builder requires downsamples to be passed in a method, whereas PyramidalOMEZarrWriter requires them to be passed to the constructor; if they aren't needed at construction, then I think defaulting to a downsample of 1 would be fine.
• OMEZarrWriter.writeImage takes no arguments, uses a background thread and returns immediately, while PyramidalOMEZarrWriter requires a path and is blocking.

Adding example scripts to the PR showing how the writers should be used would be good. I'd expect the scripts to look almost identical, so that it's easy to switch from one implementation to the other. If that's not possible, we'll also need to explain that clearly on ReadTheDocs.

I've added some comments on specific things.

I addressed all comments. The public APIs of OMEZarrWriter and PyramidalOMEZarrWriter are now similar (this required some API changes on OMEZarrWriter). Here is a script on how to use them:

import qupath.lib.images.writers.ome.zarr.OMEZarrWriter
import qupath.lib.images.writers.ome.zarr.PyramidalOMEZarrWriter

var path = "/path/to/img.ome.zarr"
var server = getCurrentServer()

new OMEZarrWriter.Builder(server)   // or new PyramidalOMEZarrWriter.Builder(server)
    .downsamples(1, 4, 16)
    .build(path)
    .writeImage()

It is now possible to provide downsamples without including 1.

Thanks, I think there's a bug in PyramidalOMEZarrWriter (I couldn't replicate it with OMEZarrWriter) connected to tile caching.

I ran the following code for 2 different images (you'll recognize them...), deleting the zarr in between:

import qupath.lib.images.writers.ome.zarr.OMEZarrWriter
import qupath.lib.images.writers.ome.zarr.PyramidalOMEZarrWriter

var server = getCurrentServer()
var path = buildPathInProject("anything.ome.zarr")

new PyramidalOMEZarrWriter.Builder(server)
    .downsamples(16, 64)
    .build(path)
    .writeImage()

It produced an image like this:

I initially thought that it was using the full image width and height, but those do in fact seem correctly downsampled (7936 x 4608px) - and the image displays properly when zoomed in so that pixels are requested from the highest available resolution.

See also #2012 - it's a longstanding bug, and perhaps fixing it would be enough.

I fixed a bug occurring when the first requested downsample isn't one.

That doesn't fix your issue though. When I run your workflow, I also get a bit a CMU-1 in the output image. However, if I clear the tile cache between the two writes, then the output image is correct. So, I think this is clearly linked with #2012.

How do you think we should fix that? Automatically clear the cache if two consecutive writes to the same location are detected?

Please check out #2013 and see if that fixes the issue - I'm not sure how exactly the modified time behaves with directories.

If you're temporarily creating a BioFormatsImageServer, then I think it would make sense to clear the cache specifically for that when it's serves its purpose - although I'm not sure how easy it is to get access to the required objects to call that method.

I'll merge both then, thanks!