Flush measurement writer after each row by alanocallaghan · Pull Request #2065 · qupath/qupath
The one I knew of was #1862 which has been addressed.
Albeit not perfectly addressed. I believe createMeasurementTable will still generate a List<String[]> where the array contains the values for each row that should be added to the file. That should be a big improvement on previously building lots of Map<String, String>, but could still be too expensive - I just hadn't personally encountered a time when it was too expensive yet.
This could certainly be improved, but it would involve dealing with / replacing a lot of legacy code, so we'd need to know where exactly the problem lurks.
One idea with minimal effort: instead of storing a row as String[] we could switch to Object[] and store either a String or Double - formatting only as the file is written. That could probably cut down the bytes quite a lot (especially if using a large number of decimal places), but would still be more expensive than figuring out another way to extract any required double values and keeping them as primitives.
Our core problem is that we don't know when opening an image will give us a whole new set of measurements that we hadn't budgeted for with the columns we created, so we build a mega-table (but not a mega string, to avoid needing a huge amount of contiguous memory). But if we accept the cost of looping through the images twice then we could probably write in a far more efficient way. We just need to know it's worth the effort.