Hi Brian, thanks for your feedback!

We have made an updated PR and include a few comments below:

1. The previous PR was focused on speed performance, rather than RAM utilization. However, our updated PR should match the RAM usage of the current implementation, while drastically improving speed and interoperability.

Below is an example comparison of performance:

Data: 10 MB finalized MF4 file

Close mimic of current implementation:

• Speed: 128.7 s
• max RAM: 14 MB

Previous PR:

• Speed: 1.3 s
• max RAM: 128 MB

Updated PR (using 1000 row chunk size):

• Speed: 7.6 s
• max RAM: 14 MB

In other words, our proposed alternative PR would yield the same RAM usage, while improving the readout speed by 20x+. If you believe the stats above and the proposed alternative PR would be acceptable, we can update the PR accordingly.

2. For the other comments, we have tried to address these in the updated PR