Fixes problems with large snapshots by alpapad · Pull Request #411 · apache/ratis
What changes were proposed in this pull request?
Hi,
I have been testing ratis for a solutions which uses rather largish snapshots for the state machine. Current implementation is not able to transfer files larger than 8Mb (I think), throwing all kinds of exceptions. I guess this is something not really used by current users of ratis.
Here are the places I modified/found to be problematic (code is more precise than words):
FileChunkReader.readFileChunk uses ByteString.readFrom(in, chunkLength); which will fully load the input stream and not just a junk of size chunkLength
Grpc.proto service installSnapshot(stream ratis.common.InstallSnapshotRequestProto) should return a stream
Modifications to SnapshotManager so chunks for the same snapshot will be placed in the same directory
Not sure if I need to create a jira issue (never used apache f. jira) I can repro and create jiras with stacktraces etc.