local: add configurable I/O block size for reads and writes by paf0186 · Pull Request #9285 · rclone/rclone
5 tasks
The write path was allocating a fresh buffer with make() for every file written, then relying on GC to clean it up. The read path already uses a buffer pool (via AsyncReader) for reuse and memory tracking. Use pool.Global() for write buffers too, so they are reused across transfers and tracked by the memory limiter. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Add --local-io-size (default 1M), --local-read-io-size, and --local-write-io-size flags to control the I/O block size used by the local backend. Values must be 4K-aligned. The write path uses pool.GlobalWithSize() to get a properly sized buffer from a per-size pool, so custom sizes still get buffer reuse and memory tracking. The pool gains GlobalWithSize() which returns the default global pool for the default 1MB size, or creates and caches a new pool for other sizes. The AsyncReader gains a NewSize() constructor that uses a matching pool, so custom read buffer sizes also get proper pool integration. Fixes: rclone#9281 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters