feat: add Uint8Array support for zero-copy worker_threads transfer by umeshmore45 · Pull Request #3183 · redis/node-redis
- Add Uint8Array to type system (BlobStringReply, SimpleStringReply, VerbatimStringReply) - Implement Uint8Array encoding/decoding in RESP protocol handlers - Support Uint8Array as command arguments across all commands - Add comprehensive test coverage for Uint8Array operations - Fix type guards in HSET, GEOSEARCH, AGGREGATE, SEARCH commands - Fix Buffer/Uint8Array handling in time-series helpers and cluster-slots - Maintain full backward compatibility with Buffer Enables zero-copy transfer to worker_threads via ArrayBuffer transfer. Closes redis#3106
marked this pull request as ready for review
…t8Array as valid key types - Updated type guard in generic-transformers to recognize Uint8Array. - Added unit tests for Uint8Array support in ZINTER, ZUNION, and ZUNIONSTORE commands. - Ensured compatibility with existing command structures while expanding argument handling. This change improves the flexibility of command arguments, allowing for zero-copy transfers with Uint8Array.
- Updated type guard to recognize Uint8Array as a valid parameter type. - Added a regression test to ensure correct handling of Uint8Array keys. - Improved argument parsing to prevent incorrect behavior with Uint8Array inputs. This change enhances the flexibility of the CONFIG SET command, aligning with recent updates for better Uint8Array support across commands.
- Updated type guard in isPlainSketches to include Uint8Array. - Added regression test to verify correct handling of Uint8Array sources in CMS.MERGE. This change improves the argument handling of the CMS.MERGE command, ensuring compatibility with Uint8Array inputs.
- Introduced tests to ensure proper invalidation of cache entries when using Uint8Array keys. - Validated that entries are evicted correctly when invalidation keys are provided as Buffer or string. - Enhanced the client-side cache functionality to handle Uint8Array keys consistently with server-side invalidation. This change improves the reliability of cache behavior in scenarios involving Uint8Array keys.
… commands - Introduced regression tests to ensure Uint8Array field keys and channel names are correctly decoded as UTF-8 strings. - Validated that the transformation of replies maintains the integrity of keys when using Uint8Array. This change enhances the reliability of key handling in the HGETALL and PUBSUB NUMSUB commands, aligning with recent updates for better Uint8Array support.
…, GET, and ZRANK_WITHSCORE commands - Introduced tests to validate that Uint8Array replies are correctly parsed as finite numbers instead of NaN. - Ensured that the transformation of replies handles Uint8Array correctly across multiple commands, addressing a coercion hazard. This change enhances the reliability of numeric handling in commands that utilize Uint8Array, improving overall data integrity.
- Refactored the handling of the reply to use a more robust check for Uint8Array. - Enhanced the condition to ensure accurate detection of replies, improving the reliability of the command queue's response handling. This change addresses potential issues with reply processing in PubSub and MONITOR modes.
…ray and Buffer - Added tests to ensure the PING command resolves correctly as Uint8Array, Buffer, or string based on the provided typeMapping. - Improved the handling of PONG replies to accurately return the expected types, enhancing the command queue's response reliability. This change addresses potential inconsistencies in the response types for the PING command, aligning with recent updates for better Uint8Array support.
…rray and Buffer - Introduced a new utility function to standardize the conversion of Redis keys from string, Buffer, or Uint8Array to UTF-8 strings. - Updated cache management in to utilize the new key conversion function, ensuring consistent key handling. - Added function to convert Redis blob replies to UTF-8 strings, improving the handling of Uint8Array and Buffer replies across multiple commands. - Refactored commands such as GEODIST, PUBSUB_NUMSUB, and ZRANK_WITHSCORE to leverage for accurate reply transformations. These changes enhance the reliability and consistency of key and reply handling, particularly for Uint8Array and Buffer types, aligning with recent updates for better data integrity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters