fs: introduce `opendir()` and `fs.Dir` by Fishrock123 · Pull Request #29349 · nodejs/node
added
c++
labels
Aug 28, 2019
addaleax
added
semver-minor
and removed lib / src
Issues and PRs related to general changes in the lib or src directory.labels
Aug 28, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs#583 Refs: libuv/libuv#2057
Fishrock123 added a commit that referenced this pull request
Oct 8, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: #583 Refs: libuv/libuv#2057 PR-URL: #29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
BridgeAR pushed a commit that referenced this pull request
Oct 9, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: #583 Refs: libuv/libuv#2057 PR-URL: #29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
addaleax pushed a commit to nodejs/quic that referenced this pull request
Oct 9, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
addaleax pushed a commit to nodejs/quic that referenced this pull request
Oct 9, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
BridgeAR added a commit that referenced this pull request
Oct 10, 2019Notable changes:
* build:
* Add `--force-context-aware` flag to prevent usage of native node
addons that aren't context aware
#29631
* deprecations:
* Add documentation-only deprecation for `process._tickCallback()`
#29781
* esm:
* Using JSON modules is experimental again
#29754
* fs:
* Introduce `opendir()` and `fs.Dir` to iterate through directories
#29349
* process:
* Add source-map support to stack traces by using
`--source-map-support` #29564
* tls:
* Honor `pauseOnConnect` option
#29635
* Add option for private keys for OpenSSL engines
#28973
PR-URL: #29919
BridgeAR added a commit that referenced this pull request
Oct 11, 2019Notable changes:
* build:
* Add `--force-context-aware` flag to prevent usage of native node
addons that aren't context aware
#29631
* deprecations:
* Add documentation-only deprecation for `process._tickCallback()`
#29781
* esm:
* Using JSON modules is experimental again
#29754
* fs:
* Introduce `opendir()` and `fs.Dir` to iterate through directories
#29349
* process:
* Add source-map support to stack traces by using
`--source-map-support` #29564
* tls:
* Honor `pauseOnConnect` option
#29635
* Add option for private keys for OpenSSL engines
#28973
PR-URL: #29919
BridgeAR added a commit that referenced this pull request
Oct 11, 2019Notable changes:
* build:
* Add `--force-context-aware` flag to prevent usage of native node
addons that aren't context aware
#29631
* deprecations:
* Add documentation-only deprecation for `process._tickCallback()`
#29781
* esm:
* Using JSON modules is experimental again
#29754
* fs:
* Introduce `opendir()` and `fs.Dir` to iterate through directories
#29349
* process:
* Add source-map support to stack traces by using
`--source-map-support` #29564
* tls:
* Honor `pauseOnConnect` option
#29635
* Add option for private keys for OpenSSL engines
#28973
PR-URL: #29919
addaleax pushed a commit to nodejs/quic that referenced this pull request
Oct 11, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
juanarbol pushed a commit to juanarbol/quic that referenced this pull request
Dec 17, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
juanarbol pushed a commit to juanarbol/quic that referenced this pull request
Dec 17, 2019This adds long-requested methods for asynchronously interacting and iterating through directory entries by using `uv_fs_opendir`, `uv_fs_readdir`, and `uv_fs_closedir`. `fs.opendir()` and friends return an `fs.Dir`, which contains methods for doing reads and cleanup. `fs.Dir` also has the async iterator symbol exposed. The `read()` method and friends only return `fs.Dirent`s for this API. Having a entry type or doing a `stat` call is deemed to be necessary in the majority of cases, so just returning dirents seems like the logical choice for a new api. Reading when there are no more entries returns `null` instead of a dirent. However the async iterator hides that (and does automatic cleanup). The code lives in separate files from the rest of fs, this is done partially to prevent over-pollution of those (already very large) files, but also in the case of js allows loading into `fsPromises`. Due to async_hooks, this introduces a new handle type of `DIRHANDLE`. This PR does not attempt to make complete optimization of this feature. Notable future improvements include: - Moving promise work into C++ land like FileHandle. - Possibly adding `readv()` to do multi-entry directory reads. - Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation. Refs: nodejs/node-v0.x-archive#388 Refs: nodejs/node#583 Refs: libuv/libuv#2057 PR-URL: nodejs/node#29349 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com>
abhishekumar-tyagi pushed a commit to abhishekumar-tyagi/node that referenced this pull request
May 5, 2024This is a follow-up to nodejs/node#29349 and a precursor to deprecating both `fs.readdir()` and `fs.readdirSync()`. This also updates the documentation which has been incorrect for some 10 years saying that these did `readdir(3)` when in reality they do `scandir(3)`. Only `scandirSync()` is introduced as "async scandir(3)" is kind of a trap, given that it returns the whole list of entries at once, regardless of how many there are. Since in many cases we'd also want to get dirents for them (i.e. `stat`-ing each and every one), this becomes a serious problem, and Node.js should encourage users to use `fs.opendir()`, which is slightly more complex but far better.
abhishekumar-tyagi pushed a commit to abhishekumar-tyagi/node that referenced this pull request
May 5, 2024It's time overdue for these to start going away. `fs.opendir()` was introduced over a year ago in nodejs/node#29349 - which stated a follow-up of: "Aliasing fs.readdir to fs.scandir and doing a deprecation." This provides the intial docs deprecation to both `fs.readdir()` and `fs.readdirSync()`, both of which are misnamed, and the former of which is a trap as it is not fully async.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters