module: cache stat() results more aggressively by bnoordhuis · Pull Request #4575 · nodejs/node
added 3 commits
January 7, 2016 21:59fs.statSync() creates and returns a heavy-weight fs.Stat object whereas fs.accessSync() simply returns nothing. The return value is ignored, the call is for its side effect of throwing an ELOOP error in case of cyclic symbolic links.
Reduce the number of stat() system calls that require() makes by caching the results more aggressively. To avoid unbounded growth without implementing a LRU cache, scope the cache to the lifetime of the first call to require(). Recursive calls (i.e. require() calls in the included code) transparently profit from the cache. The benchmarked application is the loopback-sample-app[0] and it sees the number of stat calls at start-up go down by 40%, from 4736 to 2810. [0] https://github.com/strongloop/loopback-sample-app
Avoid an unneeded ArgumentsAdaptorTrampoline stack frame by passing the
the right number of arguments to Module._load() in Module.require().
Shortens the following stack trace with one frame:
LazyCompile:~Module.load module.js:345
LazyCompile:Module._load module.js:282
Builtin:ArgumentsAdaptorTrampoline
LazyCompile:*Module.require module.js:361
LazyCompile:*require internal/module.js:11
Use internalModuleReadFile() to read the file from disk to avoid the fs.fstatSync() call that fs.readFileSync() makes. It does so to know the file size in advance so it doesn't have to allocate O(n) buffers when reading the file from disk. internalModuleReadFile() is plenty efficient though, even more so because we want a string and not a buffer. This way we also don't allocate a buffer that immediately gets thrown away again. This commit reduces the number of fstat() system calls in a benchmark application[0] from 549 to 29, all made by the application itself. [0] https://github.com/strongloop/loopback-sample-app
jasnell pushed a commit that referenced this pull request
Jan 12, 2016fs.statSync() creates and returns a heavy-weight fs.Stat object whereas fs.accessSync() simply returns nothing. The return value is ignored, the call is for its side effect of throwing an ELOOP error in case of cyclic symbolic links. PR-URL: #4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
jasnell pushed a commit that referenced this pull request
Jan 12, 2016Reduce the number of stat() system calls that require() makes by caching the results more aggressively. To avoid unbounded growth without implementing a LRU cache, scope the cache to the lifetime of the first call to require(). Recursive calls (i.e. require() calls in the included code) transparently profit from the cache. The benchmarked application is the loopback-sample-app[0] and it sees the number of stat calls at start-up go down by 40%, from 4736 to 2810. [0] https://github.com/strongloop/loopback-sample-app PR-URL: #4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
jasnell pushed a commit that referenced this pull request
Jan 12, 2016Avoid an unneeded ArgumentsAdaptorTrampoline stack frame by passing the
the right number of arguments to Module._load() in Module.require().
Shortens the following stack trace with one frame:
LazyCompile:~Module.load module.js:345
LazyCompile:Module._load module.js:282
Builtin:ArgumentsAdaptorTrampoline
LazyCompile:*Module.require module.js:361
LazyCompile:*require internal/module.js:11
PR-URL: #4575
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
jasnell pushed a commit that referenced this pull request
Jan 12, 2016Use internalModuleReadFile() to read the file from disk to avoid the fs.fstatSync() call that fs.readFileSync() makes. It does so to know the file size in advance so it doesn't have to allocate O(n) buffers when reading the file from disk. internalModuleReadFile() is plenty efficient though, even more so because we want a string and not a buffer. This way we also don't allocate a buffer that immediately gets thrown away again. This commit reduces the number of fstat() system calls in a benchmark application[0] from 549 to 29, all made by the application itself. [0] https://github.com/strongloop/loopback-sample-app PR-URL: #4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Trott
mentioned this pull request
Trott
mentioned this pull request
rvagg pushed a commit that referenced this pull request
Jan 14, 2016Avoid an unneeded ArgumentsAdaptorTrampoline stack frame by passing the
the right number of arguments to Module._load() in Module.require().
Shortens the following stack trace with one frame:
LazyCompile:~Module.load module.js:345
LazyCompile:Module._load module.js:282
Builtin:ArgumentsAdaptorTrampoline
LazyCompile:*Module.require module.js:361
LazyCompile:*require internal/module.js:11
PR-URL: #4575
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Trott added a commit to Trott/io.js that referenced this pull request
Jan 15, 2016This reverts commit 809bf5e ("change statSync to accessSync in realpathSync"). It was causing tests to fail on Windows CI. Ref: nodejs#4575 PR-URL: nodejs#4679 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Trott added a commit to Trott/io.js that referenced this pull request
Jan 15, 2016This reverts commit 7c60328. It is causing CI failures on Windows. Ref: nodejs#4575 PR-URL: nodejs#4679 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Trott added a commit that referenced this pull request
Jan 16, 2016Trott added a commit that referenced this pull request
Jan 16, 2016
rvagg
mentioned this pull request
evanlucas pushed a commit that referenced this pull request
Jan 19, 2016Reduce the number of stat() system calls that require() makes by caching the results more aggressively. To avoid unbounded growth without implementing a LRU cache, scope the cache to the lifetime of the first call to require(). Recursive calls (i.e. require() calls in the included code) transparently profit from the cache. The benchmarked application is the loopback-sample-app[0] and it sees the number of stat calls at start-up go down by 40%, from 4736 to 2810. [0] https://github.com/strongloop/loopback-sample-app PR-URL: #4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
evanlucas pushed a commit that referenced this pull request
Jan 20, 2016Reduce the number of stat() system calls that require() makes by caching the results more aggressively. To avoid unbounded growth without implementing a LRU cache, scope the cache to the lifetime of the first call to require(). Recursive calls (i.e. require() calls in the included code) transparently profit from the cache. The benchmarked application is the loopback-sample-app[0] and it sees the number of stat calls at start-up go down by 40%, from 4736 to 2810. [0] https://github.com/strongloop/loopback-sample-app PR-URL: #4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
evanlucas added a commit that referenced this pull request
Jan 20, 2016Notable changes: * events: make sure console functions exist (Dave) #4479 * fs: add autoClose option to fs.createWriteStream (Saquib) #3679 * http: improves expect header handling (Daniel Sellers) #4501 * node: allow preload modules with -i (Evan Lucas) #4696 * v8,src: expose statistics about heap spaces (`v8.getHeapSpaceStatistics()`) (Ben Ripkens) #4463 * Minor performance improvements: - lib: Use arrow functions instead of bind where possible (Minwoo Jung) #3622 - module: cache stat() results more aggressively (Ben Noordhuis) #4575 - querystring: improve parse() performance (Brian White) #4675 PR-URL: #4742
evanlucas added a commit that referenced this pull request
Jan 21, 2016Notable changes: * events: make sure console functions exist (Dave) #4479 * fs: add autoClose option to fs.createWriteStream (Saquib) #3679 * http: improves expect header handling (Daniel Sellers) #4501 * node: allow preload modules with -i (Evan Lucas) #4696 * v8,src: expose statistics about heap spaces (`v8.getHeapSpaceStatistics()`) (Ben Ripkens) #4463 * Minor performance improvements: - lib: Use arrow functions instead of bind where possible (Minwoo Jung) #3622 - module: cache stat() results more aggressively (Ben Noordhuis) #4575 - querystring: improve parse() performance (Brian White) #4675 PR-URL: #4742
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016fs.statSync() creates and returns a heavy-weight fs.Stat object whereas fs.accessSync() simply returns nothing. The return value is ignored, the call is for its side effect of throwing an ELOOP error in case of cyclic symbolic links. PR-URL: nodejs#4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016Reduce the number of stat() system calls that require() makes by caching the results more aggressively. To avoid unbounded growth without implementing a LRU cache, scope the cache to the lifetime of the first call to require(). Recursive calls (i.e. require() calls in the included code) transparently profit from the cache. The benchmarked application is the loopback-sample-app[0] and it sees the number of stat calls at start-up go down by 40%, from 4736 to 2810. [0] https://github.com/strongloop/loopback-sample-app PR-URL: nodejs#4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016Avoid an unneeded ArgumentsAdaptorTrampoline stack frame by passing the
the right number of arguments to Module._load() in Module.require().
Shortens the following stack trace with one frame:
LazyCompile:~Module.load module.js:345
LazyCompile:Module._load module.js:282
Builtin:ArgumentsAdaptorTrampoline
LazyCompile:*Module.require module.js:361
LazyCompile:*require internal/module.js:11
PR-URL: nodejs#4575
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016Use internalModuleReadFile() to read the file from disk to avoid the fs.fstatSync() call that fs.readFileSync() makes. It does so to know the file size in advance so it doesn't have to allocate O(n) buffers when reading the file from disk. internalModuleReadFile() is plenty efficient though, even more so because we want a string and not a buffer. This way we also don't allocate a buffer that immediately gets thrown away again. This commit reduces the number of fstat() system calls in a benchmark application[0] from 549 to 29, all made by the application itself. [0] https://github.com/strongloop/loopback-sample-app PR-URL: nodejs#4575 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016This reverts commit 809bf5e ("change statSync to accessSync in realpathSync"). It was causing tests to fail on Windows CI. Ref: nodejs#4575 PR-URL: nodejs#4679 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016This reverts commit 7c60328. It is causing CI failures on Windows. Ref: nodejs#4575 PR-URL: nodejs#4679 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
scovetta pushed a commit to scovetta/node that referenced this pull request
Apr 2, 2016Notable changes: * events: make sure console functions exist (Dave) nodejs#4479 * fs: add autoClose option to fs.createWriteStream (Saquib) nodejs#3679 * http: improves expect header handling (Daniel Sellers) nodejs#4501 * node: allow preload modules with -i (Evan Lucas) nodejs#4696 * v8,src: expose statistics about heap spaces (`v8.getHeapSpaceStatistics()`) (Ben Ripkens) nodejs#4463 * Minor performance improvements: - lib: Use arrow functions instead of bind where possible (Minwoo Jung) nodejs#3622 - module: cache stat() results more aggressively (Ben Noordhuis) nodejs#4575 - querystring: improve parse() performance (Brian White) nodejs#4675 PR-URL: nodejs#4742
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters