modules: significantly improve require performance by BridgeAR · Pull Request #25362 · nodejs/node

@BridgeAR added the semver-major

PRs that contain breaking changes and should be released in the next major version.

label

Jan 6, 2019

@BridgeAR BridgeAR added module

Issues and PRs related to the module subsystem.

performance

Issues and PRs related to the performance of Node.js.

and removed repl

Issues and PRs related to the REPL subsystem.

labels

Jan 6, 2019

@BridgeAR

1) It adds more benchmark options to properly verify the gains.

This makes sure the benchmark also tests requiring the same module
again instead of only loading each module only once.

2) Remove dead code:

The array check is obsolete as this function will only be called
internally with preprepared data which is always an array.

3) Simpler code

It was possible to use a more direct logic to prevent some branches.

4) Inline try catch

The function is not required anymore, since V8 is able to produce
performant code with it.

5) var -> let / const & less lines

6) Update require.extensions description

The comment was outdated.

7) Improve extension handling

This is a performance optimization to prevent loading the extensions
on each uncached require call. It uses proxies to intercept changes
and receives the necessary informations by doing that.

devsnek

addaleax

@BridgeAR

Trott

@BridgeAR

Trott

@BridgeAR

Trott

@BridgeAR

@BridgeAR

There are some issues involving the proxy. This tries to circumvent
the issue by not making any copy of the newly set object.

@BridgeAR

@BridgeAR

@BridgeAR

@BridgeAR BridgeAR added the wip

Issues and PRs that are still a work in progress.

label

Mar 26, 2019

This was referenced

Mar 28, 2019

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 1, 2019
This publicly documents that adding native module names will resolve
the added entry instead of the native module.

It also updates the description why extensions are deprecated.

PR-URL: nodejs#26971
Refs: nodejs#25362
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 4, 2019
PR-URL: nodejs#26970
Refs: nodejs#25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 4, 2019
Add more benchmark options to properly verify the gains.

This makes sure the benchmark also tests requiring the same module
again instead of only loading each module only once.

PR-URL: nodejs#26970
Refs: nodejs#25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 4, 2019
Moving `try / catch` into separate functions is not necessary
anymore due to V8 optimizations.

PR-URL: nodejs#26970
Refs: nodejs#25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 4, 2019
This adds the `path` property to the module object. It contains the
current directory as path. That is necessary to add an extra caching
layer.

It also makes sure the `id` uses a default in case it's not set.
Otherwise the `path.dirname(id)` command could fail.

PR-URL: nodejs#26970
Refs: nodejs#25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 4, 2019
This adds an extra modules caching layer that operates on the parent's
`path` property and the current require argument. That together can
be used as unique identifier to speed up loading the same module more
than once. It is a cache on top of the current modules cache.

It has the nice feature that this cache does not only work in the same
file but it works for the whole current directory. So if the same file
is loaded in any other file from the same directory, it will also hit
this cache instead of having to resolve the file again.

To keep it backwards compatible with the old modules cache, it detects
invalidation of that cache.

PR-URL: nodejs#26970
Refs: nodejs#25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

BethGriggs pushed a commit that referenced this pull request

Apr 5, 2019
This publicly documents that adding native module names will resolve
the added entry instead of the native module.

It also updates the description why extensions are deprecated.

PR-URL: #26971
Refs: #25362
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>

BridgeAR added a commit to BridgeAR/node that referenced this pull request

Apr 8, 2019
This removes a lot of code that has no functionality anymore. All
Node.js internal code calls `_resolveLookupPaths` with two arguments.

The code that validates `index.js` is not required at all as we check
for these files anyway, so it's just redundant code that should be
removed.

PR-URL: nodejs#26983
Refs: nodejs#25362
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
This publicly documents that adding native module names will resolve
the added entry instead of the native module.

It also updates the description why extensions are deprecated.

PR-URL: #26971
Refs: #25362
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
This publicly documents that adding native module names will resolve
the added entry instead of the native module.

It also updates the description why extensions are deprecated.

PR-URL: #26971
Refs: #25362
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
Add more benchmark options to properly verify the gains.

This makes sure the benchmark also tests requiring the same module
again instead of only loading each module only once.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
Moving `try / catch` into separate functions is not necessary
anymore due to V8 optimizations.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
This adds the `path` property to the module object. It contains the
current directory as path. That is necessary to add an extra caching
layer.

It also makes sure the `id` uses a default in case it's not set.
Otherwise the `path.dirname(id)` command could fail.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 9, 2019
This adds an extra modules caching layer that operates on the parent's
`path` property and the current require argument. That together can
be used as unique identifier to speed up loading the same module more
than once. It is a cache on top of the current modules cache.

It has the nice feature that this cache does not only work in the same
file but it works for the whole current directory. So if the same file
is loaded in any other file from the same directory, it will also hit
this cache instead of having to resolve the file again.

To keep it backwards compatible with the old modules cache, it detects
invalidation of that cache.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
This publicly documents that adding native module names will resolve
the added entry instead of the native module.

It also updates the description why extensions are deprecated.

PR-URL: #26971
Refs: #25362
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
Add more benchmark options to properly verify the gains.

This makes sure the benchmark also tests requiring the same module
again instead of only loading each module only once.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
Moving `try / catch` into separate functions is not necessary
anymore due to V8 optimizations.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
This adds the `path` property to the module object. It contains the
current directory as path. That is necessary to add an extra caching
layer.

It also makes sure the `id` uses a default in case it's not set.
Otherwise the `path.dirname(id)` command could fail.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>

BethGriggs pushed a commit that referenced this pull request

Apr 10, 2019
This adds an extra modules caching layer that operates on the parent's
`path` property and the current require argument. That together can
be used as unique identifier to speed up loading the same module more
than once. It is a cache on top of the current modules cache.

It has the nice feature that this cache does not only work in the same
file but it works for the whole current directory. So if the same file
is loaded in any other file from the same directory, it will also hit
this cache instead of having to resolve the file again.

To keep it backwards compatible with the old modules cache, it detects
invalidation of that cache.

PR-URL: #26970
Refs: #25362
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Signed-off-by: Beth Griggs <Bethany.Griggs@uk.ibm.com>