src: migrated to new V8 ArrayBuffer API by thangktran · Pull Request #30782 · nodejs/node

@nodejs-github-bot added c++

Issues and PRs that require attention from people who are familiar with C++.

lib / src

Issues and PRs related to general changes in the lib or src directory.

labels

Dec 3, 2019

addaleax

addaleax

@addaleax addaleax added the author ready

PRs that have at least one approval, no pending requests for changes, and a CI started.

label

Dec 6, 2019

addaleax

addaleax

@Trott Trott removed the author ready

PRs that have at least one approval, no pending requests for changes, and a CI started.

label

Dec 6, 2019

@Trott Trott added the semver-major

PRs that contain breaking changes and should be released in the next major version.

label

Dec 6, 2019

addaleax added a commit to addaleax/node that referenced this pull request

Dec 13, 2019
This fixes flaky tests that crashed because the allocations ended
up at positions of previously allocated `ArrayBuffer`s that were
still in the backing store table. In particular, there was a race
condition window between destroying a Worker thread’s `Environment`
and destroying its `Isolate` in which the underlying memory was
already released but the `ArrayBuffer` was still existent, meaning
that new memory could be allocated at the address of the previous
`ArrayBuffer`.

Refs: nodejs#30782

addaleax

gabrielschulhof pushed a commit that referenced this pull request

Dec 14, 2019
This fixes flaky tests that crashed because the allocations ended
up at positions of previously allocated `ArrayBuffer`s that were
still in the backing store table. In particular, there was a race
condition window between destroying a Worker thread’s `Environment`
and destroying its `Isolate` in which the underlying memory was
already released but the `ArrayBuffer` was still existent, meaning
that new memory could be allocated at the address of the previous
`ArrayBuffer`.

Refs: #30782
PR-URL: #30946
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 5, 2020
It looks like it’s virtually impossible at this point to
create “fake” backing stores for objects that don’t fully
own their memory allocations, like the sub-field `js_fields_`
of `Http2Session`. In particular, it turns out that an
`ArrayBuffer` cannot always be easily separated from its
backing store in that situation through by detaching it.

This commit gives the JS-exposed parts of the class its own
memory allocation and its own backing store, simplifying the
code a bit and fixing flakiness coming from it, at the cost
of one additional layer of indirection when accessing the data.

Refs: nodejs#30782
Fixes: nodejs#31107

addaleax added a commit that referenced this pull request

Feb 8, 2020
It looks like it’s virtually impossible at this point to
create “fake” backing stores for objects that don’t fully
own their memory allocations, like the sub-field `js_fields_`
of `Http2Session`. In particular, it turns out that an
`ArrayBuffer` cannot always be easily separated from its
backing store in that situation through by detaching it.

This commit gives the JS-exposed parts of the class its own
memory allocation and its own backing store, simplifying the
code a bit and fixing flakiness coming from it, at the cost
of one additional layer of indirection when accessing the data.

Refs: #30782
Fixes: #31107

PR-URL: #31648
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>

BethGriggs added a commit that referenced this pull request

Apr 20, 2020
- (SEMVER-MAJOR) crypto: move pbkdf2 without digest to EOL
  (James M Snell) [#31166](#31166)
- (SEMVER-MAJOR) fs: deprecate closing FileHandle on garbage collection
  (James M Snell) [#28396](#28396)
- (SEMVER-MAJOR) http: move OutboundMessage.prototype.flush to EOL
  (James M Snell) [#31164](#31164)
- (SEMVER-MAJOR) lib: move GLOBAL and root aliases to EOL
  (James M Snell) [#31167](#31167)
- (SEMVER-MAJOR) os: move tmpDir() to EOL
  (James M Snell)[#31169](#31169)
- (SEMVER-MAJOR) src: remove deprecated wasm type check
  (Clemens Backes) [#32116](#32116)
- (SEMVER-MAJOR) stream: move \_writableState.buffer to EOL
  (James M Snell) [#31165](#31165)
- (SEMVER-MINOR) doc: deprecate process.mainModule
  (Antoine du HAMEL) [#32232](#32232)
- (SEMVER-MINOR) doc: deprecate process.umask() with no arguments
  (Colin Ihrig) [#32499](#32499)

- src: migrate to new V8 ArrayBuffer API (Thang Tran)
  [#30782](#30782)

It is possible that this change will impact some native addons using
`ArrayBuffer`.

- (SEMVER-MAJOR) build: update macos deployment target to 10.13 for 14.x
  (AshCripps)[#32454](#32454)
- (SEMVER-MAJOR) doc: update cross compiler machine for Linux armv7
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: update Centos/RHEL releases use devtoolset-8
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: remove SmartOS from official binaries
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) win: block running on EOL Windows versions
  (João Reis) [#31954](#31954)

It is expected that there will be an ABI mismatch on ARM between the
Node.js binary and native addons.
- [#30786](#30786)

- (SEMVER-MAJOR) deps: update V8 to 8.1.307.20
  (Matheus Marchini) [#32116](#32116)

- cli, report: move --report-on-fatalerror to stable
  (Colin Ihrig) [#32496](#32496)
- deps: upgrade to libuv 1.37.0
  (Colin Ihrig) [#32866](#32866)
- fs: add fs/promises alias module
  (Gus Caplan) [#31553](#31553)
- module: remove experimental modules warning
  (Guy Bedford) [#31974](#31974)

PR-URL: #32181

BethGriggs added a commit that referenced this pull request

Apr 20, 2020
Deprecations:

- (SEMVER-MAJOR) crypto: move pbkdf2 without digest to EOL
  (James M Snell) [#31166](#31166)
- (SEMVER-MAJOR) fs: deprecate closing FileHandle on garbage collection
  (James M Snell) [#28396](#28396)
- (SEMVER-MAJOR) http: move OutboundMessage.prototype.flush to EOL
  (James M Snell) [#31164](#31164)
- (SEMVER-MAJOR) lib: move GLOBAL and root aliases to EOL
  (James M Snell) [#31167](#31167)
- (SEMVER-MAJOR) os: move tmpDir() to EOL
  (James M Snell)[#31169](#31169)
- (SEMVER-MAJOR) src: remove deprecated wasm type check
  (Clemens Backes) [#32116](#32116)
- (SEMVER-MAJOR) stream: move \_writableState.buffer to EOL
  (James M Snell) [#31165](#31165)
- (SEMVER-MINOR) doc: deprecate process.mainModule
  (Antoine du HAMEL) [#32232](#32232)
- (SEMVER-MINOR) doc: deprecate process.umask() with no arguments
  (Colin Ihrig) [#32499](#32499)

ECMAScript Modules - Experimental Warning Removal:

- module: remove experimental modules warning
  (Guy Bedford) [#31974](#31974)

In Node.js 13 we removed the need to include the --experimental-modules
flag, but when running EcmaScript Modules in Node.js, this would still
result in a warning ExperimentalWarning: The ESM module loader is
experimental.

As of Node.js 14 there is no longer this warning when using ESM in
Node.js. However, the ESM implementation in Node.js remains
experimental. As per our stability index: “The feature is not subject
to Semantic Versioning rules. Non-backward compatible changes or
removal may occur in any future release.” Users should be cautious when
using the feature in production environments.

Please keep in mind that the implementation of ESM in Node.js differs
from the developer experience you might be familiar with. Most
transpilation workflows support features such as optional file
extensions or JSON modules that the Node.js ESM implementation does not
support. It is highly likely that modules from transpiled environments
will require a certain degree of refactoring to work in Node.js. It is
worth mentioning that many of our design decisions were made with two
primary goals. Spec compliance and Web Compatibility. It is our belief
that the current implementation offers a future proof model to
authoring ESM modules that paves the path to Universal JavaScript.
Please read more in our documentation.

The ESM implementation in Node.js is still experimental but we do believe
that we are getting very close to being able to call ESM in Node.js
“stable”. Removing the warning is a huge step in that direction.

Migrate to new V8 ArrayBuffer API:

- src: migrate to new V8 ArrayBuffer API (Thang Tran)
  [#30782](#30782)

It is possible that this change will impact some native addons using
`ArrayBuffer`.

Toolchain and Compiler Upgrades:

- (SEMVER-MAJOR) build: update macos deployment target to 10.13 for 14.x
  (AshCripps)[#32454](#32454)
- (SEMVER-MAJOR) doc: update cross compiler machine for Linux armv7
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: update Centos/RHEL releases use devtoolset-8
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: remove SmartOS from official binaries
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) win: block running on EOL Windows versions
  (João Reis) [#31954](#31954)

It is expected that there will be an ABI mismatch on ARM between the
Node.js binary and native addons. Native addons are only broken if they
interact with `std::shared_ptr`. This is expected to be fixed in a
later version of Node.js 14.
- [#30786](#30786)

Update to V8 8.1:

- (SEMVER-MAJOR) deps: update V8 to 8.1.307.20
  (Matheus Marchini) [#32116](#32116)

Other Notable Changes:

- cli, report: move --report-on-fatalerror to stable
  (Colin Ihrig) [#32496](#32496)
- deps: upgrade to libuv 1.37.0
  (Colin Ihrig) [#32866](#32866)
- fs: add fs/promises alias module
  (Gus Caplan) [#31553](#31553)

PR-URL: #32181

BethGriggs added a commit that referenced this pull request

Apr 21, 2020
Deprecations:

- (SEMVER-MAJOR) crypto: move pbkdf2 without digest to EOL
  (James M Snell) [#31166](#31166)
- (SEMVER-MAJOR) fs: deprecate closing FileHandle on garbage collection
  (James M Snell) [#28396](#28396)
- (SEMVER-MAJOR) http: move OutboundMessage.prototype.flush to EOL
  (James M Snell) [#31164](#31164)
- (SEMVER-MAJOR) lib: move GLOBAL and root aliases to EOL
  (James M Snell) [#31167](#31167)
- (SEMVER-MAJOR) os: move tmpDir() to EOL
  (James M Snell)[#31169](#31169)
- (SEMVER-MAJOR) src: remove deprecated wasm type check
  (Clemens Backes) [#32116](#32116)
- (SEMVER-MAJOR) stream: move \_writableState.buffer to EOL
  (James M Snell) [#31165](#31165)
- (SEMVER-MINOR) doc: deprecate process.mainModule
  (Antoine du HAMEL) [#32232](#32232)
- (SEMVER-MINOR) doc: deprecate process.umask() with no arguments
  (Colin Ihrig) [#32499](#32499)

ECMAScript Modules - Experimental Warning Removal:

- module: remove experimental modules warning
  (Guy Bedford) [#31974](#31974)

In Node.js 13 we removed the need to include the --experimental-modules
flag, but when running EcmaScript Modules in Node.js, this would still
result in a warning ExperimentalWarning: The ESM module loader is
experimental.

As of Node.js 14 there is no longer this warning when using ESM in
Node.js. However, the ESM implementation in Node.js remains
experimental. As per our stability index: “The feature is not subject
to Semantic Versioning rules. Non-backward compatible changes or
removal may occur in any future release.” Users should be cautious when
using the feature in production environments.

Please keep in mind that the implementation of ESM in Node.js differs
from the developer experience you might be familiar with. Most
transpilation workflows support features such as optional file
extensions or JSON modules that the Node.js ESM implementation does not
support. It is highly likely that modules from transpiled environments
will require a certain degree of refactoring to work in Node.js. It is
worth mentioning that many of our design decisions were made with two
primary goals. Spec compliance and Web Compatibility. It is our belief
that the current implementation offers a future proof model to
authoring ESM modules that paves the path to Universal JavaScript.
Please read more in our documentation.

The ESM implementation in Node.js is still experimental but we do believe
that we are getting very close to being able to call ESM in Node.js
“stable”. Removing the warning is a huge step in that direction.

Migrate to new V8 ArrayBuffer API:

- src: migrate to new V8 ArrayBuffer API (Thang Tran)
  [#30782](#30782)

It is possible that this change will impact some native addons using
`ArrayBuffer`.

Toolchain and Compiler Upgrades:

- (SEMVER-MAJOR) build: update macos deployment target to 10.13 for 14.x
  (AshCripps)[#32454](#32454)
- (SEMVER-MAJOR) doc: update cross compiler machine for Linux armv7
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: update Centos/RHEL releases use devtoolset-8
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: remove SmartOS from official binaries
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) win: block running on EOL Windows versions
  (João Reis) [#31954](#31954)

It is expected that there will be an ABI mismatch on ARM between the
Node.js binary and native addons. Native addons are only broken if they
interact with `std::shared_ptr`. This is expected to be fixed in a
later version of Node.js 14.
- [#30786](#30786)

Update to V8 8.1:

- (SEMVER-MAJOR) deps: update V8 to 8.1.307.20
  (Matheus Marchini) [#32116](#32116)

Other Notable Changes:

- cli, report: move --report-on-fatalerror to stable
  (Colin Ihrig) [#32496](#32496)
- deps: upgrade to libuv 1.37.0
  (Colin Ihrig) [#32866](#32866)
- fs: add fs/promises alias module
  (Gus Caplan) [#31553](#31553)

PR-URL: #32181

BethGriggs added a commit that referenced this pull request

Apr 21, 2020
Deprecations:

- (SEMVER-MAJOR) crypto: move pbkdf2 without digest to EOL
  (James M Snell) [#31166](#31166)
- (SEMVER-MAJOR) fs: deprecate closing FileHandle on garbage collection
  (James M Snell) [#28396](#28396)
- (SEMVER-MAJOR) http: move OutboundMessage.prototype.flush to EOL
  (James M Snell) [#31164](#31164)
- (SEMVER-MAJOR) lib: move GLOBAL and root aliases to EOL
  (James M Snell) [#31167](#31167)
- (SEMVER-MAJOR) os: move tmpDir() to EOL
  (James M Snell)[#31169](#31169)
- (SEMVER-MAJOR) src: remove deprecated wasm type check
  (Clemens Backes) [#32116](#32116)
- (SEMVER-MAJOR) stream: move \_writableState.buffer to EOL
  (James M Snell) [#31165](#31165)
- (SEMVER-MINOR) doc: deprecate process.mainModule
  (Antoine du HAMEL) [#32232](#32232)
- (SEMVER-MINOR) doc: deprecate process.umask() with no arguments
  (Colin Ihrig) [#32499](#32499)

ECMAScript Modules - Experimental Warning Removal:

- module: remove experimental modules warning
  (Guy Bedford) [#31974](#31974)

In Node.js 13 we removed the need to include the --experimental-modules
flag, but when running EcmaScript Modules in Node.js, this would still
result in a warning ExperimentalWarning: The ESM module loader is
experimental.

As of Node.js 14 there is no longer this warning when using ESM in
Node.js. However, the ESM implementation in Node.js remains
experimental. As per our stability index: “The feature is not subject
to Semantic Versioning rules. Non-backward compatible changes or
removal may occur in any future release.” Users should be cautious when
using the feature in production environments.

Please keep in mind that the implementation of ESM in Node.js differs
from the developer experience you might be familiar with. Most
transpilation workflows support features such as optional file
extensions or JSON modules that the Node.js ESM implementation does not
support. It is highly likely that modules from transpiled environments
will require a certain degree of refactoring to work in Node.js. It is
worth mentioning that many of our design decisions were made with two
primary goals. Spec compliance and Web Compatibility. It is our belief
that the current implementation offers a future proof model to
authoring ESM modules that paves the path to Universal JavaScript.
Please read more in our documentation.

The ESM implementation in Node.js is still experimental but we do believe
that we are getting very close to being able to call ESM in Node.js
“stable”. Removing the warning is a huge step in that direction.

New V8 ArrayBuffer API:

* **src**: migrate to new V8 ArrayBuffer API
  (Thang Tran) [#30782](#30782)

Multiple ArrayBuffers pointing to the same base address are no longer
allowed by V8. This may impact native addons.

Toolchain and Compiler Upgrades:

- (SEMVER-MAJOR) build: update macos deployment target to 10.13 for 14.x
  (AshCripps)[#32454](#32454)
- (SEMVER-MAJOR) doc: update cross compiler machine for Linux armv7
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: update Centos/RHEL releases use devtoolset-8
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: remove SmartOS from official binaries
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) win: block running on EOL Windows versions
  (João Reis) [#31954](#31954)

It is expected that there will be an ABI mismatch on ARM between the
Node.js binary and native addons. Native addons are only broken if they
interact with `std::shared_ptr`. This is expected to be fixed in a
later version of Node.js 14.
- [#30786](#30786)

Update to V8 8.1:

- (SEMVER-MAJOR) deps: update V8 to 8.1.307.20
  (Matheus Marchini) [#32116](#32116)

Other Notable Changes:

- cli, report: move --report-on-fatalerror to stable
  (Colin Ihrig) [#32496](#32496)
- deps: upgrade to libuv 1.37.0
  (Colin Ihrig) [#32866](#32866)
- fs: add fs/promises alias module
  (Gus Caplan) [#31553](#31553)

PR-URL: #32181

BethGriggs added a commit that referenced this pull request

Apr 21, 2020
Deprecations:

- (SEMVER-MAJOR) crypto: move pbkdf2 without digest to EOL
  (James M Snell) [#31166](#31166)
- (SEMVER-MAJOR) fs: deprecate closing FileHandle on garbage collection
  (James M Snell) [#28396](#28396)
- (SEMVER-MAJOR) http: move OutboundMessage.prototype.flush to EOL
  (James M Snell) [#31164](#31164)
- (SEMVER-MAJOR) lib: move GLOBAL and root aliases to EOL
  (James M Snell) [#31167](#31167)
- (SEMVER-MAJOR) os: move tmpDir() to EOL
  (James M Snell)[#31169](#31169)
- (SEMVER-MAJOR) src: remove deprecated wasm type check
  (Clemens Backes) [#32116](#32116)
- (SEMVER-MAJOR) stream: move \_writableState.buffer to EOL
  (James M Snell) [#31165](#31165)
- (SEMVER-MINOR) doc: deprecate process.mainModule
  (Antoine du HAMEL) [#32232](#32232)
- (SEMVER-MINOR) doc: deprecate process.umask() with no arguments
  (Colin Ihrig) [#32499](#32499)

ECMAScript Modules - Experimental Warning Removal:

- module: remove experimental modules warning
  (Guy Bedford) [#31974](#31974)

In Node.js 13 we removed the need to include the --experimental-modules
flag, but when running EcmaScript Modules in Node.js, this would still
result in a warning ExperimentalWarning: The ESM module loader is
experimental.

As of Node.js 14 there is no longer this warning when using ESM in
Node.js. However, the ESM implementation in Node.js remains
experimental. As per our stability index: “The feature is not subject
to Semantic Versioning rules. Non-backward compatible changes or
removal may occur in any future release.” Users should be cautious when
using the feature in production environments.

Please keep in mind that the implementation of ESM in Node.js differs
from the developer experience you might be familiar with. Most
transpilation workflows support features such as optional file
extensions or JSON modules that the Node.js ESM implementation does not
support. It is highly likely that modules from transpiled environments
will require a certain degree of refactoring to work in Node.js. It is
worth mentioning that many of our design decisions were made with two
primary goals. Spec compliance and Web Compatibility. It is our belief
that the current implementation offers a future proof model to
authoring ESM modules that paves the path to Universal JavaScript.
Please read more in our documentation.

The ESM implementation in Node.js is still experimental but we do believe
that we are getting very close to being able to call ESM in Node.js
“stable”. Removing the warning is a huge step in that direction.

New V8 ArrayBuffer API:

* **src**: migrate to new V8 ArrayBuffer API
  (Thang Tran) [#30782](#30782)

Multiple ArrayBuffers pointing to the same base address are no longer
allowed by V8. This may impact native addons.

Toolchain and Compiler Upgrades:

- (SEMVER-MAJOR) build: update macos deployment target to 10.13 for 14.x
  (AshCripps)[#32454](#32454)
- (SEMVER-MAJOR) doc: update cross compiler machine for Linux armv7
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: update Centos/RHEL releases use devtoolset-8
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) doc: remove SmartOS from official binaries
  (Richard Lau) [#32812](#32812)
- (SEMVER-MAJOR) win: block running on EOL Windows versions
  (João Reis) [#31954](#31954)

It is expected that there will be an ABI mismatch on ARM between the
Node.js binary and native addons. Native addons are only broken if they
interact with `std::shared_ptr`. This is expected to be fixed in a
later version of Node.js 14.
- [#30786](#30786)

Update to V8 8.1:

- (SEMVER-MAJOR) deps: update V8 to 8.1.307.20
  (Matheus Marchini) [#32116](#32116)

Other Notable Changes:

- cli, report: move --report-on-fatalerror to stable
  (Colin Ihrig) [#32496](#32496)
- deps: upgrade to libuv 1.37.0
  (Colin Ihrig) [#32866](#32866)
- fs: add fs/promises alias module
  (Gus Caplan) [#31553](#31553)

PR-URL: #32181

sanosdole added a commit to sanosdole/nodeclrhost that referenced this pull request

Mar 30, 2021
Implement support for Electron 12.

Major changes:
- [node #30782](nodejs/node#30782) New ArrayBuffer backing store prevents reusing the same data pointer
- [electron #27949](electron/electron#27949) Context isolation is turned on by default and prevents `coreclr-hosting` from working properly
- [electron #18397](electron/electron#18397) & [electron #454](belnadris/angular-electron#454) lead to `app.allowRendererProcessReuse` being fixed to true on Electron 12. This required making `coreclr-hosting` context aware and implement proper re-use of the hosted CLR.

Known Issues:
- After reloading a renderer app the task scheduler sometimes hangs up, as the the `ThreadSafeFunction` does not get invoked.

Code cleanup and small memory management improvements.