src: allocate Buffer memory using ArrayBuffer allocator by addaleax · Pull Request #26207 · nodejs/node

@nodejs-github-bot added the lib / src

Issues and PRs related to general changes in the lib or src directory.

label

Feb 19, 2019

@addaleax addaleax added v8 engine

Issues and PRs related to the V8 dependency.

embedding

Issues and PRs related to embedding Node.js in another project.

labels

Feb 19, 2019

joyeecheung

@addaleax

This enables us to identify whether we are using an
allocator that we know more about than what the generic
`ArrayBuffer::Allocator` API provides, in particular
whether it is `malloc()`-compatible.
If the `ArrayBuffer::Allocator` used to create `ArrayBuffer`s
in the current `Isolate` is not a Node.js `ArrayBufferAllocator`,
we cannot know that it is `malloc()`-based, an in particular it might
not be compatible with the `ArrayBuffer::Allocator` on the receiving
end of the connection.
Add a subclass of `ArrayBufferAllocator` that performs additional
debug checking, which in particular verifies that:

- All `ArrayBuffer` backing stores have been allocated with this
  allocator, or have been explicitly marked as coming from a
  compatible source.
- All memory allocated by the allocator has been freed once it is
  destroyed.
Add a RAII utility for managing blocks of memory that have
been allocated with the `ArrayBuffer::Allocator` for a given
`Isolate`.

@addaleax

@addaleax

Always use the right allocator for memory that is turned into
an `ArrayBuffer` at a later point.

This enables embedders to use their own `ArrayBuffer::Allocator`s,
and is inspired by Electron’s electron/node@f61bae3440e. It should
render their downstream patch unnecessary.

Refs: electron/node@f61bae3

addaleax added a commit that referenced this pull request

Feb 25, 2019

addaleax added a commit that referenced this pull request

Feb 25, 2019
This enables us to identify whether we are using an
allocator that we know more about than what the generic
`ArrayBuffer::Allocator` API provides, in particular
whether it is `malloc()`-compatible.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit that referenced this pull request

Feb 25, 2019
If the `ArrayBuffer::Allocator` used to create `ArrayBuffer`s
in the current `Isolate` is not a Node.js `ArrayBufferAllocator`,
we cannot know that it is `malloc()`-based, an in particular it might
not be compatible with the `ArrayBuffer::Allocator` on the receiving
end of the connection.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit that referenced this pull request

Feb 25, 2019
Add a subclass of `ArrayBufferAllocator` that performs additional
debug checking, which in particular verifies that:

- All `ArrayBuffer` backing stores have been allocated with this
  allocator, or have been explicitly marked as coming from a
  compatible source.
- All memory allocated by the allocator has been freed once it is
  destroyed.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit that referenced this pull request

Feb 25, 2019
Add a RAII utility for managing blocks of memory that have
been allocated with the `ArrayBuffer::Allocator` for a given
`Isolate`.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit that referenced this pull request

Feb 25, 2019
Always use the right allocator for memory that is turned into
an `ArrayBuffer` at a later point.

This enables embedders to use their own `ArrayBuffer::Allocator`s,
and is inspired by Electron’s electron/node@f61bae3440e. It should
render their downstream patch unnecessary.

Refs: electron/node@f61bae3

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019
This enables us to identify whether we are using an
allocator that we know more about than what the generic
`ArrayBuffer::Allocator` API provides, in particular
whether it is `malloc()`-compatible.

PR-URL: nodejs#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019
If the `ArrayBuffer::Allocator` used to create `ArrayBuffer`s
in the current `Isolate` is not a Node.js `ArrayBufferAllocator`,
we cannot know that it is `malloc()`-based, an in particular it might
not be compatible with the `ArrayBuffer::Allocator` on the receiving
end of the connection.

PR-URL: nodejs#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019
Add a subclass of `ArrayBufferAllocator` that performs additional
debug checking, which in particular verifies that:

- All `ArrayBuffer` backing stores have been allocated with this
  allocator, or have been explicitly marked as coming from a
  compatible source.
- All memory allocated by the allocator has been freed once it is
  destroyed.

PR-URL: nodejs#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019
Add a RAII utility for managing blocks of memory that have
been allocated with the `ArrayBuffer::Allocator` for a given
`Isolate`.

PR-URL: nodejs#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

Feb 28, 2019
Always use the right allocator for memory that is turned into
an `ArrayBuffer` at a later point.

This enables embedders to use their own `ArrayBuffer::Allocator`s,
and is inspired by Electron’s electron/node@f61bae3440e. It should
render their downstream patch unnecessary.

Refs: electron/node@f61bae3

PR-URL: nodejs#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

addaleax added a commit that referenced this pull request

Mar 1, 2019

addaleax added a commit that referenced this pull request

Mar 1, 2019
This enables us to identify whether we are using an
allocator that we know more about than what the generic
`ArrayBuffer::Allocator` API provides, in particular
whether it is `malloc()`-compatible.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Backport-PR-URL: #26302
Reviewed-By: Michaël Zasso <targos@protonmail.com>

addaleax added a commit that referenced this pull request

Mar 1, 2019
If the `ArrayBuffer::Allocator` used to create `ArrayBuffer`s
in the current `Isolate` is not a Node.js `ArrayBufferAllocator`,
we cannot know that it is `malloc()`-based, an in particular it might
not be compatible with the `ArrayBuffer::Allocator` on the receiving
end of the connection.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Backport-PR-URL: #26302
Reviewed-By: Michaël Zasso <targos@protonmail.com>

addaleax added a commit that referenced this pull request

Mar 1, 2019
Add a subclass of `ArrayBufferAllocator` that performs additional
debug checking, which in particular verifies that:

- All `ArrayBuffer` backing stores have been allocated with this
  allocator, or have been explicitly marked as coming from a
  compatible source.
- All memory allocated by the allocator has been freed once it is
  destroyed.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Backport-PR-URL: #26302
Reviewed-By: Michaël Zasso <targos@protonmail.com>

addaleax added a commit that referenced this pull request

Mar 1, 2019
Add a RAII utility for managing blocks of memory that have
been allocated with the `ArrayBuffer::Allocator` for a given
`Isolate`.

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Backport-PR-URL: #26302
Reviewed-By: Michaël Zasso <targos@protonmail.com>

addaleax added a commit that referenced this pull request

Mar 1, 2019
Always use the right allocator for memory that is turned into
an `ArrayBuffer` at a later point.

This enables embedders to use their own `ArrayBuffer::Allocator`s,
and is inspired by Electron’s electron/node@f61bae3440e. It should
render their downstream patch unnecessary.

Refs: electron/node@f61bae3

PR-URL: #26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

Backport-PR-URL: #26302
Reviewed-By: Michaël Zasso <targos@protonmail.com>

addaleax added a commit to addaleax/node that referenced this pull request

May 26, 2019
Track received data correctly. Specifically, for the buffer that
is used for receiving data, we previously would try to increment
the current memory usage by its length, and later decrement it
by that, but in the meantime the buffer had been turned over to V8
and its length reset to zero. This gave the impression that more and
more memory was consumed by the HTTP/2 session when it was in fact not.

Fixes: nodejs#27416
Refs: nodejs#26207

addaleax added a commit that referenced this pull request

May 29, 2019
Track received data correctly. Specifically, for the buffer that
is used for receiving data, we previously would try to increment
the current memory usage by its length, and later decrement it
by that, but in the meantime the buffer had been turned over to V8
and its length reset to zero. This gave the impression that more and
more memory was consumed by the HTTP/2 session when it was in fact not.

Fixes: #27416
Refs: #26207

PR-URL: #27914
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

targos pushed a commit that referenced this pull request

May 31, 2019
Track received data correctly. Specifically, for the buffer that
is used for receiving data, we previously would try to increment
the current memory usage by its length, and later decrement it
by that, but in the meantime the buffer had been turned over to V8
and its length reset to zero. This gave the impression that more and
more memory was consumed by the HTTP/2 session when it was in fact not.

Fixes: #27416
Refs: #26207

PR-URL: #27914
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

MarshallOfSound pushed a commit to electron/node that referenced this pull request

Jun 18, 2019
Track received data correctly. Specifically, for the buffer that
is used for receiving data, we previously would try to increment
the current memory usage by its length, and later decrement it
by that, but in the meantime the buffer had been turned over to V8
and its length reset to zero. This gave the impression that more and
more memory was consumed by the HTTP/2 session when it was in fact not.

Fixes: nodejs/node#27416
Refs: nodejs/node#26207

PR-URL: nodejs/node#27914
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 10, 2019
Always use the right allocator for memory that is turned into
an `ArrayBuffer` at a later point.

This enables embedders to use their own `ArrayBuffer::Allocator`s,
and is inspired by Electron’s f61bae3440e. It should
render their downstream patch unnecessary.

Refs: f61bae3

PR-URL: nodejs/node#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 10, 2019
This enables us to identify whether we are using an
allocator that we know more about than what the generic
`ArrayBuffer::Allocator` API provides, in particular
whether it is `malloc()`-compatible.

PR-URL: nodejs/node#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 10, 2019
If the `ArrayBuffer::Allocator` used to create `ArrayBuffer`s
in the current `Isolate` is not a Node.js `ArrayBufferAllocator`,
we cannot know that it is `malloc()`-based, an in particular it might
not be compatible with the `ArrayBuffer::Allocator` on the receiving
end of the connection.

PR-URL: nodejs/node#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 10, 2019
Add a subclass of `ArrayBufferAllocator` that performs additional
debug checking, which in particular verifies that:

- All `ArrayBuffer` backing stores have been allocated with this
  allocator, or have been explicitly marked as coming from a
  compatible source.
- All memory allocated by the allocator has been freed once it is
  destroyed.

PR-URL: nodejs/node#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 10, 2019
Add a RAII utility for managing blocks of memory that have
been allocated with the `ArrayBuffer::Allocator` for a given
`Isolate`.

PR-URL: nodejs/node#26207
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

deepak1556 pushed a commit to electron/node that referenced this pull request

Jul 12, 2019
Track received data correctly. Specifically, for the buffer that
is used for receiving data, we previously would try to increment
the current memory usage by its length, and later decrement it
by that, but in the meantime the buffer had been turned over to V8
and its length reset to zero. This gave the impression that more and
more memory was consumed by the HTTP/2 session when it was in fact not.

Fixes: nodejs/node#27416
Refs: nodejs/node#26207

PR-URL: nodejs/node#27914
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>