worker: keep allocators for transferred SAB instances alive longer by addaleax · Pull Request #29637 · nodejs/node

@addaleax

Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance
alive for at least as long as the receiving Isolate lives, if the
`SharedArrayBuffer` instance isn’t already destroyed through GC.

This is to work around the fact that V8 7.9 started refactoring
how backing stores for `SharedArrayBuffer` instances work, changing
the timing of the call that releases the backing store to be
during Isolate disposal.

The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.

Fixes: nodejs/node-v8#115

@nodejs-github-bot added the c++

Issues and PRs that require attention from people who are familiar with C++.

label

Sep 20, 2019

@Trott Trott added the author ready

PRs that have at least one approval, no pending requests for changes, and a CI started.

label

Sep 24, 2019

Trott

bnoordhuis

@addaleax

Trott pushed a commit that referenced this pull request

Sep 25, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance
alive for at least as long as the receiving Isolate lives, if the
`SharedArrayBuffer` instance isn’t already destroyed through GC.

This is to work around the fact that V8 7.9 started refactoring
how backing stores for `SharedArrayBuffer` instances work, changing
the timing of the call that releases the backing store to be
during Isolate disposal.

The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.

Fixes: nodejs/node-v8#115
PR-URL: #29637
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

@addaleax addaleax deleted the worker-sab-isolate-dispose branch

September 25, 2019 10:10

targos pushed a commit that referenced this pull request

Oct 1, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance
alive for at least as long as the receiving Isolate lives, if the
`SharedArrayBuffer` instance isn’t already destroyed through GC.

This is to work around the fact that V8 7.9 started refactoring
how backing stores for `SharedArrayBuffer` instances work, changing
the timing of the call that releases the backing store to be
during Isolate disposal.

The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.

Fixes: nodejs/node-v8#115
PR-URL: #29637
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

pull bot pushed a commit to ksti/v8 that referenced this pull request

Oct 24, 2019
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.

For Background:

tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.

In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.

This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.

By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.

[1]: nodejs/node#30044
[2]: nodejs/node-v8#115
[3]: nodejs/node#29637

Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}

targos added a commit to targos/node that referenced this pull request

Oct 24, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

targos added a commit to targos/node that referenced this pull request

Oct 24, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

targos added a commit to targos/node that referenced this pull request

Oct 28, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

targos added a commit to targos/node that referenced this pull request

Nov 1, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

targos added a commit to targos/node that referenced this pull request

Nov 8, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

targos added a commit to targos/node that referenced this pull request

Nov 17, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

MylesBorins pushed a commit that referenced this pull request

Nov 21, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: #30044
    [2]: nodejs/node-v8#115
    [3]: #29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
    Reviewed-by: Igor Sheludko <ishell@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

Backport-PR-URL: #30513
PR-URL: #30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>