Feature or enhancement

I expect this to be implemented across multiple PRs.

For context, here is the change from the nogil-3.12 fork, but things might be done a bit differently in CPython 3.13: colesbury/nogil-3.12@df4c51f82b.

• Most operations should acquire the list object lock using the critical section API.
• Accessing a single element should optimistically avoid locking for performance
• Iterators need some special handling
• list.sort

Iterators

For performance reasons, we don't want to acquire locks while iterating over lists. This means that list iterators have a lesser notion of thread-safety: multiple threads concurrently using the same iterator should not crash but may revisit elements. We should make clear in the documentation that iterators are not generally thread-safe. From an implementation point of view, we should use relaxed atomics to update the it_index variable.

Additionally, we don't want to clear the it_seq reference to the list when the iterator is exhausted in --disable-gil builds. Doing so would pose thread-safety issues (in the "may segfault sense").

I don't expect this to pose any issues for real code. While it's fairly common to share lists between threads, it's rare to share the same iterator object between threads.

Linked PRs

• gh-112087: Update list.{pop,clear,reverse,remove} to use CS #113764
• gh-112087: Update list impl to be thread-safe with manual CS #113863
• gh-112087: Remove duplicated critical_section #114268
• gh-112087: Make list_repr and list_length to be thread-safe #114582
• gh-112087: Make PyList_{Append,Size,GetSlice} to be thread-safe #114651
• gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe #114843
• gh-112087: Make list_{count, index, contains} to be thread-safe. #114916
• [3.12] gh-112087: Fix reduce logic for the empty reverse iterator for list #115471
• [3.11] gh-112087: Fix reduce logic for the empty reverse iterator for list #115472
• gh-112087: Make list_{concat, repeat, inplace_repeat, ass_item) to be thread-safe #115605
• gh-112087: Make list_{subscript, ass_slice, slice} to be thread-safe #115854
• gh-112087: Use QSBR technique for list_new/clear for free-thread build #115875
• gh-112087: Make list_{slice, ass_slice, subscript} to be threadsafe #116233
• gh-112087: Test fix for test_sched leak #116237
• gh-112087: Update list_get_item_ref to optimistically avoid locking #116353
• gh-112087: Store memory allocation information into _PyListArray #116529
• gh-112087: Make list.sort to be thread-safe for PEP 703. #116553
• gh-112087: Make list.extend(dict) behave atomically #117438