Issue 46355: [C API] Document PyFrameObject and PyThreadState changes and explain how to port code to Python 3.11
Created on 2022-01-12 14:11 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (10)
msg410398 - (view)
Author: STINNER Victor (vstinner) * 
Date: 2022-01-12 14:11
Date: 2022-01-12 15:36
Date: 2022-01-12 16:21
Date: 2022-01-13 18:22
Date: 2022-01-13 18:23
Date: 2022-01-31 12:27
Date: 2022-02-12 00:48
Date: 2022-02-23 15:14
Date: 2022-03-03 23:47
Date: 2022-01-12 14:11
Python 3.11 made many changes in PyFrameObject and PyThreadState structures. Code which accessed directly structure members doesn't build anymore. For example, "frame->f_code" is now fails because PyFrameObject.f_code member is done. I propose to document these changes and explain how to port code. I will write a documentation PR. == PyFrameObject changes == PyFrameObject now only creates a Python frame object on demand. * f_code: removed, use PyFrame_GetCode() instead, warning: it returns a strong reference (Py_DECREF is needed) * f_lineno: changed, use PyFrame_GetLineNumber() * f_back: changed, use PyFrame_GetBack() * f_builtins: removed, get the "f_builtins" attribute in Python * f_globals: removed, get the "f_globals" attribute in Python * f_locals: removed, get the "f_locals" attribute in Python * f_valuesstack: removed * f_stackdepth: removed * f_gen: removed * f_lasti: removed, get the "f_lasti" attribute in Python? * f_iblock: removed * f_state: removed * f_blockstack: removed * f_localsplus: removed Accessing f_lineno and f_back doesn't fail with a compiler error, but these members are filled lazily. If PyFrame_GetLineNumber() is not called, it can return 0 even if the frame is running and has a line number. If PyFrame_GetBack() is not called, f_back is NULL even if the frame has a next outer frame. == PyThreadState changes == * frame: removed, use PyThreadState_GetFrame(), warning: it returns a strong reference (Py_DECREF is needed) * recursion_depth: removed, use (tstate->recursion_limit - tstate->recursion_remaining) instead * stackcheck_counter: removed * tracing: changed, use PyThreadState_EnterTracing() and PyThreadState_LeaveTracing(), added by bpo-43760 == Notes == We should also explain how to get new C API functions, like PyFrame_GetCode(), on older Python, and maybe suggest to use pythoncapi_compat to get them: https://github.com/pythoncapi/pythoncapi_compat See also: * bpo-39947: "[C API] Make the PyThreadState structure opaque (move it to the internal C API)" * bpo-40421: "[C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API" * bpo-43760: "The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)" -- add PyThreadState_EnterTracing()msg410404 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-01-12 15:36
> f_lineno: changed, use PyFrame_GetLineNumber() Oh I was wrong, PyFrame_GetLineNumber() code is the same in Python 3.10. It was already a bad idea to read directly the f_lineno member in Python 3.10: PyFrame_GetLineNumber() should always be called.msg410409 - (view) Author: Mark Shannon (Mark.Shannon) *
Date: 2022-01-12 16:21
Are there any C programming guides we can link to, that explain API use? I would hope that competent C programmers would know not to read or write to undocumented fields. But if they come from a Python background, that might not be obvious. If the advice for something is "use Python", we should probably add an API function.msg410501 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-01-13 18:22
New changeset 0885999a8e5ffad3fae0302675ad0030e33a15af by Victor Stinner in branch 'main': bpo-46355: Document PyFrameObject and PyThreadState changes (GH-30558) https://github.com/python/cpython/commit/0885999a8e5ffad3fae0302675ad0030e33a15afmsg410502 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-01-13 18:23
I merged my PR so the doc can be reviewed online at https://docs.python.org/dev/whatsnew/3.11.html once it will be rendered ;-)msg412199 - (view) Author: Petr Viktorin (petr.viktorin) *
Date: 2022-01-31 12:27
> I would hope that competent C programmers would know not to read or write to undocumented fields. But if they come from a Python background, that might not be obvious. Tread carefully when changing decades-old API, documented or not. The docs weren't always as useful and complete as they are now. > If the advice for something is "use Python", we should probably add an API function. Not really. If PyObject_GetAttrString doesn't add too much overhead, IMO it's fine to use it.msg412241 - (view) Author: miss-islington (miss-islington) Date: 2022-02-01 10:22
New changeset a4cb31927a1f0ee31025ea1ca82fcbfad44755dc by Petr Viktorin in branch 'main': bpo-46355: What's New: Note that PyFrameObject are private (GH-31032) https://github.com/python/cpython/commit/a4cb31927a1f0ee31025ea1ca82fcbfad44755dcmsg413112 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-02-12 00:48
New changeset ba5725171d9c411fc4764349205eff5cfc028797 by Erlend Egeberg Aasland in branch 'main': bpo-46355: Amend What's New in Python 3.11 C API wording (GH-31288) https://github.com/python/cpython/commit/ba5725171d9c411fc4764349205eff5cfc028797msg413800 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-02-23 15:14
I created bpo-46836: "[C API] Move PyFrameObject to the internal C API".msg414496 - (view) Author: STINNER Victor (vstinner) *
Date: 2022-03-03 23:47
New changeset ec4a580f7cada002441ae5611b909d56e3b5b613 by Victor Stinner in branch 'main': bpo-46355: Update pythoncapi_compat project URL (GH-31670) https://github.com/python/cpython/commit/ec4a580f7cada002441ae5611b909d56e3b5b613
History
Date
User
Action
Args
2022-04-11 14:59:54adminsetgithub: 90513
2022-03-03 23:47:00vstinnersetmessages:
+ msg414496
2022-03-03 22:13:39vstinnersetpull_requests:
+ pull_request29789
2022-02-23 15:14:32vstinnersetmessages:
+ msg413800
2022-02-12 00:48:33vstinnersetmessages:
+ msg413112
2022-02-11 21:14:16erlendaaslandsetnosy:
+ erlendaasland
messages: + msg412241
2022-01-31 12:27:47petr.viktorinsetnosy: vstinner, petr.viktorin, Mark.Shannon
messages: + msg412199 2022-01-31 12:23:26petr.viktorinsetnosy: + petr.viktorin
resolution: fixed
messages: + msg410502
messages: + msg410409
2022-01-12 15:36:25vstinnersetmessages: + msg410404 2022-01-12 14:59:17vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request28759 2022-01-12 14:11:24vstinnercreate
pull_requests: + pull_request29448
2022-02-01 10:22:37miss-islingtonsetnosy: + miss-islingtonmessages: + msg412241
2022-01-31 12:27:47petr.viktorinsetnosy: vstinner, petr.viktorin, Mark.Shannon
messages: + msg412199 2022-01-31 12:23:26petr.viktorinsetnosy: + petr.viktorin
pull_requests: + pull_request29215
2022-01-13 18:23:11vstinnersetstatus: open -> closedresolution: fixed
messages: + msg410502
stage: patch review -> resolved
2022-01-13 18:22:07vstinnersetmessages: + msg410501 2022-01-12 16:21:35Mark.Shannonsetnosy: + Mark.Shannonmessages: + msg410409
2022-01-12 15:36:25vstinnersetmessages: + msg410404 2022-01-12 14:59:17vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request28759 2022-01-12 14:11:24vstinnercreate