bpo-16806 introduced a regression with the following change:

commit 995d9b92979768125ced4da3a56f755bcdf80f6e (refs/bisect/bad)
Author: Anthony Sottile <asottile@umich.edu>
Date:   Sat Jan 12 20:05:13 2019 -0800

    bpo-16806: Fix `lineno` and `col_offset` for multi-line string tokens (GH-10021)

get-pip.py does now crash:

$ ./python get-pip.py --help
python: Parser/parsetok.c:266: parsetok: Assertion `(intptr_t)(int)(a - line_start) == (a - line_start)' failed.
Aborted (core dumped)

Python 3.8 is also affected.

get-pip.py comes from: https://bootstrap.pypa.io/get-pip.py

Reproducing depends if REALLOC() returns a new pointer, see below.

I attached get-pip2.py: simplified version which crash when using "./python get-pip2.py --help". If it doesn't crash, duplicate lines to make the file larger.

--

The root issue seems to be that tok->multi_line_start isn't updated on REALLOC() in tok_nextc():

                newbuf = (char *)PyMem_REALLOC(newbuf,
                                               newsize);
                if (newbuf == NULL) {
                    tok->done = E_NOMEM;
                    tok->cur = tok->inp;
                    return EOF;
                }
                tok->buf = newbuf;
                tok->cur = tok->buf + cur;
                tok->line_start = tok->cur;

I guess that multi_line_start should also be updated there?

                tok->multi_line_start = tok->cur;

PR 14433 seems to fix this issue and it contains a test.

My PR 14789 fixed get-pip.py, but it introduces another bug: I closed it.

This is marked as release blocker but since BPO-37433 is still in review, I'm releasing 3.0b3 as is. Please resolve this by b4, I will block the last beta on this issue.

> This is marked as release blocker but since BPO-37433 is still in review, I'm releasing 3.0b3 as is. Please resolve this by b4, I will block the last beta on this issue.

That sounds reasonable to me.

FYI I just approved (but not merged) PR 14433.

I just merged PR 14433

This issue is a duplicate of bpo-37433 which has been fixed.

superseder: syntax error in multiline f-string produces ~40k spaces output
stage: patch review -> resolved