For Python version 3.7 and above the following statement will end up in a segfault.

eval("1 + 100"*1000000)

Whereas Python versions 3.6 and below would tread this as a Recursion error.

Looks like it was introduced by https://github.com/python/cpython/commit/7ea143ae795a9fd57eaccf490d316bdc13ee9065:

bpo-29469: Move constant folding to AST optimizer (GH-2858)

In python 3.7/8, It's a stack overflow in the constant folding code.

On master, the overflow seems to come out of validate_expr.c.

* thread #1, name = 'python3', stop reason = signal SIGSEGV: invalid address (fault address: 0x7fffff7feff8)
    frame #0: 0x00005555557aadba python3`validate_expr(exp=0x00005555602617c0, ctx=Load) at ast.c:224:16
   221          }
   222          return validate_exprs(exp->v.BoolOp.values, Load, 0);
   223      case BinOp_kind:
-> 224          return validate_expr(exp->v.BinOp.left, Load) &&
   225              validate_expr(exp->v.BinOp.right, Load);
   226      case UnaryOp_kind:
   227          return validate_expr(exp->v.UnaryOp.operand, Load);


300,000 ish stack frames of this:

    frame #70832: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150af40, ctx=Load) at ast.c:224:16
    frame #70833: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b050, ctx=Load) at ast.c:224:16
    frame #70834: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b160, ctx=Load) at ast.c:224:16
    frame #70835: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b270, ctx=Load) at ast.c:224:16
    frame #70836: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b380, ctx=Load) at ast.c:224:16
    frame #70837: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b490, ctx=Load) at ast.c:224:16
    frame #70838: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b5a0, ctx=Load) at ast.c:224:16
    frame #70839: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b6b0, ctx=Load) at ast.c:224:16


On the one hand, pure python code should never segfault, on the other hand, `eval`ling untrusted input has bigger problems than a segfault on carefully crafted input.

This is known issue, but interesting that the cause of the crash is different in 3.7-3.8 and 3.9+.

PR 23744 adds recursion checks in the AST validator and optimizer similar to the checks in the symtable. It should not break any existing code because too deep AST tree did not pass checks in the symtable in any case.

But it does not solve all problems. A compound statement with too many "elif"s is still crashed because the new parser uses recursion in C to parse it (elif_stmt_rule). I think it should be a separate issue.

New changeset face87c94e67ad9c72b9a3724f112fd76c1002b9 by Serhiy Storchaka in branch 'master':
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744)
https://github.com/python/cpython/commit/face87c94e67ad9c72b9a3724f112fd76c1002b9

pull_requests: + pull_request22602
stage: patch review