If all you need is that

with foo:
   pass

guarantees that either both or neither of __enter__ and __exit__ are called, for C context managers, and only C context managers, then the fix is trivial.

To protect Python code would need a custom context manager wrapper

with ProtectsAgainstInterrupt(user_ctx_mngr()):
    do_stuff()

ProtectsAgainstInterrupt would need to be implemented in C and install a custom signal handler.