Python's secondary prompt in interactive mode is an artifact of the fact the command line consoles work with physical lines while python works with statements that quite possibly comprises multiple lines.

As mentioned in the its doc, IDLE works with statements.  You edit and enter complete statements, not a line at a time. The history mechanism retrieves complete statements, not a line at a time.  The side effect is what you encountered.  '>>> ' is not an indent.  '    ' is.

While working with statements instead of lines is one of Shell's best features, the indentation awkwardness is one of its worst.  I intend to open an issue about separating the prompt from statement entry to remove the latter.  I will add you as nosy so you can give an opinion, I hope in support ;-).