Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea in general -- the standard library is likely to cover a lot of ground, and is definitely "real code people actually use".

However, this benchmark is likely to change even when the contents of the stdlib changes (not just when files are added/removed). So I think the only way to make a stable benchmark out of this would be to actually include a copy of the stdlib here. That's probably reasonably forward-compatible to new versions of Python (at least as much as any random library).

Given that would be kind of a weird thing to do, I guess I'd like to get some other feedback to see what others think before merging this.