Here is a file which is accepted by Python interpreter but is rejected by the tokenize module.

$ ./python nonutf8_coding_line.py 
$ ./python -m tokenize nonutf8_coding_line.py 
nonutf8_coding_line.py: error: invalid or missing encoding declaration for 'nonutf8_coding_line.py'

Python itself checks that a line is UTF-8 encoded only if not found a magic comment. The tokenize module checks it before searching (issue14629).