Message 74296 - Python tracker

Message74296

Author	vstinner
Recipients	benjamin.peterson, brett.cannon, loewis, vstinner
Date	2008-10-04.01:42:21
SpamBayes Score	2.4342837e-08
Marked as misclassified	No
Message-id	<1223084543.07.0.547982544852.issue3574@psf.upfronthosting.co.za>
In-reply-to

Content
@brett.cannon: I found it: ast.c used a hack for iso-8859-1! Since this hack introduces a bug (your compile(...) example), I prefer to remove it to simplify to code. The new patch just removes the hack in tokenizer.c and ast.c. It does also document encoding and enc attributes of tok_state in tokenizer.h. Using tokenizer_iso-8859-1-patch2.patch, all tests (test_pep3120.py, iso.py, test_shlex.py, etc.) are OK.

Content

@brett.cannon: I found it: ast.c used a hack for iso-8859-1! Since 
this hack introduces a bug (your compile(...) example), I prefer to 
remove it to simplify to code. The new patch just removes the hack in 
tokenizer.c and ast.c. It does also document encoding and enc 
attributes of tok_state in tokenizer.h.

Using tokenizer_iso-8859-1-patch2.patch, all tests (test_pep3120.py, 
iso.py, test_shlex.py, etc.) are OK.

History
Date	User	Action	Args
2008-10-04 01:42:23	vstinner	set	recipients: + vstinner, loewis, brett.cannon, benjamin.peterson
2008-10-04 01:42:23	vstinner	set	messageid: <1223084543.07.0.547982544852.issue3574@psf.upfronthosting.co.za>
2008-10-04 01:42:22	vstinner	link	issue3574 messages
2008-10-04 01:42:22	vstinner	create