Miki Tebeka miki.tebeka at zoran.com
Thu Sep 2 02:24:19 EDT 2004

• Previous message (by thread): Basic tokenizer
• Next message (by thread): Basic tokenizer
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

Hello Dale,

> I'm looking for a simple string tokenizer. The Python tokenize module is 
> too specific and is geared to Python syntax.
> 
> I just want something that can be setup to read the basic stuff like 
> . identifiers 
> . integers (and/or reals)
> . strings
> . special characters such as operators and brackets.
> 
> Is anyone aware of such a module?
> 
> Thanks.
There are several parsing/lexing packages for Python. My favorite is PLY
(http://systems.cs.uchicago.edu/ply/).

Bye.
--
------------------------------------------------------------------------
Miki Tebeka <miki.tebeka at zoran.com>
http://tebeka.spymac.net
The only difference between children and adults is the price of the toys

---

• Previous message (by thread): Basic tokenizer
• Next message (by thread): Basic tokenizer
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

More information about the Python-list mailing list