Home Original page

Dealing with "funny" characters

Chris Mellon arkanes at gmail.com
Mon Oct 22 14:44:27 EDT 2007 
On 10/20/07, John Nagle <nagle at animats.com> wrote:
> Gert-Jan wrote:
> > sophie_newbie schreef:
> >> Hi, I want to store python text strings that characters like "é" "Č"
> >> in a mysql varchar text field. Now my problem is that mysql does not
> >> seem to accept these characters. I'm wondering if there is any way I
> >> can somehow "encode" these characters to appear as normal characters
> >> and then "decode" them when I want to get them out of the database
> >> again?
> >
> >
> > It seems you'll have to use Unicode in your program rather than 'plain'
> > strings.
> >
> > Before storing an unicode textstring in a database or a file, you must
> > encode it using an appropriate encoding/codepage, for example:
> >
> > outputstring = unicodeobject.encode('utf-8')
>
>     No, no, that's wrong.  MySQL and the Python interface to it understand
> Unicode.  You don't want to convert data to UTF-8 before putting it in a
> database; the database indexing won't work.
>

Just for the record, if MySQL supports ucs-2 and/or ucs-4,  using the
one that matches your Python build is likely to give you better
runtime performance.
More information about the Python-list mailing list