I was wandering whether the pointer returned by PyUnicode_AsString needs
to be freed after usage (It turned it doesn't since the result is
cached). However, I found out that there isn't any documentation on
docs.python.org about the PyUnicode_AsString and
PyUnicode_AsStringAndSize functions. Although, both are documented in
the public unicodeobject.h header.

I notice that the documentation for several other unicode functions is
missing. Quickly, I see:

  PyUnicode_Resize
  PyUnicode_InternImmortal
  PyUnicode_GetDefaultEncoding
  PyUnicode_SetDefaultEncoding
  PyUnicode_BuildEncodingMap
  PyUnicode_FromFormatV
  PyUnicode_*UTF7*
  PyUnicode_AsEncodedObject
  PyUnicode_FromOrdinal
  PyUnicode_DecodeFSDefault
  PyUnicode_DecodeFSDefaultAndSize

It would probably be a good idea to polish up the documentation for
PyUnicode as much as possible for Python 3000, since extension
developers will certainly need to refer to it a lot during the
transition from 2.x.

In addition to the above mentioned functions I found these to be
undocumented:

  PyUnicode_DecodeUTF7
  PyUnicode_DecodeUTF7Stateful
  PyUnicode_EncodeDecimal
  PyUnicode_EncodeUTF7
  PyUnicode_FromFormat
  PyUnicode_FromString
  PyUnicode_FromStringAndSize
  PyUnicode_GetMax
  PyUnicode_Partition
  PyUnicode_RPartition
  PyUnicode_RSplit

From the original list the following functions seem to have been removed:

  PyUnicode_InternImmortal
  PyUnicode_DecodeFSDefault
  PyUnicode_DecodeFSDefaultAndSize


I try to put together a patch for some of these during the weekend.

Ok, here is my shot at a patch for at least some of the undocumented
functions. Namely the following functions are being documented in the patch:

  PyUnicode_FromFormat
  PyUnicode_FromFormatV
  PyUnicode_FromString
  PyUnicode_FromStringAndSize
  PyUnicode_Partition
  PyUnicode_RPartition
  PyUnicode_RSplit

Please thoroughly review this patch since I didn't really digg into the
source to find out what the functions do but rather just copied old
PyString documentation or derived it from the docs for the Python API.

The patch looks alright. I don't like the documentation for
PyUnicode_FromFormatV, however. Here's my attempt to document it:

.. cfunction:: PyObject* PyUnicode_FromFormatV(const char *format,
va_list vargs)

   Equivalent to the function :cfunc:`PyUnicode_FromFormat`, except that
it takes a va_list instead of variable number of arguments.

Is it worth applying the patch given the complete rewrite of unicode for 3.3 via PEP393?

On 30.03.2013 13:09, Mark Lawrence wrote:
> 
> Is it worth applying the patch given the complete rewrite of unicode for 3.3 via PEP393?

PEP 393 only changed the way Unicode is internally stored.
The Unicode API is mostly unaffected by this change.

Remaining undocumented functions:

From this issue:

PyUnicode_RSplit
PyUnicode_Partition
PyUnicode_RPartition

From issue 10435:

PyUnicode_IsIdentifier
PyUnicode_Append
PyUnicode_AppendAndDel
PyUnicode_GetDefaultEncoding
PyUnicode_FromOrdinal
PyUnicode_Resize
PyUnicode_GetMax
PyUnicode_InternImmortal
PyUnicode_CHECK_INTERNED

From issue 18688:

Py_UNICODE_REPLACEMENT_CHARACTER
PyUnicodeIter_Type
PyUnicode_AsDecodedObject
PyUnicode_AsDecodedUnicode
PyUnicode_AsEncodedObject
PyUnicode_AsEncodedUnicode
PyUnicode_BuildEncodingMap

PyUnicode_DecodeCodePageStateful

Following functions likely should be wrapped with "#ifndef Py_LIMITED_API":

_PyUnicode_ClearStaticStrings
_PyUnicode_EQ
_PyUnicode_FromId

"""
Following functions likely should be wrapped with "#ifndef Py_LIMITED_API":

_PyUnicode_ClearStaticStrings
_PyUnicode_EQ
_PyUnicode_FromId
"""

It's already the case since at least Python 3.7. Extract of Python 3.7 Include/unicodeobject.h:

#ifndef Py_LIMITED_API
/* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/
PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*);
/* Clear all static strings. */
PyAPI_FUNC(void) _PyUnicode_ClearStaticStrings(void);

/* Fast equality check when the inputs are known to be exact unicode types
   and where the hash values are equal (i.e. a very probable match) */
PyAPI_FUNC(int) _PyUnicode_EQ(PyObject *, PyObject *);
#endif /* !Py_LIMITED_API */

pull_requests: + pull_request19322
stage: needs patch -> patch review

keywords: - easy
nosy: + berker.peksag
versions: + Python 3.5, Python 3.6, - Python 3.1, Python 2.7, Python 3.2
messages: + msg264571
stage: needs patch

nosy: + docs@python

messages: + msg86116
versions: + Python 2.6, Python 3.1, Python 2.7