Home Original page

[Python-Dev] TextIO seek and tell cookies

Ben Leslie benno at benno.id.au
Mon Sep 26 05:26:38 EDT 2016 
On 25 September 2016 at 17:21, MRAB <python at mrabarnett.plus.com> wrote:
> On 2016-09-26 00:21, Ben Leslie wrote:
>>
>> Hi all,
>>
>> I recently shot myself in the foot by assuming that TextIO.tell
>> returned integers rather than opaque cookies. Specifically I was
>> adding an offset to the value returned by TextIO.tell. In retrospect
>> this doesn't make sense/
>>
>> Now, I don't want to drive change simply because I failed to read the
>> documentation carefully, but I think the current API is very easy to
>> misuse. Most of the time TextIO.tell returns a cookie that is actually
>> an integer and adding an offset to it and seek-ing works fine.
>>
>> The only indication you get that you are mis-using the API is that
>> sometimes tell returns a cookie that when you add an integer offset to
>> it will cause seek() to fail with an OverflowError.
>>
>> Would it be possible to change the API to return something more
>> opaque? E.g.: rather than converting the C cookie structure to a long,
>> could it instead be converted to  a bytes() object.
>>
>> (I.e.: Change textiowrapper_build_cookie to use
>> PyBytes_FromStringAndSize rather than _PyLong_FromByteArray and
>> equivalent for textiowrapper_parse_cookie).
>>
>> This would ensure the return value is never mis-used and is probably
>> also faster using bytes objects than converting to/from an integer.
>>
> why would it be faster? It's an integer internally.


It isn't an integer internally though, it is a cookie:

    typedef struct {
       Py_off_t start_pos;
        int dec_flags;
        int bytes_to_feed;
        int chars_to_skip;
        char need_eof;
    } cookie_type;

The memory view of this structure is then converted to a long. Surely
converting to a PyLong is more work than converting to bytes?
In any case, performance really isn't the motivation here.

Cheers,

Ben
More information about the Python-Dev mailing list