Home Original page

Help with cPickle for deserializing datetime.datetime instances

Jp Calderone exarkun at divmod.com
Fri Oct 14 12:28:27 EDT 2005 
On Fri, 14 Oct 2005 01:25:27 -0500, Mingus Tsai <mytsai at u.washington.edu> wrote:
>Hello- please help with unpickling problem:
>
>I am using Python version 2.3.4 with IDLE version 1.0.3 on a Windows
>XPhome system.
>
>My problem is with using cPickle to deserialize my pickled arrays of
>datetime.datetime instances.  The following is the code I have written:
>
>	import cPickle, datetime
>	import Numeric
>
>#the file below contains a serialized dict with arrays of datetime
>#objects.  When these three statements run, the IDLE crashes!
>
>	input1 = open('tsm2_outa','r')
>	time1 = cPickle.load(input1)
>	input1.close()
>
>#the file below contains serialized dict with arrays of built-in objects
>#it unpickles without any problem, when I omit the above unpickling
>#operation.
>
>	input2 = open('tsm2_outb','rb')
>	data1 = cPickle.load(input2)
>	input2.close()
>
>My guess is that I need to somehow tell the pickle.load command that it
>is loading datetime instances, but I have no idea how to do this.  Any
>help would be much appreciated.

As I recall, Pickling Numeric arrays is a tricky business.  You probably shouldn't even try to do it.  Instead, pickle regular arrays or lists.

For anyone else who's interested, this can easily be reproduced without an existing pickle file:

exarkun at boson:~$ python
Python 2.4.2 (#2, Sep 30 2005, 21:19:01) 
[GCC 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Numeric, cPickle, datetime
>>> cPickle.loads(cPickle.dumps(Numeric.array([datetime.datetime.now() for n in range(50)])))
Segmentation fault
exarkun at boson:~$ 

Values smaller than 50 randomly mangle memory, but sometimes don't segfault the interpreter.  You can get exciting objects like instances of cPickle.Pdata or refcnt back from the loads() call in these cases.

So, the summary is, don't do this.

Jp
More information about the Python-list mailing list