Home Original page

python shutting down sloooooooowly/tuning dictionaries

Till Plewe till at score.is.tsukuba.ac.jp
Wed Apr 21 12:38:01 EDT 2004 
On Wed, Apr 21, 2004 at 06:12:15PM +0200, Brian Quinlan wrote:
> >Since the entire database is too large I typically run the same
> >program 10-100 times on smaller parts. The annoying bit is that the
> >time for finishing one instance of a program before starting the next
> >instance can take a lot of time.  ...
> 
> You might try disabling garbage collection using the gc module when you 
> your application is done. Explicitly closing any files bound to globals 
> might then be a good idea.
> 
> Cheers,
> Brian
> 

Thanks for the suggestions but unfortunately I already tried disabling
gc. It did not have any noticable effect (not that I was patient
enough to time it) nor did using sys.exit(). Also, I am not using
threads which excludes another possible explanation. 

To give an example of the actual time spans involved: 2-10 minutes for
running the program and > 30 minutes for shutdown. During that time
the amount of memory used by python does not seem to vary.

A far as closing files is concerned, I am using cdb for writing 
read-only databases and I am using cdb.finish() (there is no explicit
method for closing(); finish writes all key/value pairs to a temporary
file and then moves that file to wherever you want it. But as far as I can see
as soon as that file is written/has been moved no more I/O should take place.
There is no data loss if I simply kill the python program. 
 
- Till
More information about the Python-list mailing list