Home Original page

Python and Boehm-Demers GC, I have code.

Tim Peters tim_one at email.msn.com
Sun Jul 18 04:46:23 EDT 1999 
[Tim, speculates on why pystone/pybench aren't much affected by BDW w/
 refcounting left on]

[Neil Schemenauer]
> What could I use as a better benchmark?  Real applications would
> be best I suppose.

This is a real bind!  People advocating changes in core areas are always
challenged to prove they wouldn't slow things down, but there's no clear way
to do that.  Python is used by so many people for so many things on so many
platforms now, a credible study would be a huge undertaking.

So screw it <wink>.  Find someone with an important program that's leaking
memory and see whether this fixes *that*.  This (not speed) is BDW's real
strength anyway.  Chalk up a few victories there, and selling will be much
easier.

A cautionary tale:  Over the past month I've been poking away at tracking
leaks in IDLE.  After building a capable-enough cycle-finding tool (a
feature-bloated but much faster offshoot of Lars's Plumbo.py), I found
cycles all over the place.  As things turned out, though, hardly any of them
could be considered trash!  The way functions point to their module globals,
which in turn point to imported modules, some of which eventually point into
sys, from whose sys.modules everthing can be reached ... an isolated cycle
was very rare indeed.

It came as a real surprise to me that "real GC" wouldn't have helped.  This
may be a particularly nasty problem for GUI programs, so I'm not too quick
to generalize.  OTOH, don't be too quick to discount it either <wink>.

>> Python doesn't ask anyone to change anything about how they like to
>> handle their memory today, and Guido has pronounced that a Major Feature
>> on multiple occasions.  Vladimir Marangozov's PyMalloc (see his Starship
>> page) caters to that, letting Python use its own malloc without anyone
>> else getting involved.

> Perhaps I misunderstand, but I thought that the PyMem_* functions
> were created for this purpose.  Extension modules can use any
> malloc they like but they can't mix malloc/free/realloc with
> PyMem_*.

No, I misunderstood you.  But now that I don't, it's not worth going back to
explain how I did <wink>.

> ...
> Yes, one platform, no threads.  Thanks a lot Tim.  Here I was
> feeling pretty good about myself.  Now I'm depressed.

Excellent -- if you can persist beyond that, this may yet turn into
something useful instead of Yet Another 48-Hour Abandoned Grand Scheme <0.9
wink>.  Reading the BDW docs and comments closely should give you another
sharp jolt of helpful depression:  this is > a megabyte of code spread over
> 100 files, fighting a gazillion platform peculiarities:  there's nothing
easy about it.

OTOH, they've made oodles of progress over the years. and looks like it's
usable on most major platforms now.  Keep pitching it as a compile-time
config option and I think it's got a chance.

[about Tkinter callbacks]
> <homer>Mmmm, documentation</homer>.  I think I have the problem
> licked now.  I use GC_malloc_uncollectable for these things [ClientData
> structs].

It doesn't matter whether that appeared to work, it only matters whether I
*think* that should work <wink>.  Yup!  That should work.  Nice job, Neil --
that's got to be the minimal hack around this one.

>> btw-keep-in-mind-that-software-doesn't-work<wink>-ly y'rs  - tim

> No kidding.  I tried to compile this stuff under Windows.  What a
> nightmare.  I'm going to have to take some time off and lick my
> wounds.  :)

If it's any consolation, last time I looked hard at BDW was about 6 years
ago, when I gave up after about 40 hours of trying to port it to the KSR
architecture.  This was after the guy in the next office gave up after about
20 hours of trying to stop it dumping core on a SPARC/SunOS box.  It's come
a long way since then!

or-we-could-view-it-as-5x-as-many-lines-to-go-wrong<wink>-ly y'rs  - tim
More information about the Python-list mailing list