Home Original page

Jython, GILs and object locking.

Michael Chermside mcherm at mcherm.com
Fri Oct 10 15:51:19 EDT 2003 
Luis P Caamano writes:
> I have no problem with the statement that removing the GIL
> is hard and we don't have the resources to do that now.  I
> understand that.  The statement I don't like is when people
> say that "the GIL is not a problem at all and therefore,
> it doesn't need fixing."

I don't think that people ARE saying that. I think that if
you polled, most of the python developers would say that
the GIL is an unfortunate limitation, but one which is
particularly difficult to remove, since so MUCH of the
Python interpreter is written with the assumption that
the GIL is used.

> Imagine going back to your boss, customer, or investor
> to say ... "well, we need to re-architect the whole thing
> NOT to use threads because of this little detail unique
> to python.  You see, there's this GIL in python, which ..."

Ahh... but with this YOU are going too far. Just because the
GIL *is* an actual problem, doesn't mean that it's a *big*
problem. In particular, it *IS* okay to do threaded programming
in python. People do it all the time, including some very
large applications (Zope comes to mind) that make heavy use
of threading.

You see, there are multiple reasons for using threads. One
reason would be that you don't want your program to just sit
and do nothing while performing lengthy system calls (things
like waiting to read data from the disk). Python provides
for this... if one thread is blocked for I/O, then the GIL
will be released so other threads can run.

Another reason would be that you want your program to be
doing multiple things at once (eg: a web browser that 
downloads several URLs at once and also reacts to the user
input and renders the screen). Python provides for this...
if you have multiple threads in your program, then there
will be frequent context switches (used to be every 10
bytecodes, but recently changed to every 100 since it 
improved overall performance for most applications).

Another reason would be that you have a spiffy new computer
with multiple processors and you want to get the most out
of them. Python MIGHT be "not too bad" in this case. If
your program is I/O bound or user-input bound rather than
cpu-bound, then it doesn't really matter. If you are 
running several things at once, then Python can happily 
occupy one processor while the other processor(s) work on 
other tasks. And if the program is "fast enough" then you
might not mind whether it takes 0.5 seconds versus 0.25.

But if you DO have a task which is compute bound, and it's
the only task running on the machine, and the machine has
multiple processors, and if you particularly care to have
this task run faster, well then Python is not up to the
job because of the GIL. Of course, you should look into
clever tricks like writing the compute-bound bits for which
you care about speed in C/C++ (which will make it lots faster
anyhow). Then you could structure your C code to avoid
Python data structures and execute with the GIL released.
But if you ARE in such a situation, you'll need to jump
through hoops like this to make use of all your processors.

The thing is... there are lots fewer people in this situation
than there are people who THINK they are in this situation.

-- Michael Chermside
More information about the Python-list mailing list