Home Original page

Jython, GILs and object locking.

Carl Banks imbosol at aerojockey.invalid
Fri Oct 10 23:22:30 EDT 2003 
Jon Franz wrote:
>>    Python supports this. By having the I/O operations release the GIL
>> multiple CPUs may perform work simultaneously.
> 
> You mean an extension in which you hand off the I/O from your
> python program to a (c) module that handles the work in a truely
> seperate thread?

No, he doesn't.


>  Writing a c extension should not be required
> to use multiple CPUs.  If I am wrong, and python _can_ use
> multiple cpus without using such an extension, please let me know.

No, you're (mostly) wrong.

It is true that two threads cannot execute Python bytecode at the same
time, and if you want to use multiple CPUs, all but one thread has to
be executing C code.

However, you don't appear to realize that there are a lot of functions
built into Python (functions that you use every day, especially I/O
functions) that are written in C, and, because they release the GIL,
can run at the same time another thread runs bytecode on another
processor.

Your misconception is that *YOU* have to write a C extension to take
advantage of multiple CPUs.  *YOU* don't, because there is already C
code that does that built into Python.

Make sense?  Someone does have to write C code to use multiple CPUs,
but it doesn't necessarily have to be you.


Having said all that, I would rather have fine-grained locking, too.
As it is now, to take advantage of multiple CPUs, you either have to
hope a lot of your work is being done by code that releases the GIL,
or you have to write C code that does it yourself.


>> > I think, however, that until (IF) the GIL is removed, a
>> > simple documentation change could be very useful.  That
>> > is, in the thread module docs, and the tutorials (and
>> > anyplace that mentions threading) mention that python, due to
>> > the GIL, cannot use multiple processors.
>> 
>>    That would be inaccurate. The Python documents should be accurate.
> 
> See the above.  The python documentation should be accurate for 
> programming with python, not what you can do if you write an 
> extension module.

What you wrote is flatly wrong: no matter how often you don't use
extension modules, Python still *can* use multiple CPUs.

The correct statement is, "Because of the GIL, only one processor can
run Python bytecode at the same time."  And it's best to add something
like, "However, many functions, especially I/O functions, are written
in C, and release the GIL while they work, so that Python can get some
use out of multiple processors."


> Are there such modules freely in existence?

As we've discussed: pretty much all I/O code built into Python.
Probably other code, too.


-- 
CARL BANKS                   http://www.aerojockey.com/software

As the newest Lady Turnpot descended into the kitchen wrapped only in
her celery-green dressing gown, her creamy bosom rising and falling
like a temperamental souffle, her tart mouth pursed in distaste, the
sous-chef whispered to the scullery boy, "I don't know what to make of
her." 
          --Laurel Fortuner, Montendre, France 
            1992 Bulwer-Lytton Fiction Contest Winner
More information about the Python-list mailing list