Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Thread State and the Global Interpreter Lock

Thread Tools

Re: Thread State and the Global Interpreter Lock

Posts: n/a
In article <3ef8b9b2$(E-Mail Removed)>, Ryjek <(E-Mail Removed)> wrote:
>Our application currently consists of two processes: one is a database
>and one is a windowing program with a user interface. Both embed a
>Python interpreter for customization purposes (i.e. you can write your
>own GUI extensions, and your own database procedures).
>I would like to merge both processes into one process with two threads
>(mostly for performance purposes), but keep both threads' data separated
>the same way as they are now. Python practically does not allow me to do
>it because it insists that only one thread can execute Python bytecodes
>at a time. I would like to be able to run both interpreters in
>completely separate environments, where no state and no objects are
>shared. I believe the restriction of "one thread at a time" is too
>strict in such cases. What is the purpose of Py_NewInterpreter() if you
>cannot run it concurrently ..?

It was a mistake, essentially, kept around for the rare occasions when
the caveats below don't apply.

I would say that if performance is your primary goal, multi-threading
your application would be a poor idea. Threads work best in two
contexts: breaking up similar types of work for performance (and in the
case of Python, that pretty much needs to be I/O work), and increasing
the responsiveness of user-centered applications. Unless you're sharing
huge amounts of data, it's quite likely that even outside of Python you
wouldn't see much (if any) performance increase.

>I understand that making Python completely thread safe w.respect to
>native threads may not be appropriate, but it shouldn't be that
>difficult to allow for real concurrent execution in isolated
>environments. At least, this is what they did in Perl:

The problem is that Python relies heavily on DLLs, both internally and
third-party libraries. It's pretty much impossible to set things up so
that random DLLs can be correctly loaded by multiple interpreters in a
single process.

Because threading has existed in core Python for many years, a lot of
bugs and issues have been hashed out -- it's not at all clear to me the
extent to which Perl has done the same. One reason why Python sticks
with the GIL model is that multi-threading does diddly-squat for
computational threads on a single-CPU box. It's actually *more*
efficient the way Python does it.
Aahz ((E-Mail Removed)) <*>

Usenet is not a democracy. It is a weird cross between an anarchy and a
Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Regular expressions and the global interpreter lock Duncan Grisby Python 0 11-18-2005 03:47 PM
global interpreter lock Python 2 10-19-2005 06:44 AM
global interpreter lock Paul Rubin Python 59 09-15-2005 07:50 PM
Global Interpreter Lock Tomas Christiansen Python 3 09-24-2004 10:00 PM
Threading - Why Not Lock Objects Rather than lock the interpreter Fuzzyman Python 3 12-05-2003 10:43 PM