Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Will Python 3.0 remove the global interpreter lock (GIL)

Reply
Thread Tools

Will Python 3.0 remove the global interpreter lock (GIL)

 
 
king kikapu
Guest
Posts: n/a
 
      09-03-2007
I was wondering (and maybe i still do) about this GIL "problem". I am
relatively new to Python (less than a year) and when i started to
think about it i said: "Oh, this IS a problem". But when i dig a
little more, i found that "Ah, maybe it isn't".
I strongly believe that the best usage of multiple cores processor
will be achieved if programming languages are modified to support this
on their "hearts". Code blocks that would be identified by the
compiler and run in parallel and such things. Laboratories are working
on these stuff but i do not expect something in the very-near future.

So, as i mentioned above, there are solutions for that right now
("parallel python" and others) that enabled us with little effort to
spawn a new python interpreter, thus allowing the OS to schedule it on
a different core and do the job this way relatively cheap.
I wouldn't recommend going to IronPython despite the fact that the CLR
better utilize MP. The reason for this is that i would NEVER give up
the freedom that CPython gives me by exchange "better" usage of the MP
and platform lock-in.

 
Reply With Quote
 
 
 
 
king kikapu
Guest
Posts: n/a
 
      09-03-2007
I was wondering (and maybe i still do) about this GIL "problem". I am
relatively new to Python (less than a year) and when i started to
think about it i said: "Oh, this IS a problem". But when i dig a
little more, i found that "Ah, maybe it isn't".
I strongly believe that the best usage of multiple cores processor
will be achieved if programming languages are modified to support this
on their "hearts". Code blocks that would be identified by the
compiler and run in parallel and such things. Laboratories are working
on these stuff but i do not expect something in the very-near future.

So, as i mentioned above, there are solutions for that right now
("parallel python" and others) that enabled us with little effort to
spawn a new python interpreter, thus allowing the OS to schedule it on
a different core and do the job this way relatively cheap.
I wouldn't recommend going to IronPython despite the fact that the CLR
better utilize MP. The reason for this is that i would NEVER give up
the freedom that CPython gives me by exchange "better" usage of the MP
and platform lock-in.

 
Reply With Quote
 
 
 
 
TheFlyingDutchman
Guest
Posts: n/a
 
      09-19-2007
On Sep 2, 5:38 pm, "Eduardo O. Padoan" <(E-Mail Removed)>
wrote:
> > No.http://www.artima.com/weblogs/viewpo...?thread=211430

>
> Ops, I meant:http://www.artima.com/forums/threade...&thread=211200
>
> --http://www.advogato.org/person/eopadoan/
> Bookmarks:http://del.icio.us/edcrypt


"No. We're not changing the CPython implementation much. Getting rid
of the GIL would be a massive rewrite of the interpreter because all
the internal data structures (and the reference counting operations)
would have to be made thread-safe. This was tried once before (in the
late '90s by Greg Stein) and the resulting interpreter ran twice as
slow."

How much faster/slower would Greg Stein's code be on today's
processors versus CPython running on the processors of the late
1990's? And if you decide to answer, please add a true/false response
to this statement - "CPython in the late 1990's ran too slow".

 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      09-19-2007

"TheFlyingDutchman" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
| On Sep 2, 5:38 pm, "Eduardo O. Padoan" <(E-Mail Removed)>
| wrote:
| > > No.http://www.artima.com/weblogs/viewpo...?thread=211430
| >
| > Ops, I
meant:http://www.artima.com/forums/threade...&thread=211200
| >
| > --http://www.advogato.org/person/eopadoan/
| > Bookmarks:http://del.icio.us/edcrypt
|
| "No. We're not changing the CPython implementation much. Getting rid
| of the GIL would be a massive rewrite of the interpreter because all
| the internal data structures (and the reference counting operations)
| would have to be made thread-safe. This was tried once before (in the
| late '90s by Greg Stein) and the resulting interpreter ran twice as
| slow."

Since Guido wrote that, there have been put forth more ideas and interest
and promises of efforts to remove or revise the GIL or do other things to
make using multiple cores easier. (The later being the point of the
concern over GIL.)

| How much faster/slower would Greg Stein's code be on today's
| processors versus CPython running on the processors of the late
| 1990's?

Perhaps a bit faster, though processor speeds have not increased so much
the last couple of years.

|And if you decide to answer, please add a true/false response
| to this statement - "CPython in the late 1990's ran too slow".

False by late 1990's standards, True by today's standards .

Most people are not currently bothered by the GIL and would not want its
speed halved.

In any case, any of the anti-GIL people are free to update Stein's code and
distribute a GIl-less version of CPython. (Or to use Jython or
IronPython.)

tjr





 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-19-2007
On Tue, 18 Sep 2007 18:09:26 -0700, TheFlyingDutchman wrote:

> How much faster/slower would Greg Stein's code be on today's processors
> versus CPython running on the processors of the late 1990's?


I think a better question is, how much faster/slower would Stein's code
be on today's processors, versus CPython being hand-simulated in a giant
virtual machine made of clockwork?



--
Steven.
 
Reply With Quote
 
Grant Edwards
Guest
Posts: n/a
 
      09-19-2007
On 2007-09-19, Steven D'Aprano <(E-Mail Removed)> wrote:
> On Tue, 18 Sep 2007 18:09:26 -0700, TheFlyingDutchman wrote:
>
>> How much faster/slower would Greg Stein's code be on today's
>> processors versus CPython running on the processors of the
>> late 1990's?

>
> I think a better question is, how much faster/slower would
> Stein's code be on today's processors, versus CPython being
> hand-simulated in a giant virtual machine made of clockwork?


That depends on whether you have the steam-driven model or the
water-wheel driven model. The steam-drive one is quite a bit
faster once you get it going, but the water-wheel model has a
much shorter start-up time (though it is more weather
dependent).

--
Grant Edwards grante Yow! Hey, waiter! I want
at a NEW SHIRT and a PONY TAIL
visi.com with lemon sauce!
 
Reply With Quote
 
TheFlyingDutchman
Guest
Posts: n/a
 
      09-19-2007
On Sep 19, 8:51 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> On Tue, 18 Sep 2007 18:09:26 -0700, TheFlyingDutchman wrote:
> > How much faster/slower would Greg Stein's code be on today's processors
> > versus CPython running on the processors of the late 1990's?

>
> I think a better question is, how much faster/slower would Stein's code
> be on today's processors, versus CPython being hand-simulated in a giant
> virtual machine made of clockwork?
>
> --
> Steven.


Steven, You forgot this part:

"And if you decide to answer, please add a true/false response
to this statement - "CPython in the late 1990's ran too slow"'.


 
Reply With Quote
 
Paul Boddie
Guest
Posts: n/a
 
      09-19-2007
On 19 Sep, 03:09, TheFlyingDutchman <(E-Mail Removed)> wrote:
>
> How much faster/slower would Greg Stein's code be on today's
> processors versus CPython running on the processors of the late
> 1990's? And if you decide to answer, please add a true/false response
> to this statement - "CPython in the late 1990's ran too slow".


Too slow for what? And what's the fixation with CPython, anyway? Other
implementations of Python 2.x don't have the GIL. Contrary to popular
folklore, Jython has been quite a reasonable implementation of Python
for about half as long as CPython has been around, if you don't mind
the JVM. I'm sure people have lots of complaints about Jython like
they do about CPython and the GIL, thinking that complaining about it
is going to make the situation better, or that they're imparting some
kind of "wisdom" to which the people who actually wrote the code must
be oblivious, but nobody is withholding the code from anyone who wants
to actually improve it.

And there we learn something: that plenty of people are willing to
prod others into providing them with something that will make their
lives better, their jobs easier, and their profits greater, but not so
many are interested in contributing back to the cause and taking on
very much of the work themselves. Anyway, the response to your
statement is "false". Now you'll have to provide us with the insight
we're all missing. Don't disappoint!

Paul

 
Reply With Quote
 
TheFlyingDutchman
Guest
Posts: n/a
 
      09-19-2007
On Sep 19, 3:41 pm, Paul Boddie <(E-Mail Removed)> wrote:
> On 19 Sep, 03:09, TheFlyingDutchman <(E-Mail Removed)> wrote:
>
>
>
> > How much faster/slower would Greg Stein's code be on today's
> > processors versus CPython running on the processors of the late
> > 1990's? And if you decide to answer, please add a true/false response
> > to this statement - "CPython in the late 1990's ran too slow".

>
> Too slow for what? And what's the fixation with CPython, anyway? Other
> implementations of Python 2.x don't have the GIL. Contrary to popular
> folklore, Jython has been quite a reasonable implementation of Python
> for about half as long as CPython has been around, if you don't mind
> the JVM. I'm sure people have lots of complaints about Jython like
> they do about CPython and the GIL, thinking that complaining about it
> is going to make the situation better, or that they're imparting some
> kind of "wisdom" to which the people who actually wrote the code must
> be oblivious, but nobody is withholding the code from anyone who wants
> to actually improve it.


>
> And there we learn something: that plenty of people are willing to
> prod others into providing them with something that will make their
> lives better, their jobs easier, and their profits greater, but not so
> many are interested in contributing back to the cause and taking on
> very much of the work themselves. Anyway, the response to your
> statement is "false". Now you'll have to provide us with the insight
> we're all missing. Don't disappoint!
>
> Paul


Paul it's a pleasure to see that you are not entirely against
complaints.

The very fastest Intel processor of the last 1990's that I found came
out in October 1999 and had a speed around 783Mhz. Current fastest
processors are something like 3.74 Ghz, with larger caches. Memory is
also faster and larger. It appears that someone running a non-GIL
implementation of CPython today would have significantly faster
performance than a GIL CPython implementation of the late 1990's.
Correct me if I am wrong, but it seems that saying non-GIL CPython is
too slow, while once valid, has become invalid due to the increase in
computing power that has taken place.

 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      09-20-2007

"Terry Reedy" <(E-Mail Removed)> wrote in message
news:fcq352$u0a$(E-Mail Removed)...
|
| "TheFlyingDutchman" <(E-Mail Removed)> wrote in message
| news:(E-Mail Removed) oups.com...

| Since Guido wrote that, there have been put forth more ideas and interest
| and promises of efforts to remove or revise the GIL or do other things to
| make using multiple cores easier. (The later being the point of the
| concern over GIL.)

A few days ago, an undergraduate posted on the dev list that he just
started an independent study project on removing the GIL. Maybe we'll get
a report early next year.

Guido also said that he is willing to make changes to the CPython internals
to aid multiproccessor usage [as long, presumably, as it does not cut speed
in half].

|| How much faster/slower would Greg Stein's code be on today's
|| processors versus CPython running on the processors of the late
|| 1990's?
|
| Perhaps a bit faster, though processor speeds have not increased so much
| the last couple of years.

This assumes that comparing versions of 1.5 is still relevant. As far as I
know, his patch has not been maintained to apply against current Python.
This tells me that no one to date really wants to dump the GIL at the cost
of half Python's speed. Of course not. The point of dumping the GIL is to
use multiprocessors to get more speed! So with two cores and extra
overhead, Stein-patched 1.5 would not even break even.

Quad (and more) cores are a different matter. Hence, I think, the
resurgence of interest.

||And if you decide to answer, please add a true/false response
|| to this statement - "CPython in the late 1990's ran too slow".
|
| False by late 1990's standards, True by today's standards .

So now this question for you: "CPython 2.5 runs too slow in 2007: true or
false?"

If you answer false, then there is no need for GIL removal.
If you answer true, then cutting its speed for 90+% of people is bad.

| Most people are not currently bothered by the GIL and would not want its
| speed halved.

And another question: why should such people spend time they do not have to
make Python worse for themselves?

Terry Jan Reedy




 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
global interpreter lock Tommy.Ryding@gmail.com Python 2 10-19-2005 06:44 AM
global interpreter lock Paul Rubin Python 59 09-15-2005 07:50 PM
Global Interpreter Lock Tomas Christiansen Python 3 09-24-2004 10:00 PM
Threading - Why Not Lock Objects Rather than lock the interpreter Fuzzyman Python 3 12-05-2003 10:43 PM
Re: Thread State and the Global Interpreter Lock Aahz Python 0 06-28-2003 01:20 PM



Advertisments