Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > python simply not scaleable enough for google?

Reply
Thread Tools

python simply not scaleable enough for google?

 
 
Robert P. J. Day
Guest
Posts: n/a
 
      11-11-2009

http://groups.google.com/group/unlad...f544643e?pli=1

thoughts?

rday
--

================================================== ======================
Robert P. J. Day Waterloo, Ontario, CANADA

Linux Consulting, Training and Kernel Pedantry.

Web page: http://crashcourse.ca
Twitter: http://twitter.com/rpjday
================================================== ======================
 
Reply With Quote
 
 
 
 
samwyse
Guest
Posts: n/a
 
      11-11-2009
On Nov 11, 3:57*am, "Robert P. J. Day" <(E-Mail Removed)> wrote:
> http://groups.google.com/group/unlad...hread/thread/4...
>
> * thoughts?


Google's already given us its thoughts:
http://developers.slashdot.org/story...mming-Language
 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      11-14-2009
"Robert P. J. Day" <(E-Mail Removed)> writes:
> http://groups.google.com/group/unlad...f544643e?pli=1
> thoughts?


I'd bet it's not just about multicore scaling and general efficiency,
but also the suitability of the language itself for large, complex
projects. It's just not possible to be everything for everybody.
Python is beginner-friendly, has a very fast learning curve for
experienced programmers in other languages, and is highly productive
for throwing small and medium sized scripts together, that are
debugged through iterated testing. One might say it's optimized for
those purposes. I use it all the time because a lot of my programming
fits the pattern. The car analogy is the no-frills electric commuter
car, just hop in and aim it where you want to go; if you crash it,
brush yourself off and restart. But there are times (large production
applications) when you really want the Airbus A380 with the 100's of
automatic monitoring systems and checkout procedures to follow before
you take off, even if the skill level needed to use it is much higher
than the commuter car.
 
Reply With Quote
 
Vincent Manis
Guest
Posts: n/a
 
      11-14-2009
On 2009-11-13, at 19:53, Paul Rubin wrote:

> "Robert P. J. Day" <(E-Mail Removed)> writes:
>> http://groups.google.com/group/unlad...f544643e?pli=1
>> thoughts?

>
> I'd bet it's not just about multicore scaling and general efficiency,
> but also the suitability of the language itself for large, complex
> projects. It's just not possible to be everything for everybody.
> Python is beginner-friendly, has a very fast learning curve for
> experienced programmers in other languages, and is highly productive
> for throwing small and medium sized scripts together, that are
> debugged through iterated testing. One might say it's optimized for
> those purposes. I use it all the time because a lot of my programming
> fits the pattern. The car analogy is the no-frills electric commuter
> car, just hop in and aim it where you want to go; if you crash it,
> brush yourself off and restart. But there are times (large production
> applications) when you really want the Airbus A380 with the 100's of
> automatic monitoring systems and checkout procedures to follow before
> you take off, even if the skill level needed to use it is much higher
> than the commuter car.


OK. The quoted link deals with Unladen Swallow, which is an attempt to deal with the
very real performance limitations of current Python systems. The remarks above deal with
productivity scalability, which is a totally different matter. So...

People can and do write large programs in Python, not just `throwing...medium sized
scripts together'. Unlike, say, Javascript, it has the necessary machinery to build very
large programs that are highly maintainable. One can reasonably compare it with Java, C#,
and Smalltalk; the facilities are comparable, and all of those (as well as Python) are
used for building enterprise systems.

I believe that the A380's control software is largely written in Ada, which is a
perfectly fine programming language that I would prefer not to write code in. For
approximately 10 years, US DOD pretty much required the use of Ada in military (and
aerospace) software (though a a couple of years ago I discovered that there is still
one remaining source of Jovial compilers that still sells to DOD). According to a
presentation by Lt. Colonel J. A. Hamilton, `Programming Language Policy in the DOD:
After The Ada Mandate', given in 1999, `We are unlikely to see a return of a programming
language mandate' (www.drew-hamilton.com/stc99/stcAda_99.pdf). As I understand it,
the removal of the Ada mandate came from the realization (20 years after many computer
scientists *told* DOD this) that software engineering processes contribute more to
reliability than do programming language structures (c.f. Fred Brooks, `No Silver
Bullet').

So: to sum up, there are lots of large systems where Python might be totally appropriate,
especially if complemented with processes that feature careful specification and strong
automated testing. There are some large systems where Python would definitely NOT be
the language of choice, or even usable at all, because different engineering processes
were in place.

From a productivity viewpoint, there is no data to say that Python is more, less, or equally
scalable than <Language X> in that it produces correctly-tested, highly-maintainable programs
at a lower, higher, or equal cost. I would appreciate it if people who wanted to comment on
Python's scalability or lack thereof would give another programming language that they would
compare it with.

-- v



 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      11-19-2009
In article <(E-Mail Removed)>,
Robert P. J. Day <(E-Mail Removed)> wrote:
>
>http://groups.google.com/group/unlad...f544643e?pli=1
>
> thoughts?


Haven't seen this elsewhere in the thread:

http://dalkescientific.com/writings/..._tasklets.html
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it." --Brian W. Kernighan
 
Reply With Quote
 
Robin Becker
Guest
Posts: n/a
 
      11-20-2009
Aahz wrote:
> In article <(E-Mail Removed)>,
> Robert P. J. Day <(E-Mail Removed)> wrote:
>> http://groups.google.com/group/unlad...f544643e?pli=1
>>
>> thoughts?

>
> Haven't seen this elsewhere in the thread:
>
> http://dalkescientific.com/writings/..._tasklets.html



I looked at this and it looks very good in that stackless appears twice as fast
as go(lang) (I used to be in the department of computing at Imperial so I
suppose I have to side with McCabe).

Anyhow, my reading of why Pike was so proud of his set up and tear down of the
tasks example was that these were real threads.

Presumably that means they could potentially run in parallel on the 100000 cpu
machines of the future.

I'm not so clear on whether the threadless tasklets will run on separate cpus.
--
Robin Becker

 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      11-20-2009
On 20 Nov, 11:12, Robin Becker <(E-Mail Removed)> wrote:

> Presumably that means they could potentially run in parallel on the 100000 cpu
> machines of the future.
>
> I'm not so clear on whether the threadless tasklets will run on separate cpus.


You can make a user-space scheduler and run a 100000 tasklets on a
threadpool. But there is a GIL in stackless as well.

Nobody wants 100000 OS threads, not with Python, not with Go, not with
C.

Also note that Windows has native support for "taskelets", regardless
of language. They are called "fibers" (as opposed to "threads") and
are created using the CreateFiber system call. I would not be
surprised if Unix'es has this as well. We do not need Stackless for
light-weight threads. We can just take Python's threading modules' C
code and replace CreateThread with CreateFiber.












 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      11-20-2009
In article <(E-Mail Removed)>,
sturlamolden <(E-Mail Removed)> wrote:
>
>Also note that Windows has native support for "taskelets", regardless
>of language. They are called "fibers" (as opposed to "threads") and are
>created using the CreateFiber system call. I would not be surprised if
>Unix'es has this as well. We do not need Stackless for light-weight
>threads. We can just take Python's threading modules' C code and
>replace CreateThread with CreateFiber.


Are you advocating a high-fiber diet?
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it." --Brian W. Kernighan
 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      11-21-2009
On 20 Nov, 22:45, (E-Mail Removed) (Aahz) wrote:

> Are you advocating a high-fiber diet?


Only if you are a ruminant.

No really...

Windows has user-space threads natively. But you must reserve some
stack space for them (from virtual memory), which mainly makes them
useful on 64 bit systems.




 
Reply With Quote
 
Nobody
Guest
Posts: n/a
 
      11-21-2009
On Fri, 20 Nov 2009 09:51:49 -0800, sturlamolden wrote:

> You can make a user-space scheduler and run a 100000 tasklets on a
> threadpool. But there is a GIL in stackless as well.
>
> Nobody wants 100000 OS threads, not with Python, not with Go, not with
> C.
>
> Also note that Windows has native support for "taskelets", regardless
> of language. They are called "fibers" (as opposed to "threads") and
> are created using the CreateFiber system call. I would not be
> surprised if Unix'es has this as well. We do not need Stackless for
> light-weight threads. We can just take Python's threading modules' C
> code and replace CreateThread with CreateFiber.


POSIX.1-2001 and POSIX.1-2004 have makecontext(), setcontext(),
getcontext() and swapcontext(), but obsoleted by POSIX.1-2008.

They are available on Linux; I don't know about other Unices.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: python simply not scaleable enough for google? Terry Reedy Python 85 11-21-2009 02:05 AM
"pointing" to functions to execute for scaleable code metaperl Java 4 03-03-2007 06:30 PM
Scaleable naviagtion pane jsqrd ASP .Net 2 03-02-2007 04:48 PM
Scaleable Swing components Jens Piegsa Java 0 12-16-2006 09:22 AM
SVG scaleable? Sascha Kerschhofer XML 6 07-16-2003 01:37 PM



Advertisments