«

On 2011-01-18, Steven D'Aprano <steve+> wrote:
> On Mon, 17 Jan 2011 19:41:54 +0000, Tim Harig wrote:
>
>> One of the arguments for Python has always made is that you can optimize
>> it by writing the most important parts in C. Perhaps that is a crutch
>> that has held the communty back from seeking higher performance
>> solutions in the language itself.
>
> Are you aware of PyPy?

Yes I have heard of PyPy, RPython, and Cython. If they were sufficient I
probably wouldn't have to look around. These alternate Python projects
have made some small gains; but, they still have a long way to go.

> PyPy is now about double the speed of CPython for most things, and they
> have set themselves the ambitious target of being faster than C.

Let me know when they reach that target. When Java started there was hype
that the JIT would be able to run faster then native C because the JIT
could optimize at runtime based on the actual data that it encountered.
That never really panned out either.

On 2011-01-18, Stefan Behnel <> wrote:
> Tim Harig, 17.01.2011 20:41:
>> One of the arguments for Python has always made is that you can optimize
>> it by writing the most important parts in C. Perhaps that is a crutch
>> that has held the communty back from seeking higher performance solutions
>> in the language itself.
>
> The good news with Cython is that you no longer need to "write the most
> important parts in C". Instead, you type them statically and compile them.
> You don't even need to sacrifice Python source compatibility for that. So
> you get the best of Python at the speed of C (or 'c', as some would say .

Well not quite. The fact that you still need a Python interpeter should
tell you that the end result is not really just C. Effectively, you get
the same thing that you might get from writing C modules for your
performance sensitive loops. Cython just makes it a little easier to
generate the C modules; but, there is still a lot of Python operating under
the hood.

>> I prefer a single language as opposed to a creolization of two.
>
> With the possible exception of Lisp, I find it hard to think of a language
> that's still alive and not the creolisation of (at least) two other
> languages. They all inherited from each other, sometimes right from the
> start ("lessons learned") and always during their subsequent life time.

I am not talking about language influences. Cython effectively requires
two separate languages that interoperate. The end result is a mix
of two code written in two separate langauges. That is not a single
language solution.

>> I certainly don't need to require anybody who
>> wants to use a program I have compiled to install an interpreter.
>
> You will be happy to hear that you can link Cython programs against
> libpython (even statically) to build an executable that embeds the
> interpreter for you. Besides, on most operating systems, installing a

Which means that you effectively end up with a py2exe style frozen binary.
While that is a nice hack for solving some problems, it is hardly an
elegant solution. It still effectively means that I have to package a
redundant Python installation with every binary that I distribute. I
just don't have to write installation routines that check for, and
potentially install, a compatable version of Python.

> interpreter for you. Besides, on most operating systems, installing a
> dependency is done automatically, and Go code doesn't run natively on
> Windows either, without first installing and running the compiler over it.

Go works like C. If I compile my code on Windows, then I can give the
binary to another Windows user and they can execute it without having to
install any additional software (gcgo currently compiles everything using
static binaries. Since the linker only includes the fuctions you actually
use, you don't end up with the bloat of embedding entire libraries.
I am not sure about gccgo.). The same holds true for Linux, BSD, OS X,
and any other supported OS. Every go program includes a runtime componet
that provides the garbage collection; but, all of that is included as 100%
processor native code. That is quite a bit different from having to
include an interpreter to run non-native code.

Tim Harig wrote:

> You still don't see many
> companies doing large scale internal development using Python and you
> definately don't see any doing external developement using a language
> that gives the customers full access to the source code.

What you refered as "full access to the source code" only goes as far as
the license which was imposed by the copyright holders lets it to go. If
you distribute the source code along with the binaries but you only
license your code if the licencees accept that they may look at the source
code but they can't touch it then distributing the source code is
essentially meaningless. There is a good reason why "open source
software" is not the same thing as "free software".


Rui Maciel

On 2011-01-18, Rui Maciel <> wrote:
> Tim Harig wrote:
>
>> You still don't see many
>> companies doing large scale internal development using Python and you
>> definately don't see any doing external developement using a language
>> that gives the customers full access to the source code.
>
> What you refered as "full access to the source code" only goes as far as
> the license which was imposed by the copyright holders lets it to go. If
> you distribute the source code along with the binaries but you only
> license your code if the licencees accept that they may look at the source
> code but they can't touch it then distributing the source code is
> essentially meaningless. There is a good reason why "open source
> software" is not the same thing as "free software".

That argument is not going to fly with where commericial interests are
concerned. The simple fact is that there is legality and then there is
reality.

People follow licenses like they follow
the speed limit: only when they think they might get caught and punished.
When people break the licenses, the only recourse is litigation; which
is expensive. Even finding and proving license violations is difficult
where source is availible. It is therefore in the corporate interest
to make breaking licenses as difficult, uneconomical, and tracible as
possible; and that is exactly what companies do. Even companies that
don't really have any trade secrets to protect are protective about
their source keeping it locked out of public view. Access to the source
generally means signing an NDA so that if the source is somehow leaked,
they know the most likely candidates for the origin [so as not to pun with
"source"] of the leak.

Whether or not you actually agree with that economic reality is
irrelevant. Those who fund commerical projects do; and, any developement
tool which violates the security of the source is going to find itself
climbing an uphill battle in trying to gain market penetration with
commericial software producers.

On Mon, Jan 17, 2011 at 4:02 PM, Tim Harig <> wrote:

>> Go is not an ideal language for high-performance code. Despite the
>> occasional claims of others, Go is consistently outperformed by C,
>> C++, and Java on a wide variety of benchmarks. Some claim that Ada and
>> Haskell do as well, and my benchmarks (CPU bound, focused on speed of
>
> I much agree that Go doesn't beat C or C++. *I really doubt that it could
> with the runtime necessary to do garbage collction. *Nevertheless, I find
> that Go can be optimized to perform within tolerable limits of C given the
> boost it gives in development.

The question isn't whether you will use it; right or wrong, it seems
certain that you will. The question is whether it will see significant
uptake, and this is a big barrier for many.

> I really question that you get Java anywhere even close to C performance.
> Google reports they get within the same order of magnitude as C for
> their long-lived server processes where the JIT has had time to optimize
> its results. *For shorter term processes such as desktop applications,
> Java performance stinks -- even after you discount the JVM starup time.

I'm sorry, but you're wrong on this. Java's performance can be
excellent, particularly when it comes to mathematical functions. I
recall reading a pretty smug paper a few years ago describing how they
managed to beat C on a number of numerical benchmarks.

> Ada is capable of C++ like performance *if* you compile it to remove *all*
> of runtime checking. *Depending what checks you enable, it can run much
> slower.

No idea, never used it.

>> in line with the work of others. You can argue that it's good enough-
>> it is, for most cases- but taking a 20% performance hit rules it out
>> of a lot of systems work, and the C-Go gap in many cases is much
>> larger than that.
>
> I don't work anything that involves and absolute need for performance.

Then don't argue about performance, it makes you look like a hack just
eager to shill for your language.

> I could probably accept penalty several times that of C for higher
> level functionality; but, sometimes the penalty for Python is just
> too much. *Many of my programs are very quick lived such that even
> loading an interpeter or VM can eclipse the actual runtime. *Given less
> developmental effort required, I also find that I have more time to look
> for ways to optimize Go. *There are many things (such as using alternate
> data structures rather then the build in slices and immutable strings)
> that can be used to accelerate Go when time permits and I suspect many
> more will be found as the language matures.

This is inconsistent with your argument about PyPy. See my earlier comment.

>> Go is also not an ideal language for enterprise development. It
>> provides no compatibility or support guarantees; in fact, the website
>> describes it as 'an experiment' and recommends it for 'adventurous
>
> There is no doubt that it is a young project and there are a number of
> things that will need to be improved to be effective for some branches
> of programming; but, that isn't a language restriction.

It will nevertheless preclude its use in most enterprise software
development. That's most systems software.

> *Frankly, I am
> rather impressed by the sheer number of third party packages that have
> already become available. *No go isn't going to replace some of the other
> top mainstream langauges today; but, I can easily see it starting to see
> some market penetration 5 years from now.

I suppose that for some small value of market penetration ('mom uses
it!') you're right. I don't see any evidence of major movement at this
moment though.

>> users'. There are no independent experts in case something goes wrong,
>> and if it goes belly up a year from now you will have a one year old
>> piece of legacy infrastructure on your hands. Maybe you don't think
>> that this will be the case; in any event, I would not want to try
>> convincing a CFO of that.
>
> If I was trying to convince a CFO, I would ask if he really thought Google
> was likely to simply drop the time and effort that Google has already
> placed into the infrastructure.

Hahahaha. You mean like wave?

>*Furthermore, few dying languages already
> have not one, but two, open source compilers available for use.

Two compilers by the same people. Not two active projects. Big difference.

> Database bindings are another weak link outside of the more common
> open source databases. *In both cases, Go readily capable of C library
> functions as a stop-gap until a native wrapper is available. *Yes it will
> be nice once community has filled in the gaps; but, I am rather impressed
> at what is already available in less then a years time. *There are a few
> libraries you may have missed here:

Sounds like a two-language solution, ie, the thing you criticized Python for.

> I will point out that Go's libraries are miles ahead of the standard C
> library and other minimalistic standard libraries. *Many things are
> possible with even the most basic functionalities. *I don't use Python
> bindings when using GNUplot simply because its easier to access GNUplot
> directly.

And miles behind Python and other large standard libraries.

> Finally, and most importantly, nothing about any third party tools and
> libraries has any bearing on the generality language itself.

Except for its uptake.

>>> Support for concurrency is really icing on the cake. *I find it rather
>>> supprising that so many modern languages do not already support full
>>> concurrency constructs.
>>
>> Go's most-lauded feature is its goroutines. I suspect that if this
>> isn't a big deal for you, you aren't its primary use case.
>
> Actually, I would consider Go's implicit interfaces to be a far more
> important innovation. *The goroutines are nice but not ground breaking.
> They are quite recognizable to other SCP concurrancy derivatives.

I think if we did a poll of people who had heard of Go, they would
generally say that concurrency was its big selling point. I suspect
that its (ugly, IMHO) interface mechanism would not appear on the
list.

> C fails to be an object oriented language because it fails to provide
> the syntactic sugar necessary to bind functions to the data that they
> manipulate and because it doesn't provide the isolation necessary for
> encapsilation of objects.

No, it doesn't. I can add functions to structures using function
pointers, and languages like Python only provide encapsulation by
convention. There's no reason why that couldn't be true for C as well.
Ergo, if Go is OO, then C is OO.

> defer/panic/recover is conceptually a world closer to exceptions then is
> setjmp/longjmp. *It really isn't any further different then the variantions
> in exceptions between different languages.

We can agree to disagree here. As I say, I find it much closer to
things like the with statement than true exceptions, and general
internet rabble seems to agree.

As a question, given how hot you are for this language I have to
wonder how much of it you've actually written. Could you provide a
link? Google code search turned up nada.

Geremy Condra

On 1/18/2011 10:30 AM, Tim Harig wrote:

> Whether or not you actually agree with that economic reality is
> irrelevant. Those who fund commerical projects do; and, any developement
> tool which violates the security of the source is going to find itself
> climbing an uphill battle in trying to gain market penetration with
> commericial software producers.

Of course. When I submit or commit patches, I am doing it mostly for
hobby, educational, and scientific users, and maybe website makers (who
make site I can visit). If commercial users piggyback on top, ok. I do
not know how many developers, if any, are after such market penetration.

Shedskin compiles a slowly growing subset of Python to native code. But
I do not know that it has gotten any commercial support. Maybe Mark
should sell it instead of giving it away for commercial use (if he does
now).

--
Terry Jan Reedy

On 2011-01-18, geremy condra <> wrote:
> On Mon, Jan 17, 2011 at 4:02 PM, Tim Harig <> wrote:
>> I really question that you get Java anywhere even close to C performance.
>> Google reports they get within the same order of magnitude as C for
>> their long-lived server processes where the JIT has had time to optimize
>> its results. *For shorter term processes such as desktop applications,
>> Java performance stinks -- even after you discount the JVM starup time.
>
> I'm sorry, but you're wrong on this. Java's performance can be
> excellent, particularly when it comes to mathematical functions. I
> recall reading a pretty smug paper a few years ago describing how they
> managed to beat C on a number of numerical benchmarks.

I have no doubt that Java *can* occasionally beat *some* C for *some*
benchmarks; but, overall, Java has a terrible reputation for performance.

I worked with a company a few years ago that tried replacing a C VNC
client with a Java one so that its technical support contractor's wouldn't
need to have the VNC client installed on the agent's workstations.
Several of the contracters had to upgrade their systems in order to
use the Java version because it slowed down the machines so much that
the agents could not perform their jobs effectively; and that is pretty
typical from what I have seen with Java overall.

Java performs very well with some select tasks. For others, it does
exceedingly poor. That kind of hit or miss is pretty typical for JIT
compilers in general. That isn't usually the case for fully compiled
langauges where you are pretty much guaranteed to get decent, if not
always the absolute top, performance.

>> Ada is capable of C++ like performance *if* you compile it to remove *all*
>> of runtime checking. *Depending what checks you enable, it can run much
>> slower.
>
> No idea, never used it.
>
>>> in line with the work of others. You can argue that it's good enough-
>>> it is, for most cases- but taking a 20% performance hit rules it out
>>> of a lot of systems work, and the C-Go gap in many cases is much
>>> larger than that.
>>
>> I don't work anything that involves and absolute need for performance.
>
> Then don't argue about performance, it makes you look like a hack just
> eager to shill for your language.

What you don't seem to realize is there is often a performance level that
is good enough. For many things, Python is good enough. Many others,
where Python is insufficient, may still be acceptable to use Java.
Some things require the absolute best performance and will probably
always need C/C++ or equivilantly low level language.

>> I could probably accept penalty several times that of C for higher
>> level functionality; but, sometimes the penalty for Python is just
>> too much. *Many of my programs are very quick lived such that even
>> loading an interpeter or VM can eclipse the actual runtime. *Given less
>> developmental effort required, I also find that I have more time to look
>> for ways to optimize Go. *There are many things (such as using alternate
>> data structures rather then the build in slices and immutable strings)
>> that can be used to accelerate Go when time permits and I suspect many
>> more will be found as the language matures.
>
> This is inconsistent with your argument about PyPy. See my earlier comment.

I can accept 2 to 3 times the overall performance of C for almost all of
the problems that I deal with. When that multiple gets into the double
digits, it can start to cause some real headaches for some problems. When
that number reaches the upper double digits, it is acceptable for even
fewer problems.

Python is great for those problems where performance isn't critical and I
make extensive use of it. Different methods of mixing Python and C
(including manual, PyPy, SWIG, boost, etc) can extend Python's useful
range; but, I have not seen the kind of speed improvements that bring it to
less then an order of magnitude of C's speed overall.

Even assuming that PyPy does actually manage to reach within a magnitude
of C with the extra effort required to leverage two languages, why
would I bother when I can do it with one? PyPy and similar methods
where great when there was no other mid level alternative that supported
Python like features. Now it just seems like using Python as a hammer
for every problem whether or not it is the right tool for the job.

>>> Go is also not an ideal language for enterprise development. It
>>> provides no compatibility or support guarantees; in fact, the website
>>> describes it as 'an experiment' and recommends it for 'adventurous
>>
>> There is no doubt that it is a young project and there are a number of
>> things that will need to be improved to be effective for some branches
>> of programming; but, that isn't a language restriction.
>
> It will nevertheless preclude its use in most enterprise software
> development. That's most systems software.

So you conclude that because it is not quite ready for prime time yet that it
never will be? I can remember when people said C++ would never amount to
anything either.

>> Database bindings are another weak link outside of the more common
>> open source databases. *In both cases, Go readily capable of C library
>> functions as a stop-gap until a native wrapper is available. *Yes it will
>> be nice once community has filled in the gaps; but, I am rather impressed
>> at what is already available in less then a years time. *There are a few
>> libraries you may have missed here:
>
> Sounds like a two-language solution, ie, the thing you criticized Python for.

Not quite.

1. My arguments for dual language solutions where never directed at Python
proper. They were directed at PyPy. I am rather amazed at
the number of things that can be accomplished in Python without
having to bind to C.

2. There is a difference in binding to a solution that is already written
in another language so as to not reinvent a wheel and implementing
a *new* library in another language to be used exclusively
with Python.

On Tue, Jan 18, 2011 at 11:05 AM, Tim Harig <> wrote:
> Even assuming that PyPy does actually manage to reach within a magnitude
> of C with the extra effort required to leverage two languages, why
> would I bother when I can do it with one? *PyPy and similar methods
> where great when there was no other mid level alternative that supported
> Python like features. *Now it just seems like using Python as a hammer
> for every problem whether or not it is the right tool for the job.

You clearly have no idea what you're talking about regarding PyPy. You
could at least have googled it before speaking about it.

>>>> Go is also not an ideal language for enterprise development. It
>>>> provides no compatibility or support guarantees; in fact, the website
>>>> describes it as 'an experiment' and recommends it for 'adventurous
>>>
>>> There is no doubt that it is a young project and there are a number of
>>> things that will need to be improved to be effective for some branches
>>> of programming; but, that isn't a language restriction.
>>
>> It will nevertheless preclude its use in most enterprise software
>> development. That's most systems software.
>
> So you conclude that because it is not quite ready for prime time yet that it
> never will be? *I can remember when people said C++ would never amount to
> anything either.

We're a year past the initial announcement that it was ready. It's
still 'an experiment'. It doesn't have a ratified standard, committee,
or governing body. The company that developed it seems to have no
interest in enterprise support. They haven't done any serious
marketing for it since the initial release. Five years is, to put it
mildly, an overly enthusiastic timeline for the development of
broad-based industry support under those conditions.

>>> Database bindings are another weak link outside of the more common
>>> open source databases. *In both cases, Go readily capable of C library
>>> functions as a stop-gap until a native wrapper is available. *Yes it will
>>> be nice once community has filled in the gaps; but, I am rather impressed
>>> at what is already available in less then a years time. *There are a few
>>> libraries you may have missed here:
>>
>> Sounds like a two-language solution, ie, the thing you criticized Python for.
>
> Not quite.
>
> 1. My arguments for dual language solutions where never directed at Python
> * * * *proper. * They were directed at PyPy. *I am rather amazed at
> * * * *the number of things that can be accomplished in Python without
> * * * *having to bind to C.

Again, you don't know what you're talking about WRT PyPy.

> 2. There is a difference in binding to a solution that is already written
> * * * *in another language so as to not reinvent a wheel and implementing
> * * * *a *new* library in another language to be used exclusively
> * * * *with Python.

Even if that binding is done for performance reasons?

Geremy Condra

On 2011-01-18, Terry Reedy <> wrote:
> On 1/18/2011 10:30 AM, Tim Harig wrote:
>
>> Whether or not you actually agree with that economic reality is
>> irrelevant. Those who fund commerical projects do; and, any developement
>> tool which violates the security of the source is going to find itself
>> climbing an uphill battle in trying to gain market penetration with
>> commericial software producers.
>
> Of course. When I submit or commit patches, I am doing it mostly for
> hobby, educational, and scientific users, and maybe website makers (who
> make site I can visit). If commercial users piggyback on top, ok. I do
> not know how many developers, if any, are after such market penetration.

You kind of over-extended the intentions of my comment. It does not apply
to open source software in general. I agree that open source authors are
not interested in the quantitative value of market penetration. However, I
am betting that most authors of developement tools would like to be able to
use their tools on the job.

I am sure that more software developers would love to develop using
Python as part of their job. For some this is a reality; but, many more
are stuck using their employer's choice of language. One of the factors
that employers consider, when they choose a language, if they produce
retail software is that the process of compiling will sufficiently
obfiscate their code.

On 2011-01-18, geremy condra <> wrote:
> On Tue, Jan 18, 2011 at 11:05 AM, Tim Harig <> wrote:
>> Even assuming that PyPy does actually manage to reach within a magnitude
>> of C with the extra effort required to leverage two languages, why
>> would I bother when I can do it with one? *PyPy and similar methods
>> where great when there was no other mid level alternative that supported
>> Python like features. *Now it just seems like using Python as a hammer
>> for every problem whether or not it is the right tool for the job.
>
> You clearly have no idea what you're talking about regarding PyPy. You
> could at least have googled it before speaking about it.

No, I have watched several such projects over the years. Pysco, Unladen
Swallow, Cython, PyPy, Shedskin, etc. Source to source translators, JITs,
and C language integration all just add to complexity. You can't do this,
you can't do that, you have to learn a new way of doing something else,
ad nauseum.

So when something new that provided Python like capabilities without many
of Python's drawbacks came along, I jumped on it. It provides a much
cleaner solution to the problem without kludges. I will use Python for
what it does well and cleanly. For the rest, there are now better tools.
Once again, its about the right tool for the right job.

> Again, you don't know what you're talking about WRT PyPy.

Nor do I really want to. I have found a much simpler solution
to the problem. I would recommend it to many others that like the
Python language but who occassionaly struggle with its implementation
constraints.

I would say that I am sorry that it doesn't work for you; but, you seem
to prefer Java and Pypy anyway so we are both happy.

>> 2. There is a difference in binding to a solution that is already written
>> * * * *in another language so as to not reinvent a wheel and implementing
>> * * * *a *new* library in another language to be used exclusively
>> * * * *with Python.
>
> Even if that binding is done for performance reasons?

Yep, that is pretty much the summation of my dual language argument.
I don't expect a pure Python implementation of curses since there is a
perfectly good C library available to bind to. Why reinvent the wheel.

Resorting to writing packages in another language or compiling Python
code into another source language is a kludge necessary because of
the performance characteristics of the language. I suppose that is
an acceptable kludge when that is your only alternative. For me it no
longer is.