Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > The future of Python immutability

Reply
Thread Tools

The future of Python immutability

 
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-05-2009
On Fri, 04 Sep 2009 20:48:13 -0700, sturlamolden wrote:

> > Is the difference because of mutability versus immutability, or
> > because of C code in Numpy versus Matlab code? Are you comparing
> > bananas and pears?

>
> It consisted of something like this



Your code does a lot of unnecessary work if you're just trying to
demonstrate immutability is faster or slower than mutability. A simple
test that just adds one to each element would have much less overhead.

In any case, there's no doubt that immutable objects require extra time
to create compared to re-using an existing mutable object, and that time
is probably O(N) (until you approach the limits of available contiguous
memory, in which case you could see O(N**2) or worse behaviour). But an
order of magnitude difference?


> I wasn't comparing bananas against pears. Mathworks informed me they
> were using my code to investigate why Matlab was showing such slow-
> downs. I am not sure what they found out, eventially. I have also
> wondered if immutability vs. mutability was the reason, as NumPy
> generates temporary arrays. But I have not found a better explanation
> either. Anyhow, ~30 seconds for Matlab vs. ~3 seconds for Python is a
> major difference.


How does Matlab speed compare to Python in general? Ruby, for example, is
an order of magnitude slower than Python (at least it was last time I
looked), not because of immutable arrays, but just because of the speed
of the language.



--
Steven
 
Reply With Quote
 
 
 
 
sturlamolden
Guest
Posts: n/a
 
      09-05-2009
On 5 Sep, 07:04, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:

> Your code does a lot of unnecessary work if you're just trying to
> demonstrate immutability is faster or slower than mutability.


No ... I was trying to compute D4 wavelet transforms. I wanted to see
how NumPy compared with Matlab.


> How does Matlab speed compare to Python in general?


Matlab is an example of a language that only has immutable types.
While it works well if you only play with small arrays, it is horrible
for complex data structures.

Also consider that appending to an array in Matlab always becomes O
(n**2), as the array cannot mutate like Python lists. Thus it is
impossible to amortize to linear complexity.






 
Reply With Quote
 
 
 
 
sturlamolden
Guest
Posts: n/a
 
      09-05-2009
On 5 Sep, 07:04, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:

> How does Matlab speed compare to Python in general?


Speed-wise Matlab is slower, but it is not the interpreter limiting
the speed here.
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-05-2009
On Fri, 04 Sep 2009 22:30:44 -0700, sturlamolden wrote:

> On 5 Sep, 07:04, Steven D'Aprano <st...@REMOVE-THIS- cybersource.com.au>
> wrote:
>
>> How does Matlab speed compare to Python in general?

>
> Speed-wise Matlab is slower, but it is not the interpreter limiting the
> speed here.


How do you know?


--
Steven
 
Reply With Quote
 
sturlamolden
Guest
Posts: n/a
 
      09-05-2009
On 5 Sep, 08:47, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:

> How do you know?


After more than 10 years experience with scientific programming I just
do. When it comes to numerics I have a gut feeling for what is fast
and what is slow.

It's not difficult actually. You just have to imagine how often the
interpreter is invoked. Explicit loops are therefore evilness in
numerical scripts. Vectorized array expressions tend to be comparable
to C in performance, because they do a lot of work independent of the
interpreter. What do we see here? Vectorized array expressions
throughout the code, no significant amount of looping by the
interpreter (just the recursion at the end).
 
Reply With Quote
 
Adam Skutt
Guest
Posts: n/a
 
      09-05-2009
On Sep 5, 12:06*am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> On Fri, 04 Sep 2009 06:36:59 -0700, Adam Skutt wrote:
> > Nope, preventing mutation of the objects themselves is not enough. You
> > also have to forbid variables from being rebound (pointed at another
> > object). *Consider this simple example:

>
> > ---------- Thread 1 ---------- | ---------- Thread 2 ----------
> > a = "Foo"
> > spawn Thread 2
> > a = "Bar" * * * * * * * * * * * *print "thread 2: %s" % a
> > print "thread 1: %s" % a

>
> > You could see (ignoring the fact the output isn't ordered):
> > "thread 1: Bar"
> > "thread 2: Foo"
> > or:
> > "thread 1: Bar"
> > "thread 2: Bar"

>
> > so the fact "Foo" and "Bar" are immutable isn't enough to solve the
> > problem.

>
> This is a side-effect of writing code that relies on global variables.
> Global variables are generally a bad idea. Global constants are fine.


Nope, the variables don't have to be global to have this problem, they
just have to be shared:

>>> a = 3
>>> b = lambda x: x + a
>>> print b(3)

6
>>> a = 4
>>> print b(3)

7

Passing any lambda between threads will cause the problem I described
above, regardless of whether the variables captured are global or
not.

> What do you mean by "variables"? Do you mean names?

In the case of python I mean the name and the value, since all
variables in Python are pointers. (Worrying about the difference
though, is semantics)
>
> What are pointer semantics?

Assignment to the variable causes it to point to another object (as
opposed to assigning a new value to the current object, like a C++
reference) and copying the variable does not create a copy of the
referred object (which necessarily implies their lifecycles are
independent).

> Assuming you mean names must be forbidden from being rebound, no,
> incorrect. It's only names which are shared between both threads which
> must not be re-bound. If you have names local to the thread, you can
> change them all you want without affecting any other thread.

What does it matter, seeing as Python lacks the ability altogether?

Thanks,
Adam
 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      09-05-2009
Adam Skutt wrote:
\
>> This is a side-effect of writing code that relies on global variables.
>> Global variables are generally a bad idea. Global constants are fine.

>
> Nope, the variables don't have to be global to have this problem, they
> just have to be shared:
>
> >>> a = 3
> >>> b = lambda x: x + a


This is a pointless replacement for 'def b(x): return x+a'

> >>> print b(3)

> 6
> >>> a = 4
> >>> print b(3)

> 7
>
> Passing any lambda


Python does not have lambda objects. It has lambda expressions that
produce function objects identical except for .__name__ to the
equivalent def statement output.

tjr

 
Reply With Quote
 
Adam Skutt
Guest
Posts: n/a
 
      09-05-2009
On Sep 5, 11:29*am, Terry Reedy <(E-Mail Removed)> wrote:
>
> This is a pointless replacement for 'def b(x): return x+a'


And? That has nothing to do with anything I was saying whatsoever.
Point is: any mutable shared state is bad, and making objects
immutable isn't enough to remove all shared state, or even reduce it
to be available only with TEH EBIL global variables.

> Python does not have lambda objects. It has lambda expressions that
> produce function objects identical except for .__name__ to the
> equivalent def statement output.

Sure sounds like python has lambda objects to me then... the fact
they're a special case of some more general construct is mostly
semantics, /especially/ in the context of the point I was actually
making, no?
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-05-2009
On Sat, 05 Sep 2009 14:09:57 -0700, Adam Skutt wrote:

>> Python does not have lambda objects. It has lambda expressions that
>> produce function objects identical except for .__name__ to the
>> equivalent def statement output.

>
> Sure sounds like python has lambda objects to me then... the fact
> they're a special case of some more general construct is mostly
> semantics, /especially/ in the context of the point I was actually
> making, no?


No. Lambdas are a *syntactical construct*, not an object. You wouldn't
talk about "while objects" and "if objects" and "comment objects"
*because they're not objects*. Neither are lambdas -- they're syntax,
which creates ordinary functions:

>>> def f(x):

.... return x
....
>>> g = lambda x: x
>>> type(f) is type(g)

True


Functions created with def and functions created with lambda are
*precisely* the same type of object. There is no such thing as a "lambda
object" which is a "special case" of ordinary functions, there are just
functions.



--
Steven
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-05-2009
On Sat, 05 Sep 2009 04:57:21 -0700, Adam Skutt wrote:

>> > so the fact "Foo" and "Bar" are immutable isn't enough to solve the
>> > problem.

>>
>> This is a side-effect of writing code that relies on global variables.
>> Global variables are generally a bad idea. Global constants are fine.

>
> Nope, the variables don't have to be global to have this problem, they
> just have to be shared:


Yes, you're right, my bad. Globals are shared, but not all shared
variables are global.


>> What do you mean by "variables"? Do you mean names?

>
> In the case of python I mean the name and the value, since all variables
> in Python are pointers.


Let me see if I understand you...

You say that by "variable" you mean the name and the value.

Then you say that all variables are pointers.

In other words... "all names and values in Python are pointers".

Either you're not explaining yourself correctly, or you're badly confused
about Python. In Python, names are keys in namespaces, and values are
objects. Possibly some Python implementations use pointers in some way to
implement namespaces, or objects, but Python the language doesn't have
pointers, and the Python virtual machine that executes Python byte code
doesn't have pointers either.



> (Worrying about the difference though, is semantics)


Semantics are important. If we can't agree on what things mean, how can
we possibly communicate?



>> What are pointer semantics?

>
> Assignment to the variable causes it to point to another object (as
> opposed to assigning a new value to the current object, like a C++
> reference) and copying the variable does not create a copy of the
> referred object (which necessarily implies their lifecycles are
> independent).


I can *guess* what you mean by all that, but it's just a guess. You say
"assignment to the variable", but earlier you said that to you, variables
are names and values, so I'm left wondering if you mean assignment to the
name, assignment to the value, or both. Likewise for copying the variable.

Here is my guess... you're pointing out the similarities between Python
name binding and C pointers:

* when you bind a name to a new object, you cause the name be associated
with the new object rather than mutating the existing object to become
equal to the new object;

* assigning two names to a single object causes both names to be
associated with the same object, rather than each name being associated
with independent copies of the object;

while ignoring the differences between Python name binding and C pointers:

* Python names are keys, not numeric addresses, hence you can't perform
anything like pointer arithmetic on them;

* objects in Python have no idea what, if any, names are associated with
them, unlike C pointers, where you can always ask for the address of a
variable.

Okay, I accept that if you focus only on the similarities and completely
ignore the differences, Python name binding is just like pointer
semantics.



>> Assuming you mean names must be forbidden from being rebound, no,
>> incorrect. It's only names which are shared between both threads which
>> must not be re-bound. If you have names local to the thread, you can
>> change them all you want without affecting any other thread.

>
> What does it matter, seeing as Python lacks the ability altogether?


I don't understand what you mean. Python lacks the ability to do what
altogether?

If you mean that Python threads lack the ability to have local variables,
that's not true at all.




--
Steven
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Immutability and Python andrea crotti Python 8 11-08-2012 07:38 AM
Re: Immutability and Python andrea crotti Python 0 10-29-2012 03:44 PM
Python String Immutability Broken! Roy Smith Python 2 08-25-2008 02:41 AM
Easy immutability in python? Terry Hancock Python 3 03-04-2006 08:17 PM
Re: Easy immutability in python? Terry Hancock Python 2 03-04-2006 08:17 PM



Advertisments