Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > RE: anything like C++ references?

Reply
Thread Tools

RE: anything like C++ references?

 
 
Tom Plunket
Guest
Posts: n/a
 
      07-15-2003
Stephen Horne wrote:

> std::string x = "123";
> std::string y = x;
>
> // At this point, in most implementations, x and y are bound to the
> // same object due to a lazy copy optimisation.


Actually, that isn't the case these days. I'm sure there are
some who do this optimization still, but it is incredibly hard to
do so in a thread-safe manner so all of the heavies as far as I
know don't do it anymore.

-tom!

--
There's really no reason to send a copy of your
followup to my email address, so please don't.
 
Reply With Quote
 
 
 
 
Stephen Horne
Guest
Posts: n/a
 
      07-15-2003
On Tue, 15 Jul 2003 15:16:40 -0700, Tom Plunket <(E-Mail Removed)>
wrote:

>Stephen Horne wrote:
>
>> std::string x = "123";
>> std::string y = x;
>>
>> // At this point, in most implementations, x and y are bound to the
>> // same object due to a lazy copy optimisation.

>
>Actually, that isn't the case these days. I'm sure there are
>some who do this optimization still, but it is incredibly hard to
>do so in a thread-safe manner so all of the heavies as far as I
>know don't do it anymore.


Ah - makes sense. Bugger. There goes my nice neat 'but they can do it'
argument

 
Reply With Quote
 
 
 
 
Erik Max Francis
Guest
Posts: n/a
 
      07-16-2003
Stephen Horne wrote:
>
> On 15 Jul 2003 10:19:18 GMT, http://www.velocityreviews.com/forums/(E-Mail Removed) (Mark
> 'Kamikaze' Hughes) wrote:
>
> >> Really. My computer science lessons were taught in a way that
> >> respected the source of most of the theory in mathematics -
> >> algorithms, Church's lambda calculus and that kind of stuff.
> >> What did your lessons teach?

> >
> > How computer programming actually works, how to make interpreted and
> >compiled languages, how to use them to solve problems, how to analyze
> >algorithms, etc.... The usual.

>
> You say that as if it contradicts the idea of respecting the source of
> the theory. It doesn't.
>
> > Among those were classes in languages
> >other than C, which is very broadening, and I highly recommend that to
> >you.

>
> Lets see.
>
> Probably about a dozen basics, five assembler languages (6502, z80,
> 68000 series, 8086 series, 80C196KC microcontroller), forth, lisp,
> prolog, miranda, haskell, icon, ada, pascal, modula 2, cobol,
> smalltalk. Not all of them very seriously, though, and in many cases
> not recently.
>
> I forget the name of that language designed for transputers, but I
> used that a bit at college too - and depending on your inclination you
> may want to include SQL. And yes, I have used C, C++ and Java as well.
>
> That's just off the top of my head, of course.
>
> But there's no way you can call me C centric. That has been a repeated
> accusation, but it is WRONG.
>
> Python has been an everyday language for me LONGER than C++ has.
>
> >> Respect the idea of variables binding to values and suddenly the need
> >> for pointers becomes more obvious. You cannot abuse mutable objects to
> >> fake pointer functionality (another everyday fact of Python
> >> programming) if the binding of variables to values (rather than just
> >> objects) is respected.

> >
> > If you're trying to say that the trick I showed you of modifying a
> >list to reproduce the effect of reference arguments is not possible in,
> >say, C, you're wrong. Trivially proven wrong, even, since Python is
> >implemented in C. In fact, the only languages where that won't work are
> >those which don't allow passing complex arguments at all; some
> >pure-functional toy languages, perhaps.

>
> Distorting what I said, again. I have already repeatedly said that the
> point about C is what happens by default - not what happens when you
> explicitly request otherwise.
>
> Lets take an example from C++ to illustrate the point.
>
> std::string x = "123";
> std::string y = x;
>
> // At this point, in most implementations, x and y are bound to the
> // same object due to a lazy copy optimisation. However, the
> // programmer didn't request this optimisation so this
> // implementation detail is expected to be transparent to the user.
> // Therefore...
>
> y [2] = 4;
>
> // Before doing this operation, a copy was made - the so called
> // copy on write.
> // The value bound to x is still "123"
> // The value bound to y is now "124"
>
> C++ is using references here just as Python does, but despite being a
> crappy low level language it still manages to implement that without
> causing remote side-effects through mutability *unless* you explicitly
> request that.


But (ignoring for the moment Tom's point about this not really being
true in modern implementations) that's really an implementation detail
and is unimportant. C has copy semantics; whether it's optimizes some
away when they're not necessary is an implementation detail that is
unimportant from the programmer's perspective.

I did something very similar in a stack-based language I designed (and
implemented partially in C, and then more extensively in Python). There
are no pointers or references, only objects. All objects are distinct;
changing one never affects any others. But since this is a stack-based
language, there's lots of duplicating things on stack for the purposes
of a computation. As an implementation detail (and since I basically
got it for free in Python since all Python objects do the reference
counting for me), if you duplicate an object, I actually duplicate a
reference to it internally. If you mutate an object, then it mutates it
if there are no other references to it, but does an actual duplication
and then mutates that if there are -- so it implements lazy copying.
But the fact this is done at all is completely and utterly and
implementation detail. From the programmer's point of view, it makes no
difference.

Python does _not_ have copy semantics, it has binding semantics. As a
reversed example, when multiple bindings exist to the same immutable
object, it's an implementation detail whether or not those really are
all to the same object. You don't care, because you know its value
can't change.

--
Erik Max Francis && (E-Mail Removed) && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ Nine worlds I remember.
\__/ Icelandic Edda of Snorri Sturluson
 
Reply With Quote
 
Adam Ruth
Guest
Posts: n/a
 
      07-16-2003
In <(E-Mail Removed)> Stephen Horne wrote:
> On 15 Jul 2003 10:45:10 -0700, (E-Mail Removed) (Adam Ruth) wrote:
>
>>I came to Python from Ada, and all I have to say is: Pointer-like
>>semantics are evil. They're worse than goto's, they're deadly,
>>damaging, and should be avoided at all costs.
>>
>>They're an unhappy necessity, akin to stop-lights. You need them
>>because roads intersect, but if roads don't intersect, don't use them!

>
> Absolutely true. And the worst thing you can do when you really can't
> avoid pointers is to obscure the issue even more by disguising them as
> something else.


I assume that you're referring to people 'faking' pointer in Python (
such as wrapping variables in lists, etc.) I'd ammend your statement to
say:

And the worst thing you can do is to obscure the issue even more by
disguising them as something else, WHEN YOU DON'T REALLY HAVE TO.

In Python, there is no situation where "you really can't avoid pointers".
It's only when you program C or C++ in Python that you think you can't
avoid pointers. There are much better idioms to achieve the desired
results.

> The biggest problems with pointers relate to things with pointer math,
> pointers to deleted objects etc. These things need not exist in a
> scripting language that provides pointers - pointers don't have to be
> implemented using store addresses. That is just a concept from C, C++,
> Pascal, Ada etc - not something I want to see imitated. If pointers
> can be created that refer to given objects, and if pointers can be
> dereferenced, that is actually enough to support indirect referencing.


It's actually also unnecessary in Python.

Adam Ruth
 
Reply With Quote
 
Bengt Richter
Guest
Posts: n/a
 
      07-16-2003
On 15 Jul 2003 10:41:52 -0400, (E-Mail Removed) (Aahz) wrote:

>In article <(E-Mail Removed)>,
>Erik Max Francis <(E-Mail Removed)> wrote:
>>
>>The value is the object. The object is the value.

>
>Actually, that's not quite true, and it's something I've been wrestling
>with how to describe. What's the value of a file object? I think in a
>very real sense, a file object doesn't have a "value". You could try
>arguing that the file contents are the value, but that value is not
>contained within the object -- the object is a proxy. Saying that the
>filename is the value doesn't work, either, especially when you want to
>talk generically about file-like objects (such as StringIO and sockets).


ISTM the trouble is that in causual talk about "value" there is an implicit
mapping function involved which usually doesn't need mentioning. The mapping
is from a concrete "thing-supposed-to-have-value" representation to a member
of an abstract set of values, such as integers. E.g., a suitable function
can map "15" to the same abstract and unique value as another function maps "F" to.
Note that "15" can map to the same as "twentyone" depending on the function applied.

The point is that a representation has no instrinsic value, it corresponds to a
value in a particular abstract set by way of a particular valuation function.

Thus I would say that the "value" of a file object depends on the valuation function
you apply. Without specifying the latter, there can be no value.

Concretely, a valuation function can only return another representation, but that
can be useful in getting closer to a simple abstraction. E.g., StringIO.getvalue()
or f.seek(0);f.read() returning string representations, whose value may be an octet
sequence, which is easy to think of in the abstract, or it could be a sequence of
abstract glyph-set members, identified by a mapping through ord() and indexing the
ordered glyph set. I.e., a mapping function may be a composition of a conventional
series of function applications.

Anyway, ISTM "value" usually presupposes a conventional choice of mapping function.

Regards,
Bengt Richter
 
Reply With Quote
 
Donn Cave
Guest
Posts: n/a
 
      07-16-2003
Quoth Adam Ruth <(E-Mail Removed)>:
....
| In Python, there is no situation where "you really can't avoid pointers".

That's trivially true, no such situation can exist because that
Python can't be coded. I'll repeat an example that I proposed
a couple days ago, though: user implemented sequences could be
implemented with a single simple indexing function, returning
a pointer/target/whatever; Python could assign directly to that,
making "seq[i] = x" the same kind of operation as and automatically
symmetrical with "x = seq[i]".

It can't be done without pointers, as far as I can see. You may not
care if it can be done, but I think you'd agree that "there is no
situation that I care about where you really can't avoid pointers"
would be kind of a lame version of your assertion.

Donn Cave, (E-Mail Removed)
 
Reply With Quote
 
Bengt Richter
Guest
Posts: n/a
 
      07-16-2003
On Tue, 15 Jul 2003 22:14:20 +0100, Stephen Horne <(E-Mail Removed)> wrote:

>On 15 Jul 2003 20:32:30 GMT, (E-Mail Removed) (Bengt Richter) wrote:
>
>>On Tue, 15 Jul 2003 02:40:27 +0100, Stephen Horne <(E-Mail Removed)> wrote:
>>
>>>On Mon, 14 Jul 2003 00:07:44 -0700, Erik Max Francis <(E-Mail Removed)>
>>>wrote:
>>>
>>>>Stephen Horne wrote:
>>>>
>>>>> Imagine, for instance, changing all assignments and other 'copying' to
>>>>> use a copy-on-write system. Then add a pointer type and a 'newcopyof'
>>>>> operator. And nick the C-style prefix '*' for dereferencing and add
>>>>> '&' as a 'make a pointer to this object' operator (absolutely NOT a
>>>>> physical memory address).
>>>>
>>>>The problem is that this misses the point that Python is not C++. You
>>>>shouldn't try grafting syntaxes and approaches that make Python look
>>>>more like C++, you should be learning to use Python on its own merits.
>>>
>>>Not true.
>>>
>>>If you eliminate all violations of the idea of variables bound to
>>>values (basically the whole point of my thread), then mutable
>>>containers cannot achieve the same thing.
>>>
>>>If a copy-on-write scheme is added to Python, you'd get the following
>>>results from using mutable containers...
>>>
>>>>>> a = [1, 2, 3]
>>>>>> b = a
>>>>>> b[2] = 4
>>>>>> a
>>>[1, 2, 3]
>>>>>> b
>>>[1, 2, 4]
>>>

>>Yes, but it would be hugely inefficient for the case where, e.g., you are
>>modifying lines read from a 20MB log file. Imagine inducing a 20MB copy
>>every time you wanted to delete or modify a line. Now you have to invent
>>a way to program around what can be done very conveniently and efficiently
>>thanks to Python's semantics.

>
>What makes you say that!
>
>Copies are only made when they are needed. The lazy copy optimisation,
>in other words, still exists.

(BTW, don't get hung up on this part, ok? Skip forward to the ==[ code part ]==
below if you're going to skip

Ok, so the copy happens at b[2]=4 right? This is still useless copying
if the holder of the "a" reference *wants* to have it as a continuing
reference to the same single shared list. And you can program that in C++
if you want to, and you presumably would. Except now you have to create pointer variable.

Ok now how would you handle the case of multiple copies of that pointer variable?
Make it "smart" so that you get copy-on-write of what it points to? That's the discussion
we're having about the first level of Python references, ISTM.

>
>Delete or modify one string in a list of strings, and the same stuff
>would happen as happens now. Unless, perhaps somewhere you don't know

So a 20MB log file in the form of a string list might be 500,000 lines of 40-byte log entries,
which would be 2MB of 4-byte pointers. Cheap copy when you explicitly don't want it,
but want to share the same list?

>about, the caller of your function who passed that list in to you has
>a separate reference they expect to stay unchanged. In that case, the
>first changed string triggers a copy of the list - but as the list
>only contains references to strings, it doesn't trigger copying of all
>the strings. The second line changed doesn't require the list to be
>modified again because you already have your separate copy.

Sure, but the first unwanted copy cost you.
>
>C++ uses exactly this kind of approach for the std::string class among
>others. Copy-on-write is a pretty common transparent implementation
>detail in 'heavy' classes, including those written by your everyday
>programmers. Does that mean C++ is slower that Python? Of course not!
>

It could be, if the unwanted copying was big enough. Except you wouldn't
program it like that. You'd program it to share mutable data like Python
for a case like that.

In any case, the performance is a side issue, however practically important,
to the discussion of the language semantics.

===[ code part ]=================
I'm a bit disappointed in not getting a comment on class NSHorne

BTW, that was not a faked interactive log. It doesn't have lazy copy, but
that could be arranged. Though that's an implementation optimization issue, right?

Does that namespace have the semantics you are calling for or not?
If I add a mechanism to create and dereference "pointers" re that namespace, will
that do it? Here's a go at that:

>>> class NSHorne(object):

... from cPickle import dumps, loads
... class Ptr(object):
... def __init__(self, ns, name): self.ns=ns; self.name=name
... def __getitem__(self, i): return getattr(self.ns, self.name)
... def __setitem__(self, i, v): setattr(self.ns, self.name, v)
... def __setattr__(self, name, val):
... if isinstance(val, self.Ptr):
... object.__setattr__(self, name, val)
... else:
... object.__setattr__(self, name, self.loads(self.dumps(val)))
... def __getitem__(self, name): return self.Ptr(self, name)
...
>>> ns = NSHorne()

set the value of x:
>>> ns.x = [1, 2, 3]


make a "pointer" to x
>>> ns.px = ns['x']


You can use ordinary Python aliases with the "pointer"
>>> p=ns.px


You *p dereference it like:
>>> p[:]

[1, 2, 3]

And here's a function that expects such a pointer
>>> def foo(bar):

... bar[:] = 'something from bar' # like *bar=
...

You can pass it to an ordinary Python function

>>> foo(p)


And dereference it again to see what happened
>>> p[:] #like *p

'something from bar'

Or look at what it was pointing to
>>> ns.x

'something from bar'

or look via the original pointer in ns
>>> ns.px[:] # like *px

'something from bar'

and alias it some more
>>> p2 = p
>>> p2[:]=(3,4,5)
>>> ns.x

(3, 4, 5)

Or to play with one of your original examples:

>>> ns.a = [1, 2, 3]
>>> ns.b = ns.a
>>> ns.b[2] = 4
>>> ns.a

[1, 2, 3]
>>> ns.b

[1, 2, 4]

Make "pointer: to b:
>>> p = ns['b']
>>> ns.b

[1, 2, 4]
>>> ns.b[0]=-1
>>> ns.b

[-1, 2, 4]

Now dereference p:
>>> p[:]

[-1, 2, 4]

Use dereferenced pointer to mod value
>>> p[:][1]=-1


Check values
>>> ns.b

[-1, -1, 4]
>>> ns.a

[1, 2, 3]
>>> p[:]

[-1, -1, 4]
>>>


But we can set something else than that list:
>>> p[:] = 'something else'
>>> ns.b

'something else'

IOW,
p = &x is spelled ns.p = ns['x']
and
*p = x is spelled ns.p[:] = ns.x
and
y = *p is spelled ns.y = ns.p[:]
and
you can pass ns.p to a function def foo(bar): and dereference it as bar[:] there

Does this give names in ns the "variable" semantics you wanted? Is there any difference
other than "semantic sugar?" Explaining the difference, if any, will also clarify what
you are after, to me, and maybe others

Regards,
Bengt Richter
 
Reply With Quote
 
Donn Cave
Guest
Posts: n/a
 
      07-16-2003
In article <(E-Mail Removed)>,
Adam Ruth <(E-Mail Removed)> wrote:
> In <(E-Mail Removed)> Donn Cave wrote:
> > In article <(E-Mail Removed)>,
> > Adam Ruth <(E-Mail Removed)> wrote:
> >
> >> In <1058328099.572993@yasure> Donn Cave wrote:
> >> > Quoth Adam Ruth <(E-Mail Removed)>:
> >> > ....
> >> >| In Python, there is no situation where "you really can't avoid
> >> >| pointers".
> >> >
> >> > That's trivially true, no such situation can exist because that
> >> > Python can't be coded. I'll repeat an example that I proposed
> >> > a couple days ago, though: user implemented sequences could be
> >> > implemented with a single simple indexing function, returning
> >> > a pointer/target/whatever; Python could assign directly to that,
> >> > making "seq[i] = x" the same kind of operation as and automatically
> >> > symmetrical with "x = seq[i]".


....
> You are correct, 'really need' is not much of an argument. The
> statement I disagreed with was that it was bad to simulate pointers in
> those situations where "you really can't avoid pointers". I was
> expressing that really aren't any such situations, and that it's even
> worse try to simulate pointers when it's not really necessary.


OK, there really aren't any situations where you can't avoid
pointers ... so you're ready with a solution to the problem
I posed above?

Class scope containers that hold exactly one item, to pick a
commonly seen usage that I suppose is a simulated pointer -
that's really not necessary, and it's "even worse" (than what?)

Donn Cave, (E-Mail Removed)
 
Reply With Quote
 
Stephen Horne
Guest
Posts: n/a
 
      07-16-2003
On 16 Jul 2003 05:18:48 GMT, (E-Mail Removed) (Bengt Richter) wrote:

>>Copies are only made when they are needed. The lazy copy optimisation,
>>in other words, still exists.

>(BTW, don't get hung up on this part, ok? Skip forward to the ==[ code part ]==
>below if you're going to skip
>
>Ok, so the copy happens at b[2]=4 right? This is still useless copying
>if the holder of the "a" reference *wants* to have it as a continuing
>reference to the same single shared list. And you can program that in C++
>if you want to, and you presumably would. Except now you have to create pointer variable.


If you want to share a single object, you need an explicit way to do
it.

Let me use an analogy from a discussion held elsewhere but on a
related subject.

In Algol and Fortran were among the earliest high level languages.
There were, I believe, called 'third generation' (assembler as opposed
to machine code being the second generation). Hence the name of
'forth' (which was supposed to be ever higher level, though IMO it is
a lower level language) and also the term '4GL' that was heavily used
for languages like SQL at one time.

Anyway, Algol and Fortran apparently (I didn't know this until today)
*always* used call-by-reference.

So why is it that all C parameters are call-by-value (though the value
may be a pointer) and Pascal parameters are call-by-value by default?

This is new to me - my first experiences of C and Pascal was at an age
when the whos-language-is-better rows were about Pascal providing
call-by-reference. To me, the distinction is that C is lower level -
closer to what you'd do in assembler or machine code - whereas Pascal
is higher level. You can pass pointers in Pascal to achieve the same
goal as a 'var' parameter, but 'var' better expresses why you are
doing it.

I had assumed, therefore, that there had been a natural progression
from low-level assembler - that earlier high level languages would
either be like C or like Pascal, depending on just how high level they
are. But that's not true. Algol and Fortran both used
call-by-reference, and that somewhat established tradition was
rejected when C and Pascal were being developed.

Why the rejection? On a point of principle. When parameters are
call-by-reference by default, it allows accidental side-effects.
Sometimes, functions change parameters that callers aren't expecting
them to change.

When changing one identifier can affect another by default, errors can
happen. There are tools for doing this explicitly. They are called
pointers or references.


>
>Ok now how would you handle the case of multiple copies of that pointer variable?
>Make it "smart" so that you get copy-on-write of what it points to? That's the discussion
>we're having about the first level of Python references, ISTM.


Easy enough.

Reference counting for the object keeps track of how many variables
are currently using it as their value. Assign to as many variables as
you like - all you get is all those variables referencing one object
with a higher reference count.

Before modifying the object, you check the reference count. If greater
than one, you need to make a copy.

Pointer/reference objects have to point to the 'variable' rather than
the object. Access the object through that, and they are then
guarenteed to access the correct object that they were associated
with.

Pointers and references don't increment the reference count as they
are not supposed to get copies.

ASCII art warning...

+---+ +-------+ +---------------+
| x |--->| x var |----->| object : RC=2 |
+---+ +-------+ +---------------+
^
+---+ +-------+ |
| y |--->| y var |----+
+---+ +-------+
^
|
+---+ +-------+ +-------------------+
| z |-->| z var |-->| ptr object : RC=1 |
+---+ +-------+ +-------------------+


Garbage collection continues to work as it already does, having
nothing to do with the new reference count in the object. The
intermediate 'variable' thing means that even if the original variable
goes out of scope, the pointer can still refer to it - and a new
variable created with the same name doesn't have to re-use the old
variable thingy.

This involves an extra layer of indirection compared with Python at
the moment. It would involve a (small) performance hit as a result.
But there is no copying of the object unless the copy is needed.


>>C++ uses exactly this kind of approach for the std::string class among
>>others. Copy-on-write is a pretty common transparent implementation
>>detail in 'heavy' classes, including those written by your everyday
>>programmers. Does that mean C++ is slower that Python? Of course not!
>>

>It could be, if the unwanted copying was big enough.


No, it can't. It *never* creates an unnecessary copy. That is the
whole point of lazy copying.

>I'm a bit disappointed in not getting a comment on class NSHorne


I *know* that you can use a class to get pointer behaviour. The issue
of pointers becomes important if you don't get that behaviour because
of copy-on-write.

>Does this give names in ns the "variable" semantics you wanted? Is there any difference
>other than "semantic sugar?" Explaining the difference, if any, will also clarify what
>you are after, to me, and maybe others


The issue is what happens by default.

 
Reply With Quote
 
Stephen Horne
Guest
Posts: n/a
 
      07-16-2003
On Tue, 15 Jul 2003 17:29:29 -0700, Erik Max Francis <(E-Mail Removed)>
wrote:

>Python does _not_ have copy semantics, it has binding semantics.


As you know, that's precisely what I see as a wart.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
object-like macro used like function-like macro Patrick Kowalzick C++ 5 03-14-2006 03:30 PM
Anything like IPKall for sipgate? John Smith UK VOIP 6 09-20-2005 05:43 PM
Is there anything like Oracle ADF in other Java IDE tools ? krislioe@gmail.com Java 10 06-30-2005 03:27 PM
Re: never heard anything like it in my life? Jumbo C++ 5 01-23-2004 03:18 AM
Re: Anything like Atomica/GuruNet ? Mir Computer Support 1 08-17-2003 12:12 PM



Advertisments