Sign in to follow this  
Gage64

[Python] Slice notation and object copies

Recommended Posts

Gage64    1235
I've decided to try and learn Python. I'm using the tutorial in the documentation and there's something I don't quite understand. When using slice notation, it looks like I am sometimes getting a copy of the object and other times I get the object itself. Sounds vague, I know, but the following example shows what I mean. If I write the following in the python shell:
a = [1, 2, 3]
a[:] = [4, 5, 6]
a
the output is: [4, 5, 6], so the object 'a' was modified, but if I write the following:
a[:].append(7)
a
the output is: [4, 5, 6], so 'a' was not modified, which I assume is because append() was called on a copy of 'a' returned by the [:]. So what's going on? Another question: Why does Python allow me to write the second code snippet? I mean, modifying a temporary copy of 'a' doesn't make sense, so shouldn't this be flagged as an error (like it does in C++)? Thanks in advance.

Share this post


Link to post
Share on other sites
bubu LV    1436
Quote:
Original post by Gage64
modifying a temporary copy of 'a' doesn't make sense, so shouldn't this be flagged as an error (like it does in C++)?

Modifying temporary copies is completely legal in C++:

std::vector<int> a;
a.push_back(1);

std::vector<int>(a).push_back(2); // !!!



Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by bubu LV
Modifying temporary copies is completely legal in C++


I just tried a few examples and you're right. I was going by this (under the Temporaries section), but it looks like it's incorrect (even though what is written there makes sense).

Share this post


Link to post
Share on other sites
CrimsonSun    336
Python objects have several functions available to them in order to deal with slices including the __getslice__ and __setslice__ methods. What you're doing in your first example is (implicitly) calling the __setslice__ method - it takes a slice of an object, modifies it, and reinserts it into the object. In your second example, you are implicitly calling __getslice__ which returns a copy of the slice selected.

Basically, whenever you have a assign operation following a slice, the __setslice__ method is called. Otherwise the __getslice__ method is called.

Share this post


Link to post
Share on other sites
Gage64    1235
CrimsonSun - Thank you for the thorough explanation, I understand now, though I still think it's strange that the second snippet doesn't cause an error.

Share this post


Link to post
Share on other sites
CrimsonSun    336
In your second example, you're just appending 7 to a temporary copy. If you don't do anything with this temporary, then the temporary is just thrown away. You can assign this modified temporary to a variable:

a = [1, 2, 3]
b = a[:].append(7)


Now b contains [1, 2, 3, 7].

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
a = [1, 2, 3]

b = a[:].append(7)



Now b contains [1, 2, 3, 7].


I just typed this, and the value of b is None because append() doesn't return anything.

Share this post


Link to post
Share on other sites
CrimsonSun    336
You're absolutely right, I made a mistake above - append doesn't return anything, so b gets the value of None. Please disregard my above post.

However, you can always assign a variable to the return value of a slice. Notice also that this is the only way to get a copy of a sequence such as a list.

If you have:
a = [1, 2, 3]

And then assign b to a:
b = a

Then you've assigned b to the same list as that of a. So anything that modifies the list accessed from a also modifies the list accessed from b (they are the same list!):

b[1] = 4
Now both a and b contain the list [1, 4, 3]. If you want to have a copy of the sequence in a (such that modification of the list accessed by a does not modify the list accessed by b), you just take the default slice:

b = a[:]

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by CrimsonSun
However, you can always assign a variable to the return value of a slice. Notice also that this is the only way to get a copy of a sequence such as a list.

If you have:
a = [1, 2, 3]

And then assign b to a:
b = a

Then you've assigned b to the same list as that of a. So anything that modifies the list accessed from a also modifies the list accessed from b (they are the same list!):

b[1] = 4
Now both a and b contain the list [1, 4, 3]. If you want to have a copy of the sequence in a (such that modification of the list accessed by a does not modify the list accessed by b), you just take the default slice:

b = a[:]


I don't understand how any of that is relevant to my question :)

If you write:

a[:].append(5)

you are modifying a temporary object that will cease to exist after that statement (or at least it will no longer be accessible). I can't think of an example where you would want to do this, so if you wrote something like this, you probably made a mistake, which is why I think the interpreter should flag this as an error.

Thanks for your help so far.

Share this post


Link to post
Share on other sites
Kylotan    10008
It's not trivial for an interpreter to spot that such things are mistakes. Sometimes the function call on the right (eg. append() ) has side-effects that you're relying upon, so it can't assume it was an error. Maybe you were just testing the append() functionality, for example!

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by Kylotan
It's not trivial for an interpreter to spot that such things are mistakes. Sometimes the function call on the right (eg. append() ) has side-effects that you're relying upon, so it can't assume it was an error.


I don't know if Python supports const member functions (I just started learning it), but if it does, a temporary could be made automatically const and then such an error could be caught fairly easily (I think).

Either way, I'm more interested in the conceptual reason (for lack of a better term) - is it because of limitations in the implementation of the interpreter, or is my logic incorrect and there are times when you'll want to do this?

Quote:
Maybe you were just testing the append() functionality, for example!


IMO, that's not a good enough reason to leave out such a potentially helpful feature.

Share this post


Link to post
Share on other sites
swiftcoder    18437
Quote:
Original post by Gage64
Either way, I'm more interested in the conceptual reason (for lack of a better term) - is it because of limitations in the implementation of the interpreter, or is my logic incorrect and there are times when you'll want to do this?


There really is no good reason to disallow the current behaviour, unless you come from a Java-style world where the programmer must be protected even from himself. Python gives the programmer absolute power, and it is up to the programmer not to do stupid things with that power.

In a similar vein, you will discover that python has no const, no privacy, and allows the programmer to freely add and delete functions, classes and instance variables at any time.

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by swiftcoder
There really is no good reason to disallow the current behaviour


Quote:
Original post by Gage64
If you write:

a[:].append(5)

you are modifying a temporary object that will cease to exist after that statement (or at least it will no longer be accessible). I can't think of an example where you would want to do this, so if you wrote something like this, you probably made a mistake, which is why I think the interpreter should flag this as an error.


Quote:
Python gives the programmer absolute power


And what sort of "power" does this "feature" give you?

Quote:
you will discover that python has no const, no privacy


You're not serious... [wow]

Share this post


Link to post
Share on other sites
Zahlman    1682
Quote:
Original post by Gage64
Quote:
Original post by swiftcoder
There really is no good reason to disallow the current behaviour


Quote:
Original post by Gage64
If you write:

a[:].append(5)

you are modifying a temporary object that will cease to exist after that statement (or at least it will no longer be accessible). I can't think of an example where you would want to do this, so if you wrote something like this, you probably made a mistake, which is why I think the interpreter should flag this as an error.



And as indicated, there are many ways to do similar things in C++ and other languages, too. Regardless, the language design philosophy demands that Python flag as little as possible as an error in compilation (although it is certainly more structured than Perl ;) ), and it would be bizarre to raise an exception for something that can, in fact, be evaluated.

Consider also, for example, in Java:

class MyClass {
public static void main(String[] args) {
MyClass(args).run();
}
}


This is in fact a standard idiom, for minimizing the footprint of the (necessarily non-"OO") entry point of the program.

Quote:
Quote:
you will discover that python has no const, no privacy


You're not serious... [wow]


Dead serious. Python is largely about trust: you can make things sort of private by putting a double underscore at the start of the member name (which triggers some name mangling stuff), but the name mangling is easy to figure out if you really need to. But most Pythonistas prefer to just put a single underscore (reserving the double underscore for special names like __init__) and trust each other.

But instead of const, Python does have immutable objects, which are often even more useful. And because there is no low-level memory access, it can be trusted - there is no "const_cast". And thus, for example, no worry about mutating the keys of an dictionary and corrupting the hash: the dictionary only lets you change the set of keys, not the keys themselves, as it will only let you use immutable things as keys.

Share this post


Link to post
Share on other sites
SiCrane    11839
Python does have a variety of ways to make pseudo-const variables. For example, you can create read only properties for names:

class C:
def __init__(self):
__a = 5;
@property
def a(self):
return __a;

Alternately, you can override __getattr__ to map "a" to "__a" and __setattr__ to prevent anyone writing to "a", though that's slightly more annoying to code.

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by Zahlman
the language design philosophy demands that Python flag as little as possible as an error in compilation


A broad statement that doesn't address my question at all.

Quote:
it would be bizarre to raise an exception for something that can, in fact, be evaluated.


If by "can be evaluated" you mean that it's valid syntex, I disagree. This code:


float f = 2.5f;
int i = f;


can also be evaluated, but does that mean that the compiler should not flag that as an error (or at least a warning)? It's not that you are not able to perform the above assignment, it's just that if you do, you have to state that you are aware of the risks involved (i.e., use an explicit cast).

Quote:
Consider also, for example, in Java:



class MyClass {

public static void main(String[] args) {

MyClass(args).run();

}

}

This is in fact a standard idiom, for minimizing the footprint of the (necessarily non-"OO") entry point of the program.


I don't think it's that much trouble to replace that code with:

class MyClass {

public static void main(String[] args) {

MyClass(args) mc;
mc.run();

}

}


Quote:
most Pythonistas prefer to just put a single underscore (reserving the double underscore for special names like __init__) and trust each other.


Wasn't it the violation of this trust in early languages that caused later OOP languages to add privacy in the first place? Why should it be any different in Python?

Share this post


Link to post
Share on other sites
Kylotan    10008
Quote:
Original post by Gage64
Quote:
Original post by Kylotan
It's not trivial for an interpreter to spot that such things are mistakes. Sometimes the function call on the right (eg. append() ) has side-effects that you're relying upon, so it can't assume it was an error.


Either way, I'm more interested in the conceptual reason (for lack of a better term) - is it because of limitations in the implementation of the interpreter, or is my logic incorrect and there are times when you'll want to do this?


There are times when you might want to do this, or something similar, and the interpreter has no way of knowing if it's a mistake or not. You might really want to create something purely for the purpose of running one command on it and then have it disappear.

DBConnectionFactory().executeSQL("delete from stuff")


Quote:
I don't think it's that much trouble to replace that code with:

class MyClass {

public static void main(String[] args) {

MyClass(args) mc;
mc.run();

}

}

That's because you'd rather have the compiler force you write more code in an attempt to make it easier to check for errors. That's not the Python philosophy. If you want to continue writing extra code because it makes you feel safer, then Python is probably not the language for you.

Quote:
Wasn't it the violation of this trust in early languages that caused later OOP languages to add privacy in the first place? Why should it be any different in Python?


Different languages have different philosophies though. They're not just alternate syntax for the same approaches. The Python approach is to give you expressiveness, not to limit you in the name of safety.

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by Kylotan
Different languages have different philosophies though. They're not just alternate syntax for the same approaches. The Python approach is to give you expressiveness, not to limit you in the name of safety.


This seems to be the running theme, so I'm not going to argue with it any more. I don't have a lot of programming experience, and most of the time I learned C++, which might be the reason I think about safety so much. I don't know what philosophy works better in practice because I haven't had enough practice (if there's a pun in there, it's intended). Like I said, I'm just starting out with Python, and maybe after I learn more of it, I'll start seeing things in a different way.

Thanks everyone for their input.

Share this post


Link to post
Share on other sites
SiCrane    11839
Another thing to keep in mind is that Python is dynamically typed; and almost militantly so. For example, once you create an object as a certain class, you can actually change the type of the object via a function call.

class C:
def foo(self):
print "C!"
class D:
def bar(self):
print "D!"
def transmute(s):
s.__class__ = D
c = C()
transmute(c)
c.bar() # perfectly valid

In an environment where this is possible, strict compile time checking goes out the window. The number of things that the phrase "can be valid" encompasses is quite large.

Share this post


Link to post
Share on other sites
Zahlman    1682
Quote:
Original post by Gage64
Quote:
Original post by Zahlman
the language design philosophy demands that Python flag as little as possible as an error in compilation


A broad statement that doesn't address my question at all.


Er, what question? The implied one of "shouldn't you be prevented from doing this?" Then the answer is "it's a lot of effort, potentially prevents valid and convenient code constructs, and is actually quite unlikely to catch anything." After all, how could you accidentally do something like your 'a[:].append(5)' example?

Quote:
it would be bizarre to raise an exception for something that can, in fact, be evaluated.


If by "can be evaluated" you mean that it's valid syntex, I disagree. This code:


float f = 2.5f;
int i = f;


can also be evaluated, but does that mean that the compiler should not flag that as an error (or at least a warning)? It's not that you are not able to perform the above assignment, it's just that if you do, you have to state that you are aware of the risks involved (i.e., use an explicit cast).[/quote]

Not analogous. Python is dynamically and strongly typed; the equivalent code does require you to "cast" (actually convert) explicitly.

There are very few languages/compilers/etc. that actively try to prevent infinite loops, either, BTW.

Quote:
I don't think it's that much trouble to replace that code with


Which buys you nothing, and costs giving a name to something that is not actually important.

Quote:
Quote:
most Pythonistas prefer to just put a single underscore (reserving the double underscore for special names like __init__) and trust each other.


Wasn't it the violation of this trust in early languages that caused later OOP languages to add privacy in the first place? Why should it be any different in Python?


Um, no. This decision is re-made each time according to the language designer's philosophy.

Share this post


Link to post
Share on other sites
Oluseyi    2111
I'm late to the conversation, but I'd like to point out a few glaring flaws:

Quote:
Original post by Gage64
Another question: Why does Python allow me to write the second code snippet? I mean, modifying a temporary copy of 'a' doesn't make sense...

Sure it does.

Quote:
...so shouldn't this be flagged as an error (like it does in C++)?

Sorry, modifying a temporary in C++ is not flagged as an error.

You're mixing concepts, and that's why you're confused. The C++ concept you have an imprecise grasp of here is rvalues and lvalues. In C++ (and C), an lvalue is any object that can appear on the left-hand side of an assignment, while an rvalue is anything that can appear on the right. Almost all values in C++ are rvalues, whereas lvalues are more restricted.

Python doesn't really have this distinction. Instead, Python's limitation is when an assignment statement is syntactically valid, with the most prominent restriction being that it can not occur in a conditional, so the common C/C++ idiom of:
if((var = generatedValue) == referenceValue) { ... }
is not possible.

Python embodies a philosophy of respecting the programmer. It presumes that you know what you're doing, and notifies you when things go wrong rather than trying to prevent you from ever doing wrong. It's the same reason why Python has no const and no privacy - the language philosophy holds that such things are crutches people rely on to guarantee facets they actually can't (private, for example, is circumvented at the binary level, and even C++ lets you const_cast with impunity). It's certainly a different way to program, and the near-reflexive rejection of it is a common initial reaction. It was mine when I first encountered Python, in 2002. Now Python's the first language I reach for.

Happy hacking.

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by Oluseyi
You're mixing concepts, and that's why you're confused. The C++ concept you have an imprecise grasp of here is rvalues and lvalues. In C++ (and C), an lvalue is any object that can appear on the left-hand side of an assignment, while an rvalue is anything that can appear on the right. Almost all values in C++ are rvalues, whereas lvalues are more restricted.


I'm familiar with that distinction and that's not what I meant. I merely suggested that temporary objects should automatically be const, so that you'll only be able to call const member functions on them (which I thought was the way it works in C++, but I already admitted that I was wrong). The reason being that if you call a non-const member function on a temporary, it's probably a mistake on your part (not necessarily a typo; it might be a logical error). This may occasionally force you to write a couple of extra lines of code, but the added safety might be worth it.

Again, I would like to emphasize that this is just a theoretical statement from someone who is mostly only familiar with C++'s philosophy and doesn't have a lot of programming experience. I'm not sure if the above enforcement is a good idea in practice (and in every language), though I'm still not really convinced that it isn't.

Quote:
It's certainly a different way to program, and the near-reflexive rejection of it is a common initial reaction. It was mine when I first encountered Python, in 2002. Now Python's the first language I reach for.


Like I said, I'm just starting out with Python (though it's nice to hear that I'm not the only one who had these objections when starting out). When I'll learn more of it, I might start seeing things your way.

Share this post


Link to post
Share on other sites
SiCrane    11839
Quote:
Original post by Gage64
The reason being that if you call a non-const member function on a temporary, it's probably a mistake on your part (not necessarily a typo; it might be a logical error). This may occasionally force you to write a couple of extra lines of code, but the added safety might be worth it.


There are plenty of instances of idiomatic C++ that involve calling non-const functions on temporaries. For example, the swap trick to reduce memory usage on a std::vector:

std::vector<T> original;
// stuff that leaves original with a capacity greater than it's size
std::vector<T>(original).swap(original); // removes excess capacity

Or dereferencing a temporary iterator:

*itr++ = foo;

Non-pointer iterators overload operator * so *itr++ involves calling a non-const function on a temporary.

Share this post


Link to post
Share on other sites
Gage64    1235
Quote:
Original post by SiCrane
...


Finally, code examples I can't argue with. Now I'm convinced. [smile]

Thank you for posting this.

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.
Sign in to follow this