• Advertisement
Sign in to follow this  

Python question: Default Pass by reference, or value?

This topic is 4246 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have been working on programming a game entirely in python, using pygame, in an attempt to learn python (I already know c++). I have run into a problem with a recursive function I wrote (pathfinding). Basically, it seems to me that the variables are passing by reference, instead of by value (as is the default in c++). As you can imagine, this is causing problems since I wrote the function assuming that pass by value was the default. Is the default indeed pass by reference? And if so, is there any way to pass by value? Otherwise, I'm going to need a messy work-around for the function (as if my code isn't messy enough) Thanks in advance

Share this post


Link to post
Share on other sites
Advertisement
It's a little more complicated - see the fairly involved discussion on the Python mailing list. It's similar to pass-by-reference semantics, but more involved that that. The discussion linked has a decent explanation.


Personally, I haven't used the language enough to know if there's a way to force copy semantics when passing arguments to a function; I'm sure someone else can answer that half, though [smile]

Share this post


Link to post
Share on other sites
Quote:
Original post by PimpWilly
Is the default indeed pass by reference? And if so, is there any way to pass by value?


It's pass by reference, but assignment is not overwriting but is rebinding. So although you get a reference in the called function, if you assign to that object, it won't affect the original object. However, if you modify that object, it does affect the original.

Basically, if you want a copy, make a copy. This might involve the copy module, or whatever it's called. Generally, it's actually very rare that you actually do need to make that copy. I've never used that module myself. You can probably express the problem in a different way that doesn't require a copy.

Share this post


Link to post
Share on other sites
As far as argument passing goes, the only available Python way is pass-by-value. All Python variables are references to objects of course, but they themselves are not passed by reference. It's clearly references that are passed by value. Don't confuse passing by reference and passing a reference.

Share this post


Link to post
Share on other sites
You're blurring the 2 concepts beyond the point of usefulness however. It's exactly the same in C or C++, or Java - the called function ends up with a copy of the reference or the pointer. At the low level it's copying a value which represents a memory location, but in language terms it's still "pass by reference" because changes made to an argument in the called function are reflected in the corresponding object in the calling function.

Share this post


Link to post
Share on other sites
Quote:
Original post by Kylotan
You're blurring the 2 concepts beyond the point of usefulness however. It's exactly the same in C or C++, or Java - the called function ends up with a copy of the reference or the pointer. At the low level it's copying a value which represents a memory location, but in language terms it's still "pass by reference" because changes made to an argument in the called function are reflected in the corresponding object in the calling function.


Not at all. Consider this simple Python function that gets an angle and returns both the sine and the cosine, ie it needs to return 2 values:


import math

def sin_cos(angle,sin,cos):
sin=math.sin(angle);
cos=math.cos(angle);

angle=45.0;
sin=None;
cos=None;
sin_cos(angle,sin,cos);
print "The sine of ",angle,"is: ",sin
print "The sine of ",angle,"is: ",cos








Claiming that Python uses pass-by-reference would be a big mistake, as the programmer would naturally expect the above code to work just as it works with C++(where references are regularly used to for multiple return values). He must realize the difference between pass-by-reference and pass-by-value of a reference and use the correct way to do this, that is return a tuple that contains both results.

See this for a couple of more ways to emulate pass-by-reference in Python, although most of them are overcomplicating things and most probably can be avoided.

[Edited by - mikeman on July 3, 2006 2:12:37 PM]

Share this post


Link to post
Share on other sites
It's passing references by value, in the same way that Java does, except that Python doesn't have any "primitives" - everything is an object, including "ints".

If you need to copy (clone) things, then do so, with the copy module. However, you normally should be trying to write things in such a way that you don't need to do this.

I don't suppose you could show the code in question? :s

Share this post


Link to post
Share on other sites
I'll go ahead and post the path function. Might be a bit convuluted, but eh, here it goes:

Note: I tried changing leftData to self.leftData, to see if that would help the problem, but it didn't seem to do the trick. so thats why it's referenced different right now

def findPath(self, currentData, currentSquare, destinationSquare, movementLeft):
#subtract current tile movement cost from movement spaces left
# check if movement cost < 0.
# If so, I cant get here, so append with a -1 and return it
#print "Updating Data for debug:", currentData
tileInfo = self.getTile(currentSquare[0], currentSquare[1])
currentTileMovementCosts = self.mapKey.getMovementCost(tileInfo)
print " TileCosts:", currentTileMovementCosts
if movementLeft < 0:
currentData[0] = -1
return currentData
#Check if I am on the destination square
if currentSquare == destinationSquare:
print "Found the dest square, returning data ->", currentData
return currentData
else:
currentData[0] += currentTileMovementCosts
movementLeft -= currentTileMovementCosts
self.leftData = currentData
self.leftData.append('L')
tempSquare = currentSquare
tempSquare[0] -= 1
self.leftData = self.findPath(self.leftData, tempSquare, destinationSquare, movementLeft)

upData = currentData
upData.append('U')
tempSquare = currentSquare
tempSquare[1] -= 1
upData = self.findPath(upData, tempSquare, destinationSquare, movementLeft)

rightData = currentData
rightData.append('R')
tempSquare = currentSquare
tempSquare[0] += 1
rightData = self.findPath(rightData, tempSquare, destinationSquare, movementLeft)

downData = currentData
downData.append('D')
tempSquare = currentSquare
tempSquare[1] += 1
downData = self.findPath(downData, tempSquare, destinationSquare, movementLeft)

#Find which path is >0 and which has the lowest movement costs
#print "all data:", self.leftData
if upData[0] >= 0:
lowestData = upData
elif rightData[0] >= 0:
lowestData = rightData
elif self.leftData[0] >= 0:
print "@@@@@@@@@@@@@Left Data", self.leftData
lowestData = rightData
elif downData[0] >= 0:
lowestData = downData
else:
#no paths found
# set currentData[0] to -1 and return
currentData[0] = -1
#print "Path not found"
return currentData
print "!!!!!!!!!!!!!!!!!!!!!!!!!!Left Data,", self.leftData
if upData[0] > 0 and upData[0] < lowestData[0]:
lowestData = upData
if rightData[0] > 0 and rightData[0] < lowestData[0]:
lowestData = rightData
if self.leftData[0] > 0 and self.leftData[0] < lowestData[0]:
print "#####################Left Data", self.leftData
lowestData = leftData
if downtData[0] > 0 and downData[0] < lowestData[0]:
lowestData = downData

#now return our shortest path
print " Path found, heres data ->", lowestData
return lowestData




As the algorithm runs, it ends up just adding a whole lot of data to leftData, rightData,upData,downData, and currentData, so that when it finally returns with what I hoped would be the shortest path, it ends up just being a path with like 60+ moves (even for just one square over)

Share this post


Link to post
Share on other sites
Just an update, but I haven't had any luck so far. Looks like I am going to have to go ahead and just create a temp variable before each recursion call, then re-assign it after it returns, unless anybody has any better alternatives?

Thanks,

--Steve

Share this post


Link to post
Share on other sites
This is just a brute-force depth-first search with some limit on allowed cost, AFAICT.

The problem is the statements like:


upData = currentData


This does not clone the list, but instead aliases it. Again, Java object model.

However, Python is friendlier here in that it offers a general-purpose copy module. Here you could solve the problem by invoking the module like:


# At the top, or just inside findPath
import copy
# ...
upData = copy.copy(currentData)
# Use upData


When the structure is complex (i.e. contains nested lists), there is also copy.deepcopy which applies recursively to the contained things.

However, there is a much simpler and idiomatic solution: instead of "copying" the list and appending to it (the reason you get the garbage results is that all the changes get applied to the passed-in currentData via the upData etc. aliases), do the append in a way that implicitly makes a copy:


upData = currentData + ['U']


That said, lots of cleanup is possible here. The directions are all treated in similar ways, so there's code duplication there that we can clean up. Also, we can represent currentSquare not as a list of two ints (as it currently appears to be) but instead represent points with complex numbers, and then use plain old addition to generate the new points.


def findPath(self, currentData, currentSquare, destinationSquare, remaining):
# I renamed 'movementLeft' because the 'left' is confusing ;)
# Similarly, change the getTile interface to accept the coordinate directly.
tileInfo = self.getTile(currentSquare)
currentTileMovementCosts = self.mapKey.getMovementCost(tileInfo)

# Guard clause: Can we no longer get there?
if movementLeft < 0:
currentData[0] = -1
return currentData

# Guard clause: Did we just get there?
if currentSquare == destinationSquare:
return currentData

# Otherwise, recurse over four neighbouring directions.
currentData[0] += currentTileMovementCosts
remaining -= currentTileMovementCosts

# set the default ("failed") result, and improve it with the recursion
# results if possible.
bestResult = [-1] + currentData[1:]

for direction in ([['L'], complex(-1, 0),
['U'], complex(0, -1),
['D'], complex(0, 1),
['R'], complex(1, 0)]):
tempData = currentData + direction[0] # append direction code
tempSquare = currentSquare + direction[1] # add direction vector

result = self.findPath(tempData, tempSquare, destinationSquare, remaining)
# If that's better than what we have, set it.
if result[0] != -1 and (bestResult[0] == -1 or result[0] < bestResult[0]):
bestResult = result # don't need to copy; we won't modify it,
# but just possibly replace it :)

return bestResult

Share this post


Link to post
Share on other sites
There is no passing by value in Python. EVERYTHING is passed by reference. This differs from Java, as even integers, floats, and booleans are passed by reference.

However, integers, floats, and booleans, (and tuples as well) are all immutable. That means that you cannot change their value at all, and thus, you cannot propogate changes back to the caller.

Consider mulling over these examples:
def increment(x):
x += 1
print x

myvar = 4
increment(myvar) # prints 5
print myvar # prints 4



def increment_list(mylist):
mylist[0] += 1
print mylist[0]

myvar = [4]
increment_list(myvar) # prints 5
print myvar[0] # prints 5 as well


If you want to pass by value, you can simply copy the data once you get to the function.

def increment_list_by_value(mylist):
mylist = mylist[:] # [:] is a shorthand to copy lists and tuples, dictionaries also support the copy() method.
mylist[0] += 1
print mylist[0]

myvar = [4]
increment_list_by_value(myvar) # prints 5
print myvar[0] # prints 4, because changes were only made to the copy

Share this post


Link to post
Share on other sites
Quote:
Original post by Tac-Tics
However, integers, floats, and booleans, (and tuples as well) are all immutable.

Whoa, nelly!

Tuples and strings are immutable. Integers, floats and booleans are not immutable in the sense your examples imply.

s = "Oluseyi" # we would traditionally think of s as a string
s = "Olu, the Evil" # s was reassigned, not modified
s[2] = 'y' # TypeError: object does not support item assignment
s += " Man" # OK. Again, s was reassigned to the result of the concatenation

i = 1 # we would traditionally think of i as an integer
i = 5
i += 2 # both OK; i is being reassigned

Given that there is no way to modify an integer without reassigning it, the notion that it is immutable is imprecise. It's the same for floats. Mutability is only relevant, in this context, to aggregate or sequence types - lists, tuples, strings, dictionaries. Of those four, only strings and tuples are immutable (you can not add new elements to a tuple, nor can you modify an element already in a tuple).

Your increment example doesn't prove the immutability of integers, for instance; it proves that Python argument lists are populated with references to copies of the values of the supplied parameters (because all objects are accessed by reference in Python).

Share this post


Link to post
Share on other sites
Quote:

This differs from Java, as even integers, floats, and booleans are passed by reference.


No. You don't pass "integers,floats and booleans". Not by value, not by reference, not by anything. You don't pass objects in general. You pass references *by value*. It doesn't matter if it's a reference to an integer,a string, a list or a user-defined object. It's all the same to Python. All the examples you gave are explained by that simple notion: If you pass copies of references, any reassignment of the reference isn't visible outside of the scope of the function, but of course any operation done to the referenced object is.

Share this post


Link to post
Share on other sites
Quote:
Original post by mikeman
Quote:
Original post by Kylotan
You're blurring the 2 concepts beyond the point of usefulness however. It's exactly the same in C or C++, or Java - the called function ends up with a copy of the reference or the pointer. At the low level it's copying a value which represents a memory location, but in language terms it's still "pass by reference" because changes made to an argument in the called function are reflected in the corresponding object in the calling function.


Not at all. Consider this simple Python function that gets an angle and returns both the sine and the cosine, ie it needs to return 2 values:

*** Source Snippet Removed ***

Claiming that Python uses pass-by-reference would be a big mistake, as the programmer would naturally expect the above code to work just as it works with C++(where references are regularly used to for multiple return values).


But that is not because the objects are not passed by reference, but because assignment works differently to the way it does in C++. The references themselves are handled in the function call exactly as they would be in C++. Only the assignment operation differs.

Share this post


Link to post
Share on other sites
Quote:
Original post by Kylotan
Quote:
Original post by mikeman
Quote:
Original post by Kylotan
You're blurring the 2 concepts beyond the point of usefulness however. It's exactly the same in C or C++, or Java - the called function ends up with a copy of the reference or the pointer. At the low level it's copying a value which represents a memory location, but in language terms it's still "pass by reference" because changes made to an argument in the called function are reflected in the corresponding object in the calling function.


Not at all. Consider this simple Python function that gets an angle and returns both the sine and the cosine, ie it needs to return 2 values:

*** Source Snippet Removed ***

Claiming that Python uses pass-by-reference would be a big mistake, as the programmer would naturally expect the above code to work just as it works with C++(where references are regularly used to for multiple return values).


But that is not because the objects are not passed by reference, but because assignment works differently to the way it does in C++. The references themselves are handled in the function call exactly as they would be in C++. Only the assignment operation differs.


Assignment just copies the value from one variable to another. There's nothing special about that. For example:

-----------------
a=str("Hello");
b=a
-----------------

1)a=str("Hello"):
str() creates a string object and returns a reference to it. Then, the assignment operator copies the value of the reference to "a".

2)b=a
The value of "a" is copied to "b". Now a and b reference the same object.

I don't understand why you need to think in terms of "overwriting and not rebinding" as you mentioned earlier when things are pretty clear. Variables in python are "objects" that hold a reference. If you want an equivalent in C++, you can observe the behaviour of Boost::Python. For instance:


void foo(boost::python::object a)
{
a=str("World");
}

boost::python::object myvar;
myvar=str("Hello");
foo(myvar);
//myvar is still "Hello"




boost::python::object represents a Python variable(integer,string,list,callable,anything) and is passed by value, exactly like Python does. Now:



void foo(boost::python::object& a)
{
a=str("World");
}

boost::python::object myvar;
myvar=str("Hello");
foo(myvar);
//myvar is "World"




That is passing by reference. The thing is, Python doesn't have that option.

[Edited by - mikeman on July 8, 2006 3:21:24 PM]

Share this post


Link to post
Share on other sites
Er, I think you're both right, based on my memory of all the previous times I thought about this; it's just different ways of looking at it. Assignment does work differently - rebinding rather than changing all state - and thus it's *as if* it were pass by reference but with a different assignment operator (that - well - didn't actually assign back to the reference, which makes it rather strange to call it pass by reference after all), whereas it's more clearly thought of as pass-reference-by-pointer (especially if you're writing an implementation).

Share this post


Link to post
Share on other sites
Quote:
Original post by Zahlman
Er, I think you're both right, based on my memory of all the previous times I thought about this; it's just different ways of looking at it. Assignment does work differently - rebinding rather than changing all state - and thus it's *as if* it were pass by reference but with a different assignment operator (that - well - didn't actually assign back to the reference, which makes it rather strange to call it pass by reference after all), whereas it's more clearly thought of as pass-reference-by-pointer (especially if you're writing an implementation).


I just don't understand this "rebinding" thing. When I do 'a=str("mikeman")' it's not like the left side of the assignment is a reference and the right side is an actual object, so the assignment operator "binds" a to "mikeman". Both sides are references, so in my mind the assignment works pretty straightforward, all you have to do is keep in mind that everything you manipulate in Python is a reference. Kind of like a smart pointer in C++, except that it's untyped(it can reference any type of object).

Share this post


Link to post
Share on other sites
*sigh*

The "rebinding" process is that of cloning a remote control and putting it into a coffee cup (replacing whatever remote control was there before). The "assignment" process, if Java (or Python) provided it, would be that of overwriting all the contents of the Cat object with those of some other Cat object (possibly making adjustments for the fact that its location is different). That's what happens when you invoke a C++ operator= via a reference.

The difference being, Python only has objects and no primitives, so you don't need as complicated of an explanation.

We all know how it works here; I'm just trying to translate the different explanations to each other's proponents :s

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement