Now for the reason for this blog entry. As I stated I have really been honing my programming skills as of late and dealing with some odd languages mostly in the functional paradigm. One thing about functional languages is that you don't have OOP so you need to find alternate ways to create somewhat of a similar effect and it turns out Closures are just that. Many people ask why not just use OOP then. Well the issue really arises because of the way most books typically teach OOP which leads to very sloppy inheritance and can cause deep hierarchies. This issue with this is it makes your code a maintenance nightmare. The other with the way most books teach OOP is they create a notion of taxonomies which leads to people creating Classes that should have never been classes to begin with. Note I am not saying OOP is evil I am saying the way OOP is often taught is evil. On another note OOP can lead to issues with parallelism where the state of the object becomes out of sync when multiple threads are involved and closures solve this problem quite well.
[Edit Thank you TheUnbeliever]
Just recently I read an post in For Beginners about this issue this post brings up. The OP was sent to StackOverflow where there was a good explanation of the issue at hand with Python's Scope resolution and closures. The one solution I present in this article is the one used in the StackOverflow article this article is actually one that I used when learning how to implement closures in Python 2.7. Hopefully this post will be useful to help people understand closures and how to implement them in both Python 2.7 and Python 3.x.
So what is a closure. Most definitions of what a closure is uses odd jargon like lexical structure and referencing environment and such. The definitions are not very clear unless you have a strong functional programming background so here is a more clean definition that I stumbled across.
A closure is a block of code that meets 3 particular criteria...
1. It can be passed around as a value
2. Can be executed on demand by anyone who has this value
3. It can refer to variables from the context in which it was created in (lexical scope, referencing environment)
So why are these useful... For one they allow you to maintain state between calls much like an object, very useful for creating callback, can be used to hide utility function in the function providing a cleaner api and they can even be used to create many programming constructs such as loops. These constructs are very useful over all and can really simplify to code you write and need to maintain in large complex systems.
Python has had support for closures since version 2.2, however, there are a few issues. Issueless support comes in Python 3.x. The issue in Python 2.7 is a Python Scoping issue that forces the closed over variable to be read only.
Here is an example of a closure in implemented in Python with the exact issue of Scoping...
def counter(start_pt):
def inc():
start_pt += 1
return start_pt
return inc
if __name__ == '__main__':
func = counter(1)
for i in range(10):
print(func())
So what does the code do it is simple it counts up 10 numbers from an arbitrary start point. So what is the issue well the issue is with pythons scope this actually generates an error the exact error is UnboundLocalError: local variable 'start_pt' referenced before assignment. This is the same issue the author of the post was having and it is because of the way python defines the scope of a variable. Basically python determines the scope from the innermost first assignment it finds which means when we increment start_pt we are accessing a variable with no initial value yet because python has the scope confused and forces the variable to be read only.
Python 3.x solves this scope problem by implementing a new statement called nonlocal here is how it looks...
def counter(start_pt):
def inc():
nonlocal start_pt
start_pt += 1
return start_pt
return inc
if __name__ == '__main__':
func = counter(1)
for i in range(10):
print(func())
The output of this code will be as expected 2 3 4 5 6 7 8 9 10 11
But again this is only in Python 3.x what if we can't use Python 3.x because we need support for API's that are not Python 3.x compatible yet. We want to use Closures but also want to avoid the scope issue. There are a few solutions to this problem. Lets go through 2 of them one which seems kind of hacky and other which is less hacky but not a true closure but instead just mimics the closure concept. Here is the first which is a true closure that uses a Python mutable list to work around the Scope issue.
def counter(start_pt):
c = [0]
c[0] = start_pt
def inc():
c[0] += 1
return c[0]
return inc
if __name__ == '__main__':
func = counter(1)
for i in range(10):
print(func())
Basically what we do is create a new variable that is a 1 element mutable list and place our start_pt in this location. We can't use a standard variable c that is not a list because c becomes read only inside the inner function which results in the same issue as before giving us the same error as before so using a mutable list works around the issue and we get the expected output of 2 3 4 5 6 7 8 9 10 11. This is kind of sloppy looking so lets look at the not a true closure but mimics a closure alternative. Note I am violating standard Python naming conventions here because we want the user to think they are using a lexical closure so here is the code.
class counter(object):
def __init__(self, start_pt):
self.start_pt = start_pt
def __call__(self):
self.start_pt += 1
return self.start_pt
if __name__ == '__main__':
func = counter(1)
for i in range(10):
print(func())
So here we turn the closure into a mimicked closure by using a class to preserve the state instead of the mutable list.
We also force our class to be executed only as a function by using the __call__ magic method. So here we have a closure but it is not a true closure but actually and object disguised as a closure.
So there you have it we were able to discover 2 ways to work around the Python 2.7 issues with Scoping and still be able to use our non 3.x compatible libraries. Both methods are valid you can choose which one you want to use. Sorry the example is so simplistic but it really is an easy example to get the point across now you can use your favorite language and use the powerful feature of closures the method you choose is up to you or you can be a real man and jump on Python 3.x
