Jump to content
  • Advertisement
  • 08/16/13 07:41 PM
    Sign in to follow this  

    Advanced Python Part One: List Comprehensions

    General and Gameplay Programming

    superman3275
    This is an extension of my blog article about list comprehensions and the first part of an advanced python series. We will cover List Comprehensions, Lambdas, Functional Programming, Map / Reduce / Filter, Decorators, and Descriptors.

    Python List Comprehensions

    List comprehensions are Python's solution to the large loops required to do tasks which should be easy. They transform the simple task of getting all the even numbers between zero and one-thousand from: my_list = [] for index in range(1000): if index % 2 == 0: my_list.append(index) to my_list = [x for x in range(1000) if x % 2 == 0]

    The use for list comprehensions

    List comprehensions are essentially: list_name = [expression values filter] Values - A for loop of values to check against the filter Filter - All values go through this. It is an if statement, and the expression is appended to the end of the list when it is true. Expression - Data to append onto the list if Filter is true It's important to note here that the expression could be anything. We could get the squares of all even numbers between zero and one-thousand just as easily: my_list = [x**2 for x in range(1000) if x % 2 == 0]

    Interesting Points

    List comprehensions work on strings, and the for loop can loop through any list. The expression can be as complex as you want. The expression is what gets appended, and it can even be another list. Look at this piece of code: my_list = [[x*y for y in range(1,11) if y % 2 == 0] for x in range(1,11) if x % 2 == 0] This generates a list of lists (which is stored in my_list) where every even number between one and ten is multiplied by every other even number between one and ten.

    Conclusion

    To practice list comprehensions you can try out Code Academy's interactive course. You can also learn more by looking at the Python Docs List Comprehensions are a very useful tool which can replace many loops if you look out for it. Cheers :)!

    Article Update Log

    June 20 2013 - Released Article August 19 2013 - Added in Python Docs Link.


      Report Article
    Sign in to follow this  


    User Feedback


    In my_list = [x**2 for x in range(1000) if x % 2 == 0]

     

    What does x**2 mean?

    Share this comment


    Link to comment
    Share on other sites

    In my_list = [x**2 for x in range(1000) if x % 2 == 0]

     

    What does x**2 mean?

    To the Power: 2 ** 2 = 4

    Share this comment


    Link to comment
    Share on other sites

     

    In my_list = [x**2 for x in range(1000) if x % 2 == 0]

     

    What does x**2 mean?

    To the Power: 2 ** 2 = 4

     

    Oh. That's a nice operator. Neither C++ nor Java has this type of operator :)

    I'm really starting to like Python! :)

    Share this comment


    Link to comment
    Share on other sites
    This is a nice article but it is a bit short.
     
    Here is some code that uses list comprehensions to filter all of the jpg files in the current directory:
     
    import re
    import os
    
    filtered_list = [f for f in os.listdir(os.getcwd()) if re.match(r".*\.jpg", f, re.IGNORECASE)]
    
    for filename in filtered_list:
        print(filename)
    
    Without list comprehensions it would look like this:
     
    import re
    import os
    
    filtered_list = []
    
    for f in os.listdir(os.getcwd()):
        if re.match(r".*\.jpg", f, re.IGNORECASE):
            filtered_list.append(f)
            
    for filename in filtered_list:
        print(filename)
    
    and yes, I know I could compile the regular expression but this is just an example smile.png .

    Share this comment


    Link to comment
    Share on other sites

    So I thought about this for a while and I wanted to add my opinion to this article.

     

    Such things like list comprehensions, decorators, ect can be useful in the right context, but they can also be misused. I will even go so far as to say that in a good majority of situations you should avoid using such things unless you understand enough of what you are doing and have a well thought out reason to introduce them.

     

    Keep in mind that one of the wonderful things about Python is the readability. You can do some neat things with Python but it can make others on your team or other people not as able to understand the code that you have written. This may not be a concern for everyone, but it is a concern that I deal with when working with Python.

     

    For instance in the example I posted previously the second code is really in my mind easier to understand even though it is more lines of Python. We saved some line count but now we have to spend more time to read and understand the code.

     

    Even with the ** operator it is not immediately clear that this raises something to a power, where pow(x, 2) can be more clear (pow is similar to ** but it is NOT the same as math.pow... I know that is confusing).

     

    Just remember that the brilliant code that you write today will be an annoying puzzle to you in the future :) .

    Share this comment


    Link to comment
    Share on other sites

    Shadowisadog, I would be very surprised if someone didn't know what the ** operator did. It is one of the operators taught in all the beginning courses I've taken, and if someone hasn't studied python enough to know what ** means (or doesn't know how to google it) then I shouldn't be importing the math module and using an extra function just in case.

     

    List comprehensions are almost always better than the longer and harder to read solutions. I haven't ever looked at a list comprehensions and not known what it did, and that's because I've studied them.

     

    As another example, in the pygame library here's how you get events:

    import pygame
    
    # List to hold keydown events
    keydown_events = []
    
    # Game Loop
    while True:
        # Get all the keydown events
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                keydown_events.append(event)
    

    it could be easily simplified to:

    import pygame
    
    keydown_events = [event for event in pygame.event.get() if event.type == pygame.KEYDOWN]
    

    Once someone learns about list comprehensions I doubt that there are ones that they can't understand just because they're too complicated.

     

    Cheers smile.png!

     

    (Also, sorry for downvoting you. It was spur of the moment and unfortunately gamedev.net won't let me take it back :0!)

    Share this comment


    Link to comment
    Share on other sites

    You should mention:

    • that there are memory-friendly generator expressions, which can replace many list uses
    • that there are dict and set comprehensions with a very similar syntax
    • that multiple for and if clauses are allowed, and obviously very useful to navigate and join data structures
    • that such nested clauses are evaluated left to right (the most important non-obvious detail)

    Clumsy style:

    my_list = [[x*y for y in range(1,11) if y % 2 == 0] for x in range(1,11) if x % 2 == 0]

    Almost normal style:

    my_list = [x*y for x in range(1,11) if x % 2 == 0 for y in range(1,11) if y % 2 == 0]
    my_list = [x*y for x in range(1,11) for y in range(1,11) if if (x % 2 == 0 and y % 2 == 0)]

    Good style:

    my_list = [x*y for x in range(2,11,2) for y in range(2,11,2)]
    • More practical and developed examples would also be an improvement.

    Share this comment


    Link to comment
    Share on other sites

    Shadowisadog, I would be very surprised if someone didn't know what the ** operator did. It is one of the operators taught in all the beginning courses I've taken, and if someone hasn't studied python enough to know what ** means (or doesn't know how to google it) then I shouldn't be importing the math module and using an extra function just in case.

     

    pow is not part of math. I mentioned that there is pow and math.pow. I think using pow is clearer because it is more explicit and it is can prevent typos where you accidentally type * instead of **. Someone on these very comments specifically asked what ** as well so I guess feel free to be surprised?

     

     

    List comprehensions are almost always better than the longer and harder to read solutions. I haven't ever looked at a list comprehensions and not known what it did, and that's because I've studied them.

     

    I disagree from the standpoint that I think the longer solution is easier to read. The list comprehension approach may save line count and avoid "extra" code, but I do not think it is easier to read. List comprehensions may not be hard for you to read but that is precisely because you have studied them! This is an "advanced python" article after all?

     

     

    As another example, in the pygame library here's how you get events:

    import pygame
    
    # List to hold keydown events
    keydown_events = []
    
    # Game Loop
    while True:
        # Get all the keydown events
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                keydown_events.append(event)
    

    it could be easily simplified to:

    import pygame
    
    keydown_events = [event for event in pygame.event.get() if event.type == pygame.KEYDOWN]
    

    Once someone learns about list comprehensions I doubt that there are ones that they can't understand just because they're too complicated.

     

    I am not sure that is a great simplification from the stand point that I may want to add extra event types (and probably would in context). I think the code would be simpler if I did not use a list comprehension in this context... because then I might repeat it for the KEYUP, ect and then I am repeating the event for event in pygame.event.get() ?

     

     


     

    Cheers smile.png!

     

    (Also, sorry for downvoting you. It was spur of the moment and unfortunately gamedev.net won't let me take it back :0!)

     

    No worries.

    Share this comment


    Link to comment
    Share on other sites

     

    Shadowisadog, I would be very surprised if someone didn't know what the ** operator did. It is one of the operators taught in all the beginning courses I've taken, and if someone hasn't studied python enough to know what ** means (or doesn't know how to google it) then I shouldn't be importing the math module and using an extra function just in case.

     

    pow is not part of math. I mentioned that there is pow and math.pow. I think using pow is clearer because it is more explicit and it is can prevent typos where you accidentally type * instead of **. Someone on these very comments specifically asked what ** as well so I guess feel free to be surprised?

     

     

    List comprehensions are almost always better than the longer and harder to read solutions. I haven't ever looked at a list comprehensions and not known what it did, and that's because I've studied them.

     

    I disagree from the standpoint that I think the longer solution is easier to read. The list comprehension approach may save line count and avoid "extra" code, but I do not think it is easier to read. List comprehensions may not be hard for you to read but that is precisely because you have studied them! This is an "advanced python" article after all?

     

     

    As another example, in the pygame library here's how you get events:

    import pygame
    
    # List to hold keydown events
    keydown_events = []
    
    # Game Loop
    while True:
        # Get all the keydown events
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
                keydown_events.append(event)
    

    it could be easily simplified to:

    import pygame
    
    keydown_events = [event for event in pygame.event.get() if event.type == pygame.KEYDOWN]
    

    Once someone learns about list comprehensions I doubt that there are ones that they can't understand just because they're too complicated.

     

    I am not sure that is a great simplification from the stand point that I may want to add extra event types (and probably would in context). I think the code would be simpler if I did not use a list comprehension in this context... because then I might repeat it for the KEYUP, ect and then I am repeating the event for event in pygame.event.get() ?

     

     


     

    Cheers smile.png!

     

    (Also, sorry for downvoting you. It was spur of the moment and unfortunately gamedev.net won't let me take it back :0!)

     

    No worries.

     

    Well, for getting various types of events:

    import pygame
    
    keyboard_events = [event for event in pygame.event.get() if event.type == pygame.KEYDOWN or event.type == pygame.KEYUP]
    

    List comprehensions are also rapidly replacing things like Map, Replace, and Filter. I honestly would be very confused as to why someone wouldn't understand list comprehensions unless they were learning from an outdates source, also, considering almost every tutorial I've looked at (including Code Academy's) has included sections on it.

     

    Also, the whole entire power argument doesn't make sense. All pow does is call **, and it is one of the basic operators of the language. Even if someone didn't know what it did, a quick google of "Python operators" would help.

    Share this comment


    Link to comment
    Share on other sites
    Everything LorenzoGatti said.

    A treatment of list comprehensions that is named "advanced" really needs to explain generator expressions, the relation of the one to the other, and the performance tradeoffs of the two approaches.

    I'd also expect to see some discussion of iterators and generators, and how they relate to the above.

    (Or, potentially, renaming this this to "Beginners Python")

    Share this comment


    Link to comment
    Share on other sites

    This really was meant to be an introduction, not to cover every single aspect of list comprehensions.

     

    Nevertheless, I'm going to edit in a link to the python Documentation on the subject.

    Share this comment


    Link to comment
    Share on other sites


    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!

    Register a new account

    Sign in

    Already have an account? Sign in here.

    Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!