Python Lists: An In-Depth Tutorial

You can't talk about lists without talking about list comprehension. List comprehensions are a concise way to create lists.

What you might have written this way:

result = []
for x in range(3):
    for y in range(3):
        result.append((x, y,))
result
> [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]

Can be written like this:

result = [(x, y,) for x in range(3) for y in range(3)]

List comprehensions are used primarily to make your code cleaner and more readable.

If you'd like to read more about this, we've written a more comprehensive (pun intended) guide on list comprehensions.

Python's ability to slice lists (or strings) is quite powerful.

Here is the basic breakdown of how lists can be sliced, given the list a:

a[start:end] # start index through end-1 index
a[start:] # start index through the end of the list
a[:end] # beginning of the list through end-1 index
a[:] # a copy of the whole array

For example, if you have a list of names and want to return a subset, it's quite easy:

names = ['Lancelot', 'Galahad', 'Arthur', 'Robin']
names[1:3]
> ['Galahad', 'Arthur']
names[1:]
> ['Galahad', 'Arthur', 'Robin']
names[:2]
> ['Lancelot', 'Galahad']
names[:]
> ['Lancelot', 'Galahad', 'Arthur', 'Robin']

Python's slicing syntax also supports a third argument, step. This specifies how to step through the list. The default step is 1 meaning that it will hit every index. But you can specify any integer, positive or negative. A negative step will step through the list backwards, which is useful for reversing a list or string.

numbers = list(range(10))
numbers[::2] # should print only even numbers
> [0, 2, 4, 6, 8]

When you've got a list of strings, sometimes you'll want to join all of the elements to create a single string. To do this you'll need to provide a delimiter and use the join function.

words = ['knights', 'who', 'say', 'ni']
sentence = ' '.join(words)
sentence
> 'knights who say ni'

Python has a built in function called sum that accepts an iterable and adds together each item. So adding elements in a list is cake:

primes = [0, 1, 2, 3, 5, 7, 11]
sum(primes)
> 29

Python makes it very easy to find the max and min values in a list - using the built-in max and min functions.

l = [0, 10, 100, 50, 500, 30]
max(l)
> 500
min(l)
> 0

If you have two lists and want to combine them into a single list, you can do so simply using the + operator.

a = ['Lancelot', 'Galahad', 'Robin']
b = ['Patsy', 'Arthur']
result = a + b
result
> ['Lancelot', 'Galahad', 'Robin', 'Patsy', 'Arthur']

Again, built-in functions make life easy.

Suppose you have this list:

numbers = [0, 10, 4, -4, 22, 2]

And you'd like to return a sorted list:

sorted(numbers)
> [-4, 0, 2, 4, 10, 22]

Or if you want to sort the list rather than returning a new, sorted list:

numbers.sort()
numbers
> [-4, 0, 2, 4, 10, 22]

Lists can be used as a stack or a queue. A stack is first-in-last-out (like a stack of plates) while a queue is first-in-first-out (like a line at the grocery store).

In either case, stack or queue, you will add elements to the end of the list. What changes is how you remove elements.

Stack

stack = []
stack.append('Robin')
stack.append('Lancelot')
stack.append('Arthur')
stack
> ['Robin', 'Lancelot', 'Arthur']

And we can use the pop method to remove elements from the end of the list.

stack.pop()
> 'Arthur'
stack.pop()
> 'Lancelot'
stack.pop()
> 'Robin'

Queue

For a queue, we'll actually need to use a deque from the collections module.

from collections import deque

The deque can be populated directly from a list like this:

queue = deque(['Robin', 'Lancelot', 'Arthur'])

Or it can be instantiated empty, and use the append method to add elements.

queue =deque()
queue.append('Robin')
queue.append('Lancelot')
queue.append('Arthur')
queue
> deque(['Robin', 'Lancelot', 'Arthur'])

And we can use the popleft method to remove elements from the beginning.

queue.popleft()
> 'Robin'
queue
> deque(['Lancelot', 'Arthur'])

queue.popleft()
> 'Lancelot'
queue
> deque([ 'Arthur'])

queue.popleft()
> 'Arthur'
queue
> deque([])

There are often cases where you'll want to count the frequency of elements in a list. Consider a problem where you need to find the frequency of words in a document.

We can use this short paragraph as an example.

paragraph = "I am.  And this my trusty servant Patsy. We have ridden the length and breadth of the land in search of knights who will join me in my court of Camelot.  I must speak with your lord and master."

First we want to make sure this document only contains words and spaces (no punctuation).

paragraph = re.sub(r'[^\w\s]', '', paragraph)

And then we want to split the paragraph by whitespace and return a list.

words = re.split(r'\s+', paragraph)

And now with our list of words, we can use Counter to give us our word frequencies.

from collections import Counter

counter = Counter(words)

From this we can get the 5 most commonly used words:

counter.most_common(5)
> [('of', 3), ('and', 2), ('in', 2), ('I', 2), ('the', 2)]

Or we can return the frequency of a desired word.

counter['of']
> 3

Map

Python's map function lets you run a function on every element in a list. This is often used in conjunction with lambda functions.

Suppose we have a list of numbers and we want to replace each element in with it's square.

numbers = [0, 1, 2, 3, 5, 7, 11]
squared = map(lambda x: x**2, numbers)
squared
> [0, 1, 4, 9, 25, 49, 121]

Filter

Filter lets you do exactly that - filter out elements from a list.

Suppose we have a list and we only want to keep numbers greater than 0.

numbers = [-1, 0, -5, 10, 8, -20, -3, 16]
positives = filter(lambda x: x > 0, numbers)
positives
> [10, 8, 16]

Reduce

Reduce allows you to write a function that accepts two parameters, the first is the current product (generated from all previous iterations) and the second is the current element. As opposed to map and filter, reduce returns a single value.

Suppose we have a list of numbers that we want to multiply together - but only if the number is not 0.

numbers = [-1, 0, -5, 10, 8, -20, -3, 16]
product = reduce(lambda x, y: x*y if not y == 0 else x, numbers)
product
> 384000

Last but not least, itertools is essential for working with iterable data - like lists. You may not use it often, but it's valuable to know what is available in the module.

I won't cover everything in this step, but here is a link to the documentation for itertools.

But to give an idea for the sorts of things you can do with itertools, check out a few of these examples.

accumulate

Accumulate allows you to perform some binary function as you iterate through the list. It's very similar to something you might write with map but a bit easier to read. Each element in the resulting iterable is the accumulated value.

By default accumulate adds the values.

list(accumulate([1, 2, 3, 4]))
> [1, 3, 6, 10]

But you can also pass in a binary function as well.

list(accumulate([1, 2, 3, 4], operator.mul))
> [1, 2, 6, 24]

compress

Compress takes data and an iterable, and it return the values from data only when the corresponding value the iterable evalutes to true.

Suppose you have a binary string "Lancelot" and you only want to return values in this string that correspond to 1's in the following list [1, 1, 1, 0, 0, 0, 1, 1].

You could write:

list(compress('Lancelot', [1, 1, 1, 0, 0, 0, 1, 1]))
> ['L', 'a', 'n', 'o', 't']