Syntax behind sorted(key=lambda: ...) [duplicate]

Question

I don't quite understand the syntax behind the sorted() argument:

key=lambda variable: variable[0]

Isn't lambda arbitrary? Why is variable stated twice in what looks like a dict?

Extravasate

There are two separate questions here: what the lambda variable: variable[0] part means (it is totally separate syntax that has nothing to do with sorted, and what sorted does with its key keyword argument. I added a more specific duplicate link for each. – Bianca

@KarlKnechtel per meta.stackoverflow.com/a/255411 I believe this question has more valuable answers, would it make more sense to link those questions as duplicates of this one? – Extravasate

The short version is that I disagree that it does, because the other questions are more focused. I would be happy to discuss this on Meta or in the Python canon discussion chat room, but not here. – Bianca

Answer 1

key is a function that will be called to transform the collection's items before they are compared. The parameter passed to key must be something that is callable.

The use of lambda creates an anonymous function (which is callable). In the case of sorted the callable only takes one parameters. Python's lambda is pretty simple. It can only do and return one thing really.

The syntax of lambda is the word lambda followed by the list of parameter names then a single block of code. The parameter list and code block are delineated by colon. This is similar to other constructs in python as well such as while, for, if and so on. They are all statements that typically have a code block. Lambda is just another instance of a statement with a code block.

We can compare the use of lambda with that of def to create a function.

adder_lambda = lambda parameter1,parameter2: parameter1+parameter2
def adder_regular(parameter1, parameter2): return parameter1+parameter2

lambda just gives us a way of doing this without assigning a name. Which makes it great for using as a parameter to a function.

variable is used twice here because on the left hand of the colon it is the name of a parameter and on the right hand side it is being used in the code block to compute something.

Friesen

note (to OP): it should be avoided to assign a lambda to a name (i.e. using it in a way other than as anonymous function). if you find yourself doing this you should probably just use a def. – Nickola

Answer 2

I think all of the answers here cover the core of what the lambda function does in the context of sorted() quite nicely, however I still feel like a description that leads to an intuitive understanding is lacking, so here is my two cents.

For the sake of completeness, I'll state the obvious up front: sorted() returns a list of sorted elements and if we want to sort in a particular way or if we want to sort a complex list of elements (e.g. nested lists or a list of tuples) we can invoke the key argument.

For me, the intuitive understanding of the key argument, why it has to be callable, and the use of lambda as the (anonymous) callable function to accomplish this comes in two parts.

1. Using lamba ultimately means you don't have to write (define) an entire function. Lambda functions are created, used, and immediately destroyed - so they don't funk up your code with more code that will only ever be used once. This, as I understand it, is the core utility of the lambda function and its application for such a role is broad. Its syntax is purely a convention, which is in essence the nature of programmatic syntax in general. Learn the syntax and be done with it.

Lambda syntax is as follows:

lambda input_variable(s): tasty one liner

where lambda is a python keyword.

e.g.

In [1]: f00 = lambda x: x/2

In [2]: f00(10)
Out[2]: 5.0

In [3]: (lambda x: x/2)(10)
Out[3]: 5.0

In [4]: (lambda x, y: x / y)(10, 2)
Out[4]: 5.0

In [5]: (lambda: 'amazing lambda')() # func with no args!
Out[5]: 'amazing lambda'

2. The idea behind the key argument is that it should take in a set of instructions that will essentially point the 'sorted()' function at those list elements which should be used to sort by. When it says key=, what it really means is: As I iterate through the list, one element at a time (i.e. for e in some_list), I'm going to pass the current element to the function specifed by the key argument and use that to create a transformed list which will inform me on the order of the final sorted list.

Check it out:

In [6]: mylist = [3, 6, 3, 2, 4, 8, 23]  # an example list
# sorted(mylist, key=HowToSort)  # what we will be doing

Base example:

# mylist = [3, 6, 3, 2, 4, 8, 23]
In [7]: sorted(mylist)
Out[7]: [2, 3, 3, 4, 6, 8, 23]  
# all numbers are in ascending order (i.e.from low to high).

Example 1:

# mylist = [3, 6, 3, 2, 4, 8, 23]
In [8]: sorted(mylist, key=lambda x: x % 2 == 0)

# Quick Tip: The % operator returns the *remainder* of a division
# operation. So the key lambda function here is saying "return True 
# if x divided by 2 leaves a remainer of 0, else False". This is a 
# typical way to check if a number is even or odd.

Out[8]: [3, 3, 23, 6, 2, 4, 8]  
# Does this sorted result make intuitive sense to you?

Notice that my lambda function told sorted to check if each element e was even or odd before sorting.

BUT WAIT! You may (or perhaps should) be wondering two things.

First, why are the odd numbers coming before the even numbers? After all, the key value seems to be telling the sorted function to prioritize evens by using the mod operator in x % 2 == 0.

Second, why are the even numbers still out of order? 2 comes before 6, right?

By analyzing this result, we'll learn something deeper about how the 'key' argument really works, especially in conjunction with the anonymous lambda function.

Firstly, you'll notice that while the odds come before the evens, the evens themselves are not sorted. Why is this?? Lets read the docs:

Key Functions Starting with Python 2.4, both list.sort() and sorted() added a key parameter to specify a function to be called on each list element prior to making comparisons.

We have to do a little bit of reading between the lines here, but what this tells us is that the sort function is only called once, and if we specify the key argument, then we sort by the value that key function points us to.

So what does the example using a modulo return? A boolean value: True == 1, False == 0. So how does sorted deal with this key? It basically transforms the original list to a sequence of 1s and 0s.

[3, 6, 3, 2, 4, 8, 23] becomes [0, 1, 0, 1, 1, 1, 0]

Now we're getting somewhere. What do you get when you sort the transformed list?

[0, 0, 0, 1, 1, 1, 1]

Okay, so now we know why the odds come before the evens. But the next question is: Why does the 6 still come before the 2 in my final list? Well that's easy - it is because sorting only happens once! Those 1s still represent the original list values, which are in their original positions relative to each other. Since sorting only happens once, and we don't call any kind of sort function to order the original even numbers from low to high, those values remain in their original order relative to one another.

The final question is then this: How do I think conceptually about how the order of my boolean values get transformed back in to the original values when I print out the final sorted list?

Sorted() is a built-in method that (fun fact) uses a hybrid sorting algorithm called Timsort that combines aspects of merge sort and insertion sort. It seems clear to me that when you call it, there is a mechanic that holds these values in memory and bundles them with their boolean identity (mask) determined by (...!) the lambda function. The order is determined by their boolean identity calculated from the lambda function, but keep in mind that these sublists (of one's and zeros) are not themselves sorted by their original values. Hence, the final list, while organized by Odds and Evens, is not sorted by sublist (the evens in this case are out of order). The fact that the odds are ordered is because they were already in order by coincidence in the original list. The takeaway from all this is that when lambda does that transformation, the original order of the sublists are retained.

So how does this all relate back to the original question, and more importantly, our intuition on how we should implement sorted() with its key argument and lambda?

That lambda function can be thought of as a pointer that points to the values we need to sort by, whether its a pointer mapping a value to its boolean transformed by the lambda function, or if its a particular element in a nested list, tuple, dict, etc., again determined by the lambda function.

Lets try and predict what happens when I run the following code.

In [9]: mylist = [(3, 5, 8), (6, 2, 8), (2, 9, 4), (6, 8, 5)]
In[10]: sorted(mylist, key=lambda x: x[1])

My sorted call obviously says, "Please sort this list". The key argument makes that a little more specific by saying, 'for each element x in mylist, return the second index of that element, then sort all of the elements of the original list mylist by the sorted order of the list calculated by the lambda function. Since we have a list of tuples, we can return an indexed element from that tuple using the lambda function.

The pointer that will be used to sort would be:

[5, 2, 9, 8] # the second element of each tuple

Sorting this pointer list returns:

[2, 5, 8, 9]

Applying this to mylist, we get:

Out[10]: [(6, 2, 8), (3, 5, 8), (6, 8, 5), (2, 9, 4)]
# Notice the sorted pointer list is the same as the second index of each tuple in this final list

Run that code, and you'll find that this is the order. Try sorting a list of integers using this key function and you'll find that the code breaks (why? Because you cannot index an integer of course).

This was a long winded explanation, but I hope this helps to sort your intuition on the use of lambda functions - as the key argument in sorted(), and beyond.

Placida

Thank you for the deep explanation. I read the docs and sorting how-to and it wasn't clear to me the idea behind the key function. If you are trying to understand sorted function, them the lambda syntax just gets into the way of understanding. – Athalie

This is the best explanation here. This was really useful in helping me understand how this actually worked - I sorta grasped the lambda function, but using it in context of sorted() made no sense. This really helped, thanks! – Kimmi

Answer 3

lambda is a Python keyword that is used to generate anonymous functions.

>>> (lambda x: x+2)(3)
5

Diamante

Why are there parentheses around each? – Extravasate

The parens are around 3 because it's being passed to a function. The parens are around the lambda so that the expression isn't parsed as lambda x: x+2(3), which is invalid since 2 isn't a function. – Diamante

I am not sure I like the term "anonymous" functions. I mean, its true that they are not named, so anonymous is "technically" accurate. I'd rather refer to them as "temporary functions." But then, I am a pedant. – Inebriant

Answer 4

The variable left of the : is a parameter name. The use of variable on the right is making use of the parameter.

Means almost exactly the same as:

def some_method(variable):
  return variable[0]

Artemus

Answer 5

One more example of usage sorted() function with key=lambda. Let's consider you have a list of tuples. In each tuple you have a brand, model and weight of the car and you want to sort this list of tuples by brand, model or weight. You can do it with lambda.

cars = [('citroen', 'xsara', 1100), ('lincoln', 'navigator', 2000), ('bmw', 'x5', 1700)]

print(sorted(cars, key=lambda car: car[0]))
print(sorted(cars, key=lambda car: car[1]))
print(sorted(cars, key=lambda car: car[2]))

Results:

[('bmw', 'x5', '1700'), ('citroen', 'xsara', 1100), ('lincoln', 'navigator', 2000)]
[('lincoln', 'navigator', 2000), ('bmw', 'x5', '1700'), ('citroen', 'xsara', 1100)]
[('citroen', 'xsara', 1100), ('bmw', 'x5', 1700), ('lincoln', 'navigator', 2000)]

Juba

Answer 6

Simple and not time consuming answer with an example relevant to the question asked Follow this example:

 user = [{"name": "Dough", "age": 55}, 
            {"name": "Ben", "age": 44}, 
            {"name": "Citrus", "age": 33},
            {"name": "Abdullah", "age":22},
            ]
    print(sorted(user, key=lambda el: el["name"]))
    print(sorted(user, key= lambda y: y["age"]))

Look at the names in the list, they starts with D, B, C and A. And if you notice the ages, they are 55, 44, 33 and 22. The first print code

print(sorted(user, key=lambda el: el["name"]))

Results to:

[{'name': 'Abdullah', 'age': 22}, 
{'name': 'Ben', 'age': 44}, 
{'name': 'Citrus', 'age': 33}, 
{'name': 'Dough', 'age': 55}]

sorts the name, because by key=lambda el: el["name"] we are sorting the names and the names return in alphabetical order.

The second print code

print(sorted(user, key= lambda y: y["age"]))

Result:

[{'name': 'Abdullah', 'age': 22},
 {'name': 'Citrus', 'age': 33},
 {'name': 'Ben', 'age': 44}, 
 {'name': 'Dough', 'age': 55}]

sorts by age, and hence the list returns by ascending order of age.

Try this code for better understanding.

Aliquant

It could relate to your answer the most: short and sweet. I have a question for you though: in light of your above question, can it be said that the "key" in the sorted() tells the sorted() on which criterion the sorting needs to be done cause the sorting of an iterable can be done in multiple ways. Thanks again. – Odds

Answer 7

Another usage of lambda and sorted is like the following:

Given the input array: people = [[7,0],[4,4],[7,1],[5,0],[6,1],[5,2]]

The line: people_sort = sorted(people, key = lambda x: (-x[0], x[1])) will give a people_sort list as [[7,0],[7,1],[6,1],[5,0],[5,2],[4,4]]

In this case, key=lambda x: (-x[0], x[1]) basically tells sorted to firstly sort the array based on the value of the first element of each of the instance(in descending order as the minus sign suggests), and then within the same subgroup, sort based on the second element of each of the instance(in ascending order as it is the default option).

Hope this is some useful information to you!

Clos

Answer 8

lambda is an anonymous function, not an arbitrary function. The parameter being accepted would be the variable you're working with, and the column in which you're sorting it on.

Mainly

Answer 9

Since the usage of lambda was asked in the context of sorted(), take a look at this as well https://wiki.python.org/moin/HowTo/Sorting/#Key_Functions

Almazan

Answer 10

Just to rephrase, the key (Optional. A Function to execute to decide the order. Default is None) in sorted functions expects a function and you use lambda.

To define lambda, you specify the object property you want to sort and python's built-in sorted function will automatically take care of it.

If you want to sort by multiple properties then assign key = lambda x: (property1, property2).

To specify order-by, pass reverse= true as the third argument(Optional. A Boolean. False will sort ascending, True will sort descending. Default is False) of sorted function.

Cyclopean