5 Advanced Python Concepts: Explanations and Applications

1. Lambda Functions Lambda functions are also called anonymous functions in Python. Some people simply refer to them as lambdas. They have the following syntax: lambda arguments: expression. In essence, we use the lambda keyword to signify the declaration of a lambda function. Then we list the arguments, the number of which can be zero or more. After the colon, we list the expression that uses these arguments for any applicable operations. Lambda functions are particularly useful in cases where we need to have a short one-time use function. For instance, several built-in functions have the key argument, to which we can set a lambda function. Sorting With a Lambda FunctionIn the above code, we wanted to sort a list of tuples. By default, the tuples will be sorted based on each of the items contained. In this case, the sorting was based on the names’ first letters. However, we wanted to solve by the scores, which are the second items of the tuples. To accomplish it, we took advantage of the lambda function, in which the x argument refers to each tuple that was to be sorted. Because the score was the second item in each tuple, we just needed to specify the index of 1 to access the second item.

2. Comprehensions Probably the most Pythonic example that is mentioned a lot is the comprehension technique. In essence, this technique allows us to create a list, dictionary, or set using an exiting iterable, which are named list comprehension, dictionary comprehension, and set comprehension, respectively. The following code snippet shows you these usages. ComprehensionsThe syntax of these comprehensions looks similar. Here’s a quick highlight of the differential forms. It should be noted that you can add conditions to keep the items you need. List Comprehension: [expr for x in iterable] Dictionary Comprehension: {key_expr: value_expr for x in iterable} Set Comprehension: {expr for x in iterable}With Optional Conditions: [expr for x in iterable if condition] {key_expr: value_expr for x in iterable if condition} {expr for x in iterable if condition} Instead of implementing for-loops, these comprehensions are very handy to create these container data types. More importantly, they’re usually faster than the for-loops, and thus they’re more performant and should be the preferred way for these operations.

3. Generators Previously, I’ve mentioned iterables, which refer to Python objects that can be iterated. During the iteration, the iterable is converted to an iterator, such that the iterator can render elements when needed. One particular kind of iterator is a generator. Unlike typical iterables, such as lists and dictionaries — which have all their elements loaded in the memory — generators produce elements in a lazy fashion without the need of loading all elements in the memory, and thus they’re memory-efficient iterables. A trivial example is shown in the following code snippet. Simple generator exampleAs you can see, the generator function makes a generator involving the use of the yield keyword. During iteration, these elements are rendered sequentially. One practical use case of generators is to deal with a large amount of data — when all loaded, it can slow down the computer or simply can’t be loaded at all because of an enormously large size. For instance, a trivial example would be to calculate the sum of integers 1–10,000,000,000. I tried 1 billion on my computer and found out that the size was about 8 GB. So, 10 billion would be about 80 GB if I had tried it, which would probably crash the program or even my computer. Without being able to create the list, it was impossible for me to calculate the sum using the list. In this case, we should consider generators. Generator Use CaseAs shown above, we can create a generator that produces an integer once at a time, which is memory-efficient. The above code snippet shows you another useful technique called generator expression, which has the following format: (expr for item in iterable).

4. Decorators Decorators are higher-order functions that modify other functions’ behavior without affecting their core functionalities. You can think of other functions as plain donuts; the decoration is the process of applying coatings to the donuts. No matter how you decorate them, they’re still donuts. In other words, decorators are just to add some tweaks in terms of the functions’ look or some other non-essential aspects without changing their internal algorithm. Let’s look at decorators with a trivial example. Decorators ExampleThe above example shows you how to declare a decorator function and how to use it to decorate other functions. As you can see, the decorator function timing takes another function as a parameter, which logs the running time of the decorated function. Notably, the decorated function returns the function as its output. To use the decorator, we simply place it above another function’s top by using a @ sign as a prefix, which signals that the following declared function is decorated by this specified decorator function. The following code shows you what the decoration is like. Timing the FunctionsOne thing that I found out that many tutorials on decorators failed to mention is the use of @wraps decorator in the declaration of a decorator function. I highly recommend using it (see Line 6 of the previous code snippet), for various reasons that have been covered in my previous blog article. Why You Should Wrap Decorators in Python Take advantage of functools.wraps towardsdatascience.com

5. Hashability When we learn Python dictionaries, we get to know that the keys need to be hashable. What does hashable mean? Hashable simply means a particular Python object can be hashed, the process of which is known as hashing. The following diagram shows you a simplified flow of how hashing works. The general process of hashing (Wikipedia, Public Domain)Essentially, hashing is the process of using a hash function (sometimes referred to as hasher) to convert Python objects (called keys in the diagram) to numeric hashed values (called hashes in the diagram). A quick way to know if a particular Python object is to use the built-in hash() function to fetch the hash value. If the object is not hashable, Python will raise a TypeError exception for us. Hashability and HashNotably, hashing takes time and can be slower than building lists and tuples. So the question is — why do we bother implementing dictionaries using hashing? On a related note, you may have heard that the elements in a set need to be hashable too. Under the hood, both creating dictionaries and sets require the building of hash tables. The following code snippet shows you how the hashability of particular objects may affect their qualifications to be used as dictionary keys. The biggest advantage of using hashes is instant look-up time (i.e., O(1) time complexity) for fetching an element in the dictionary. Similarly, checking whether a particular item is in the set takes a constant time, too. In other words, using hashing as the implementation mechanism provides high efficiency for various common operations, such as item retrieving, item insertion, and item checking, at the expense of the overhead of having the hash table under the hood. Constant Lookup TimeTo mimic a real situation, we generate some random integers to get an average lookup time for item fetching. As you can see, even with 100,000 items in the dictionary, the lookup time stays about the same, which highlights the advantage of implementing a hash table as the storage mechanism for the dictionaries.

Conclusions In this article, we reviewed five advanced concepts in Python. Here’s a quick recap of the most key takeaway information.

• Lambda functions. You use lambda functions to have a simple operation, usually within another function call, such as sorted() or max().

• Comprehensions. They’re convenient and performant ways to create lists, dictionaries, and sets from iterables.

• Generators. Generators are lazily-evaluated iterators that render items only when requested, and thus they’re very memory efficient. They should be used when you’re dealing with a large amount of data sequentially.

• Decorators. Decorators are useful when you want to have some non-algorithmic modifications with current functions. In addition, decorators can be used repeatedly. Once defined, they can decorate as many functions as you want.

• Hashability. Hashability is the required characteristic of Python objects that can be used as dictionary keys or set elements. They provide a mechanism for efficient item retrieving and insertion, as well as membership checking.