List Comprehensions, Generators & Functional Tools

# List comprehension — stores all values in memory:
squares_list = [x**2 for x in range(1000000)]
# Uses ~8MB memory

# Generator expression — computes on demand:
squares_gen = (x**2 for x in range(1000000))
# Uses <1KB memory

next(squares_gen)  # 0
next(squares_gen)  # 1

Generators are lazy — they compute one value at a time only when needed. For large datasets (processing 10M rows), generators prevent memory overflow. Use () instead of [] to create a generator expression.

nums = [1, 2, 3, 4, 5]

# map — apply function to each element:
doubled = list(map(lambda x: x * 2, nums))  # [2,4,6,8,10]

# filter — keep elements where function returns True:
evens = list(filter(lambda x: x % 2 == 0, nums))  # [2,4]

# Pythonic equivalents (preferred):
doubled = [x * 2 for x in nums]
evens = [x for x in nums if x % 2 == 0]

map() and filter() are functional programming tools. List comprehensions are generally preferred in Python for readability. However, map/filter are faster for large iterables and appear in many codebases — know both.

from functools import reduce

nums = [1, 2, 3, 4, 5]

# Sum using reduce:
total = reduce(lambda acc, x: acc + x, nums)  # 15

# Product:
product = reduce(lambda acc, x: acc * x, nums)  # 120

# Equivalent to:
total = sum(nums)  # prefer built-in when available

reduce() applies a function cumulatively to reduce a sequence to a single value. It is a functional programming concept — useful for custom aggregations. For standard operations, use built-ins (sum, max, min) which are faster.

import time

def timer(func):
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end-start:.2f}s")
        return result
    return wrapper

@timer
def load_data():
    # simulate slow operation
    time.sleep(1)
    return "done"

load_data()  # prints: load_data took 1.00s

A decorator is a function that wraps another function to add behaviour. Common uses: timing, logging, caching (functools.lru_cache), authentication checks. The @syntax is syntactic sugar for load_data = timer(load_data).

from functools import lru_cache

@lru_cache(maxsize=128)
def expensive_lookup(product_id):
    # Simulate slow database query:
    time.sleep(0.5)
    return f"Product {product_id} data"

expensive_lookup(42)  # slow — fetches from DB
expensive_lookup(42)  # instant — cached result

lru_cache memoises function results — the first call computes and stores the result, subsequent identical calls return the cached value instantly. Use for repeated expensive operations with the same inputs (API calls, DB lookups, complex calculations).