Python Full Stack Web Developer Course

Week 3: Object-Oriented Programming Advanced Concepts

Magic/Dunder Methods

Understanding Magic/Dunder Methods in Python

Magic methods, also known as "dunder" methods (short for "double underscore"), are special methods in Python that have double underscores at the beginning and end of their names, like __init__ or __str__. These methods enable Python's unique features and allow your classes to integrate seamlessly with Python's built-in functions and operators.

Dunder methods are what make Python's object-oriented programming so powerful and expressive. They allow you to define how your objects behave when used with Python's syntax and built-in functions. For example, by defining the __add__ method, you can make your objects work with the + operator, or by defining __str__, you control how your objects are represented as strings.

In today's session, we'll explore these powerful methods in detail, learning how they work and how to implement them effectively in your own classes.

Today's File Structure

For today's lesson, we'll create a new Python module in our project. Ensure you have the following directory structure:

project_root/
├── dunder_methods/
│   ├── __init__.py  (empty file to make the folder a package)
│   ├── basic_dunder.py
│   ├── operators.py
│   ├── container_methods.py
│   ├── comparison_methods.py
│   ├── attribute_access.py
│   ├── callable_objects.py
│   └── real_world_examples.py

All code examples will be saved in these files, allowing you to organize and revisit these concepts easily.

Basic Dunder Methods

Let's start with some of the most fundamental dunder methods that help define the core behavior of your objects. Create a file named basic_dunder.py with the following code:

# File: dunder_methods/basic_dunder.py

class Person:
    """A class to demonstrate basic dunder methods"""
    
    def __init__(self, first_name, last_name, age):
        """
        Initialize a Person object.
        
        This is called whenever you create a new instance of the class.
        """
        self.first_name = first_name
        self.last_name = last_name
        self.age = age
    
    def __str__(self):
        """
        Return a string representation of the Person object.
        
        This is called by the str() function and when you use print().
        """
        return f"{self.first_name} {self.last_name}, {self.age} years old"
    
    def __repr__(self):
        """
        Return a developer-friendly string representation of the Person object.
        
        This is called by the repr() function and in interactive mode.
        It should ideally return a string that, when executed, would recreate the object.
        """
        return f"Person('{self.first_name}', '{self.last_name}', {self.age})"
    
    def __format__(self, format_spec):
        """
        Return a formatted string representation of the Person object.
        
        This is called by the format() function and when using f-strings with format specifiers.
        The format_spec is the string after the colon in the format specifier.
        """
        if format_spec == 'short':
            return f"{self.first_name[0]}. {self.last_name}"
        elif format_spec == 'full':
            return f"{self.first_name} {self.last_name}, aged {self.age}"
        else:
            return str(self)
    
    def __bool__(self):
        """
        Return a boolean representation of the Person object.
        
        This is called by the bool() function and in boolean contexts like if statements.
        """
        # Consider a person as "truthy" if they are an adult (18+)
        return self.age >= 18
    
    def __del__(self):
        """
        Clean up resources when the object is about to be destroyed.
        
        This is called when the object is about to be garbage collected.
        It's rarely needed in practice but can be useful for cleanup.
        """
        print(f"Person object for {self.first_name} {self.last_name} is being destroyed")


# Demo basic usage
if __name__ == "__main__":
    # Create a Person object
    person = Person("John", "Doe", 30)
    
    # __str__ method
    print("Demonstrating __str__:")
    print(str(person))
    print(person)  # print() calls str() implicitly
    
    # __repr__ method
    print("\nDemonstrating __repr__:")
    print(repr(person))
    
    # __format__ method
    print("\nDemonstrating __format__:")
    print(f"Default format: {person}")
    print(f"Short format: {person:short}")
    print(f"Full format: {person:full}")
    
    # __bool__ method
    print("\nDemonstrating __bool__:")
    adult = Person("Jane", "Smith", 25)
    minor = Person("Billy", "Kid", 15)
    
    print(f"Is {adult.first_name} an adult? {bool(adult)}")
    print(f"Is {minor.first_name} an adult? {bool(minor)}")
    
    if adult:
        print(f"{adult.first_name} is considered truthy")
    
    if not minor:
        print(f"{minor.first_name} is considered falsy")
    
    # __del__ method
    print("\nDemonstrating __del__:")
    print("Creating a temporary person")
    temp_person = Person("Temp", "User", 20)
    print("Deleting the temporary person")
    del temp_person
    
    print("\nProgram finished")

Code Breakdown:

When to use these methods:

Real-world analogy: Think of these dunder methods as the social etiquette of your objects. __init__ is like introducing yourself when you meet someone for the first time. __str__ is how you present yourself to regular people, while __repr__ is how you present yourself to colleagues in your profession with more technical details. __format__ is like adjusting how you present yourself in different social contexts. __bool__ determines whether people consider you "present" or "absent" in a situation.

Operator Overloading with Dunder Methods

One of the most powerful features of dunder methods is operator overloading, which allows your objects to work with Python's operators like +, -, *, etc. Create a file named operators.py with the following code:

# File: dunder_methods/operators.py

class Vector2D:
    """A 2D vector class with operator overloading"""
    
    def __init__(self, x, y):
        self.x = x
        self.y = y
    
    def __str__(self):
        return f"Vector2D({self.x}, {self.y})"
    
    def __repr__(self):
        return f"Vector2D({self.x}, {self.y})"
    
    # Arithmetic operators
    def __add__(self, other):
        """Vector addition: self + other"""
        if isinstance(other, Vector2D):
            return Vector2D(self.x + other.x, self.y + other.y)
        else:
            raise TypeError("Can only add another Vector2D")
    
    def __sub__(self, other):
        """Vector subtraction: self - other"""
        if isinstance(other, Vector2D):
            return Vector2D(self.x - other.x, self.y - other.y)
        else:
            raise TypeError("Can only subtract another Vector2D")
    
    def __mul__(self, other):
        """
        Vector scaling or dot product: self * other
        If other is a scalar, returns a scaled vector.
        If other is a Vector2D, returns the dot product (a scalar).
        """
        if isinstance(other, (int, float)):
            return Vector2D(self.x * other, self.y * other)
        elif isinstance(other, Vector2D):
            return self.x * other.x + self.y * other.y  # Dot product
        else:
            raise TypeError("Can only multiply by a scalar or another Vector2D")
    
    def __rmul__(self, other):
        """
        Right multiplication: other * self
        Called when the left operand doesn't support the operation.
        """
        # We just invoke __mul__ since vector multiplication is commutative with a scalar
        return self.__mul__(other)
    
    def __truediv__(self, other):
        """Vector division by scalar: self / other"""
        if isinstance(other, (int, float)):
            if other == 0:
                raise ZeroDivisionError("Cannot divide by zero")
            return Vector2D(self.x / other, self.y / other)
        else:
            raise TypeError("Can only divide by a scalar")
    
    def __neg__(self):
        """Negation: -self"""
        return Vector2D(-self.x, -self.y)
    
    def __abs__(self):
        """Absolute value (magnitude): abs(self)"""
        return (self.x ** 2 + self.y ** 2) ** 0.5
    
    # Augmented assignment operators
    def __iadd__(self, other):
        """In-place addition: self += other"""
        if isinstance(other, Vector2D):
            self.x += other.x
            self.y += other.y
            return self
        else:
            raise TypeError("Can only add another Vector2D")
    
    def __isub__(self, other):
        """In-place subtraction: self -= other"""
        if isinstance(other, Vector2D):
            self.x -= other.x
            self.y -= other.y
            return self
        else:
            raise TypeError("Can only subtract another Vector2D")
    
    def __imul__(self, other):
        """In-place multiplication: self *= other (only for scalar)"""
        if isinstance(other, (int, float)):
            self.x *= other
            self.y *= other
            return self
        else:
            raise TypeError("In-place multiplication only supports scalars")


# Demo operator overloading
if __name__ == "__main__":
    # Create some vectors
    v1 = Vector2D(3, 4)
    v2 = Vector2D(1, 2)
    
    # Addition and subtraction
    print(f"v1 = {v1}")
    print(f"v2 = {v2}")
    print(f"v1 + v2 = {v1 + v2}")
    print(f"v1 - v2 = {v1 - v2}")
    
    # Multiplication
    print(f"v1 * 2 = {v1 * 2}")  # Scaling
    print(f"2 * v1 = {2 * v1}")  # Right multiplication
    print(f"v1 * v2 = {v1 * v2}")  # Dot product
    
    # Division
    print(f"v1 / 2 = {v1 / 2}")
    
    # Negation and absolute value
    print(f"-v1 = {-v1}")
    print(f"|v1| = {abs(v1)}")
    
    # Augmented assignment
    v3 = Vector2D(3, 4)
    print(f"v3 = {v3}")
    v3 += Vector2D(1, 1)
    print(f"After v3 += Vector2D(1, 1): {v3}")
    v3 -= Vector2D(0, 2)
    print(f"After v3 -= Vector2D(0, 2): {v3}")
    v3 *= 2
    print(f"After v3 *= 2: {v3}")
    
    # Error handling
    try:
        v1 + 5  # Can't add a vector and a scalar
    except TypeError as e:
        print(f"Error: {e}")
    
    try:
        v1 / 0  # Can't divide by zero
    except ZeroDivisionError as e:
        print(f"Error: {e}")


class Money:
    """A class to represent money with currency, demonstrating operator overloading"""
    
    def __init__(self, amount, currency="USD"):
        self.amount = amount
        self.currency = currency
    
    def __str__(self):
        return f"{self.currency} {self.amount:.2f}"
    
    def __repr__(self):
        return f"Money({self.amount}, '{self.currency}')"
    
    def __add__(self, other):
        """Add money: self + other"""
        if isinstance(other, Money):
            if self.currency != other.currency:
                raise ValueError("Cannot add different currencies")
            return Money(self.amount + other.amount, self.currency)
        elif isinstance(other, (int, float)):
            return Money(self.amount + other, self.currency)
        else:
            raise TypeError("Can only add Money or a number")
    
    def __radd__(self, other):
        """Right addition: other + self"""
        if isinstance(other, (int, float)):
            return Money(self.amount + other, self.currency)
        else:
            raise TypeError("Can only add Money or a number")
    
    def __sub__(self, other):
        """Subtract money: self - other"""
        if isinstance(other, Money):
            if self.currency != other.currency:
                raise ValueError("Cannot subtract different currencies")
            return Money(self.amount - other.amount, self.currency)
        elif isinstance(other, (int, float)):
            return Money(self.amount - other, self.currency)
        else:
            raise TypeError("Can only subtract Money or a number")
    
    def __mul__(self, other):
        """Multiply money by a scalar: self * other"""
        if isinstance(other, (int, float)):
            return Money(self.amount * other, self.currency)
        else:
            raise TypeError("Can only multiply by a number")
    
    def __rmul__(self, other):
        """Right multiplication: other * self"""
        return self.__mul__(other)
    
    def __truediv__(self, other):
        """Divide money: self / other"""
        if isinstance(other, (int, float)):
            if other == 0:
                raise ZeroDivisionError("Cannot divide by zero")
            return Money(self.amount / other, self.currency)
        elif isinstance(other, Money):
            if self.currency != other.currency:
                raise ValueError("Cannot divide different currencies")
            if other.amount == 0:
                raise ZeroDivisionError("Cannot divide by zero")
            return self.amount / other.amount  # Returns a scalar ratio
        else:
            raise TypeError("Can only divide by Money or a number")
    
    def __lt__(self, other):
        """Less than: self < other"""
        if isinstance(other, Money):
            if self.currency != other.currency:
                raise ValueError("Cannot compare different currencies")
            return self.amount < other.amount
        elif isinstance(other, (int, float)):
            return self.amount < other
        else:
            raise TypeError("Can only compare with Money or a number")
    
    def __eq__(self, other):
        """Equal to: self == other"""
        if isinstance(other, Money):
            return self.currency == other.currency and self.amount == other.amount
        elif isinstance(other, (int, float)):
            return self.amount == other
        else:
            return False


# Demo the Money class
print("\nMoney class examples:")
m1 = Money(100, "USD")
m2 = Money(50, "USD")
m3 = Money(200, "EUR")

print(f"m1 = {m1}")
print(f"m2 = {m2}")
print(f"m3 = {m3}")

print(f"m1 + m2 = {m1 + m2}")
print(f"m1 - m2 = {m1 - m2}")
print(f"m1 * 2 = {m1 * 2}")
print(f"m1 / 2 = {m1 / 2}")
print(f"m1 / m2 = {m1 / m2}")  # Ratio

print(f"m1 + 50 = {m1 + 50}")
print(f"75 + m2 = {75 + m2}")

print(f"m1 < m2: {m1 < m2}")
print(f"m1 == Money(100, 'USD'): {m1 == Money(100, 'USD')}")
print(f"m1 == 100: {m1 == 100}")

try:
    m1 + m3  # Different currencies
except ValueError as e:
    print(f"Error: {e}")

Code Breakdown:

We've created two classes that demonstrate operator overloading:

  1. Vector2D: Represents a 2D vector with mathematical operations
    • __add__(self, other): Handles vector addition with the + operator
    • __sub__(self, other): Handles vector subtraction with the - operator
    • __mul__(self, other): Handles multiplication (scaling or dot product) with the * operator
    • __rmul__(self, other): Handles right multiplication (when the vector is on the right side)
    • __truediv__(self, other): Handles division by a scalar with the / operator
    • __neg__(self): Handles negation with the - unary operator
    • __abs__(self): Handles the abs() function (vector magnitude)
    • __iadd__(self, other), __isub__(self, other), __imul__(self, other): Handle augmented assignment operators (+=, -=, *=)
  2. Money: Represents monetary values with currency
    • Similar operators as Vector2D but with currency-specific logic
    • Also implements comparison operators like __lt__ and __eq__

Common Arithmetic Operator Methods

Method Operator Description
__add__(self, other) + Addition
__sub__(self, other) - Subtraction
__mul__(self, other) * Multiplication
__truediv__(self, other) / Division
__floordiv__(self, other) // Floor division
__mod__(self, other) % Modulo
__pow__(self, other) ** Exponentiation
__neg__(self) - (unary) Negation
__pos__(self) + (unary) Positive
__abs__(self) abs() Absolute value

Reflected Operation Methods

These methods handle operations where your object is on the right side.

Method Operator Description
__radd__(self, other) other + self Reflected addition
__rsub__(self, other) other - self Reflected subtraction
__rmul__(self, other) other * self Reflected multiplication
__rtruediv__(self, other) other / self Reflected division

Augmented Assignment Methods

These methods handle in-place operations.

Method Operator Description
__iadd__(self, other) self += other In-place addition
__isub__(self, other) self -= other In-place subtraction
__imul__(self, other) self *= other In-place multiplication
__itruediv__(self, other) self /= other In-place division

Best practices for operator overloading:

Real-world analogy: Operator overloading is like teaching your objects a new language. Just as "+" means different things when adding numbers (5 + 3 = 8) versus concatenating strings ("Hello" + "World" = "HelloWorld"), you're teaching your objects what "+" and other symbols mean in their context. It's like teaching your Money objects that when you "add" them, they should check if the currencies match and then sum the amounts.

Container and Sequence Methods

Python provides dunder methods that allow your objects to behave like containers or sequences, supporting operations like indexing, slicing, length checking, and iteration. Create a file named container_methods.py with the following code:

# File: dunder_methods/container_methods.py

class CustomList:
    """A custom list-like container class"""
    
    def __init__(self, items=None):
        self.items = items if items is not None else []
    
    def __str__(self):
        return f"CustomList({self.items})"
    
    def __repr__(self):
        return f"CustomList({self.items})"
    
    # Container methods
    def __len__(self):
        """Return the number of items in the container"""
        return len(self.items)
    
    def __getitem__(self, key):
        """Get an item or slice from the container"""
        if isinstance(key, slice):
            # Handle slicing (e.g., custom_list[1:5])
            start, stop, step = key.indices(len(self))
            return CustomList([self.items[i] for i in range(start, stop, step)])
        elif isinstance(key, int):
            # Handle integer index (positive or negative)
            if key < 0:  # Handle negative indices
                key += len(self)
            if key < 0 or key >= len(self):
                raise IndexError("CustomList index out of range")
            return self.items[key]
        else:
            raise TypeError("CustomList indices must be integers or slices")
    
    def __setitem__(self, key, value):
        """Set an item or slice in the container"""
        if isinstance(key, slice):
            # Handle slice assignment (e.g., custom_list[1:3] = [4, 5])
            start, stop, step = key.indices(len(self))
            if step != 1:
                raise ValueError("Step must be 1 for slice assignment")
            if not hasattr(value, '__iter__'):
                raise TypeError("Can only assign an iterable to a slice")
            value_list = list(value)
            self.items[start:stop] = value_list
        elif isinstance(key, int):
            # Handle integer index
            if key < 0:  # Handle negative indices
                key += len(self)
            if key < 0 or key >= len(self):
                raise IndexError("CustomList index out of range")
            self.items[key] = value
        else:
            raise TypeError("CustomList indices must be integers or slices")
    
    def __delitem__(self, key):
        """Delete an item or slice from the container"""
        if isinstance(key, slice):
            # Handle slice deletion (e.g., del custom_list[1:3])
            start, stop, step = key.indices(len(self))
            if step != 1:
                # For simplicity, we're only handling step=1
                # A real implementation would be more robust
                raise ValueError("Step must be 1 for slice deletion")
            del self.items[start:stop]
        elif isinstance(key, int):
            # Handle integer index
            if key < 0:  # Handle negative indices
                key += len(self)
            if key < 0 or key >= len(self):
                raise IndexError("CustomList index out of range")
            del self.items[key]
        else:
            raise TypeError("CustomList indices must be integers or slices")
    
    def __iter__(self):
        """Return an iterator for the container"""
        return iter(self.items)
    
    def __contains__(self, item):
        """Check if an item is in the container (for 'in' operator)"""
        return item in self.items
    
    def __reversed__(self):
        """Return a reversed iterator (for 'reversed()' function)"""
        return reversed(self.items)
    
    # Additional methods that sequence-like objects often implement
    def append(self, item):
        """Add an item to the end of the list"""
        self.items.append(item)
    
    def extend(self, items):
        """Extend the list with an iterable"""
        self.items.extend(items)
    
    def insert(self, index, item):
        """Insert an item at a specific position"""
        self.items.insert(index, item)
    
    def remove(self, item):
        """Remove the first occurrence of an item"""
        self.items.remove(item)
    
    def pop(self, index=-1):
        """Remove and return an item at a specific position (default: last)"""
        return self.items.pop(index)
    
    def clear(self):
        """Remove all items"""
        self.items.clear()
    
    def index(self, item, start=0, end=None):
        """Return the index of the first occurrence of an item"""
        if end is None:
            end = len(self)
        return self.items.index(item, start, end)
    
    def count(self, item):
        """Return the number of occurrences of an item"""
        return self.items.count(item)
    
    def sort(self, key=None, reverse=False):
        """Sort the list in place"""
        self.items.sort(key=key, reverse=reverse)
    
    def reverse(self):
        """Reverse the list in place"""
        self.items.reverse()


# Demo container methods
if __name__ == "__main__":
    # Create a CustomList
    custom_list = CustomList([1, 2, 3, 4, 5])
    print(f"Initial list: {custom_list}")
    
    # Length
    print(f"Length: {len(custom_list)}")
    
    # Indexing
    print(f"custom_list[2]: {custom_list[2]}")
    print(f"custom_list[-1]: {custom_list[-1]}")
    
    # Slicing
    print(f"custom_list[1:4]: {custom_list[1:4]}")
    print(f"custom_list[::2]: {custom_list[::2]}")
    
    # Setting values
    custom_list[2] = 10
    print(f"After custom_list[2] = 10: {custom_list}")
    
    custom_list[1:3] = [20, 30]
    print(f"After custom_list[1:3] = [20, 30]: {custom_list}")
    
    # Deleting items
    del custom_list[0]
    print(f"After del custom_list[0]: {custom_list}")
    
    del custom_list[1:3]
    print(f"After del custom_list[1:3]: {custom_list}")
    
    # Iteration
    print("Iterating over the list:")
    for item in custom_list:
        print(f"  {item}")
    
    # Membership testing
    print(f"Is 30 in the list? {30 in custom_list}")
    print(f"Is 100 in the list? {100 in custom_list}")
    
    # Reversed
    print("Iterating in reverse:")
    for item in reversed(custom_list):
        print(f"  {item}")
    
    # Other sequence methods
    custom_list.append(60)
    print(f"After append(60): {custom_list}")
    
    custom_list.extend([70, 80])
    print(f"After extend([70, 80]): {custom_list}")
    
    custom_list.insert(1, 25)
    print(f"After insert(1, 25): {custom_list}")
    
    custom_list.remove(70)
    print(f"After remove(70): {custom_list}")
    
    popped = custom_list.pop()
    print(f"Popped: {popped}, After pop(): {custom_list}")
    
    index = custom_list.index(60)
    print(f"Index of 60: {index}")
    
    count = custom_list.count(25)
    print(f"Count of 25: {count}")
    
    custom_list.sort(reverse=True)
    print(f"After sort(reverse=True): {custom_list}")
    
    custom_list.reverse()
    print(f"After reverse(): {custom_list}")
    
    custom_list.clear()
    print(f"After clear(): {custom_list}")


class Matrix:
    """A simple matrix class demonstrating container methods"""
    
    def __init__(self, rows, cols, data=None):
        self.rows = rows
        self.cols = cols
        if data is None:
            self.data = [[0 for _ in range(cols)] for _ in range(rows)]
        else:
            if len(data) != rows or any(len(row) != cols for row in data):
                raise ValueError(f"Data must be a {rows}x{cols} matrix")
            self.data = [row[:] for row in data]  # Make a deep copy
    
    def __str__(self):
        matrix_str = "\n".join(
            " ".join(f"{self.data[i][j]:4}" for j in range(self.cols))
            for i in range(self.rows)
        )
        return f"Matrix({self.rows}x{self.cols}):\n{matrix_str}"
    
    def __repr__(self):
        return f"Matrix({self.rows}, {self.cols}, {self.data})"
    
    def __getitem__(self, key):
        """Get an item from the matrix using row, col notation: matrix[row, col]"""
        if isinstance(key, tuple) and len(key) == 2:
            row, col = key
            if 0 <= row < self.rows and 0 <= col < self.cols:
                return self.data[row][col]
            else:
                raise IndexError("Matrix indices out of range")
        elif isinstance(key, int):
            if 0 <= key < self.rows:
                return self.data[key]  # Return a row
            else:
                raise IndexError("Matrix row index out of range")
        else:
            raise TypeError("Matrix indices must be integers or 2-tuples")
    
    def __setitem__(self, key, value):
        """Set an item in the matrix using row, col notation: matrix[row, col] = value"""
        if isinstance(key, tuple) and len(key) == 2:
            row, col = key
            if 0 <= row < self.rows and 0 <= col < self.cols:
                self.data[row][col] = value
            else:
                raise IndexError("Matrix indices out of range")
        elif isinstance(key, int):
            if 0 <= key < self.rows:
                if len(value) != self.cols:
                    raise ValueError(f"Row must have {self.cols} elements")
                self.data[key] = value[:]  # Set a row (copy the value)
            else:
                raise IndexError("Matrix row index out of range")
        else:
            raise TypeError("Matrix indices must be integers or 2-tuples")
    
    def __len__(self):
        """Return the number of rows in the matrix"""
        return self.rows
    
    def __iter__(self):
        """Return an iterator over the rows of the matrix"""
        return iter(self.data)
    
    # Matrix addition using operator overloading
    def __add__(self, other):
        """Matrix addition"""
        if not isinstance(other, Matrix):
            raise TypeError("Can only add another Matrix")
        if self.rows != other.rows or self.cols != other.cols:
            raise ValueError("Matrices must have the same dimensions")
        
        result = Matrix(self.rows, self.cols)
        for i in range(self.rows):
            for j in range(self.cols):
                result[i, j] = self[i, j] + other[i, j]
        return result


# Demo the Matrix class
print("\nMatrix examples:")
mat1 = Matrix(3, 3, [[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print(f"Matrix 1:\n{mat1}")

# Accessing elements
print(f"mat1[1, 2]: {mat1[1, 2]}")
print(f"mat1[0]: {mat1[0]}")  # Get a row

# Setting elements
mat1[0, 0] = 10
print(f"After mat1[0, 0] = 10:\n{mat1}")

mat1[1] = [11, 12, 13]  # Set a row
print(f"After mat1[1] = [11, 12, 13]:\n{mat1}")

# Matrix operations
mat2 = Matrix(3, 3, [[9, 8, 7], [6, 5, 4], [3, 2, 1]])
print(f"Matrix 2:\n{mat2}")

mat3 = mat1 + mat2
print(f"Matrix 1 + Matrix 2:\n{mat3}")

# Iteration
print("Matrix rows:")
for row in mat1:
    print(row)

Code Breakdown:

We've created two classes that demonstrate container and sequence behavior:

  1. CustomList: A custom list-like container implementing various sequence methods:
    • __len__(self): Returns the length, enabling len(obj)
    • __getitem__(self, key): Enables indexing and slicing with obj[key]
    • __setitem__(self, key, value): Enables assignment with obj[key] = value
    • __delitem__(self, key): Enables deletion with del obj[key]
    • __iter__(self): Makes the object iterable with for item in obj
    • __contains__(self, item): Enables membership testing with item in obj
    • __reversed__(self): Supports the reversed() function
  2. Matrix: A 2D matrix class that demonstrates container methods for multi-dimensional data:
    • Uses tuple indexing matrix[row, col] for 2D access
    • Supports row-based iteration and row extraction
    • Implements matrix addition with __add__

Common Container and Sequence Methods

Method Operation Description
__len__(self) len(obj) Return the length of the container
__getitem__(self, key) obj[key] Get an item or slice
__setitem__(self, key, value) obj[key] = value Set an item or slice
__delitem__(self, key) del obj[key] Delete an item or slice
__iter__(self) for x in obj Return an iterator
__contains__(self, item) item in obj Membership test
__reversed__(self) reversed(obj) Return a reversed iterator

Best practices for container methods:

Real-world analogy: Container methods turn your objects into collections that can be used like boxes or shelves. __getitem__ is like retrieval instructions (e.g., "get me the item in the third drawer"), __setitem__ is like placement instructions (e.g., "put this book on the second shelf"), and __len__ is like counting how many items are stored. __iter__ is like providing a guided tour through all items in the collection.

Comparison Dunder Methods

Python provides dunder methods for making your objects comparable with operators like ==, !=, <, >, etc. Create a file named comparison_methods.py with the following code:

# File: dunder_methods/comparison_methods.py

class Version:
    """A class to represent software versions (e.g., 1.2.3)"""
    
    def __init__(self, major, minor=0, patch=0):
        self.major = major
        self.minor = minor
        self.patch = patch
    
    def __str__(self):
        return f"{self.major}.{self.minor}.{self.patch}"
    
    def __repr__(self):
        return f"Version({self.major}, {self.minor}, {self.patch})"
    
    # Comparison methods
    def __eq__(self, other):
        """Equal to: self == other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) == (other.major, other.minor, other.patch)
    
    def __ne__(self, other):
        """Not equal to: self != other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) != (other.major, other.minor, other.patch)
    
    def __lt__(self, other):
        """Less than: self < other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) < (other.major, other.minor, other.patch)
    
    def __le__(self, other):
        """Less than or equal to: self <= other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) <= (other.major, other.minor, other.patch)
    
    def __gt__(self, other):
        """Greater than: self > other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) > (other.major, other.minor, other.patch)
    
    def __ge__(self, other):
        """Greater than or equal to: self >= other"""
        if not isinstance(other, Version):
            return NotImplemented
        return (self.major, self.minor, self.patch) >= (other.major, other.minor, other.patch)
    

# Demo comparison methods
if __name__ == "__main__":
    # Create some versions
    v1 = Version(1, 0, 0)
    v2 = Version(1, 1, 0)
    v3 = Version(1, 1, 0)  # Same as v2
    v4 = Version(2, 0, 0)
    
    print(f"v1 = {v1}")
    print(f"v2 = {v2}")
    print(f"v3 = {v3}")
    print(f"v4 = {v4}")
    
    # Equality and inequality
    print(f"v1 == v2: {v1 == v2}")
    print(f"v2 == v3: {v2 == v3}")
    print(f"v1 != v2: {v1 != v2}")
    
    # Less than and greater than
    print(f"v1 < v2: {v1 < v2}")
    print(f"v2 < v4: {v2 < v4}")
    print(f"v1 <= v2: {v1 <= v2}")
    print(f"v2 <= v3: {v2 <= v3}")
    
    print(f"v4 > v2: {v4 > v2}")
    print(f"v3 > v2: {v3 > v2}")
    print(f"v4 >= v2: {v4 >= v2}")
    print(f"v3 >= v2: {v3 >= v2}")
    
    # Other uses of comparison methods
    versions = [v4, v1, v3, v2]
    print(f"Unsorted versions: {versions}")
    sorted_versions = sorted(versions)
    print(f"Sorted versions: {sorted_versions}")
    
    print(f"Min version: {min(versions)}")
    print(f"Max version: {max(versions)}")


class Person:
    """A class to represent a person, with rich comparison methods"""
    
    def __init__(self, first_name, last_name, age):
        self.first_name = first_name
        self.last_name = last_name
        self.age = age
    
    def __str__(self):
        return f"{self.first_name} {self.last_name}, {self.age} years old"
    
    def __repr__(self):
        return f"Person('{self.first_name}', '{self.last_name}', {self.age})"
    
    def __eq__(self, other):
        """Equal to: self == other"""
        if not isinstance(other, Person):
            return NotImplemented
        return ((self.last_name, self.first_name, self.age) == 
                (other.last_name, other.first_name, other.age))
    
    def __lt__(self, other):
        """Less than: self < other
        
        Sorts by last name, then first name, then age.
        """
        if not isinstance(other, Person):
            return NotImplemented
        return ((self.last_name, self.first_name, self.age) < 
                (other.last_name, other.first_name, other.age))
    
    # We only need to implement __eq__ and __lt__ if we use the @total_ordering decorator
    from functools import total_ordering
    
    @total_ordering
    class PersonEfficient:
        """A version of Person using the total_ordering decorator"""
        
        def __init__(self, first_name, last_name, age):
            self.first_name = first_name
            self.last_name = last_name
            self.age = age
        
        def __str__(self):
            return f"{self.first_name} {self.last_name}, {self.age} years old"
        
        def __repr__(self):
            return f"PersonEfficient('{self.first_name}', '{self.last_name}', {self.age})"
        
        def __eq__(self, other):
            """Equal to: self == other"""
            if not isinstance(other, PersonEfficient):
                return NotImplemented
            return ((self.last_name, self.first_name, self.age) == 
                    (other.last_name, other.first_name, other.age))
        
        def __lt__(self, other):
            """Less than: self < other"""
            if not isinstance(other, PersonEfficient):
                return NotImplemented
            return ((self.last_name, self.first_name, self.age) < 
                    (other.last_name, other.first_name, other.age))


# Demo the Person class
print("\nPerson examples:")
p1 = Person("John", "Doe", 30)
p2 = Person("Jane", "Doe", 28)
p3 = Person("Jane", "Doe", 28)  # Same as p2
p4 = Person("Alice", "Smith", 35)

print(f"p1 = {p1}")
print(f"p2 = {p2}")
print(f"p3 = {p3}")
print(f"p4 = {p4}")

print(f"p1 == p2: {p1 == p2}")
print(f"p2 == p3: {p2 == p3}")

print(f"p1 < p2: {p1 < p2}")  # Compare last name, first name, age
print(f"p2 < p4: {p2 < p4}")  # Doe comes before Smith
print(f"p4 > p1: {p4 > p1}")  # Smith comes after Doe

# Sorting
people = [p1, p2, p4, p3]
print(f"Unsorted people: {people}")
sorted_people = sorted(people)
print(f"Sorted people: {sorted_people}")

# Using the efficient version with total_ordering
from functools import total_ordering

pe1 = PersonEfficient("John", "Doe", 30)
pe2 = PersonEfficient("Jane", "Doe", 28)

print(f"\nUsing total_ordering:")
print(f"pe1 <= pe2: {pe1 <= pe2}")  # This is derived from __lt__ and __eq__
print(f"pe1 >= pe2: {pe1 >= pe2}")  # This is derived from __lt__ and __eq__

Code Breakdown:

We've created two classes that demonstrate comparison methods:

  1. Version: Represents software versions (e.g., 1.2.3) with complete comparison support:
    • __eq__(self, other): Equality comparison (==)
    • __ne__(self, other): Inequality comparison (!=)
    • __lt__(self, other): Less than comparison (<)
    • __le__(self, other): Less than or equal to comparison (<=)
    • __gt__(self, other): Greater than comparison (>)
    • __ge__(self, other): Greater than or equal to comparison (>=)
  2. Person and PersonEfficient: Demonstrate comparison methods for sorting people:
    • Person implements all comparison methods manually
    • PersonEfficient uses the @total_ordering decorator to generate the remaining comparison methods after implementing just __eq__ and __lt__

Comparison Method Table

Method Operator Description
__eq__(self, other) == Equality
__ne__(self, other) != Inequality
__lt__(self, other) < Less than
__le__(self, other) <= Less than or equal to
__gt__(self, other) > Greater than
__ge__(self, other) >= Greater than or equal to

Best practices for comparison methods:

The @total_ordering decorator:

The @total_ordering decorator from the functools module provides a way to reduce the boilerplate code when implementing comparison methods. If you implement __eq__ and any one of __lt__, __le__, __gt__, or __ge__, the decorator will generate the other comparison methods for you.

Real-world analogy: Comparison methods are like the rules for sorting and comparing objects in the real world. For example, a library might sort books by author's last name, then first name, then title. Similarly, your comparison methods define the natural ordering of your custom objects, allowing them to be sorted and compared just like built-in types.

Attribute Access Methods

Python provides special methods to customize attribute access, which can be used for validation, computed properties, and more. Create a file named attribute_access.py with the following code:

# File: dunder_methods/attribute_access.py

class ProtectedAttributes:
    """A class demonstrating attribute access customization"""
    
    def __init__(self, **kwargs):
        # Initialize with any provided attributes
        self._data = {}
        for key, value in kwargs.items():
            self._data[key] = value
    
    def __getattr__(self, name):
        """Called when an attribute lookup fails with AttributeError
        
        This is called when the attribute is not found through normal lookup.
        """
        if name in self._data:
            return self._data[name]
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")
    
    def __setattr__(self, name, value):
        """Called when an attribute is set: obj.name = value"""
        # Allow _data and other attributes starting with _ to be set normally
        if name.startswith('_'):
            object.__setattr__(self, name, value)
        else:
            # Store other attributes in the _data dictionary
            if hasattr(self, '_data'):
                self._data[name] = value
            else:
                # During initialization, _data might not exist yet
                object.__setattr__(self, name, value)
    
    def __delattr__(self, name):
        """Called when an attribute is deleted: del obj.name"""
        if name.startswith('_'):
            object.__delattr__(self, name)
        elif name in self._data:
            del self._data[name]
        else:
            raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")
    
    def __dir__(self):
        """Return the list of attributes (used by the dir() function)"""
        # Combine standard attributes with our custom attributes
        return list(set(dir(self.__class__)) | set(self._data.keys()))


class ValidatedPerson:
    """A class that validates attributes when they are set"""
    
    def __init__(self, name=None, age=None, email=None):
        # Use the descriptor property for validation
        self.name = name
        self.age = age
        self.email = email
    
    def __setattr__(self, name, value):
        """Validate attributes before setting them"""
        if name == 'name' and value is not None:
            if not isinstance(value, str):
                raise TypeError("Name must be a string")
            if len(value) < 2:
                raise ValueError("Name must be at least 2 characters")
        elif name == 'age' and value is not None:
            if not isinstance(value, int):
                raise TypeError("Age must be an integer")
            if value < 0 or value > 120:
                raise ValueError("Age must be between 0 and 120")
        elif name == 'email' and value is not None:
            if not isinstance(value, str):
                raise TypeError("Email must be a string")
            if '@' not in value:
                raise ValueError("Email must contain an @ symbol")
        
        # Set the attribute after validation
        object.__setattr__(self, name, value)


class LazyProperties:
    """A class demonstrating lazy-loaded properties"""
    
    def __init__(self):
        self._data = {}
        self._computed = {}
    
    def __getattr__(self, name):
        """Handle lazy-loaded properties"""
        if name.startswith('compute_'):
            # If it's a compute_* method, run it and cache the result
            property_name = name[8:]  # Remove 'compute_' prefix
            compute_method = getattr(self.__class__, name)
            result = compute_method(self)
            self._computed[property_name] = result
            return result
        
        if name in self._computed:
            # Return pre-computed property
            return self._computed[name]
        
        # Check if there's a compute_* method for this property
        compute_method_name = f"compute_{name}"
        if hasattr(self.__class__, compute_method_name):
            # Compute the property, cache it, and return it
            compute_method = getattr(self.__class__, compute_method_name)
            result = compute_method(self)
            self._computed[name] = result
            return result
        
        # Default behavior: look in _data or raise AttributeError
        if name in self._data:
            return self._data[name]
        
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")
    
    def __setattr__(self, name, value):
        """Handle attribute assignment"""
        if name.startswith('_'):
            # Allow internal attributes to be set normally
            object.__setattr__(self, name, value)
        else:
            # Store other attributes in the _data dictionary
            if hasattr(self, '_data'):
                self._data[name] = value
                # Clear any computed value for this property
                if name in self._computed:
                    del self._computed[name]
            else:
                # During initialization, _data might not exist yet
                object.__setattr__(self, name, value)
    
    # Example compute methods
    def compute_full_name(self):
        """Compute the full name from first_name and last_name"""
        first_name = self._data.get('first_name', '')
        last_name = self._data.get('last_name', '')
        return f"{first_name} {last_name}".strip()
    
    def compute_age_in_days(self):
        """Compute age in days from the age in years"""
        age_years = self._data.get('age', 0)
        return age_years * 365


class Point:
    """A simple Point class with a descriptor for coordinate validation"""
    
    def __init__(self, x=0, y=0):
        self.x = x
        self.y = y
    
    def __str__(self):
        return f"Point({self.x}, {self.y})"
    
    def __getattribute__(self, name):
        """Called for all attribute access (even attributes that exist)"""
        # Log all attribute access (for demonstration purposes)
        print(f"Accessing attribute: {name}")
        
        # Call the original __getattribute__ to actually get the attribute
        return object.__getattribute__(self, name)


# Demo attribute access methods
if __name__ == "__main__":
    # ProtectedAttributes example
    print("ProtectedAttributes example:")
    obj = ProtectedAttributes(a=1, b=2, c=3)
    print(f"obj.a = {obj.a}")
    print(f"obj.b = {obj.b}")
    
    obj.d = 4
    print(f"After setting obj.d = 4: obj.d = {obj.d}")
    
    del obj.c
    try:
        print(obj.c)
    except AttributeError as e:
        print(f"Error: {e}")
    
    print(f"dir(obj): {dir(obj)}")
    
    # ValidatedPerson example
    print("\nValidatedPerson example:")
    person = ValidatedPerson(name="John Doe", age=30, email="john@example.com")
    print(f"Initial person: name={person.name}, age={person.age}, email={person.email}")
    
    try:
        person.age = -5  # Invalid age
    except ValueError as e:
        print(f"Error setting invalid age: {e}")
    
    try:
        person.email = "invalid-email"  # Invalid email
    except ValueError as e:
        print(f"Error setting invalid email: {e}")
    
    # LazyProperties example
    print("\nLazyProperties example:")
    props = LazyProperties()
    props.first_name = "John"
    props.last_name = "Doe"
    props.age = 30
    
    # Access a computed property
    print(f"props.full_name = {props.full_name}")  # This will compute and cache the full name
    print(f"props.full_name = {props.full_name}")  # This will use the cached value
    
    # Change an input and the computed property should update
    props.first_name = "Jane"
    print(f"After changing first_name: props.full_name = {props.full_name}")
    
    # Try accessing the compute method directly
    print(f"props.compute_age_in_days() = {props.compute_age_in_days()}")
    print(f"props.age_in_days = {props.age_in_days}")
    
    # Point example with __getattribute__
    print("\nPoint example:")
    p = Point(3, 4)
    x = p.x  # This will log the access
    y = p.y  # This will log the access
    print(f"p = {p}")

Code Breakdown:

We've created several classes that demonstrate customizing attribute access:

  1. ProtectedAttributes: A class that stores attributes in a protected dictionary:
    • __getattr__(self, name): Called when an attribute lookup fails through normal mechanisms.
    • __setattr__(self, name, value): Called when an attribute is set.
    • __delattr__(self, name): Called when an attribute is deleted.
    • __dir__(self): Called to get the list of attributes (used by dir()).
  2. ValidatedPerson: A class that validates attributes when set:
    • __setattr__(self, name, value): Validates attributes before setting them.
  3. LazyProperties: A class with lazy-loaded computed properties:
    • __getattr__(self, name): Handles lazy computation of properties.
    • __setattr__(self, name, value): Clears cached computed values when dependencies change.
  4. Point: A simple class demonstrating __getattribute__:
    • __getattribute__(self, name): Called for all attribute access, even existing attributes.

Attribute Access Method Table

Method Called When Primary Use
__getattr__(self, name) When normal attribute lookup fails Fallback for missing attributes
__getattribute__(self, name) For all attribute access (even existing attributes) Intercept all attribute access
__setattr__(self, name, value) When an attribute is set Validate or transform attributes before setting
__delattr__(self, name) When an attribute is deleted Custom deletion behavior
__dir__(self) When dir(obj) is called Customize the list of attributes

Important Differences

Best practices for attribute access methods:

Real-world analogy: Attribute access methods are like receptionists or security guards for your objects. __getattr__ is like a helpful receptionist who steps in only when a visitor asks for someone who isn't in the directory. __getattribute__ is like a security guard who checks every single person entering the building, regardless of whether they're in the system or not. __setattr__ is like a mail room that inspects and processes all packages before they're delivered.

Callable Objects with __call__

Python allows you to make your objects callable, like functions, by implementing the __call__ method. Create a file named callable_objects.py with the following code:

# File: dunder_methods/callable_objects.py

class Counter:
    """A simple counter that can be called to increment"""
    
    def __init__(self, start=0, step=1):
        self.value = start
        self.step = step
    
    def __call__(self, step=None):
        """Makes the object callable, incrementing the counter"""
        if step is None:
            step = self.step
        self.value += step
        return self.value
    
    def reset(self, value=0):
        """Reset the counter to a specific value"""
        self.value = value
    
    def __str__(self):
        return str(self.value)
    
    def __repr__(self):
        return f"Counter(start={self.value}, step={self.step})"


class Multiplier:
    """A callable object that multiplies its arguments by a fixed factor"""
    
    def __init__(self, factor):
        self.factor = factor
    
    def __call__(self, *args):
        """Multiply all arguments by the factor"""
        if len(args) == 1:
            return args[0] * self.factor
        return tuple(arg * self.factor for arg in args)
    
    def __str__(self):
        return f"Multiplier(factor={self.factor})"
    
    def __repr__(self):
        return f"Multiplier({self.factor})"


class FunctionCache:
    """A class that caches function results based on arguments"""
    
    def __init__(self, func):
        self.func = func
        self.cache = {}
    
    def __call__(self, *args):
        """Call the function with caching"""
        # We can only cache if all arguments are hashable
        if args in self.cache:
            print(f"Cache hit for {args}")
            return self.cache[args]
        
        # Call the original function and cache the result
        result = self.func(*args)
        self.cache[args] = result
        print(f"Cache miss for {args}, cached result")
        return result
    
    def clear_cache(self):
        """Clear the cache"""
        self.cache.clear()
    
    def __str__(self):
        return f"FunctionCache({self.func.__name__})"
    
    def __repr__(self):
        return f"FunctionCache(func={self.func.__name__})"


# A complex function to be cached
def fibonacci(n):
    """Calculate the nth Fibonacci number"""
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)


class HTMLElement:
    """A class representing an HTML element that can be called to add content"""
    
    def __init__(self, tag, **attrs):
        self.tag = tag
        self.attrs = attrs
        self.content = []
    
    def __call__(self, *content):
        """Add content to the element and return self for chaining"""
        self.content.extend(content)
        return self
    
    def __str__(self):
        """Render the HTML element as a string"""
        # Create the opening tag with attributes
        attrs_str = ' '.join(f'{k}="{v}"' for k, v in self.attrs.items())
        if attrs_str:
            opening_tag = f"<{self.tag} {attrs_str}>"
        else:
            opening_tag = f"<{self.tag}>"
        
        # Create the closing tag
        closing_tag = f""
        
        # If there's no content, return a self-closing tag
        if not self.content:
            return opening_tag + closing_tag
        
        # Join the content (converting objects to strings)
        content_str = ''.join(str(item) for item in self.content)
        
        # Return the complete element
        return opening_tag + content_str + closing_tag


# Demo callable objects
if __name__ == "__main__":
    # Counter example
    print("Counter example:")
    counter = Counter()
    print(f"Initial counter: {counter}")
    print(f"counter(): {counter()}")
    print(f"counter(): {counter()}")
    print(f"counter(10): {counter(10)}")
    print(f"Final counter: {counter}")
    
    counter.reset()
    print(f"After reset: {counter}")
    
    # Multiplier example
    print("\nMultiplier example:")
    double = Multiplier(2)
    triple = Multiplier(3)
    half = Multiplier(0.5)
    
    print(f"double(5): {double(5)}")
    print(f"triple(5): {triple(5)}")
    print(f"half(5): {half(5)}")
    
    print(f"double(1, 2, 3): {double(1, 2, 3)}")
    print(f"triple('hello'): {triple('hello')}")
    
    # FunctionCache example
    print("\nFunctionCache example:")
    # Create a cached version of the fibonacci function
    cached_fibonacci = FunctionCache(fibonacci)
    
    # First call (cache miss)
    print(f"cached_fibonacci(10): {cached_fibonacci(10)}")
    
    # Second call (cache hit)
    print(f"cached_fibonacci(10): {cached_fibonacci(10)}")
    
    # Different argument (cache miss)
    print(f"cached_fibonacci(12): {cached_fibonacci(12)}")
    
    # Clear the cache
    cached_fibonacci.clear_cache()
    print("Cache cleared")
    
    # This will be a miss again
    print(f"cached_fibonacci(10): {cached_fibonacci(10)}")
    
    # HTMLElement example
    print("\nHTMLElement example:")
    # Create HTML elements
    div = HTMLElement('div', id='content', class_='container')
    p = HTMLElement('p', style='color: blue;')
    h1 = HTMLElement('h1')
    
    # Add content using the call syntax
    h1('Hello, World!')
    p('This is a paragraph with ', b := HTMLElement('b')('bold'), ' text.')
    div(h1, p)
    
    # Render the HTML
    print(div)
    
    # Chain calls
    html = HTMLElement('html')(
        HTMLElement('head')(
            HTMLElement('title')('My Page')
        ),
        HTMLElement('body')(
            HTMLElement('h1')('Welcome'),
            HTMLElement('p')('This is my page.')
        )
    )
    
    print("\nHTML document created with chaining:")
    print(html)

Code Breakdown:

We've created several classes that demonstrate the __call__ method, which makes objects callable like functions:

  1. Counter: A simple counter that increments when called:
    • counter() increments the counter by the default step
    • counter(10) increments the counter by a specified amount
  2. Multiplier: A callable object that multiplies its arguments by a fixed factor:
    • double = Multiplier(2) creates a function-like object that doubles its arguments
    • double(5) -> 10
  3. FunctionCache: A class that caches function results based on arguments:
    • Wraps an existing function and caches its results
    • Demonstrates a practical use of callable objects for function decoration
  4. HTMLElement: A class representing an HTML element that can be called to add content:
    • Uses the callable pattern to create a fluent interface for building HTML
    • Demonstrates method chaining with __call__

When to use __call__:

Best practices for __call__:

Real-world analogy: The __call__ method is like giving your objects the ability to be "activated" or "triggered" like a button. Just as a button can be pressed to perform an action, an object with __call__ can be invoked like a function to perform its primary action. For example, a Camera object might be callable to take a picture, or a Database object might be callable to execute a query.

Real-World Examples

Let's look at some real-world examples of how dunder methods are used in practice. Create a file named real_world_examples.py with the following code:

# File: dunder_methods/real_world_examples.py

import json
import os
from datetime import datetime, timedelta
from functools import wraps
import time


class APIResponse:
    """A class representing an API response"""
    
    def __init__(self, data=None, status_code=200, message=None):
        self.data = data
        self.status_code = status_code
        self.message = message
        self.timestamp = datetime.now()
    
    def __str__(self):
        """String representation for users"""
        if self.message:
            return f"[{self.status_code}] {self.message}"
        return f"[{self.status_code}] {'Success' if self.status_code < 400 else 'Error'}"
    
    def __repr__(self):
        """Detailed representation for developers"""
        return f"APIResponse(status_code={self.status_code}, message='{self.message}', data={self.data})"
    
    def __bool__(self):
        """Return True if the response is successful (status_code < 400)"""
        return self.status_code < 400
    
    def __getitem__(self, key):
        """Allow accessing data attributes using dictionary notation"""
        if not self.data:
            raise KeyError(key)
        if isinstance(self.data, dict):
            return self.data[key]
        raise TypeError("Response data is not a dictionary")
    
    def to_dict(self):
        """Convert the response to a dictionary"""
        return {
            'status_code': self.status_code,
            'message': self.message,
            'data': self.data,
            'timestamp': self.timestamp.isoformat()
        }
    
    def to_json(self):
        """Convert the response to a JSON string"""
        return json.dumps(self.to_dict(), default=str)


class ExponentialBackoff:
    """A callable object implementing exponential backoff for retries"""
    
    def __init__(self, initial_delay=1, max_delay=60, factor=2, max_retries=5):
        self.initial_delay = initial_delay
        self.max_delay = max_delay
        self.factor = factor
        self.max_retries = max_retries
        self.retries = 0
    
    def __call__(self):
        """Return the next delay time in seconds"""
        if self.retries >= self.max_retries:
            raise Exception(f"Maximum retries ({self.max_retries}) exceeded")
        
        delay = min(self.initial_delay * (self.factor ** self.retries), self.max_delay)
        self.retries += 1
        return delay
    
    def reset(self):
        """Reset retry counter"""
        self.retries = 0


def retry_with_backoff(func):
    """Decorator to retry a function with exponential backoff"""
    @wraps(func)
    def wrapper(*args, **kwargs):
        backoff = ExponentialBackoff()
        while True:
            try:
                return func(*args, **kwargs)
            except Exception as e:
                try:
                    delay = backoff()
                    print(f"Error: {e}. Retrying in {delay} seconds...")
                    time.sleep(delay)
                except Exception as backoff_error:
                    print(f"Giving up after {backoff.retries} retries: {e}")
                    raise e
    return wrapper


class ConfigDict:
    """A dictionary-like configuration class with dot notation access"""
    
    def __init__(self, config=None):
        self._config = {}
        if config:
            for key, value in config.items():
                self[key] = value
    
    def __getitem__(self, key):
        """Access config items with dictionary notation"""
        return self._config[key]
    
    def __setitem__(self, key, value):
        """Set config items with dictionary notation"""
        # If the value is a dictionary, convert it to a ConfigDict
        if isinstance(value, dict):
            value = ConfigDict(value)
        self._config[key] = value
    
    def __delitem__(self, key):
        """Delete config items with dictionary notation"""
        del self._config[key]
    
    def __contains__(self, key):
        """Check if config contains a key"""
        return key in self._config
    
    def __getattr__(self, name):
        """Access config items with dot notation"""
        if name in self._config:
            return self._config[name]
        raise AttributeError(f"No configuration setting named '{name}'")
    
    def __setattr__(self, name, value):
        """Set config items with dot notation"""
        if name == '_config':
            # Allow setting the internal _config dictionary
            super().__setattr__(name, value)
        else:
            # Set other attributes in the config dictionary
            self[name] = value
    
    def __iter__(self):
        """Iterate over config keys"""
        return iter(self._config)
    
    def __len__(self):
        """Get the number of config items"""
        return len(self._config)
    
    def get(self, key, default=None):
        """Get a config item with a default value"""
        return self._config.get(key, default)


class Cache:
    """A simple cache with expiring entries"""
    
    def __init__(self, default_ttl=300):  # Default TTL: 5 minutes
        self._cache = {}
        self._expiry = {}
        self.default_ttl = default_ttl
    
    def __getitem__(self, key):
        """Get an item from the cache, checking for expiration"""
        if key not in self._cache:
            raise KeyError(key)
        
        # Check if the entry has expired
        if key in self._expiry and datetime.now() > self._expiry[key]:
            # Remove expired entry
            del self._cache[key]
            del self._expiry[key]
            raise KeyError(f"Cache entry '{key}' has expired")
        
        return self._cache[key]
    
    def __setitem__(self, key, value):
        """Set an item in the cache with default TTL"""
        self.set(key, value, self.default_ttl)
    
    def __delitem__(self, key):
        """Delete an item from the cache"""
        if key in self._cache:
            del self._cache[key]
        if key in self._expiry:
            del self._expiry[key]
    
    def __contains__(self, key):
        """Check if key is in the cache and not expired"""
        if key not in self._cache:
            return False
        
        # Check if the entry has expired
        if key in self._expiry and datetime.now() > self._expiry[key]:
            # Remove expired entry
            del self._cache[key]
            del self._expiry[key]
            return False
        
        return True
    
    def set(self, key, value, ttl=None):
        """Set an item in the cache with a specific TTL"""
        self._cache[key] = value
        
        if ttl is not None:
            self._expiry[key] = datetime.now() + timedelta(seconds=ttl)
    
    def get(self, key, default=None):
        """Get an item from the cache or return default"""
        try:
            return self[key]
        except KeyError:
            return default
    
    def clear(self):
        """Clear the cache"""
        self._cache.clear()
        self._expiry.clear()


class FileSystemStorage:
    """A class for managing file storage with context manager support"""
    
    def __init__(self, base_path):
        self.base_path = base_path
        self.file = None
    
    def __enter__(self):
        """Context manager entry point"""
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        """Context manager exit point"""
        if self.file:
            self.file.close()
            self.file = None
    
    def __getitem__(self, file_path):
        """Read file content using dictionary notation"""
        full_path = os.path.join(self.base_path, file_path)
        try:
            with open(full_path, 'r') as f:
                return f.read()
        except FileNotFoundError:
            raise KeyError(f"File not found: {file_path}")
    
    def __setitem__(self, file_path, content):
        """Write file content using dictionary notation"""
        full_path = os.path.join(self.base_path, file_path)
        os.makedirs(os.path.dirname(full_path), exist_ok=True)
        with open(full_path, 'w') as f:
            f.write(content)
    
    def __delitem__(self, file_path):
        """Delete a file using dictionary notation"""
        full_path = os.path.join(self.base_path, file_path)
        if not os.path.exists(full_path):
            raise KeyError(f"File not found: {file_path}")
        os.remove(full_path)
    
    def __contains__(self, file_path):
        """Check if a file exists using 'in' operator"""
        full_path = os.path.join(self.base_path, file_path)
        return os.path.exists(full_path)
    
    def open(self, file_path, mode='r'):
        """Open a file and return the file object"""
        full_path = os.path.join(self.base_path, file_path)
        os.makedirs(os.path.dirname(full_path), exist_ok=True)
        self.file = open(full_path, mode)
        return self.file
    
    def list_files(self):
        """List all files in the storage"""
        files = []
        for root, _, filenames in os.walk(self.base_path):
            for filename in filenames:
                full_path = os.path.join(root, filename)
                rel_path = os.path.relpath(full_path, self.base_path)
                files.append(rel_path)
        return files


# Demo real-world examples
if __name__ == "__main__":
    # APIResponse example
    print("APIResponse example:")
    success_response = APIResponse(
        data={"user_id": 123, "username": "john_doe"},
        status_code=200,
        message="User retrieved successfully"
    )
    
    error_response = APIResponse(
        status_code=404,
        message="User not found"
    )
    
    print(f"Success response: {success_response}")
    print(f"Error response: {error_response}")
    
    # Using __bool__
    if success_response:
        print("Success response evaluated as True")
    
    if not error_response:
        print("Error response evaluated as False")
    
    # Using __getitem__
    print(f"User ID: {success_response['user_id']}")
    print(f"Username: {success_response['username']}")
    
    # ExponentialBackoff example
    print("\nExponentialBackoff example:")
    backoff = ExponentialBackoff(initial_delay=0.1, max_delay=2, factor=2, max_retries=5)
    
    for _ in range(5):
        delay = backoff()
        print(f"Retry in {delay} seconds")
    
    try:
        backoff()  # This should exceed max retries
    except Exception as e:
        print(f"Error: {e}")
    
    # ConfigDict example
    print("\nConfigDict example:")
    config = ConfigDict({
        'app': {
            'name': 'My App',
            'version': '1.0.0'
        },
        'database': {
            'host': 'localhost',
            'port': 5432,
            'user': 'admin'
        },
        'debug': True
    })
    
    # Access with dot notation
    print(f"App name: {config.app.name}")
    print(f"Database host: {config.database.host}")
    print(f"Debug mode: {config.debug}")
    
    # Access with dictionary notation
    print(f"App version: {config['app']['version']}")
    print(f"Database port: {config['database']['port']}")
    
    # Modify with dot notation
    config.app.version = '1.1.0'
    print(f"Updated app version: {config.app.version}")
    
    # Add new settings
    config.logging = {'level': 'INFO'}
    print(f"Logging level: {config.logging.level}")
    
    # Cache example
    print("\nCache example:")
    cache = Cache(default_ttl=2)  # 2 seconds TTL for testing
    
    cache['key1'] = 'value1'
    cache.set('key2', 'value2', ttl=5)  # 5 seconds TTL
    
    print(f"key1 in cache: {cache.get('key1')}")
    print(f"key2 in cache: {cache.get('key2')}")
    print(f"key3 in cache: {cache.get('key3', 'default')}")
    
    print("Waiting for key1 to expire...")
    time.sleep(3)
    
    print(f"key1 in cache: {cache.get('key1', 'expired')}")
    print(f"key2 in cache: {cache.get('key2', 'expired')}")
    
    # FileSystemStorage example (commented out to avoid actual file operations)
    """
    print("\nFileSystemStorage example:")
    storage = FileSystemStorage("./temp_storage")
    
    with storage:
        # Write files
        storage['test.txt'] = 'Hello, World!'
        storage['folder/nested.txt'] = 'Nested file content'
        
        # Read files
        print(f"test.txt content: {storage['test.txt']}")
        print(f"folder/nested.txt content: {storage['folder/nested.txt']}")
        
        # Check if file exists
        print(f"'test.txt' exists: {'test.txt' in storage}")
        print(f"'nonexistent.txt' exists: {'nonexistent.txt' in storage}")
        
        # List files
        print(f"All files: {storage.list_files()}")
        
        # Delete a file
        del storage['test.txt']
        print(f"After deletion, 'test.txt' exists: {'test.txt' in storage}")
    """

Code Breakdown:

We've created several practical classes that demonstrate how dunder methods can be used in real-world applications:

  1. APIResponse: Represents an API response with methods for string representation, truthiness checking, and data access:
    • __str__ and __repr__ for different string representations
    • __bool__ to determine success based on status code
    • __getitem__ for dictionary-like access to response data
  2. ExponentialBackoff: A callable object implementing exponential backoff for retry logic:
    • __call__ for function-like behavior
    • Used in the retry_with_backoff decorator
  3. ConfigDict: A configuration dictionary supporting both dictionary and dot notation access:
    • Container methods like __getitem__, __setitem__, etc.
    • Attribute access methods like __getattr__ and __setattr__
  4. Cache: A simple caching system with expiring entries:
    • Container methods for dictionary-like access
    • Time-based expiration of cache entries
  5. FileSystemStorage: A file storage system with context manager support:
    • __enter__ and __exit__ for context manager (with statement) support
    • Container methods for file access using dictionary notation

These examples demonstrate how dunder methods can be combined to create intuitive, Pythonic interfaces for complex functionality. By implementing the appropriate dunder methods, we can make our classes behave like built-in types, making them more familiar and easier to use for other developers.

Key Takeaways

Best Practices for Dunder Methods

  1. Follow Python's conventions: Implement dunder methods that align with Python's intended semantics. For example, __add__ should do something addition-like.
  2. Provide complementary methods: If you implement __add__, consider implementing __radd__ too. If you implement __eq__, consider implementing __ne__ as well.
  3. Use NotImplemented for unsupported operations: Return NotImplemented (not to be confused with NotImplementedError) when an operation is not supported for a given type.
  4. Keep dunder methods consistent: Your dunder methods should behave consistently with each other and with Python's built-in types.
  5. Be careful with performance: Some dunder methods like __getattribute__ are called frequently, so keep them efficient.
  6. Don't abuse dunder methods: Only implement the dunder methods that make sense for your class's domain.
  7. Avoid infinite recursion: Be careful when implementing methods like __getattribute__ and __setattr__ to avoid calling themselves recursively.
  8. Document your dunder methods: Especially if they behave in non-standard ways.
  9. Use appropriate return types: For example, __len__ should return an integer, __bool__ should return a boolean.
  10. Consider immutability: For value-like objects, consider making them immutable and implementing dunder methods accordingly.

Assignment: Create a Financial Dashboard with Dunder Methods

For this assignment, you'll create a financial dashboard system that makes extensive use of dunder methods to provide a clean, intuitive interface.

Requirements:

  1. Create a Currency class:
    • Support for different currency symbols (USD, EUR, GBP, etc.)
    • Operator overloading for arithmetic operations
    • String representation with proper formatting
    • Comparison operators for comparing monetary values
  2. Create a Portfolio class:
    • Container methods for managing investments (add, remove, get by name)
    • Iteration support for looping through investments
    • Length checking for counting investments
    • Attribute access for accessing portfolio statistics
  3. Create an Investment class:
    • Basic dunder methods for string representation
    • Callable behavior for updating the investment value
    • Comparison methods for ranking investments
  4. Create a Transaction class:
    • String representation for transaction details
    • Attribute access with validation
    • Boolean evaluation based on transaction status
  5. Create a Report class:
    • Context manager support for report generation
    • Callable behavior for filtering report data
    • Container methods for accessing report sections
  6. Integration of these classes into a cohesive financial dashboard system
  7. Documentation and examples showing how the dunder methods enhance the user experience

Bonus Challenges:

  1. Implement a TimeRange class with dunder methods for date/time operations
  2. Create a DataFrameView class that uses dunder methods to provide a pandas-like interface
  3. Implement lazy-loading of portfolio data using __getattr__
  4. Create a simple GUI or web interface that leverages your dunder methods for display and interaction
  5. Add persistence with file or database operations using dunder methods

Submit your work as a Python module with clear structure and organization. Be prepared to explain how the dunder methods you've implemented enhance the usability and expressiveness of your code.

Further Reading and Resources

# If it's a compute_* method, run it and cache the result