Handling Multiple Exceptions in Python

Week 3: Python Fundamentals - Exception Management

Introduction to Handling Multiple Exceptions

Welcome to our exploration of handling multiple exceptions in Python! As we've learned, exceptions are Python's way of dealing with unexpected events during program execution. In real-world applications, multiple things can go wrong, and we need strategies for handling different types of errors appropriately.

Think of exception handling as the emergency response system for your code. Just like a city has different teams for handling fires, medical emergencies, and police matters, your code can have specific handlers for different types of exceptions.

Folder Structure for Today's Examples

multiple_exceptions/
├── basics/
│   ├── simple_multiple_except.py
│   ├── exception_groups.py
│   ├── exception_tuple.py
│   └── exception_hierarchy.py
├── advanced/
│   ├── exception_chaining.py
│   ├── unified_handler.py
│   └── exception_filtering.py
├── practical/
│   ├── file_processing.py
│   ├── network_operations.py
│   ├── database_access.py
│   └── data_conversion.py
└── exercises/
    ├── exercise1.py
    ├── exercise2.py
    └── exercise3.py
                

Basic Approaches to Handling Multiple Exceptions

Python offers several ways to handle multiple exception types. Let's explore these approaches starting with the most basic.

Multiple except Blocks

# File: basics/simple_multiple_except.py

def divide_and_process_numbers(a, b):
    try:
        # Several operations that could raise different exceptions
        result = a / b            # Potential ZeroDivisionError
        processed = int(result)   # Potential ValueError
        return processed
    
    except ZeroDivisionError:
        print("Error: Cannot divide by zero!")
        return None
    
    except ValueError:
        print("Error: Could not convert the result to an integer!")
        return None
    
    except Exception as e:
        # Catch-all for any other exceptions
        print(f"Unexpected error: {e}")
        return None

# Test cases
print(divide_and_process_numbers(10, 2))    # Works: 5
print(divide_and_process_numbers(10, 0))    # ZeroDivisionError
print(divide_and_process_numbers(10, 3.5))  # Works: 2 (truncated to int)
print(divide_and_process_numbers("10", 2))  # TypeError (caught by generic Exception)
                

This is the simplest approach, where each exception type has its own except block. Python tries each block in sequence until it finds a matching exception type.

Key points:

  • The order of except blocks matters. More specific exceptions should come first, followed by more general ones.
  • Once an exception is caught by a matching block, no other blocks are checked.
  • Using Exception as a catch-all should generally be the last block to avoid masking more specific errors.

Handling Multiple Exception Types in a Single Block

# File: basics/exception_tuple.py

def process_data(data):
    try:
        # Operations that might raise different exceptions
        value = data['key']                  # Potential KeyError
        result = 100 / value                 # Potential ZeroDivisionError
        return f"Processed result: {result}"
    
    except (KeyError, TypeError) as e:
        # Handle data structure issues together
        print(f"Data structure error: {e}")
        return None
    
    except (ValueError, ZeroDivisionError) as e:
        # Handle calculation issues together
        print(f"Calculation error: {e}")
        return None

# Test cases
print(process_data({'key': 5}))        # Works: Processed result: 20.0
print(process_data({'different_key': 5}))  # KeyError
print(process_data(None))              # TypeError
print(process_data({'key': 0}))        # ZeroDivisionError
print(process_data({'key': '5'}))      # TypeError (can't divide by string)
                

You can group similar exceptions by providing a tuple of exception types in a single except clause. This is useful when you want to handle related exceptions in the same way.

Advantages:

  • Reduces code duplication when multiple exceptions require similar handling
  • Groups logically related errors together
  • Creates cleaner code with fewer repetitive blocks

Leveraging the Exception Hierarchy

# File: basics/exception_hierarchy.py

def read_config(filename):
    try:
        with open(filename, 'r') as file:
            content = file.read()
            config = eval(content)  # Danger: eval is used for demonstration only!
            return config
    
    except FileNotFoundError:
        # Specific handling for missing files
        print(f"Config file not found: {filename}")
        return {}
    
    except OSError as e:
        # Handles broader file system errors (parent of FileNotFoundError)
        print(f"OS error when reading config: {e}")
        return {}
    
    except SyntaxError as e:
        # Handle invalid Python syntax in the config
        print(f"Invalid syntax in config file: {e}")
        return {}
    
    except Exception as e:
        # Generic fallback
        print(f"Unexpected error reading config: {e}")
        return {}

# Show the exception hierarchy
import inspect

def show_exception_hierarchy(exception_class, indent=0):
    print(" " * indent + exception_class.__name__)
    for subclass in exception_class.__subclasses__():
        show_exception_hierarchy(subclass, indent + 4)

# Show a partial hierarchy starting from OSError
print("Partial Exception Hierarchy:")
show_exception_hierarchy(OSError)

# Test with different files
print("\nTesting with different files:")
print(read_config("config.txt"))         # Assuming it doesn't exist
print(read_config("/nonexistent_dir/config.txt"))  # Permission error on some systems
                

Python's exceptions form a hierarchy, which you can leverage to handle exceptions at different levels of specificity. Understanding this hierarchy helps you structure your exception handling more effectively.

Hierarchical handling strategy:

  • Catch specific exceptions first for specialized handling
  • Use parent exception classes to handle groups of related exceptions
  • Remember that child exceptions won't be caught by handlers that appear after their parent exception handler

For example, since FileNotFoundError is a subclass of OSError, a handler for OSError would catch FileNotFoundError too, unless the FileNotFoundError handler comes first.

Advanced Exception Handling Techniques

Let's explore more sophisticated approaches for managing multiple exceptions that provide better organization and error propagation.

Exception Chaining (Exception from Exception)

# File: advanced/exception_chaining.py

def parse_config_value(text):
    try:
        # Try to parse as an integer
        return int(text)
    except ValueError:
        # Chain the original exception to a new, more specific one
        raise ConfigParsingError(f"Invalid numeric value: {text}") from ValueError

def read_config_value(filename, key):
    try:
        with open(filename, 'r') as file:
            for line in file:
                if line.startswith(f"{key}="):
                    value_text = line.split('=', 1)[1].strip()
                    return parse_config_value(value_text)
                    
        # If we get here, the key wasn't found
        raise KeyError(f"Config key not found: {key}")
        
    except FileNotFoundError as e:
        # Wrap the original exception with more context
        raise ConfigError(f"Could not read config file: {filename}") from e
    
    except KeyError as e:
        # Propagate KeyError with the original as the cause
        raise ConfigError(f"Missing configuration: {e}") from e

# Custom exception classes
class ConfigError(Exception):
    """Base class for configuration-related errors"""
    pass

class ConfigParsingError(ConfigError):
    """Raised when a config value can't be parsed"""
    pass

# Usage example
def demonstrate_exception_chaining():
    try:
        # Try to read a configuration value
        value = read_config_value("settings.ini", "timeout")
        print(f"Timeout setting: {value}")
        
    except ConfigError as e:
        print(f"Configuration error: {e}")
        
        # Access the original exception that caused this one
        if e.__cause__:
            print(f"Original error: {e.__cause__}")

# Run the demonstration
demonstrate_exception_chaining()
                

Exception chaining allows you to raise a new exception while preserving the original exception as the cause. This is done using the raise ... from ... syntax.

Benefits of exception chaining:

  • Preserves the original exception information
  • Adds higher-level context to low-level exceptions
  • Builds a clear chain of what went wrong
  • Makes debugging easier by showing the complete error path

This is like telling someone not just that their package delivery failed, but exactly why: "Your delivery failed because the truck broke down because it ran out of gas."

Unified Exception Handling with Error Classification

# File: advanced/unified_handler.py

def classify_exception(e):
    """Classify an exception into a general category for handling."""
    
    # Check for data access errors
    if isinstance(e, (FileNotFoundError, PermissionError, IOError)):
        return "DATA_ACCESS_ERROR", f"Could not access required data: {e}"
    
    # Check for data format errors
    elif isinstance(e, (ValueError, TypeError, AttributeError)):
        return "DATA_FORMAT_ERROR", f"Invalid data format: {e}"
    
    # Check for resource errors
    elif isinstance(e, (MemoryError, TimeoutError, ConnectionError)):
        return "RESOURCE_ERROR", f"System resource error: {e}"
    
    # Default classification
    else:
        return "UNKNOWN_ERROR", f"An unexpected error occurred: {e}"

def process_with_unified_handling(func, *args, **kwargs):
    """
    Execute a function with unified exception handling.
    
    Args:
        func: The function to execute
        *args, **kwargs: Arguments to pass to the function
        
    Returns:
        A tuple of (success, result, error_code, error_message)
    """
    try:
        # Try to execute the function
        result = func(*args, **kwargs)
        return True, result, None, None
    
    except Exception as e:
        # Classify and handle the exception
        error_code, error_message = classify_exception(e)
        
        # Log the error (in a real system, use a proper logger)
        print(f"ERROR [{error_code}]: {error_message}")
        print(f"Exception type: {type(e).__name__}")
        
        # Return failure result with error information
        return False, None, error_code, error_message

# Example functions to test with
def read_data(filename):
    with open(filename, 'r') as file:
        return file.read()

def parse_json(text):
    import json
    return json.loads(text)

def calculate_average(numbers):
    return sum(numbers) / len(numbers)

# Demonstrate unified handling
def demonstrate_unified_handling():
    # Test with file not found error
    success, data, error_code, error_msg = process_with_unified_handling(
        read_data, "nonexistent_file.txt"
    )
    print(f"Operation succeeded: {success}")
    if not success:
        print(f"Error code: {error_code}")
    print()
    
    # Test with JSON parsing error
    success, data, error_code, error_msg = process_with_unified_handling(
        parse_json, "{ invalid json }"
    )
    print(f"Operation succeeded: {success}")
    if not success:
        print(f"Error code: {error_code}")
    print()
    
    # Test with division by zero
    success, data, error_code, error_msg = process_with_unified_handling(
        calculate_average, []  # Empty list will cause division by zero
    )
    print(f"Operation succeeded: {success}")
    if not success:
        print(f"Error code: {error_code}")

# Run the demonstration
demonstrate_unified_handling()
                

This approach creates a unified system for handling exceptions by classifying them into general categories. It's especially useful in larger applications where you want consistent error handling across many components.

Advantages of unified error handling:

  • Consistent error responses throughout the application
  • Centralizes error handling logic
  • Makes it easier to standardize logging, reporting, and user feedback
  • Simplifies code at the call site

This is like having a central dispatch service that routes all emergency calls to the appropriate department based on the type of emergency.

Exception Filtering with Predicates

# File: advanced/exception_filtering.py

def match_exception(e, **conditions):
    """
    Check if an exception matches specific conditions.
    
    Args:
        e: The exception to check
        **conditions: Key-value pairs of attributes to match
        
    Returns:
        True if all conditions match, False otherwise
    """
    for attr, value in conditions.items():
        # Check if the exception has the attribute
        if not hasattr(e, attr):
            return False
        
        # Check if the attribute value matches
        if getattr(e, attr) != value:
            return False
    
    # All conditions matched
    return True

def handle_database_error(func):
    """
    A decorator that handles database-related exceptions.
    
    Args:
        func: The function to wrap
        
    Returns:
        A wrapped function with error handling
    """
    def wrapper(*args, **kwargs):
        try:
            return func(*args, **kwargs)
            
        except Exception as e:
            # Handle connection errors
            if isinstance(e, ConnectionError) and match_exception(e, code='timeout'):
                print("Database connection timed out. Please try again later.")
                return None
                
            elif isinstance(e, ConnectionError) and match_exception(e, code='auth'):
                print("Database authentication failed. Please check credentials.")
                return None
                
            # Handle constraint violations
            elif isinstance(e, ValueError) and 'constraint' in str(e).lower():
                print("Database constraint violation. Please check your input data.")
                return None
                
            # Re-raise unhandled exceptions
            raise
    
    return wrapper

# Simulated database functions and exceptions
class DatabaseError(Exception):
    """Base class for database errors."""
    pass

class ConnectionError(DatabaseError):
    """Database connection errors."""
    def __init__(self, message, code=None, severity=None):
        super().__init__(message)
        self.code = code
        self.severity = severity

class QueryError(DatabaseError):
    """Database query errors."""
    def __init__(self, message, query=None, params=None):
        super().__init__(message)
        self.query = query
        self.params = params

# Simulated database functions
@handle_database_error
def connect_to_database(host, user, password):
    """Simulate connecting to a database."""
    if not host:
        raise ConnectionError("No host specified", code='param')
    
    if user == "invalid":
        raise ConnectionError("Authentication failed", code='auth', severity='high')
    
    if host == "slow-server":
        raise ConnectionError("Connection timed out", code='timeout', severity='medium')
    
    print(f"Connected to database at {host}")
    return {"connection": "object"}

@handle_database_error
def execute_query(connection, query, params=None):
    """Simulate executing a database query."""
    if not connection:
        raise ConnectionError("Not connected to database", code='state')
    
    if not query:
        raise QueryError("Empty query", query=query, params=params)
    
    if "INSERT" in query and params and 'value' in params and params['value'] > 100:
        raise ValueError("Constraint violation: value must be <= 100")
    
    print(f"Executed query: {query}")
    return ["result1", "result2"]

# Demonstrate exception filtering
def demonstrate_exception_filtering():
    # Test with different scenarios
    print("Scenario 1: Valid connection")
    conn = connect_to_database("db-server", "user", "password")
    print()
    
    print("Scenario 2: Authentication failure")
    conn = connect_to_database("db-server", "invalid", "password")
    print()
    
    print("Scenario 3: Connection timeout")
    conn = connect_to_database("slow-server", "user", "password")
    print()
    
    print("Scenario 4: Valid query")
    if conn:
        results = execute_query(conn, "SELECT * FROM users")
    print()
    
    print("Scenario 5: Constraint violation")
    if conn:
        results = execute_query(conn, "INSERT INTO items VALUES (:value)", {'value': 200})
    print()
    
    print("Scenario 6: Unhandled exception")
    try:
        if conn:
            results = execute_query(None, "SELECT * FROM users")
    except Exception as e:
        print(f"Caught unhandled exception: {e}")

# Run the demonstration
demonstrate_exception_filtering()
                

Exception filtering lets you handle exceptions based not just on their type but also on their attributes or other conditions. This gives you fine-grained control over exception handling.

When to use exception filtering:

  • When you need to differentiate between exceptions of the same type
  • When exception details determine how to handle the error
  • When working with APIs that use rich exception objects
  • For implementing more sophisticated error recovery strategies

This is like a doctor not just identifying that you have an infection, but determining exactly what kind of infection and prescribing the appropriate treatment based on its specific characteristics.

Real-World Examples of Multiple Exception Handling

Let's explore some practical examples of handling multiple exceptions in common programming scenarios.

File Processing with Multiple Error Conditions

# File: practical/file_processing.py

import os
import json
import csv
from datetime import datetime

def process_data_file(input_file, output_file, error_log=None):
    """
    Process a data file and write the results to an output file.
    
    Args:
        input_file: Path to the input file (JSON or CSV)
        output_file: Path to the output file
        error_log: Optional path to an error log file
    
    Returns:
        A dictionary with processing statistics
    """
    stats = {
        'input_file': input_file,
        'output_file': output_file,
        'start_time': datetime.now(),
        'end_time': None,
        'records_processed': 0,
        'records_succeeded': 0,
        'records_failed': 0,
        'status': 'failed',  # Default to failed, update on success
        'error': None
    }
    
    # Create or open the error log file if specified
    error_log_file = None
    if error_log:
        try:
            error_log_file = open(error_log, 'a')
            error_log_file.write(f"\n\n--- Processing {input_file} at {stats['start_time']} ---\n")
        except Exception as e:
            print(f"Warning: Could not open error log file: {e}")
    
    try:
        # Make sure the output directory exists
        output_dir = os.path.dirname(output_file)
        if output_dir and not os.path.exists(output_dir):
            os.makedirs(output_dir)
        
        # Determine the file type based on extension
        file_extension = os.path.splitext(input_file)[1].lower()
        
        # Read the input file
        data = []
        
        if file_extension == '.json':
            # Handle JSON file
            try:
                with open(input_file, 'r') as f:
                    data = json.load(f)
                    
                    # Ensure data is a list
                    if not isinstance(data, list):
                        if isinstance(data, dict) and 'records' in data:
                            # Handle common format where data is in a "records" field
                            data = data['records']
                        else:
                            # Wrap single object in a list
                            data = [data]
            
            except json.JSONDecodeError as e:
                raise ValueError(f"Invalid JSON format: {e}")
        
        elif file_extension == '.csv':
            # Handle CSV file
            try:
                with open(input_file, 'r', newline='') as f:
                    reader = csv.DictReader(f)
                    data = list(reader)
            
            except csv.Error as e:
                raise ValueError(f"Invalid CSV format: {e}")
        
        else:
            raise ValueError(f"Unsupported file type: {file_extension}")
        
        # Process the data
        output_data = []
        
        for i, record in enumerate(data):
            try:
                # Process the record (in a real system, this would do something useful)
                processed_record = {
                    'id': record.get('id', i),
                    'timestamp': datetime.now().isoformat(),
                    'original_data': record,
                    'processed': True
                }
                
                output_data.append(processed_record)
                stats['records_succeeded'] += 1
            
            except Exception as e:
                # Log the error for this record
                error_message = f"Error processing record {i}: {e}"
                
                if error_log_file:
                    error_log_file.write(f"{error_message}\n")
                else:
                    print(error_message)
                
                stats['records_failed'] += 1
                
                # Include the failed record with error information
                output_data.append({
                    'id': record.get('id', i),
                    'timestamp': datetime.now().isoformat(),
                    'original_data': record,
                    'processed': False,
                    'error': str(e)
                })
            
            # Update the records processed count
            stats['records_processed'] += 1
        
        # Write the output data
        output_extension = os.path.splitext(output_file)[1].lower()
        
        if output_extension == '.json':
            with open(output_file, 'w') as f:
                json.dump(output_data, f, indent=2)
        
        elif output_extension == '.csv':
            if output_data:
                # Get all possible field names from all records
                fieldnames = set()
                for record in output_data:
                    fieldnames.update(record.keys())
                
                with open(output_file, 'w', newline='') as f:
                    writer = csv.DictWriter(f, fieldnames=sorted(fieldnames))
                    writer.writeheader()
                    writer.writerows(output_data)
        
        else:
            raise ValueError(f"Unsupported output file type: {output_extension}")
        
        # Update statistics
        stats['status'] = 'success'
    
    except FileNotFoundError as e:
        stats['error'] = f"File not found: {e}"
        if error_log_file:
            error_log_file.write(f"ERROR: {stats['error']}\n")
        raise
    
    except PermissionError as e:
        stats['error'] = f"Permission denied: {e}"
        if error_log_file:
            error_log_file.write(f"ERROR: {stats['error']}\n")
        raise
    
    except ValueError as e:
        stats['error'] = f"Value error: {e}"
        if error_log_file:
            error_log_file.write(f"ERROR: {stats['error']}\n")
        raise
    
    except Exception as e:
        stats['error'] = f"Unexpected error: {e}"
        if error_log_file:
            error_log_file.write(f"ERROR: {stats['error']}\n")
        raise
    
    finally:
        # Update end time
        stats['end_time'] = datetime.now()
        
        # Close the error log file if it was opened
        if error_log_file:
            error_log_file.write(f"Completed with status: {stats['status']}\n")
            error_log_file.write(f"Processed {stats['records_processed']} records: "
                              f"{stats['records_succeeded']} succeeded, "
                              f"{stats['records_failed']} failed\n")
            error_log_file.write(f"Total time: {stats['end_time'] - stats['start_time']}\n")
            error_log_file.close()
    
    return stats

# Function to demonstrate the file processor
def demonstrate_file_processor():
    # Create some sample data for testing
    sample_data = [
        {"id": 1, "name": "Alice", "score": 95},
        {"id": 2, "name": "Bob", "score": "invalid"},  # Will cause an error during processing
        {"id": 3, "name": "Charlie", "score": 85}
    ]
    
    # Write sample data to a JSON file
    os.makedirs("sample_data", exist_ok=True)
    with open("sample_data/input.json", 'w') as f:
        json.dump(sample_data, f)
    
    # Write sample data to a CSV file
    with open("sample_data/input.csv", 'w', newline='') as f:
        writer = csv.DictWriter(f, fieldnames=["id", "name", "score"])
        writer.writeheader()
        writer.writerows(sample_data)
    
    # Process the files
    print("Processing JSON file:")
    try:
        stats = process_data_file(
            "sample_data/input.json",
            "sample_data/output.json",
            "sample_data/errors.log"
        )
        print(f"Processing completed with status: {stats['status']}")
        print(f"Processed {stats['records_processed']} records: "
              f"{stats['records_succeeded']} succeeded, "
              f"{stats['records_failed']} failed")
    except Exception as e:
        print(f"Processing failed: {e}")
    
    print("\nProcessing CSV file:")
    try:
        stats = process_data_file(
            "sample_data/input.csv",
            "sample_data/output.csv",
            "sample_data/errors.log"
        )
        print(f"Processing completed with status: {stats['status']}")
        print(f"Processed {stats['records_processed']} records: "
              f"{stats['records_succeeded']} succeeded, "
              f"{stats['records_failed']} failed")
    except Exception as e:
        print(f"Processing failed: {e}")
    
    print("\nTesting error handling with nonexistent file:")
    try:
        stats = process_data_file(
            "sample_data/nonexistent.json",
            "sample_data/output.json",
            "sample_data/errors.log"
        )
    except FileNotFoundError as e:
        print(f"Expected error caught: {e}")
    
    print("\nTesting error handling with invalid JSON:")
    # Create an invalid JSON file
    with open("sample_data/invalid.json", 'w') as f:
        f.write("{this is not valid JSON}")
    
    try:
        stats = process_data_file(
            "sample_data/invalid.json",
            "sample_data/output.json",
            "sample_data/errors.log"
        )
    except ValueError as e:
        print(f"Expected error caught: {e}")

# Run the demonstration
# demonstrate_file_processor()
                

This example demonstrates a robust file processing system that handles multiple types of exceptions at different levels. It shows how to:

  • Handle file-level errors (missing files, permission issues)
  • Handle format-specific parsing errors (invalid JSON, CSV)
  • Handle record-level errors to allow processing to continue
  • Log errors for later analysis
  • Provide detailed statistics on the processing results

This pattern is common in data processing pipelines, ETL (Extract, Transform, Load) processes, and batch processing systems.

Network Operations with Multiple Failure Modes

# File: practical/network_operations.py

import socket
import json
import time
import random
from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError
from urllib.parse import urlencode

class APIError(Exception):
    """Base class for API-related errors."""
    pass

class ConnectionFailure(APIError):
    """Raised when connection to the API fails."""
    pass

class AuthenticationError(APIError):
    """Raised when authentication with the API fails."""
    pass

class RateLimitError(APIError):
    """Raised when the API rate limit is exceeded."""
    pass

class APIClientError(APIError):
    """Raised when the API reports a client error (4xx status code)."""
    pass

class APIServerError(APIError):
    """Raised when the API reports a server error (5xx status code)."""
    pass

class SimpleAPIClient:
    """
    A simple API client that demonstrates handling multiple network-related exceptions.
    """
    
    def __init__(self, base_url, api_key=None, timeout=10, max_retries=3):
        self.base_url = base_url.rstrip('/')
        self.api_key = api_key
        self.timeout = timeout
        self.max_retries = max_retries
    
    def request(self, endpoint, method='GET', params=None, data=None, headers=None):
        """
        Send a request to the API.
        
        Args:
            endpoint: API endpoint (without base URL)
            method: HTTP method (GET, POST, PUT, DELETE)
            params: Query parameters
            data: Request body data for POST/PUT
            headers: Custom headers
            
        Returns:
            The parsed JSON response
            
        Raises:
            Various APIError subclasses depending on what went wrong
        """
        # Build the full URL
        url = f"{self.base_url}/{endpoint.lstrip('/')}"
        
        # Add query parameters
        if params:
            query_string = urlencode(params)
            url = f"{url}?{query_string}"
        
        # Prepare headers
        request_headers = headers or {}
        
        if self.api_key:
            request_headers['Authorization'] = f"Bearer {self.api_key}"
        
        if data:
            request_headers['Content-Type'] = 'application/json'
        
        # Convert data to JSON if needed
        json_data = None
        if data:
            json_data = json.dumps(data).encode('utf-8')
        
        # Create the request
        request = Request(
            url=url,
            data=json_data,
            headers=request_headers,
            method=method
        )
        
        # Try to send the request with retries
        retries = 0
        
        while True:
            try:
                # Attempt to send the request
                response = urlopen(request, timeout=self.timeout)
                
                # Parse the response
                response_data = response.read().decode('utf-8')
                
                # Return the parsed JSON data
                return json.loads(response_data) if response_data else {}
                
            except HTTPError as e:
                # Handle HTTP error responses
                if e.code == 401 or e.code == 403:
                    raise AuthenticationError(f"Authentication failed: {e}")
                
                elif e.code == 429:
                    if retries < self.max_retries:
                        # Rate limited, wait and retry
                        retry_after = int(e.headers.get('Retry-After', 5))
                        print(f"Rate limited. Waiting {retry_after} seconds before retry...")
                        time.sleep(retry_after)
                        retries += 1
                        continue
                    else:
                        raise RateLimitError(f"API rate limit exceeded: {e}")
                
                elif 400 <= e.code < 500:
                    # Client errors
                    raise APIClientError(f"Client error (HTTP {e.code}): {e}")
                
                elif 500 <= e.code < 600:
                    # Server errors
                    if retries < self.max_retries:
                        # Server error, wait and retry
                        backoff = (2 ** retries) + random.uniform(0, 1)
                        print(f"Server error. Retrying in {backoff:.2f} seconds...")
                        time.sleep(backoff)
                        retries += 1
                        continue
                    else:
                        raise APIServerError(f"Server error (HTTP {e.code}): {e}")
                
                # Re-raise any other HTTP errors
                raise APIError(f"HTTP error: {e}")
                
            except URLError as e:
                # Handle URL errors (connection problems)
                if isinstance(e.reason, socket.timeout):
                    if retries < self.max_retries:
                        # Timeout, wait and retry
                        backoff = (2 ** retries) + random.uniform(0, 1)
                        print(f"Request timed out. Retrying in {backoff:.2f} seconds...")
                        time.sleep(backoff)
                        retries += 1
                        continue
                    else:
                        raise ConnectionFailure(f"Connection timed out after {self.max_retries} retries")
                
                elif "connection refused" in str(e.reason).lower():
                    raise ConnectionFailure(f"Connection refused: {e.reason}")
                
                # Re-raise other URL errors
                raise ConnectionFailure(f"Connection failed: {e.reason}")
                
            except socket.timeout:
                # Handle socket timeouts
                if retries < self.max_retries:
                    # Timeout, wait and retry
                    backoff = (2 ** retries) + random.uniform(0, 1)
                    print(f"Request timed out. Retrying in {backoff:.2f} seconds...")
                    time.sleep(backoff)
                    retries += 1
                    continue
                else:
                    raise ConnectionFailure(f"Connection timed out after {self.max_retries} retries")
                
            except json.JSONDecodeError as e:
                # Handle invalid JSON responses
                raise APIError(f"Invalid JSON response: {e}")
                
            except Exception as e:
                # Handle any other unexpected errors
                raise APIError(f"Unexpected error: {e}")
    
    def get(self, endpoint, params=None, headers=None):
        """Send a GET request to the API."""
        return self.request(endpoint, method='GET', params=params, headers=headers)
    
    def post(self, endpoint, data=None, params=None, headers=None):
        """Send a POST request to the API."""
        return self.request(endpoint, method='POST', params=params, data=data, headers=headers)
    
    def put(self, endpoint, data=None, params=None, headers=None):
        """Send a PUT request to the API."""
        return self.request(endpoint, method='PUT', params=params, data=data, headers=headers)
    
    def delete(self, endpoint, params=None, headers=None):
        """Send a DELETE request to the API."""
        return self.request(endpoint, method='DELETE', params=params, headers=headers)

# Function to demonstrate the API client
def demonstrate_api_client():
    # Create an API client for a public test API
    client = SimpleAPIClient(
        base_url="https://jsonplaceholder.typicode.com",
        timeout=5,
        max_retries=2
    )
    
    # Example 1: Successful GET request
    try:
        print("Example 1: Successful GET request")
        response = client.get("/posts/1")
        print(f"Response: {response}")
    except APIError as e:
        print(f"API Error: {e}")
    print()
    
    # Example 2: Resource not found (404)
    try:
        print("Example 2: Resource not found (404)")
        response = client.get("/nonexistent-endpoint")
        print(f"Response: {response}")
    except APIClientError as e:
        print(f"Expected client error: {e}")
    except APIError as e:
        print(f"API Error: {e}")
    print()
    
    # Example 3: POST request
    try:
        print("Example 3: POST request")
        new_post = {
            "title": "Test Post",
            "body": "This is a test post",
            "userId": 1
        }
        response = client.post("/posts", data=new_post)
        print(f"Response: {response}")
    except APIError as e:
        print(f"API Error: {e}")
    print()
    
    # Example 4: Connection failure (invalid hostname)
    try:
        print("Example 4: Connection failure (invalid hostname)")
        bad_client = SimpleAPIClient("https://nonexistent-api-host.invalid")
        response = bad_client.get("/endpoint")
        print(f"Response: {response}")
    except ConnectionFailure as e:
        print(f"Expected connection failure: {e}")
    except APIError as e:
        print(f"API Error: {e}")
    print()
    
    # Example 5: Handle a timeout
    try:
        print("Example 5: Handle a timeout")
        # Simulate a timeout by using a very short timeout value
        timeout_client = SimpleAPIClient(
            base_url="https://httpbin.org/delay/3",  # This endpoint delays 3 seconds
            timeout=1,  # Only wait 1 second
            max_retries=1
        )
        response = timeout_client.get("")
        print(f"Response: {response}")
    except ConnectionFailure as e:
        print(f"Expected timeout: {e}")
    except APIError as e:
        print(f"API Error: {e}")

# Run the demonstration
# demonstrate_api_client()
                

This example demonstrates handling the many types of errors that can occur during network operations and API calls, including:

  • Connection failures (timeouts, refused connections)
  • HTTP errors (client errors, server errors)
  • Authentication failures
  • Rate limiting
  • JSON parsing errors

The example also shows how to implement retry logic for transient errors, with exponential backoff to avoid overwhelming the server.

Best Practices for Handling Multiple Exceptions

Key Principles

  1. Order exceptions from most specific to most general - Always catch the most specific exceptions first, followed by more general ones.
  2. Don't catch exceptions you can't handle properly - If you can't take appropriate action for an exception, let it propagate up the call stack.
  3. Keep error handling separate from business logic - This makes both the error handling and the main code easier to understand.
  4. Use custom exceptions for domain-specific errors - Create a hierarchy of exception classes that makes sense for your application.
  5. Be specific about which exceptions you catch - Avoid catching Exception or worse, all exceptions with a bare except: clause.
  6. Always clean up resources - Use finally blocks or context managers (with statement) to ensure resources are properly released.
  7. Provide context in exceptions - Include relevant information that helps understand what went wrong and how to fix it.
  8. Consider retries for transient errors - Some errors (network timeouts, rate limiting) are temporary and can be resolved by trying again.
  9. Log exceptions with appropriate detail - Include enough information to debug the issue, but be careful about sensitive data.
  10. Use appropriate abstraction levels - Lower-level exceptions should be caught and translated to higher-level exceptions that make sense in the current context.

Common Anti-Patterns to Avoid

  • Bare except: clauses - This catches all exceptions including keyboard interrupts and system exits.
  • Catching Exception too broadly - Unless you're at a very high level in your application, this probably catches too much.
  • Empty except blocks - Silently ignoring errors makes debugging nearly impossible.
  • Long code blocks in try - Keep try blocks focused on the specific operations that might raise exceptions.
  • Handling exceptions in the wrong place - Catch exceptions where you have enough context to handle them properly.
  • Raising string exceptions - Always raise instances of the Exception class or its subclasses, not strings.
  • Catching and re-raising without adding value - If you're just going to re-raise, consider whether you need to catch the exception at all.
  • Using exceptions for flow control - Exceptions should be for exceptional conditions, not normal program flow.

Before and After: Improving Exception Handling

# BEFORE: Problematic exception handling
def problematic_function(filename, value):
    try:
        # Long block with multiple potential exceptions
        file = open(filename, 'r')
        data = file.read()
        result = int(data) / value
        file.close()
        return result
    except:
        # Bare except, no specific handling
        print("An error occurred")
        # No cleanup
        return None

# AFTER: Improved exception handling
def improved_function(filename, value):
    try:
        # Using a context manager for proper resource handling
        with open(filename, 'r') as file:
            data = file.read()
        
        # Separate try block for different operation
        try:
            return int(data) / value
        except ValueError:
            # Specific handling for integer conversion
            print(f"Error: File does not contain a valid integer: {data}")
            return None
        except ZeroDivisionError:
            # Specific handling for division by zero
            print("Error: Cannot divide by zero")
            return float('inf')  # Or another appropriate default
            
    except FileNotFoundError:
        # Specific handling for missing file
        print(f"Error: File not found: {filename}")
        return None
    except PermissionError:
        # Specific handling for permission issues
        print(f"Error: Permission denied for file: {filename}")
        return None
    except Exception as e:
        # General handler as last resort, with useful information
        print(f"Unexpected error processing file {filename}: {e}")
        return None

# Test both functions
def compare_functions():
    print("Testing problematic function:")
    print(f"Missing file: {problematic_function('nonexistent.txt', 5)}")
    print(f"Invalid data: {problematic_function('invalid_data.txt', 5)}")
    print(f"Zero value: {problematic_function('valid_data.txt', 0)}")
    print()
    
    print("Testing improved function:")
    print(f"Missing file: {improved_function('nonexistent.txt', 5)}")
    print(f"Invalid data: {improved_function('invalid_data.txt', 5)}")
    print(f"Zero value: {improved_function('valid_data.txt', 0)}")

# Create test files
def create_test_files():
    with open('valid_data.txt', 'w') as f:
        f.write('10')
    
    with open('invalid_data.txt', 'w') as f:
        f.write('not a number')

# Uncomment to run
# create_test_files()
# compare_functions()
                

This example contrasts poor exception handling with a better approach, highlighting several key improvements:

  • Using context managers (with statement) for resource management
  • Separating different operations into different try/except blocks
  • Catching specific exceptions rather than using a bare except
  • Providing informative error messages
  • Handling different error conditions differently
  • Using Exception as a last resort, not the primary handler

Exercises to Reinforce Learning

Exercise 1: Database Connection Wrapper

Create a database connection wrapper class that handles multiple types of database-related exceptions, providing a unified interface for error handling and retries.

# File: exercises/database_connection.py

class DatabaseConnection:
    """
    A wrapper for database connections with unified error handling.
    
    Your task:
    1. Implement the connect, execute, and close methods
    2. Add appropriate exception handling for different database error scenarios
    3. Implement retry logic for transient errors
    4. Ensure proper resource cleanup
    """
    
    def __init__(self, connection_string, max_retries=3):
        self.connection_string = connection_string
        self.max_retries = max_retries
        self.connection = None
    
    def connect(self):
        """
        Establish a connection to the database.
        
        Returns:
            True if connection successful, False otherwise
        
        Raises:
            Various exceptions depending on what went wrong
        """
        # Your implementation here
        pass
    
    def execute(self, query, params=None):
        """
        Execute a query on the database.
        
        Args:
            query: SQL query to execute
            params: Parameters for the query
            
        Returns:
            Query results
            
        Raises:
            Various exceptions depending on what went wrong
        """
        # Your implementation here
        pass
    
    def close(self):
        """
        Close the database connection.
        """
        # Your implementation here
        pass
    
    def __enter__(self):
        """Support for use as a context manager."""
        self.connect()
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        """Clean up resources when used as a context manager."""
        # Your implementation here
        pass
                

Exercise 2: Robust Configuration Manager

Create a configuration manager that can read configuration from multiple sources (environment variables, config files, defaults) and handle various error conditions gracefully.

# File: exercises/config_manager.py

class ConfigManager:
    """
    A configuration manager that reads from multiple sources with error handling.
    
    Your task:
    1. Implement methods to read configuration from different sources
    2. Handle different types of errors (missing files, invalid formats, etc.)
    3. Implement fallback mechanisms
    4. Provide meaningful error messages
    """
    
    def __init__(self, app_name, config_files=None, env_prefix=None):
        self.app_name = app_name
        self.config_files = config_files or []
        self.env_prefix = env_prefix or app_name.upper()
        self.config = {}
    
    def load_config(self):
        """
        Load configuration from all sources.
        
        Returns:
            The merged configuration dictionary
        """
        # Your implementation here
        pass
    
    def load_from_file(self, filename):
        """
        Load configuration from a file.
        
        Args:
            filename: Path to the configuration file
            
        Returns:
            Configuration dictionary from the file
            
        Raises:
            Various exceptions depending on what went wrong
        """
        # Your implementation here
        pass
    
    def load_from_env(self):
        """
        Load configuration from environment variables.
        
        Returns:
            Configuration dictionary from environment variables
        """
        # Your implementation here
        pass
    
    def get(self, key, default=None):
        """
        Get a configuration value.
        
        Args:
            key: Configuration key
            default: Default value if key not found
            
        Returns:
            The configuration value or default
        """
        # Your implementation here
        pass
    
    def set(self, key, value):
        """
        Set a configuration value.
        
        Args:
            key: Configuration key
            value: Configuration value
        """
        # Your implementation here
        pass
    
    def save(self, filename=None):
        """
        Save the current configuration to a file.
        
        Args:
            filename: Path to the output file (optional)
            
        Returns:
            True if successful, False otherwise
        """
        # Your implementation here
        pass
                

Exercise 3: HTTP Request Validator

Create a function that validates HTTP requests, checking various aspects of the request and raising appropriate exceptions for different validation failures.

# File: exercises/request_validator.py

class ValidationError(Exception):
    """Base class for request validation errors."""
    pass

class InvalidMethodError(ValidationError):
    """Raised when the HTTP method is not allowed."""
    pass

class InvalidContentTypeError(ValidationError):
    """Raised when the content type is not supported."""
    pass

class MissingRequiredFieldError(ValidationError):
    """Raised when a required field is missing."""
    pass

class InvalidFieldValueError(ValidationError):
    """Raised when a field has an invalid value."""
    pass

class RequestTooLargeError(ValidationError):
    """Raised when the request payload is too large."""
    pass

def validate_request(method, headers, body, allowed_methods=None, max_size=None, required_fields=None, field_validators=None):
    """
    Validate an HTTP request.
    
    Args:
        method: HTTP method (GET, POST, etc.)
        headers: Dictionary of HTTP headers
        body: Request body (dictionary)
        allowed_methods: List of allowed HTTP methods
        max_size: Maximum allowed request body size
        required_fields: List of required fields in the body
        field_validators: Dictionary mapping field names to validator functions
    
    Returns:
        None if validation succeeds
    
    Raises:
        Various ValidationError subclasses depending on what's invalid
    """
    # Your implementation here
    pass

# Example validator functions
def validate_email(value):
    """Validator for email fields."""
    if '@' not in value:
        raise InvalidFieldValueError("Invalid email format")

def validate_age(value):
    """Validator for age fields."""
    try:
        age = int(value)
        if age < 0 or age > 120:
            raise InvalidFieldValueError("Age must be between 0 and 120")
    except (ValueError, TypeError):
        raise InvalidFieldValueError("Age must be a number")

# Test the validator
def test_validator():
    """Test cases for the request validator."""
    
    # Example 1: Valid request
    try:
        validate_request(
            method="POST",
            headers={"Content-Type": "application/json"},
            body={"name": "Alice", "email": "alice@example.com", "age": 30},
            allowed_methods=["POST", "PUT"],
            max_size=1024,
            required_fields=["name", "email"],
            field_validators={"email": validate_email, "age": validate_age}
        )
        print("Example 1: Valid request passed validation")
    except ValidationError as e:
        print(f"Example 1 failed: {e}")
    
    # Example 2: Invalid method
    try:
        validate_request(
            method="DELETE",
            headers={"Content-Type": "application/json"},
            body={"name": "Alice", "email": "alice@example.com"},
            allowed_methods=["POST", "PUT"],
            required_fields=["name", "email"]
        )
        print("Example 2: Should have failed but didn't")
    except InvalidMethodError as e:
        print(f"Example 2: Correctly caught {type(e).__name__}: {e}")
    except ValidationError as e:
        print(f"Example 2: Caught wrong error type: {type(e).__name__}: {e}")
    
    # Additional test cases for other validation errors...
    
# Uncomment to run the tests
# test_validator()
                

Summary

In this comprehensive guide to handling multiple exceptions in Python, we've covered:

  • The basic approaches to handling different exception types using multiple except blocks and exception tuples
  • Leveraging Python's exception hierarchy for more elegant error handling
  • Advanced techniques like exception chaining, unified error handling, and exception filtering
  • Real-world examples of handling multiple exceptions in file processing and network operations
  • Best practices and common anti-patterns in exception handling

Effective exception handling is key to building robust, maintainable Python applications. By understanding the different approaches and when to use them, you can write code that gracefully handles errors, provides helpful feedback, and ensures your applications remain stable even when unexpected situations arise.

Remember that good exception handling is not just about preventing crashes—it's about making your code more resilient, easier to debug, and ultimately more reliable. By applying the principles and techniques covered in this guide, you'll be well-equipped to handle the diverse range of errors that can occur in real-world applications.