Python Full Stack Web Developer Course

Week 2: Thursday Afternoon - Error Handling with try/except

Introduction to Error Handling

Welcome to our deep dive into error handling in Python! Today, we'll explore one of the most practical and essential aspects of programming: how to gracefully handle errors and exceptions. This skill separates novice programmers from professionals - no matter how well-designed your code is, unexpected situations will occur in real-world applications.

This tutorial will be stored in your course files as: /week2_thursday_error_handling.html

Understanding Errors and Exceptions

Before we dive into handling errors, let's understand what they are. In Python, there are generally two kinds of errors:

Syntax errors need to be fixed before your code will run, but exceptions can be caught and handled, allowing your program to continue running even when something goes wrong.

Real-world Analogy: Driving a Car

Think of exceptions like unexpected events while driving:

  • A syntax error is like trying to start a car with no key - you won't even get moving.
  • An exception is like a flat tire while driving - an unexpected problem that requires handling.
  • Exception handling is like having a spare tire and knowing how to change it - you've prepared for the problem and can continue your journey.

Why Error Handling Matters

Error handling is not just about preventing crashes - it's about creating robust, user-friendly software. Here's why it's crucial:

Real-world Application: Web Form Submission

When a user submits a form on your website:

  • Without error handling: If the database connection fails, the user sees a server error page with confusing technical details, has no idea if their data was saved, and your application crashes.
  • With error handling: If the database connection fails, the user sees a friendly message like "We couldn't process your submission right now. Your data has been saved and we'll try again soon." Your application logs the error and continues serving other users.

Basic try/except Structure

Python's primary mechanism for handling exceptions is the try/except block:

try:
    # Code that might raise an exception
    result = 10 / 0  # This will raise a ZeroDivisionError
except:
    # Code to handle the exception
    print("An error occurred!")

The basic structure works like this:

  1. The code inside the try block is executed.
  2. If no exception occurs, the except block is skipped.
  3. If an exception occurs, the rest of the try block is skipped and the except block is executed.
  4. After either case, execution continues after the try/except block.

Warning: Avoid Bare Except!

Using a bare except: without specifying which exceptions to catch is generally considered bad practice. It can hide bugs and catch exceptions you didn't intend to handle. Always specify which exceptions you expect to catch.

Catching Specific Exceptions

In practice, you should always catch specific exceptions rather than using a bare except:

try:
    number = int(input("Enter a number: "))
    result = 10 / number
    print(f"10 divided by {number} is {result}")
except ZeroDivisionError:
    print("You can't divide by zero!")
except ValueError:
    print("That's not a valid number!")

This approach has several advantages:

Common Built-in Exceptions

Exception Description Example Scenario
ValueError Inappropriate value for operation Converting a string like "hello" to an integer
TypeError Operation or function applied to inappropriate type Adding a string to an integer without conversion
ZeroDivisionError Division or modulo by zero Dividing any number by zero
FileNotFoundError Trying to access a file that doesn't exist Opening a file that was deleted or moved
IndexError Sequence index out of range Accessing list[10] in a list with only 5 elements
KeyError Dictionary key not found Accessing dict['key'] when 'key' isn't in the dictionary
AttributeError Object has no such attribute or method Calling string.append() when strings don't have an append method

The Exception Hierarchy

Python's exceptions form a hierarchy, with the base class BaseException at the top, and Exception as the parent class for most exceptions you'll catch. This hierarchy is important because when you catch an exception, you also catch all its subclasses.

try:
    # This code could raise various types of exceptions
    with open("data.txt", "r") as file:
        data = file.read()
        value = int(data)
        result = 100 / value
except (ValueError, ZeroDivisionError):
    # Catch multiple specific exceptions
    print("Invalid data in file!")
except OSError as e:
    # The 'as' clause captures the exception object
    print(f"File error: {e}")

Exception Hierarchy (Simplified)

  • BaseException
    • SystemExit
    • KeyboardInterrupt
    • Exception (almost all other exceptions derive from this)
      • ArithmeticError
        • ZeroDivisionError
        • OverflowError
      • LookupError
        • IndexError
        • KeyError
      • OSError
        • FileNotFoundError
        • PermissionError
      • ValueError
      • TypeError
      • And many more...

Note: When catching exceptions, order matters! Always catch more specific exceptions before more general ones.

The else and finally Clauses

The try/except statement can include two additional clauses: else and finally.

The else Clause

The else clause executes only if the try block completes without raising an exception:

try:
    number = int(input("Enter a number: "))
    result = 100 / number
except ValueError:
    print("That's not a valid number!")
except ZeroDivisionError:
    print("You can't divide by zero!")
else:
    # This runs only if no exceptions were raised
    print(f"The result is {result}")
    # You can also put code here that might raise different exceptions

The else clause is useful when you want some code to run only if no exceptions occur, but you still want to catch exceptions in the try block.

The finally Clause

The finally clause always executes, whether an exception was raised or not (even if there's a return statement in the try or except blocks):

try:
    file = open("important_data.txt", "w")
    file.write("Critical information")
    # Some code that might raise an exception
    result = calculate_something()
except Exception as e:
    print(f"An error occurred: {e}")
finally:
    # This ALWAYS runs, ensuring the file is closed
    file.close()
    print("File has been closed")

The finally clause is perfect for cleanup code that must run regardless of whether an exception occurred.

Try/Except/Else/Finally Analogy: Baking a Cake

  • try: Attempt to bake a cake following a recipe
  • except: Handle specific problems (out of eggs? use a substitute)
  • else: If the cake baked perfectly, add frosting and decoration
  • finally: No matter what happened, clean up the kitchen

Complete Example: File Processing

Let's pull everything together in a practical example of processing a data file:

def process_data_file(filename):
    processed_data = []
    file = None
    
    try:
        # Attempt to open and process the file
        file = open(filename, "r")
        
        for line_number, line in enumerate(file, 1):
            try:
                # Process each line (might fail for various reasons)
                data = line.strip().split(',')
                if len(data) != 3:
                    raise ValueError(f"Line {line_number}: Expected 3 values but got {len(data)}")
                
                name = data[0]
                age = int(data[1])
                score = float(data[2])
                
                if age < 0 or age > 120:
                    raise ValueError(f"Line {line_number}: Invalid age {age}")
                
                if score < 0 or score > 100:
                    raise ValueError(f"Line {line_number}: Invalid score {score}")
                
                processed_data.append({
                    'name': name,
                    'age': age,
                    'score': score
                })
                
            except ValueError as e:
                # Handle data format errors for this line
                print(f"Error processing line {line_number}: {e}")
                # Continue processing the next line
            
    except FileNotFoundError:
        print(f"The file {filename} does not exist.")
    except PermissionError:
        print(f"No permission to read the file {filename}.")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")
    else:
        print(f"Successfully processed {len(processed_data)} records.")
        return processed_data
    finally:
        # Ensure file is closed even if an exception occurred
        if file is not None:
            file.close()
            print(f"File {filename} has been closed.")
    
    # This only executes if an exception occurred in the main try block
    return None

# Example usage
data = process_data_file("student_data.txt")

This example demonstrates several key concepts:

Raising Exceptions

Sometimes you need to raise exceptions yourself, either to report errors in your code or to re-raise caught exceptions:

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError("Cannot divide by zero")
    return a / b

def calculate_average(numbers):
    if not numbers:
        raise ValueError("Cannot calculate average of empty list")
    return sum(numbers) / len(numbers)

# Re-raising with additional information
def process_user_data(user_id):
    try:
        user = get_user_from_database(user_id)
        return process_user(user)
    except DatabaseError as e:
        # Add context to the exception
        raise RuntimeError(f"Error processing user {user_id}") from e

The raise ... from ... syntax creates an exception chain, preserving the original exception as the cause of the new one. This is very helpful for debugging because it shows the complete exception history.

When to Raise Exceptions

Creating Custom Exceptions

For complex applications, it's often valuable to create your own exception types:

class ApplicationError(Exception):
    """Base class for all exceptions in this application."""
    pass

class ConfigurationError(ApplicationError):
    """Raised when there's a problem with application configuration."""
    pass

class DatabaseConnectionError(ApplicationError):
    """Raised when the application cannot connect to the database."""
    pass

class UserNotFoundError(ApplicationError):
    """Raised when a requested user doesn't exist."""
    def __init__(self, user_id):
        self.user_id = user_id
        super().__init__(f"User with ID {user_id} not found")

Custom exceptions have several benefits:

Using Custom Exceptions

def get_user(user_id):
    try:
        # Attempt to get user from database
        if not valid_connection():
            raise DatabaseConnectionError("Database connection failed")
        
        user = find_user(user_id)
        if user is None:
            raise UserNotFoundError(user_id)
        
        return user
    
    except ApplicationError as e:
        # We can catch all our custom exceptions together
        log_error(e)
        # Or we could have specific handling for different subclasses
        raise

Context Managers and the with Statement

Python's with statement provides a clean way to handle resource cleanup, even if exceptions occur:

# Without with statement
file = open("data.txt", "r")
try:
    data = file.read()
    # Process data...
finally:
    file.close()

# With the with statement (much cleaner!)
with open("data.txt", "r") as file:
    data = file.read()
    # Process data...
# File is automatically closed, even if an exception occurs

The with statement works with context managers - objects that define __enter__ and __exit__ methods. When the with block ends (normally or due to an exception), the __exit__ method is called, ensuring cleanup.

Common Context Managers

Creating Your Own Context Managers

You can create custom context managers using a class or the contextlib module:

from contextlib import contextmanager

# Using a class
class Timer:
    def __enter__(self):
        import time
        self.start_time = time.time()
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        import time
        self.end_time = time.time()
        print(f"Operation took {self.end_time - self.start_time:.2f} seconds")
        # Return False to let any exceptions propagate
        return False

# Using the contextmanager decorator
@contextmanager
def timer():
    import time
    start_time = time.time()
    try:
        # The yield statement is where the with block's content executes
        yield
    finally:
        end_time = time.time()
        print(f"Operation took {end_time - start_time:.2f} seconds")

# Usage
with Timer():
    # Code to time...
    process_large_file()

# Or with the decorator version
with timer():
    # Code to time...
    process_large_file()

Error Handling Best Practices

Do's

Don'ts

Common Error Handling Patterns

1. Look Before You Leap (LBYL) vs. Easier to Ask Forgiveness than Permission (EAFP)
# LBYL style (check conditions first)
if os.path.exists(filename):
    with open(filename) as f:
        data = f.read()
else:
    data = default_data

# EAFP style (try it and handle exceptions)
try:
    with open(filename) as f:
        data = f.read()
except FileNotFoundError:
    data = default_data

Python generally favors the EAFP style, which can be more concise and avoids race conditions.

2. Reraise Pattern
try:
    result = complex_operation()
except SomeError as e:
    logger.error(f"Operation failed: {e}")
    raise  # Re-raise the caught exception

This pattern lets you take action (like logging) while still propagating the exception.

3. Conversion Pattern
try:
    result = api.get_data()
except APIError as e:
    # Convert to a more appropriate exception for this layer
    raise DataUnavailableError("Could not retrieve data") from e

This pattern translates low-level exceptions into more appropriate ones for your application layer.

Error Handling in Web Development

In web applications, error handling is particularly important. Here are some specific applications:

Form Validation and User Input

def process_user_registration(data):
    try:
        # Validate and process user data
        username = data['username']
        if not username:
            raise ValueError("Username cannot be empty")
        
        email = data['email']
        if '@' not in email:
            raise ValueError("Invalid email format")
        
        age = int(data.get('age', 0))
        if age < 18:
            raise ValueError("User must be at least 18 years old")
        
        # ... more validation
        
        # If everything is valid, create user
        return create_user(username, email, age)
        
    except KeyError as e:
        # Missing required field
        return {"success": False, "error": f"Missing required field: {e}"}
    except ValueError as e:
        # Invalid data format or validation error
        return {"success": False, "error": str(e)}
    except DatabaseError as e:
        # Database error
        log_error(f"Database error during registration: {e}")
        return {"success": False, "error": "Registration failed due to a system error. Please try again later."}
    except Exception as e:
        # Unexpected error
        log_error(f"Unexpected error during registration: {e}")
        return {"success": False, "error": "An unexpected error occurred. Please try again later."}

API Responses and Status Codes

@app.route('/api/users/')
def get_user_api(user_id):
    try:
        user = get_user_from_database(user_id)
        return jsonify(user), 200
    except UserNotFoundError:
        return jsonify({"error": "User not found"}), 404
    except DatabaseConnectionError:
        return jsonify({"error": "Database connection error"}), 503
    except Exception as e:
        log_error(f"Unexpected error in get_user_api: {e}")
        return jsonify({"error": "Internal server error"}), 500

Graceful Fallbacks in Templates

# In your Flask route
@app.route('/dashboard')
def dashboard():
    try:
        user_stats = get_user_statistics()
        recent_activity = get_recent_activity()
        notifications = get_notifications()
    except DataUnavailableError:
        # Fallback to empty data if not available
        user_stats = {}
        recent_activity = []
        notifications = []
    
    return render_template(
        'dashboard.html',
        user_stats=user_stats,
        recent_activity=recent_activity,
        notifications=notifications
    )

Security Considerations

Always be careful with exception messages in production:

  • Don't expose sensitive details in user-facing error messages
  • Log detailed exceptions for debugging, but show users generic messages
  • Be aware that exception stack traces can reveal implementation details
  • Consider using different error handling in development vs. production

Debugging with Exceptions

Exceptions are not just for error handling - they're also valuable debugging tools:

Assertion Statements

def calculate_discount(price, rate):
    # Verify inputs meet expectations
    assert price >= 0, f"Price must be non-negative, got {price}"
    assert 0 <= rate <= 1, f"Rate must be between 0 and 1, got {rate}"
    
    return price * rate

Assertions are checked only when Python is run without the -O (optimize) flag, making them perfect for development-time checks.

Logging Exceptions

import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(levelname)s - %(message)s',
    filename='app.log'
)

try:
    result = complex_calculation()
except Exception as e:
    # Log the exception with traceback
    logging.exception("Error in complex_calculation")
    # This is equivalent to:
    # logging.error("Error in complex_calculation", exc_info=True)
    
    # Handle the exception
    result = fallback_value

Proper logging of exceptions provides crucial information for debugging production issues.

Debugging with the traceback Module

import traceback

try:
    problematic_function()
except Exception as e:
    # Print detailed traceback
    traceback.print_exc()
    
    # Or get traceback as a string for logging/display
    traceback_str = traceback.format_exc()
    log_error(traceback_str)

Practice Exercises

Exercise 1: File Reader

Write a function that safely reads a file, handles potential errors, and returns the content or a default value.

Solution
def safe_read_file(filename, default=""):
    try:
        with open(filename, 'r') as file:
            return file.read()
    except FileNotFoundError:
        print(f"Warning: File {filename} not found.")
        return default
    except PermissionError:
        print(f"Warning: No permission to read {filename}.")
        return default
    except Exception as e:
        print(f"Unexpected error reading {filename}: {e}")
        return default

Exercise 2: Division Calculator

Create a function that divides two numbers, handling all potential errors gracefully.

Solution
def safe_divide():
    try:
        num1 = float(input("Enter the numerator: "))
        num2 = float(input("Enter the denominator: "))
        result = num1 / num2
        print(f"{num1} divided by {num2} is {result}")
        return result
    except ValueError:
        print("Error: Please enter valid numbers.")
    except ZeroDivisionError:
        print("Error: Cannot divide by zero.")
    except KeyboardInterrupt:
        print("\nOperation cancelled by user.")
    except Exception as e:
        print(f"Unexpected error: {e}")
    
    # If we get here, an exception occurred
    return None

Exercise 3: Custom Exception

Create a custom exception for a banking application that validates withdrawals.

Solution
class BankingError(Exception):
    """Base exception for banking operations."""
    pass

class InsufficientFundsError(BankingError):
    """Raised when a withdrawal would result in a negative balance."""
    def __init__(self, account_id, requested, available):
        self.account_id = account_id
        self.requested = requested
        self.available = available
        self.deficit = requested - available
        super().__init__(
            f"Account {account_id} has insufficient funds: "
            f"requested ${requested:.2f}, available ${available:.2f}, "
            f"deficit ${self.deficit:.2f}"
        )

def withdraw(account_id, amount):
    # Get account balance (in a real app, this would be from a database)
    balance = get_account_balance(account_id)
    
    if amount <= 0:
        raise ValueError("Withdrawal amount must be positive")
    
    if balance < amount:
        raise InsufficientFundsError(account_id, amount, balance)
    
    # Perform withdrawal
    new_balance = perform_withdrawal(account_id, amount)
    return new_balance

# Usage
try:
    new_balance = withdraw("ACC123456", 500.00)
    print(f"Withdrawal successful. New balance: ${new_balance:.2f}")
except ValueError as e:
    print(f"Invalid withdrawal: {e}")
except InsufficientFundsError as e:
    print(f"Withdrawal failed: {e}")
    print(f"Would you like to withdraw ${e.available:.2f} instead?")
except BankingError as e:
    print(f"Banking error: {e}")
except Exception as e:
    print(f"Unexpected error: {e}")

Exercise 4: Context Manager

Create a context manager that measures and logs the execution time of a block of code.

Solution
import time
import logging
from contextlib import contextmanager

# Configure logging
logging.basicConfig(level=logging.INFO)

@contextmanager
def timing(description):
    start_time = time.time()
    try:
        yield
    finally:
        elapsed_time = time.time() - start_time
        logging.info(f"{description}: {elapsed_time:.4f} seconds")

# Usage
with timing("Data processing"):
    # Code to time
    data = process_large_dataset()

with timing("API request"):
    # Another operation to time
    response = api.get_data()

Assignment: Robust Data Processing Pipeline

Create a robust data processing pipeline that:

  1. Reads data from a CSV file
  2. Validates and transforms the data
  3. Processes the transformed data
  4. Writes results to an output file
  5. Handles all potential errors gracefully
  6. Creates a comprehensive log of the operation

Requirements:

Tips:

Submission location: /week2_thursday_error_handling_assignment.py

Conclusion

Robust error handling is what separates professional applications from hobby projects. By mastering Python's exception handling, you'll create software that's more reliable, user-friendly, and maintainable.

Remember these key points:

As you develop more complex applications, particularly web applications, these techniques will become an essential part of your toolkit, ensuring your software can handle the unpredictability of the real world.

"Defensive programming is about expecting the unexpected and being prepared for it."