Python Full Stack Web Developer Course

Week 2 Wednesday: Lists and List Comprehensions

Introduction to Lists

Welcome to our deep dive into one of Python's most powerful and versatile data structures: lists. Think of lists as the Swiss Army knife in your Python toolkit - they're incredibly flexible, allowing you to store collections of items in a single container that you can manipulate, iterate through, and transform.

Lists in Python are like shelves in a well-organized pantry. You can store different types of items together, rearrange them at will, add new items, or remove ones you no longer need. This flexibility makes lists foundational to Python programming.

List Fundamentals

Creating Lists

Lists in Python are created using square brackets, with comma-separated values:

# Creating empty lists
empty_list = []
another_empty = list()  # Using the list constructor

# Lists with elements
numbers = [1, 2, 3, 4, 5]
mixed_data = [42, "hello", True, 3.14]
nested_list = [1, [2, 3], [4, [5, 6]]]

Notice how Python lists can contain elements of different data types, including other lists. This heterogeneous nature makes lists incredibly flexible for real-world applications.

List Indexing

Like a bookshelf where each position has a specific number, Python lists use zero-based indexing:

fruits = ["apple", "banana", "cherry", "date", "elderberry"]

# Accessing elements
first_fruit = fruits[0]  # "apple"
third_fruit = fruits[2]  # "cherry"

# Negative indexing (counting from the end)
last_fruit = fruits[-1]  # "elderberry"
second_to_last = fruits[-2]  # "date"

The zero-based indexing is similar to how floors are numbered in many countries - the ground floor is floor 0, not floor 1. This can be confusing if you're coming from languages that use 1-based indexing, but you'll quickly get used to it.

Real-World Analogy: Apartment Building

Think of a list as an apartment building. Each apartment (element) has a unique apartment number (index). In Python's system, the first apartment is apartment 0, the second is apartment 1, and so on. Negative indices are like counting from the roof down - apartment -1 is the top floor, apartment -2 is the second from the top, etc.

List Slicing

One of Python's most powerful features is slice notation, which allows you to access a range of elements:

numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

# Basic slicing: list[start:end] (end is exclusive)
first_three = numbers[0:3]  # [0, 1, 2]
middle = numbers[3:7]  # [3, 4, 5, 6]

# Omitting start/end
beginning = numbers[:5]  # [0, 1, 2, 3, 4] - from start to index 4
end_portion = numbers[7:]  # [7, 8, 9] - from index 7 to end

# With step value: list[start:end:step]
every_second = numbers[::2]  # [0, 2, 4, 6, 8] - every 2nd element
reversed_list = numbers[::-1]  # [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] - reverse

Slicing is like asking for a specific range of books on your bookshelf. The slice [0:3] means "give me items at positions 0, 1, and 2, but stop before position 3."

Practical Usage: Data Processing

Slicing is invaluable when working with large datasets. For example, when analyzing sales data, you might use slicing to isolate quarterly data:

# Monthly sales data for a year
monthly_sales = [45000, 52000, 58000, 60000, 62500, 71000, 
                 73000, 69000, 65000, 58000, 52000, 59000]

# Extract quarterly data
q1_sales = monthly_sales[0:3]  # First quarter
q2_sales = monthly_sales[3:6]  # Second quarter
q3_sales = monthly_sales[6:9]  # Third quarter
q4_sales = monthly_sales[9:]   # Fourth quarter

# Calculate quarterly averages
q1_avg = sum(q1_sales) / len(q1_sales)  # Average sales in Q1

Common List Operations

Modifying Lists

Unlike strings, lists are mutable, meaning you can change their content without creating a new list:

shopping_list = ["milk", "eggs", "bread", "fruit"]

# Changing an element
shopping_list[3] = "apples"  # More specific than just "fruit"
print(shopping_list)  # ["milk", "eggs", "bread", "apples"]

# Adding elements
shopping_list.append("cheese")  # Adds to the end
print(shopping_list)  # ["milk", "eggs", "bread", "apples", "cheese"]

shopping_list.insert(2, "butter")  # Insert at specific position
print(shopping_list)  # ["milk", "eggs", "butter", "bread", "apples", "cheese"]

# Extending with another list
more_items = ["juice", "cereal"]
shopping_list.extend(more_items)  # Adds all items from another list
print(shopping_list)  # ["milk", "eggs", "butter", "bread", "apples", "cheese", "juice", "cereal"]

# Removing elements
shopping_list.remove("eggs")  # Removes first occurrence of value
print(shopping_list)  # ["milk", "butter", "bread", "apples", "cheese", "juice", "cereal"]

last_item = shopping_list.pop()  # Removes and returns last item
print(last_item)  # "cereal"
print(shopping_list)  # ["milk", "butter", "bread", "apples", "cheese", "juice"]

specific_item = shopping_list.pop(1)  # Remove at specific index
print(specific_item)  # "butter"
print(shopping_list)  # ["milk", "bread", "apples", "cheese", "juice"]

# Clearing a list
shopping_list.clear()  # Removes all items
print(shopping_list)  # []

Think of these operations as managing a real shopping list - you can cross items off, add new ones, or even tear off the entire list and start over.

List Concatenation and Replication

Lists can be combined using the + operator and replicated using the * operator:

list1 = [1, 2, 3]
list2 = [4, 5, 6]

# Concatenation
combined = list1 + list2
print(combined)  # [1, 2, 3, 4, 5, 6]

# Replication
repeated = list1 * 3
print(repeated)  # [1, 2, 3, 1, 2, 3, 1, 2, 3]

Concatenation is like putting two bookshelves side by side, while replication is like making multiple copies of the same bookshelf.

Finding Elements

There are several ways to locate and verify elements within a list:

colors = ["red", "green", "blue", "yellow", "green", "purple"]

# Checking membership
if "green" in colors:
    print("Green is in the list!")

# Finding index of an element
first_green_index = colors.index("green")  # Returns 1
# colors.index("black")  # Would raise ValueError as "black" is not in the list

# Counting occurrences
green_count = colors.count("green")  # Returns 2

# Finding all occurrences (more advanced)
green_indices = [i for i, color in enumerate(colors) if color == "green"]
print(green_indices)  # [1, 4]
Real-World Example: Email Management

Imagine you're building an email filtering system. You might use lists to manage categories, search for specific senders, or count messages from particular domains:

# List of email addresses
emails = [
    "alice@example.com", 
    "bob@company.org", 
    "charlie@example.com",
    "dana@company.org", 
    "eve@personal.net"
]

# Find all company emails
company_emails = [email for email in emails if "company.org" in email]
print(company_emails)  # ['bob@company.org', 'dana@company.org']

# Count emails from each domain
domains = [email.split('@')[1] for email in emails]
unique_domains = set(domains)
domain_counts = {domain: domains.count(domain) for domain in unique_domains}
print(domain_counts)  # {'example.com': 2, 'company.org': 2, 'personal.net': 1}

Essential List Methods

Python lists come with built-in methods that provide powerful functionality:

Method Description Example
append() Adds an element to the end of the list fruits.append("fig")
insert() Adds an element at a specific position fruits.insert(1, "apricot")
extend() Adds all elements from another iterable fruits.extend(["grape", "kiwi"])
remove() Removes the first occurrence of a value fruits.remove("banana")
pop() Removes and returns element at given position last = fruits.pop()
clear() Removes all elements fruits.clear()
index() Returns index of first occurrence pos = fruits.index("cherry")
count() Counts occurrences of a value num = fruits.count("apple")
sort() Sorts the list in place fruits.sort()
reverse() Reverses the list in place fruits.reverse()
copy() Returns a shallow copy of the list new_list = fruits.copy()

Sorting Lists

Sorting is one of the most common operations performed on lists:

numbers = [42, 13, 7, 55, 21, 101, 8]

# Simple sorting
numbers.sort()  # In-place sorting
print(numbers)  # [7, 8, 13, 21, 42, 55, 101]

# Sorting in reverse
numbers.sort(reverse=True)  # In-place reverse sorting
print(numbers)  # [101, 55, 42, 21, 13, 8, 7]

# Using sorted() to create a new list
original = [42, 13, 7, 55, 21, 101, 8]
sorted_numbers = sorted(original)  # Creates a new sorted list
print(original)  # Unchanged: [42, 13, 7, 55, 21, 101, 8]
print(sorted_numbers)  # [7, 8, 13, 21, 42, 55, 101]

# Custom sorting with key function
students = [
    {"name": "Alice", "grade": 85},
    {"name": "Bob", "grade": 72},
    {"name": "Charlie", "grade": 91},
    {"name": "Diana", "grade": 88}
]

# Sort by grade
students.sort(key=lambda student: student["grade"])
print([student["name"] for student in students])  # ['Bob', 'Alice', 'Diana', 'Charlie']

# Sort by name
students.sort(key=lambda student: student["name"])
print([student["name"] for student in students])  # ['Alice', 'Bob', 'Charlie', 'Diana']
Practical Application: Data Analysis

Sorting is crucial in data analysis. Imagine you're analyzing customer data:

customers = [
    {"id": 1, "name": "Smith, John", "purchases": 5, "total_spent": 325.50},
    {"id": 2, "name": "Johnson, Mary", "purchases": 12, "total_spent": 842.75},
    {"id": 3, "name": "Williams, David", "purchases": 3, "total_spent": 152.30},
    {"id": 4, "name": "Brown, Linda", "purchases": 8, "total_spent": 597.80},
    {"id": 5, "name": "Jones, Patricia", "purchases": 7, "total_spent": 421.15}
]

# Find top customers by total spent
top_spenders = sorted(customers, key=lambda c: c["total_spent"], reverse=True)
print("Top 3 customers by spending:")
for customer in top_spenders[:3]:
    print(f"{customer['name']}: ${customer['total_spent']:.2f}")

# Find most frequent customers
most_frequent = sorted(customers, key=lambda c: c["purchases"], reverse=True)
print("\nTop 3 customers by purchase frequency:")
for customer in most_frequent[:3]:
    print(f"{customer['name']}: {customer['purchases']} purchases")

List Comprehensions: Python's Secret Weapon

List comprehensions are one of Python's most elegant and powerful features. They provide a concise way to create lists based on existing iterables, combining the functionality of map and filter into a readable one-line expression.

Basic Syntax

The basic syntax of a list comprehension is:

[expression for item in iterable if condition]

Where:

Simple Examples

Let's look at some basic list comprehensions compared to traditional for loops:

# Creating a list of squares using a for loop
squares_loop = []
for i in range(1, 11):
    squares_loop.append(i * i)
print(squares_loop)  # [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

# Same operation with a list comprehension
squares_comp = [i * i for i in range(1, 11)]
print(squares_comp)  # [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

# Creating a list of even numbers using a for loop with conditional
evens_loop = []
for i in range(1, 21):
    if i % 2 == 0:
        evens_loop.append(i)
print(evens_loop)  # [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

# Same operation with a list comprehension
evens_comp = [i for i in range(1, 21) if i % 2 == 0]
print(evens_comp)  # [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

As you can see, list comprehensions condense multiline operations into a single, readable line. They're like a shorthand notation for creating lists, similar to how we use mathematical set notation.

Real-World Analogy: Recipe Book

Think of a list comprehension as a recipe that tells you exactly how to create a new dish from existing ingredients. The recipe specifies which ingredients to use (iterable), how to prepare each ingredient (expression), and which ingredients to include or exclude (condition).

Advanced List Comprehensions

List comprehensions can become quite sophisticated, handling complex transformations and multiple conditions:

# Using conditional expressions (ternary operator)
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
result = ["even" if x % 2 == 0 else "odd" for x in numbers]
print(result)  # ['odd', 'even', 'odd', 'even', 'odd', 'even', 'odd', 'even', 'odd', 'even']

# Multiple conditions
filtered = [x for x in range(1, 51) if x % 2 == 0 if x % 5 == 0]
print(filtered)  # [10, 20, 30, 40, 50] - numbers divisible by both 2 and 5

# Equivalent to:
filtered_loop = []
for x in range(1, 51):
    if x % 2 == 0:
        if x % 5 == 0:
            filtered_loop.append(x)

# Nested list comprehensions
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]
print(flattened)  # [1, 2, 3, 4, 5, 6, 7, 8, 9]

# Equivalent to:
flattened_loop = []
for row in matrix:
    for num in row:
        flattened_loop.append(num)

# Creating a matrix (2D list) using list comprehensions
matrix = [[i * j for j in range(1, 4)] for i in range(1, 4)]
print(matrix)
# [[1, 2, 3],
#  [2, 4, 6],
#  [3, 6, 9]]
Practical Usage: Data Cleaning and Transformation

List comprehensions excel at data processing tasks, making them invaluable for web development:

# Cleaning and normalizing data from a web form
user_inputs = [
    "  john.doe@example.com  ",
    "jane_smith@company.org",
    "",
    "invalid@email",
    "  admin@site.net  "
]

# Clean and validate email addresses
valid_emails = [
    email.strip().lower() 
    for email in user_inputs 
    if email.strip() and "@" in email and "." in email.split("@")[1]
]
print(valid_emails)
# ['john.doe@example.com', 'jane_smith@company.org', 'admin@site.net']

# Extracting data from HTML (simplified example)
html_snippets = [
    "<div class='user'>Alice</div>",
    "<div class='admin'>Bob</div>",
    "<div class='user'>Charlie</div>",
    "<div class='guest'>Diana</div>"
]

# Extract names and roles
users = [
    {
        "name": snippet.split('>')[1].split('<')[0],
        "role": snippet.split("class='")[1].split("'")[0]
    }
    for snippet in html_snippets
]

print(users)
# [{'name': 'Alice', 'role': 'user'}, 
#  {'name': 'Bob', 'role': 'admin'}, 
#  {'name': 'Charlie', 'role': 'user'}, 
#  {'name': 'Diana', 'role': 'guest'}]

# Filter for specific roles
admins = [user["name"] for user in users if user["role"] == "admin"]
print(admins)  # ['Bob']

Performance and Best Practices

When to Use List Comprehensions

While list comprehensions are powerful, they're not always the best choice:

Use list comprehensions when:

  • The operation is simple and clearly expressed in one line
  • You're creating a new list from an existing iterable
  • The transformation logic is straightforward
  • Performance is important (they're generally faster than equivalent for loops)

Consider traditional loops when:

  • The operation is complex or requires multiple steps
  • You need to handle exceptions within the loop
  • Readability would suffer with a list comprehension
  • You're not actually creating a new list (use generators instead)

Memory Considerations

List comprehensions create the entire result list in memory. For very large datasets, consider using generator expressions instead:

# List comprehension - creates entire list in memory
numbers = [x * x for x in range(10000000)]  # Could use a lot of memory

# Generator expression - generates values on demand
numbers_gen = (x * x for x in range(10000000))  # More memory efficient

# Processing large files with generators
def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line.strip()

# Process lines without loading entire file into memory
for line in read_large_file('very_large_log.txt'):
    if 'ERROR' in line:
        print(line)
Real-World Analogy: Assembly Line vs. Warehouse

Think of list comprehensions like building an entire inventory in a warehouse before delivering it (using memory to store everything at once). Generator expressions are more like an assembly line that produces each item just in time as it's needed, without storing the entire inventory.

Common List Patterns in Web Development

Let's explore some typical patterns you'll encounter when working with Python lists in web development:

Processing Form Data

# Processing multiple checkbox selections from a form
selected_options = ["option1", "option3", "option5"]
all_options = ["option1", "option2", "option3", "option4", "option5"]

# Create a list of checkbox states (checked/unchecked)
checkbox_states = [opt in selected_options for opt in all_options]
print(checkbox_states)  # [True, False, True, False, True]

# Create a mapping of options to their selected state
option_map = {opt: opt in selected_options for opt in all_options}
print(option_map)  
# {'option1': True, 'option2': False, 'option3': True, 'option4': False, 'option5': True}

Working with JSON Data

import json

# Sample JSON data from an API response
json_data = '''
{
    "users": [
        {"id": 1, "username": "alice", "active": true, "role": "admin"},
        {"id": 2, "username": "bob", "active": false, "role": "user"},
        {"id": 3, "username": "charlie", "active": true, "role": "user"},
        {"id": 4, "username": "diana", "active": true, "role": "moderator"}
    ]
}
'''

# Parse JSON and extract user information
data = json.loads(json_data)
users = data["users"]

# Get all active usernames
active_users = [user["username"] for user in users if user["active"]]
print(active_users)  # ['alice', 'charlie', 'diana']

# Create a mapping of user IDs to usernames
user_lookup = {user["id"]: user["username"] for user in users}
print(user_lookup)  # {1: 'alice', 2: 'bob', 3: 'charlie', 4: 'diana'}

# Group users by role
users_by_role = {}
for user in users:
    role = user["role"]
    if role not in users_by_role:
        users_by_role[role] = []
    users_by_role[role].append(user["username"])
    
print(users_by_role)  
# {'admin': ['alice'], 'user': ['bob', 'charlie'], 'moderator': ['diana']}

Building HTML Elements

# Creating a navigation menu
menu_items = [
    {"text": "Home", "url": "/", "active": True},
    {"text": "Products", "url": "/products", "active": False},
    {"text": "Services", "url": "/services", "active": False},
    {"text": "About", "url": "/about", "active": False},
    {"text": "Contact", "url": "/contact", "active": False}
]

# Generate HTML for menu items
menu_html = [
    f"<li class='{item['active'] and 'active' or ''}'><a href='{item['url']}'>{item['text']}</a></li>"
    for item in menu_items
]

# Join the HTML items into a complete menu
nav_html = "<ul class='nav'>" + "".join(menu_html) + "</ul>"
print(nav_html)
# 

Database Query Processing

# Simulated database query results
query_results = [
    (1, "Product A", 29.99, "Electronics"),
    (2, "Product B", 49.99, "Home & Kitchen"),
    (3, "Product C", 15.50, "Electronics"),
    (4, "Product D", 99.99, "Electronics"),
    (5, "Product E", 35.25, "Clothing")
]

# Transform query results to dictionaries
products = [
    {
        "id": row[0],
        "name": row[1],
        "price": row[2],
        "category": row[3]
    }
    for row in query_results
]

# Filter for electronics products
electronics = [p for p in products if p["category"] == "Electronics"]
print(electronics)

# Calculate average price by category
categories = set(p["category"] for p in products)
avg_prices = {
    category: sum(p["price"] for p in products if p["category"] == category) / 
              sum(1 for p in products if p["category"] == category)
    for category in categories
}
print(avg_prices)
# {'Electronics': 48.49, 'Home & Kitchen': 49.99, 'Clothing': 35.25}

Practice Exercises

Exercise 1: List Manipulation

Create a function that takes a list of numbers and returns a new list containing:

  • The squares of all even numbers
  • The cubes of all odd numbers

Example input: [1, 2, 3, 4, 5]
Expected output: [1, 4, 27, 16, 125]

Solution
def transform_numbers(numbers):
    return [num**2 if num % 2 == 0 else num**3 for num in numbers]

result = transform_numbers([1, 2, 3, 4, 5])
print(result)  # [1, 4, 27, 16, 125]

Exercise 2: Data Filtering

Given a list of dictionaries representing products:

products = [
    {"id": 1, "name": "Laptop", "price": 999.99, "in_stock": True, "category": "Electronics"},
    {"id": 2, "name": "Headphones", "price": 149.50, "in_stock": False, "category": "Electronics"},
    {"id": 3, "name": "Coffee Maker", "price": 89.99, "in_stock": True, "category": "Kitchen"},
    {"id": 4, "name": "Desk Chair", "price": 249.95, "in_stock": True, "category": "Furniture"},
    {"id": 5, "name": "Blender", "price": 79.50, "in_stock": True, "category": "Kitchen"},
    {"id": 6, "name": "Smartphone", "price": 799.00, "in_stock": False, "category": "Electronics"}
]

Write list comprehensions to:

  1. Find all in-stock products with prices under $100
  2. Create a list of all product names, sorted alphabetically
  3. Calculate the total value of all in-stock products
  4. Find the most expensive product in each category
Solution
# 1. In-stock products under $100
affordable_products = [p for p in products if p["in_stock"] and p["price"] < 100]
print("Affordable products:", [p["name"] for p in affordable_products])
# ['Coffee Maker', 'Blender']

# 2. All product names, sorted alphabetically
all_names = sorted([p["name"] for p in products])
print("Sorted product names:", all_names)
# ['Blender', 'Coffee Maker', 'Desk Chair', 'Headphones', 'Laptop', 'Smartphone']

# 3. Total value of all in-stock products
total_value = sum(p["price"] for p in products if p["in_stock"])
print(f"Total inventory value: ${total_value:.2f}")
# Total inventory value: $1419.43

# 4. Most expensive product in each category
categories = set(p["category"] for p in products)
most_expensive_by_category = {
    category: max([p for p in products if p["category"] == category], 
                  key=lambda p: p["price"])
    for category in categories
}

print("Most expensive by category:")
for category, product in most_expensive_by_category.items():
    print(f"{category}: {product['name']} (${product['price']:.2f})")
# Electronics: Laptop ($999.99)
# Kitchen: Coffee Maker ($89.99)
# Furniture: Desk Chair ($249.95)

Exercise 3: Web Development Task

You're building a simple e-commerce website. Given the following data:

cart_items = [
    {"product_id": 101, "name": "Product A", "price": 19.99, "quantity": 2},
    {"product_id": 103, "name": "Product B", "price": 29.99, "quantity": 1},
    {"product_id": 107, "name": "Product C", "price": 9.99, "quantity": 3}
]

shipping_options = [
    {"id": "standard", "name": "Standard Shipping", "cost": 4.99, "days": "3-5"},
    {"id": "express", "name": "Express Shipping", "cost": 9.99, "days": "1-2"},
    {"id": "overnight", "name": "Overnight Shipping", "cost": 19.99, "days": "1"}
]

user = {
    "premium_member": True,
    "shipping_addresses": [
        {"id": 1, "address": "123 Main St", "default": True},
        {"id": 2, "address": "456 Oak Ave", "default": False}
    ]
}

Create list comprehensions to:

  1. Calculate the total price for each item (price * quantity)
  2. Generate option elements for a shipping method dropdown
  3. Get the user's default shipping address
  4. Apply a 10% discount for premium members
Solution
# 1. Calculate total price for each item
item_totals = [item["price"] * item["quantity"] for item in cart_items]
print("Item totals:", item_totals)  # [39.98, 29.99, 29.97]

# 2. Generate shipping method dropdown options
shipping_options_html = [
    f'<option value="{option["id"]}">{option["name"]} (${option["cost"]:.2f}, {option["days"]} days)</option>'
    for option in shipping_options
]
shipping_dropdown = "<select name='shipping'>" + "".join(shipping_options_html) + "</select>"
print("Shipping dropdown:", shipping_dropdown)

# 3. Get user's default shipping address
default_address = [addr for addr in user["shipping_addresses"] if addr["default"]][0]
print("Default address:", default_address["address"])  # 123 Main St

# 4. Apply 10% discount for premium members
discount_multiplier = 0.9 if user["premium_member"] else 1.0
discounted_prices = [
    {
        "name": item["name"],
        "original_price": item["price"],
        "quantity": item["quantity"],
        "total": item["price"] * item["quantity"] * discount_multiplier
    }
    for item in cart_items
]
print("Discounted prices:", discounted_prices)
# [{'name': 'Product A', 'original_price': 19.99, 'quantity': 2, 'total': 35.982},
#  {'name': 'Product B', 'original_price': 29.99, 'quantity': 1, 'total': 26.991},
#  {'name': 'Product C', 'original_price': 9.99, 'quantity': 3, 'total': 26.973}]

# Calculate cart total with discount
cart_total = sum(item["price"] * item["quantity"] for item in cart_items) * discount_multiplier
print(f"Cart total with discount: ${cart_total:.2f}")  # $89.946

Further Topics to Explore

These topics will become increasingly important as you progress in your Python journey, especially when working with data processing, API responses, and database interactions in web development.

Key Takeaways

As you continue through this course, you'll find that lists and list comprehensions become essential tools in your Python toolkit, allowing you to write cleaner, more efficient code for web development.