Python Full Stack Web Developer Course

Exercise: Requirements.txt Management for a Web Application

Let's apply what we've learned with a practical exercise:

Exercise: Create a Requirements Structure for a Web Application

Scenario: You're starting a new Flask web application that will have these features:

• Web interface with forms and templates
• Database integration with SQLAlchemy
• User authentication
• RESTful API
• Background task processing with Celery

Tasks:

1. Create a virtual environment for the project
2. Create a directory structure with separate requirements files for different environments
3. Populate the requirements files with appropriate packages
4. Create a script to install dependencies for different environments
5. Add security checking to ensure dependencies are up-to-date and secure

Step 1: Project Setup

# Create project directory
mkdir flask_project
cd flask_project

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Create directory structure
mkdir -p requirements app/templates app/static tests

Step 2: Create Requirements Files

Create these files in the requirements directory:

requirements/base.txt:

# Base requirements for Flask Web Application
# Python 3.9+

# Web Framework
flask==2.0.1

# Database
flask-sqlalchemy==2.5.1
sqlalchemy==1.4.25
alembic==1.7.3
flask-migrate==3.1.0

# Forms
flask-wtf==1.0.0
wtforms==2.3.3

# Authentication
flask-login==0.5.0
flask-bcrypt==0.7.1

# API
flask-restful==0.3.9
marshmallow==3.13.0

# Background Tasks
celery==5.1.2
redis==3.5.3

# Utilities
python-dotenv==0.19.0

requirements/development.txt:

# Development requirements
-r base.txt

# Debugging
flask-debugtoolbar==0.11.0

# Code Quality
black==21.8b0
flake8==3.9.2
isort==5.9.3

# Documentation
sphinx==4.2.0

# Development Server
watchdog==2.1.5

# REPL
ipython==7.27.0

requirements/testing.txt:

# Testing requirements
-r base.txt

# Testing
pytest==6.2.5
pytest-cov==2.12.1
pytest-flask==1.2.0
pytest-sugar==0.9.4

# Mocking
faker==8.13.2
factory-boy==3.2.0

requirements/production.txt:

# Production requirements
-r base.txt

# WSGI Server
gunicorn==20.1.0

# Performance
flask-caching==1.10.1

# Monitoring and Logging
sentry-sdk[flask]==1.3.1
prometheus-flask-exporter==0.18.2

Step 3: Create Installation Script

Create a script to help with dependency installation:

scripts/install_deps.sh:

#!/bin/bash
# Script to install dependencies for different environments

# Check if virtual environment is activated
if [[ "$VIRTUAL_ENV" == "" ]]; then
    echo "Virtual environment not activated. Please activate it first."
    echo "Run: source venv/bin/activate"
    exit 1
fi

# Process command line arguments
ENV=${1:-development}

case $ENV in
    development|dev)
        REQUIREMENTS="requirements/development.txt"
        ;;
    testing|test)
        REQUIREMENTS="requirements/testing.txt"
        ;;
    production|prod)
        REQUIREMENTS="requirements/production.txt"
        ;;
    *)
        echo "Unknown environment: $ENV"
        echo "Usage: $0 [development|testing|production]"
        exit 1
        ;;
esac

echo "Installing dependencies for $ENV environment..."
pip install -r $REQUIREMENTS

# Check for security vulnerabilities
echo "Checking for security vulnerabilities..."
pip install safety
safety check -r $REQUIREMENTS

echo "Done!"

Make the script executable:

chmod +x scripts/install_deps.sh

Step 4: Create a Project Initialization Script

scripts/setup_project.sh:

#!/bin/bash
# Script to set up a new project environment

# Create virtual environment if it doesn't exist
if [ ! -d "venv" ]; then
    echo "Creating virtual environment..."
    python -m venv venv
fi

# Activate virtual environment
source venv/bin/activate

# Install dependencies
./scripts/install_deps.sh development

# Initialize git repository if it doesn't exist
if [ ! -d ".git" ]; then
    echo "Initializing git repository..."
    git init
    
    # Create .gitignore
    echo "Creating .gitignore..."
    cat > .gitignore << EOL
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
venv/
ENV/
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg

# Flask
instance/
.webassets-cache

# Celery
celerybeat-schedule
celerybeat.pid

# Testing
.coverage
htmlcov/
.pytest_cache/

# IDE
.idea/
.vscode/
*.swp
*.swo

# Environment
.env
.env.local

# Logs
logs/
*.log
EOL
    
    # Initial commit
    git add .
    git commit -m "Initial project setup"
fi

echo "Project setup complete!"
echo "Activate the virtual environment with: source venv/bin/activate"

Make the script executable:

chmod +x scripts/setup_project.sh

Step 5: Create a Requirements Updating Script

scripts/update_deps.sh:

#!/bin/bash
# Script to update dependencies

# Check if virtual environment is activated
if [[ "$VIRTUAL_ENV" == "" ]]; then
    echo "Virtual environment not activated. Please activate it first."
    echo "Run: source venv/bin/activate"
    exit 1
fi

# Install pip-tools if not installed
pip install pip-tools

# Check outdated packages
echo "Checking for outdated packages..."
pip list --outdated

# Ask for confirmation
read -p "Do you want to update packages? (y/n) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
    echo "Update canceled."
    exit 0
fi

# Create requirements.in files if they don't exist
if [ ! -f "requirements/base.in" ]; then
    echo "Creating requirements.in files from current requirements.txt files..."
    
    # Extract direct dependencies from requirements.txt
    grep -v "^-r" requirements/base.txt | grep -v "^#" | grep -v "^$" > requirements/base.in
    grep -v "^-r" requirements/development.txt | grep -v "^#" | grep -v "^$" > requirements/development.in
    echo "-r base.in" > requirements/development.in.tmp
    grep -v "^-r" requirements/development.in >> requirements/development.in.tmp
    mv requirements/development.in.tmp requirements/development.in
    
    # Same for testing and production
    # ... similar commands for testing.in and production.in ...
fi

# Update base requirements
echo "Updating base requirements..."
pip-compile --upgrade requirements/base.in

# Update environment-specific requirements
for env in development testing production; do
    echo "Updating $env requirements..."
    pip-compile --upgrade requirements/${env}.in
done

# Check for security vulnerabilities
echo "Checking for security vulnerabilities..."
pip install safety
safety check -r requirements/base.txt

echo "Dependencies updated successfully!"

Make the script executable:

chmod +x scripts/update_deps.sh

Step 6: Create a Basic Flask Application

app/__init__.py:

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_login import LoginManager
from flask_wtf.csrf import CSRFProtect

# Initialize extensions
db = SQLAlchemy()
migrate = Migrate()
login_manager = LoginManager()
csrf = CSRFProtect()

def create_app(config_object='app.config.DevelopmentConfig'):
    """Create and configure the Flask application."""
    app = Flask(__name__)
    app.config.from_object(config_object)
    
    # Initialize extensions
    db.init_app(app)
    migrate.init_app(app, db)
    login_manager.init_app(app)
    csrf.init_app(app)
    
    # Register blueprints
    from app.routes import main_bp
    app.register_blueprint(main_bp)
    
    return app

app/config.py:

import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

class Config:
    """Base configuration class."""
    SECRET_KEY = os.environ.get('SECRET_KEY', 'dev-key-please-change')
    SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL', 'sqlite:///app.db')
    SQLALCHEMY_TRACK_MODIFICATIONS = False

class DevelopmentConfig(Config):
    """Development configuration."""
    DEBUG = True
    SQLALCHEMY_ECHO = True

class TestingConfig(Config):
    """Testing configuration."""
    TESTING = True
    SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
    WTF_CSRF_ENABLED = False

class ProductionConfig(Config):
    """Production configuration."""
    DEBUG = False
    TESTING = False

app/routes.py:

from flask import Blueprint, render_template

main_bp = Blueprint('main', __name__)

@main_bp.route('/')
def index():
    return render_template('index.html')

app/templates/index.html:

<html>
<head>
    <title>Flask App</title>
</head>
<body>
    <h1>Welcome to Flask App</h1>
    <p>This is a Flask application with proper dependency management.</p>
</body>
</html>

run.py:

from app import create_app

app = create_app()

if __name__ == '__main__':
    app.run()

Running the Application

# Set up the project
./scripts/setup_project.sh

# Activate virtual environment
source venv/bin/activate

# Run the application
python run.py

This exercise demonstrates:

• Structured requirements management for different environments
• Scripts for automating dependency installation and updates
• Security checking of dependencies
• Integration with a Flask application
• Project initialization and setup automation

By completing this exercise, you've created a robust dependency management system for a Python web application that follows best practices and can scale as the project grows.