Building Your Foundation: Python Installation and Environment Management
Welcome to a crucial session in your journey as a Python developer! Today, we'll set up the core tools that will power your development work throughout this course and your future career: Python itself and virtual environments.
Think of Python as the engine that powers your applications, while virtual environments are like specialized workspaces that keep your projects organized, clean, and isolated from each other. Together, they form the foundation of professional Python development.
By the end of this session, you'll have a properly installed Python environment and understand how to create, activate, and manage virtual environments for your projects. These skills are absolutely essential for modern Python development and will save you countless hours of troubleshooting and configuration headaches down the road.
Understanding Python Versions
Before we install Python, let's understand the Python version landscape, which will help you make informed decisions about which version to use.
Python 2 vs. Python 3
Python has two major versions: Python 2 and Python 3. The most important thing to know is that Python 2 reached end-of-life on January 1, 2020 and is no longer supported. All new development should use Python 3.
The transition from Python 2 to 3 involved some significant changes that broke backward compatibility, including:
- Print is now a function:
print("Hello")instead ofprint "Hello" - Unicode strings by default
- Division returns floating-point values by default
- Many library improvements and reorganizations
While you might encounter legacy Python 2 code in some older systems, for this course, we'll be using Python 3 exclusively.
Which Python 3 Version?
Python 3 has several minor versions (3.7, 3.8, 3.9, 3.10, 3.11, etc.). For this course, we recommend using Python 3.10 or newer, which includes:
- Improved error messages
- Structural pattern matching
- Type hint improvements
- Significant performance enhancements (especially in 3.11)
- Better support for modern libraries and frameworks
Most major Python libraries and frameworks now support Python 3.8 and above, with many optimizing for 3.9+.
Release Cycle and Support
Python follows a predictable release schedule:
- New minor versions are released approximately yearly (3.10 in October 2021, 3.11 in October 2022, etc.)
- Each version receives bug fix support for about 18 months, followed by security updates for an additional 18 months
- This means each version has about a 3-year support window
Think of Python versions like car models – newer versions have improved features and performance, but any recent model will still get you where you need to go. For professional development, staying reasonably current is important.
Installing Python
Let's install Python on your system. The installation process varies by operating system, so follow the section relevant to your platform.
Windows Installation
On Windows, we have several options for installing Python:
Option 1: Official Installer (Recommended)
- Visit the Python downloads page
- Click on the "Download Python 3.x.x" button (where 3.x.x is the latest version)
- Run the downloaded installer
- Important: Check the box that says "Add Python to PATH" during installation
- Choose "Customize installation" for more options
- Make sure "pip" and "tcl/tk" are selected
- Complete the installation
The "Add Python to PATH" option is crucial—it allows you to run Python from any command prompt. Think of it like putting your favorite tools on your workbench instead of having to fetch them from a drawer every time.
Option 2: Microsoft Store
- Open the Microsoft Store app
- Search for "Python"
- Select the version you want to install (e.g., "Python 3.10")
- Click "Get" or "Install"
This method is simpler but gives you less control over the installation options. It's convenient for beginners but may have limitations for some advanced development scenarios.
Option 3: Windows Subsystem for Linux (WSL)
If you're using WSL (which provides a Linux environment on Windows):
- Open your WSL terminal
- Update the package list:
sudo apt update - Install Python:
sudo apt install python3 python3-pip python3-venv
This approach gives you a Linux-like Python environment, which can be advantageous if you're deploying to Linux servers.
macOS Installation
macOS comes with Python pre-installed, but it's usually an older version. Here's how to install a current version:
Option 1: Homebrew (Recommended)
Homebrew is a package manager for macOS that makes installing software simple.
- Install Homebrew if you don't have it:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" - Install Python:
brew install python
Homebrew handles all the path setup automatically and gives you the latest Python version.
Option 2: Official Installer
- Visit the Python downloads page
- Download the macOS installer for the latest version
- Run the installer and follow the instructions
The official installer works well but doesn't integrate with macOS package management, so future updates will need to be managed manually.
Option 3: pyenv (Advanced)
For developers who need to work with multiple Python versions:
- Install pyenv with Homebrew:
brew install pyenv - Add pyenv to your shell configuration
- Install a Python version:
pyenv install 3.11.0 - Set it as global default:
pyenv global 3.11.0
Think of pyenv as a closet with multiple sets of tools for different jobs—you can easily switch between them as needed.
Linux Installation
Most Linux distributions come with Python pre-installed, but it may not be the latest version. Here's how to install a current version:
Debian/Ubuntu and Derivatives
sudo apt update
sudo apt install python3 python3-pip python3-venv
Fedora
sudo dnf install python3 python3-pip
Arch Linux
sudo pacman -S python python-pip
Using pyenv on Linux (for multiple versions)
If you need to manage multiple Python versions on Linux:
# Install dependencies
sudo apt install -y build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev curl libncursesw5-dev xz-utils \
tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
# Install pyenv
curl https://pyenv.run | bash
# Add to your shell configuration file (.bashrc, .zshrc, etc.)
# Then install and set Python version
pyenv install 3.11.0
pyenv global 3.11.0
Introduction to Virtual Environments
Now that Python is installed, let's learn about virtual environments—one of the most important concepts for professional Python development.
What Are Virtual Environments?
A virtual environment is an isolated Python environment where you can install packages and dependencies specific to a particular project without affecting other projects or the system-wide Python installation.
Think of virtual environments like individual workshops for different projects. Each workshop contains only the specific tools needed for that project, preventing clutter and conflicts between projects.
Why Use Virtual Environments?
Virtual environments solve several critical problems in Python development:
- Dependency Isolation: Different projects can use different versions of the same library without conflict. For example, Project A can use Django 3.2 while Project B uses Django 4.1.
- Clean Testing Environment: Ensures your application works with only the specified dependencies, which helps prevent "it works on my machine" problems.
- Simplified Dependency Management: Makes it easy to track and install all the packages a project needs.
- Easy Sharing: You can generate a list of dependencies that others can use to recreate your exact environment.
- Protection Against System Changes: System upgrades or other installations won't break your project's environment.
Without virtual environments, Python package installations are global, which leads to conflicts, upgrade issues, and difficulty in reproducing development environments—much like having all your tools for different hobbies mixed together in one messy pile.
Virtual Environment Tools
Several tools can create and manage virtual environments in Python:
- venv: Built into Python 3.3+ and the simplest option for most users
- virtualenv: An external package that works with all Python versions and has more features
- conda: Part of the Anaconda distribution, handles non-Python dependencies well (popular in data science)
- pipenv: Combines pip and virtualenv with a higher-level workflow
- poetry: Modern dependency management with an emphasis on deterministic builds
For this course, we'll focus on venv as it's built into Python and is the officially recommended tool for creating virtual environments. However, knowing that alternatives exist is valuable as you grow in your Python journey.
Creating and Using Virtual Environments with venv
Let's learn how to use the built-in venv module to create and manage virtual environments.
Creating a Virtual Environment
To create a virtual environment, use the venv module with the Python interpreter:
# On Windows
python -m venv myenv
# On macOS/Linux
python3 -m venv myenv
This creates a directory called myenv (you can use any name) containing the virtual environment. The environment includes:
- A Python interpreter (copy of your system Python)
- The pip package manager
- A basic set of installed packages
- Scripts to activate/deactivate the environment
It's a common practice to name your virtual environment venv or .venv (with a dot to make it hidden), but you can use any name that makes sense for your project.
Activating the Virtual Environment
Before you can use the virtual environment, you need to activate it. The activation process varies by operating system:
# On Windows (Command Prompt)
myenv\Scripts\activate.bat
# On Windows (PowerShell)
myenv\Scripts\Activate.ps1
# On macOS/Linux
source myenv/bin/activate
When activated, you'll notice your command prompt changes to show the virtual environment name, like (myenv) C:\Users\username>. This indicates that you're now working within the virtual environment.
Activation is like putting on a specialized worksuit for a particular project—it changes your environment so that when you use Python, you're using the specific version and packages installed in that environment.
Installing Packages in the Virtual Environment
With the virtual environment activated, you can install packages using pip:
# Make sure your environment is activated first!
pip install package_name
# Install specific version
pip install package_name==1.2.3
# Install from requirements file
pip install -r requirements.txt
These packages will only be installed in the virtual environment, not globally. You can verify this by checking the installation location:
pip list # Shows packages installed in the current environment
pip show package_name # Shows details about a specific package
Deactivating the Virtual Environment
When you're done working on the project, you can deactivate the virtual environment:
deactivate
This returns you to using your system-wide Python installation. It's like taking off your specialized worksuit and going back to regular clothes.
Creating Requirements Files
One of the benefits of virtual environments is the ability to easily share your environment configuration. You can generate a list of installed packages with versions:
pip freeze > requirements.txt
This creates a file called requirements.txt that lists all installed packages and their exact versions. Others can then recreate your environment with:
python -m venv new_env
# Activate the new environment
pip install -r requirements.txt
Think of the requirements file as a recipe that lists all the ingredients (packages) needed to recreate a specific dish (your project environment).
Virtual Environment Best Practices
Let's explore some best practices for using virtual environments effectively in your development workflow.
One Environment Per Project
Create a separate virtual environment for each project you work on. This keeps dependencies clean and prevents conflicts between projects.
For example, if you have a blog project and a data analysis project, create separate environments for each:
# Blog project
python -m venv blog_env
# Data analysis project
python -m venv data_analysis_env
Virtual Environment Location
There are two common approaches to where you should store your virtual environments:
-
Inside Project Directory: Create the virtual environment inside your project folder (often named
.venvorvenvwith a leading dot to hide it)cd my_project python -m venv .venvAdvantages: The environment travels with the project, making it clear which environment belongs to which project.
Disadvantages: Can increase project size when backing up or transferring projects (though you typically don't include the virtual environment in version control).
-
Central Location: Store all virtual environments in a central directory
# Create a directory for all environments mkdir ~/virtualenvs python -m venv ~/virtualenvs/my_project_envAdvantages: Keeps projects smaller, easy to manage all environments from one location.
Disadvantages: Less obvious which environment belongs to which project.
For most projects, especially when learning, the first approach (inside the project directory) is usually simpler and clearer.
Version Control Considerations
When using version control (like Git), you should never commit your virtual environment to the repository. It's large, contains platform-specific files, and can be easily recreated with the requirements file.
Add your virtual environment directory to your .gitignore file:
# .gitignore
venv/
.venv/
env/
ENV/
Instead, commit your requirements.txt file, which allows others to recreate the environment easily.
Keeping Track of Direct Dependencies
pip freeze lists all installed packages, including indirect dependencies. Sometimes it's useful to track only the direct dependencies—those you explicitly installed. You can manually create a requirements file like:
# requirements.txt
Django==4.1.7
requests==2.28.2
pytest==7.2.2
More advanced tools like Pipenv and Poetry automatically distinguish between direct and indirect dependencies, which can be helpful for larger projects.
Updating Packages Safely
To update packages in your virtual environment:
# Update a single package
pip install --upgrade package_name
# Update all packages listed in requirements.txt
pip install --upgrade -r requirements.txt
For critical projects, create a new virtual environment and test updates there before applying them to your main environment.
Environment Activation in IDEs
Most modern IDEs like VS Code and PyCharm can automatically detect and use virtual environments. In VS Code:
- Open your project folder
- VS Code should detect the virtual environment
- Select the Python interpreter from the virtual environment when prompted
- Or manually select it with the "Python: Select Interpreter" command
This integration allows your IDE to provide accurate auto-completion, linting, and other features specific to your project's environment.
Alternative Environment Management Tools
While venv is sufficient for most projects and is our focus in this course, it's worth knowing about some alternative tools that offer additional features for environment management.
virtualenv
virtualenv is a third-party package that works similarly to venv but offers more features and works with older Python versions:
# Install virtualenv
pip install virtualenv
# Create environment
virtualenv my_project_env
# Activate (same as venv)
The main advantages are compatibility with older Python versions and some additional customization options.
Pipenv
Pipenv combines pip and virtualenv into a single tool with a higher-level interface:
# Install Pipenv
pip install pipenv
# Install packages (automatically creates a virtual environment)
cd my_project
pipenv install django
# Activate the environment
pipenv shell
# Install development-only dependencies
pipenv install pytest --dev
Pipenv's key features include:
- Automatic virtual environment management
- Generates Pipfile and Pipfile.lock (more comprehensive than requirements.txt)
- Separates development and production dependencies
- Security vulnerability checking
Pipenv is like an all-in-one workshop management system—it not only organizes your tools but also handles inventory, safety checks, and equipment maintenance.
Poetry
Poetry is a modern dependency management and packaging tool:
# Install Poetry
pip install poetry
# Create a new project
poetry new my_project
cd my_project
# Or initialize in an existing project
cd existing_project
poetry init
# Add dependencies
poetry add django
# Add development dependencies
poetry add pytest --dev
# Activate the environment
poetry shell
Poetry's strengths include:
- Deterministic builds with lock files
- Simplified dependency resolution
- Built-in package building and publishing
- Modern, intuitive interface
Poetry is gaining popularity, especially for library development and more complex applications.
Conda
Conda is both a package manager and environment manager that handles not just Python packages but also C libraries and other external dependencies:
# Create environment
conda create --name my_project_env python=3.10
# Activate environment
conda activate my_project_env
# Install packages
conda install numpy pandas matplotlib
Conda is particularly popular in data science because it handles complex packages with C dependencies (like NumPy, SciPy, and TensorFlow) very well. It's like a specialized workshop system designed for interdisciplinary projects that require tools from different fields.
When to Consider Alternatives
While venv is sufficient for learning and many projects, consider exploring alternatives when:
- You need to manage complex dependencies with system-level components (consider Conda)
- You're developing a Python library for distribution (consider Poetry)
- You want more robust dependency locking and security features (consider Pipenv or Poetry)
- You're working on a team that has standardized on a particular tool
For this course, we'll stick with venv to keep things simple and focus on the core concepts that apply across all tools.
Troubleshooting Common Virtual Environment Issues
Let's address some common issues you might encounter when working with virtual environments.
Activation Problems
-
Script cannot be loaded because running scripts is disabled (PowerShell):
# Run PowerShell as Administrator and execute: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser # Then try activating again -
bash: activate: No such file or directory:
Ensure you're using the correct path to the activate script and that you created the virtual environment correctly.
-
Command not found: activate:
Use
sourcebefore the path on Unix-like systems:source myenv/bin/activate
Package Installation Issues
-
Packages installed but not available:
Ensure your virtual environment is activated. Check with
which pythonon Unix-like systems orwhere pythonon Windows to verify you're using the environment's Python. -
Permission errors during installation:
If you're getting permission errors, you might be trying to install to the system Python. Ensure your virtual environment is activated, or use the
--userflag with pip as a last resort. -
Connection errors with pip:
If pip can't connect to PyPI (the Python Package Index), check your internet connection and any proxy settings. You can also try using a mirror or downloading packages manually.
Environment Corruption
Sometimes virtual environments can become corrupted or behave unexpectedly. Signs include:
- Unexpected import errors for installed packages
- Activation works but Python uses the wrong interpreter
- Strange path issues or missing executables
The simplest solution is often to recreate the environment:
# Deactivate current environment
deactivate
# Save your dependencies
pip freeze > saved_requirements.txt
# Remove the old environment
rm -rf myenv # Unix-like systems
rmdir /s /q myenv # Windows
# Create a new environment
python -m venv myenv
# Activate and reinstall packages
# (activate the new environment)
pip install -r saved_requirements.txt
This is like cleaning out and reorganizing a messy workshop when things get too disorganized—sometimes starting fresh is more efficient than trying to fix every small issue.
IDE Integration Issues
If your IDE isn't recognizing your virtual environment:
- VS Code: Use the Python: Select Interpreter command and manually navigate to the virtual environment's Python executable
- PyCharm: Go to Settings/Preferences → Project → Python Interpreter → Add → Existing Environment and locate the interpreter
-
Jupyter Notebooks: Install ipykernel in your virtual environment and register it:
python -m ipykernel install --user --name=myenv
Virtual Environments in Real-World Projects
Let's look at how virtual environments are used in real-world Python projects, from small personal tools to large enterprise applications.
Simple Web Application
For a Flask-based web application, a typical environment setup might look like:
# Create project structure
mkdir my_flask_app
cd my_flask_app
# Create and activate virtual environment
python -m venv .venv
# Activate the environment
# Install core dependencies
pip install flask flask-sqlalchemy flask-wtf python-dotenv
# Create requirements file
pip freeze > requirements.txt
# Create basic application files
touch app.py config.py
mkdir templates static
This approach keeps the virtual environment with the project, making it clear which environment belongs to the application.
Data Science Workflow
Data scientists often work with multiple environments for different types of analysis:
# Create environment for data analysis
python -m venv data_analysis_env
# Activate
pip install pandas numpy matplotlib jupyter seaborn scikit-learn
# Create environment for deep learning
python -m venv deep_learning_env
# Activate
pip install tensorflow keras jupyter matplotlib pandas
Many data scientists prefer Conda for these use cases due to its superior handling of complex numerical packages with C dependencies.
Enterprise Web Application
Large enterprise applications often use more sophisticated tools:
# Initialize a Poetry project
poetry new enterprise_app
cd enterprise_app
# Add dependencies with specific versions
poetry add django==4.1.7 djangorestframework==3.14.0 celery==5.2.7
poetry add --dev pytest pytest-django coverage black flake8
# Create detailed documentation of the environment
mkdir docs
echo "# Environment Setup" > docs/environment.md
poetry export -f requirements.txt --output docs/requirements.txt
Enterprise projects often have strict requirements for dependency management, security scanning, and environment reproducibility, which tools like Poetry or Pipenv can help address.
Microservices Architecture
In a microservices setup, each service might have its own environment:
# User service
cd user_service
python -m venv .venv
# Activate
pip install fastapi sqlalchemy pydantic uvicorn
# Product service
cd ../product_service
python -m venv .venv
# Activate
pip install fastapi motor pydantic uvicorn
# Shared libraries
cd ../shared_lib
python -m venv .venv
# Activate
pip install pydantic pytest httpx
This isolation ensures each service has exactly the dependencies it needs, making them more maintainable and deployable independently.
Continuous Integration/Deployment
In CI/CD pipelines, virtual environments are often created programmatically:
# Example GitHub Actions workflow
name: Python Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install pytest pytest-cov
- name: Run tests
run: |
source .venv/bin/activate
pytest --cov=myapp tests/
Here, the CI system creates a fresh virtual environment for each test run, ensuring a clean, consistent testing environment.
Beyond Virtual Environments: Containerization
While virtual environments isolate Python dependencies, containers like Docker take isolation to the next level by including the entire operating system environment. Let's briefly explore how Docker relates to Python environments.
Virtual Environments vs. Containers
Here's how virtual environments and containers compare:
-
Virtual Environments:
- Isolate Python packages only
- Share the system's Python interpreter
- Lightweight and quick to create
- Great for development on a single machine
-
Containers (Docker):
- Isolate the entire application environment, including system libraries and even the operating system
- Package everything needed to run the application
- More resource-intensive than virtual environments
- Excellent for deployment and ensuring consistency across different environments
If virtual environments are like having different workshops for different projects, containers are like having entire separate buildings, each with its own infrastructure, utilities, and facilities.
Using Both Together
Many projects use both approaches:
- Virtual environments for local development
- Containers for testing, staging, and production deployment
A typical workflow might look like:
# Local development with virtual environment
python -m venv .venv
# Activate
pip install -r requirements.txt
# Develop, test locally...
# Build Docker image for deployment
docker build -t myapp:latest .
# Run container for testing
docker run -p 8000:8000 myapp:latest
# Deploy to production (e.g., with Kubernetes or a cloud service)
Example Dockerfile for a Python Application
Here's a simple Dockerfile for a Python web application:
# Use an official Python runtime as the base image
FROM python:3.10-slim
# Set working directory
WORKDIR /app
# Copy requirements file
COPY requirements.txt .
# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application
COPY . .
# Command to run the application
CMD ["python", "app.py"]
This Dockerfile:
- Starts with a base Python image
- Installs the application's dependencies
- Copies the application code
- Defines how to run the application
Notice that it doesn't use a virtual environment inside the container—it's not necessary because the container itself provides isolation.
When to Consider Containers
Consider using containers when:
- You need to ensure consistent deployment across different environments
- Your application has complex system dependencies beyond Python packages
- You're working with microservices architecture
- You need to run the same application on different operating systems
- You want to isolate applications for security reasons
We'll explore containerization with Docker more deeply in a future session. For now, focus on mastering virtual environments as they're an essential stepping stone to understanding more advanced isolation concepts.
Practical Exercise: Setting Up a Python Project Environment
Let's put what we've learned into practice with a hands-on exercise. We'll create a complete Python project environment from scratch.
Exercise Goals
By the end of this exercise, you will have:
- Created a new Python project with a proper directory structure
- Set up a virtual environment
- Installed project dependencies
- Created a requirements file
- Written a simple Python script to verify everything works
Step 1: Create Project Directory Structure
# Create main project directory
mkdir weather_app
cd weather_app
# Create subdirectories
mkdir src tests docs
Step 2: Create and Activate a Virtual Environment
# Create virtual environment
python -m venv .venv
# Activate the environment
# On Windows:
.venv\Scripts\activate
# On macOS/Linux:
source .venv/bin/activate
# Verify activation (you should see the environment name in your prompt)
Step 3: Install Dependencies
# Install required packages
pip install requests pytest
# Create requirements file
pip freeze > requirements.txt
Step 4: Create a Simple Python Script
Create a file src/weather.py with the following content:
import requests
def get_weather(city, api_key="demo"):
"""
Get current weather for a city using the OpenWeatherMap API.
Args:
city (str): Name of the city
api_key (str): OpenWeatherMap API key (default: "demo" for testing)
Returns:
dict: Weather data if successful, None otherwise
"""
base_url = "https://api.openweathermap.org/data/2.5/weather"
params = {
"q": city,
"appid": api_key,
"units": "metric"
}
try:
response = requests.get(base_url, params=params)
response.raise_for_status() # Raise exception for HTTP errors
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error fetching weather data: {e}")
return None
def format_weather(weather_data):
"""
Format weather data into a readable string.
Args:
weather_data (dict): Weather data from the API
Returns:
str: Formatted weather information
"""
if not weather_data:
return "No weather data available."
try:
city = weather_data["name"]
country = weather_data["sys"]["country"]
temp = weather_data["main"]["temp"]
description = weather_data["weather"][0]["description"]
humidity = weather_data["main"]["humidity"]
return f"Weather in {city}, {country}:\n" \
f"Temperature: {temp}°C\n" \
f"Conditions: {description}\n" \
f"Humidity: {humidity}%"
except KeyError:
return "Error: Weather data format is unexpected."
def main():
"""Main function to run the weather app."""
city = input("Enter city name: ")
weather_data = get_weather(city)
weather_info = format_weather(weather_data)
print("\n" + weather_info)
if __name__ == "__main__":
main()
Step 5: Create a Simple Test
Create a file tests/test_weather.py with the following content:
import pytest
from src.weather import format_weather
def test_format_weather_valid_data():
# Sample weather data
weather_data = {
"name": "London",
"sys": {"country": "GB"},
"main": {"temp": 15.5, "humidity": 76},
"weather": [{"description": "cloudy"}]
}
result = format_weather(weather_data)
assert "London, GB" in result
assert "15.5°C" in result
assert "cloudy" in result
assert "76%" in result
def test_format_weather_no_data():
result = format_weather(None)
assert result == "No weather data available."
def test_format_weather_invalid_data():
result = format_weather({})
assert "Error" in result
Step 6: Create a README File
Create a file README.md with information about your project:
# Weather App
A simple command-line weather application that fetches current weather data.
## Setup
1. Create and activate a virtual environment:
```
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
2. Install dependencies:
```
pip install -r requirements.txt
```
## Usage
Run the application:
```
python src/weather.py
```
## Running Tests
Run tests with pytest:
```
pytest tests/
```
## Project Structure
- `src/`: Source code
- `tests/`: Test files
- `docs/`: Documentation
- `.venv/`: Virtual environment (not included in repository)
Step 7: Verify Everything Works
# Run the tests
pytest tests/
# Run the application
python src/weather.py
Step 8: Clean Up (Optional)
When you're done, you can deactivate the virtual environment:
deactivate
This exercise demonstrates a complete Python project setup using a virtual environment. The structure follows best practices and includes basic project components: a functional script, tests, requirements, and documentation.
Advanced Topics and Future Learning
As you grow as a Python developer, you might explore these advanced topics related to Python environments:
Development vs. Production Dependencies
For larger projects, it's common to separate dependencies into those needed for development (testing, linting, debugging) and those needed for production:
# Using pip with multiple requirements files
pip install -r requirements.txt # Base requirements
pip install -r requirements-dev.txt # Development extras
# With Poetry
poetry add django requests # Production dependencies
poetry add --dev pytest black mypy # Development dependencies
Dependency Pinning and Locking
For reproducible builds, exact versions should be specified:
# requirements.txt with pinned versions
django==4.1.7
requests==2.28.2
Tools like Poetry and Pipenv create lock files that capture not just direct dependencies but also their dependencies, ensuring exact reproduction of environments.
Compiled Extensions and Binary Packages
Some Python packages include compiled C code, which can cause installation issues. Strategies include:
- Using pre-compiled wheels when available
- Installing system-level development libraries
- Using Conda, which handles many binary dependencies
- Using Docker containers with all necessary compilation tools
Environment Management in CI/CD Pipelines
Continuous Integration and Deployment systems need reliable, reproducible environments:
- Using requirements.txt with pinned versions
- Caching dependencies to speed up builds
- Using dedicated service containers for dependencies
- Implementing matrix testing across Python versions
Python Version Management
Tools like pyenv allow managing multiple Python versions on a single system:
# Install multiple Python versions
pyenv install 3.8.16
pyenv install 3.10.10
pyenv install 3.11.2
# Set global or local Python version
pyenv global 3.10.10
pyenv local 3.11.2 # Sets version for current directory
# Create virtual environments with specific Python versions
pyenv virtualenv 3.8.16 legacy_project_env
pyenv virtualenv 3.11.2 modern_project_env
Conclusion: Your Python Foundation Is Ready
Congratulations! You've now set up Python and learned how to create and manage virtual environments—two foundational skills that will serve you throughout your Python development journey.
Let's recap what we've covered:
- Installing Python across different operating systems
- Understanding Python versions and their support lifecycle
- Creating isolated virtual environments with
venv - Managing packages and dependencies
- Following best practices for project organization
- Exploring alternative tools for more advanced needs
- Troubleshooting common issues
- Getting a glimpse of how these concepts apply in real-world projects
With this knowledge, you're now prepared to start developing Python applications without worrying about dependency conflicts or environment issues. As you progress in your Python journey, you'll find that these skills become second nature, and you'll appreciate the clean, isolated environments that make Python development a pleasure.
In our next sessions, we'll build upon this foundation as we dive into Python programming fundamentals and web development concepts. Each of these topics will benefit from the clean, reproducible environments you now know how to create.
Assignment: Python Environment Setup
To reinforce your understanding and skills, complete the following tasks:
-
Python Installation Verification:
- Verify your Python installation using the command line
- Check the installed version
- Confirm that pip is installed and working
- Submit a screenshot showing these checks
-
Basic Project Setup:
- Create a new project directory for a simple Python utility
- Set up a virtual environment using
venv - Activate the environment
- Install at least three packages (e.g., requests, pytest, black)
- Generate a requirements.txt file
- Create a simple Python script that uses at least one of the installed packages
- Submit your project files (excluding the virtual environment) and requirements.txt
-
Multiple Environments Exercise:
- Create two separate virtual environments:
- One for a web development project (include Flask or Django)
- One for a data analysis project (include pandas and matplotlib)
- Document the process of creating and switching between environments
- Create simple test scripts in each environment that import the relevant packages
- Submit your documentation and test scripts
- Create two separate virtual environments:
-
Environment Management Challenge:
- Download the provided project skeleton
- Set up a virtual environment
- Install the dependencies listed in the provided requirements.txt
- Fix any installation issues you encounter (the requirements contain some deliberate challenges)
- Run the provided test script to verify your environment works correctly
- Document any issues you encountered and how you resolved them
-
Exploration of Alternative Tools (Bonus):
- Install and try one alternative tool (Pipenv, Poetry, or Conda)
- Set up a small project using this tool
- Write a brief comparison of your experience with this tool versus
venv - Consider aspects like ease of use, features, and documentation quality
Submit your completed assignment before the next class session. The assignment is designed to give you hands-on practice with the key concepts we've covered, ensuring you're comfortable with Python environments before we dive into more advanced topics.
Additional Resources
- Python Official Documentation: Virtual Environments
- Python Packaging User Guide: Installing Packages Using pip and Virtual Environments
- Real Python: Python Virtual Environments: A Primer
- Stack Overflow: Comparing Different Virtual Environment Tools
- Docker: Build Python Applications
- Pipenv Documentation
- Poetry Documentation
- Conda Documentation
- pyenv on GitHub
- Real Python: Dependency Management with Poetry
Books and Courses:
- "Python Packaging and Virtual Environments" chapters in "Python Crash Course" by Eric Matthes
- "The Python Workshop" by Packt Publishing (Chapters on environments and packaging)
- LinkedIn Learning and Pluralsight have several courses on Python environments and packaging
YouTube Channels and Tutorials:
- Corey Schafer - Great Python environment tutorials
- Real Python - Focused Python tutorials
- mCoding - Advanced Python concepts