Managing Secrets and API Keys in Python Projects (.env Guide)

Image by Author

# Introduction to Keeping Secrets

 
Storing sensitive information like API keys, database passwords, or tokens directly in your Python code is dangerous. If these secrets are leaked, attackers can break into your systems, and your organization can suffer loss of trust, financial and legal consequences. Instead, you should externalize secrets so they never appear in code or version control. A common best practice is to store secrets in environment variables (outside your code). This way, secrets never appear in the codebase. Though, manual environment variables work, for local development it’s convenient to keep all secrets in a single .env file.

This article explains seven practical techniques for managing secrets in Python projects, with code examples and explanations of common pitfalls.

# Technique 1: Using a .env File Locally (And Loading it Safely)

 
A .env file is a text file of KEY=value pairs that you keep locally (not in version control). It lets you define environment-specific settings and secrets for development. For example, a recommended project layout is:

my_project/
  app/
    main.py
    settings.py
  .env              # NOT committed – contains real secrets
  .env.example      # committed – lists keys without real values
  .gitignore
  pyproject.toml

 
Your actual secrets go into .env locally, e.g.:

# .env (local only, never commit)
OPENAI_API_KEY=your_real_key_here
DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
DEBUG=true

In contrast, .env.example is a template that you commit, for other developers to see which keys are needed:

# .env.example (commit this)
OPENAI_API_KEY=
DATABASE_URL=
DEBUG=false

Add patterns to ignore these files in Git:

.env
.env.*

So that your secret .env never gets accidentally checked in. In Python, the common practice is to use the python-dotenv library, which will load the .env file at runtime. For example, in app/main.py you might write:

# app/main.py
import os
from dotenv import load_dotenv

load_dotenv()  # reads variables from .env into os.environ

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise RuntimeError("Missing OPENAI_API_KEY. Set it in your environment or .env file.")

print("App started (key loaded).")

Here, load_dotenv() automatically finds .env in the working directory and sets each key=value into os.environ (unless that variable is already set). This approach avoids common mistakes like committing .env or sharing it insecurely, while giving you a clean, reproducible development environment. You can switch between machines or dev setups without changing code, and local secrets stay safe.

# Technique 2: Read Secrets from the Environment

 
Some developers put placeholders like API_KEY="test" in their code or assume variables are always set in development. This can work on their machine but fail in production. If a secret is missing, the placeholder could end up running and create a security risk. Instead, always fetch secrets from environment variables at runtime. In Python, you can use os.environ or os.getenv to get the values safely. For example:

def require_env(name: str) -> str:
    value = os.getenv(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value

OPENAI_API_KEY = require_env("OPENAI_API_KEY")

 
This makes your app fail fast on startup if a secret is missing, which is far safer than proceeding with a missing or dummy value.

# Technique 3: Validate Configuration with a Settings Module

 
As projects grow, many scattered os.getenv calls become messy and error-prone. Using a settings class like Pydantic’s BaseSettings centralizes configuration, validates types, and loads values from .env and the environment. For example:

# app/settings.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", extra="ignore")

    openai_api_key: str = Field(min_length=1)
    database_url: str = Field(min_length=1)
    debug: bool = False

settings = Settings()

 
Then in your app:

# app/main.py
from app.settings import settings

if settings.debug:
    print("Debug mode on")
api_key = settings.openai_api_key

 
This prevents mistakes like mistyping keys, misparsing types ("false" vs False), or duplicating environment lookups. Using a settings class ensures your app fails fast if secrets are missing and avoids “works on my machine” problems.

# Technique 4: Using Platform/CI secrets for Deployments

 
When you deploy to production, you should not copy your local .env file. Instead, use your hosting/CI platform’s secret management. For example, if you’re using GitHub Actions for CI, you can store secrets encrypted in the repository settings and then inject them into workflows. This way, your CI or cloud platform injects the real values at runtime, and you never see them in code or logs.

# Technique 5: Docker

 
In Docker, avoid baking secrets into images or using plain ENV. Docker and Kubernetes provide secrets mechanisms that are safer than environment variables, which can leak through process listings or logs. For local dev, .env plus python-dotenv works, but in production containers, mount secrets or use docker secret. Avoid ENV API_KEY=... in Dockerfiles or committing Compose files with secrets. Doing so lowers the risk of secrets being permanently exposed in images and simplifies rotation.

# Technique 6: Adding Guardrails

 
Humans make mistakes, so automate secret protection. GitHub push protection can block commits containing secrets, and CI/CD secret-scanning tools like TruffleHog or Gitleaks detect leaked credentials before merging. Beginners often rely on memory or speed, which leads to accidental commits. Guardrails prevent leaks before they enter your repo, making it much safer to work with .env and environment variables across development and deployment.

# Technique 7: Using a Real Secrets Manager

 
For larger applications, it makes sense to use a proper secrets manager like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. These tools control who can access secrets, log every access, and rotate keys automatically. Without one, teams often reuse passwords or forget to rotate them, which is risky. A secrets manager keeps everything under control, makes rotation simple, and protects your production systems even if a developer’s computer or local .env file is exposed.

# Wrapping Up

 
Keeping secrets safe is more than following rules. It is about building a workflow that makes your projects secure, easy to maintain, and portable across different environments. To make this easier, I have put together a checklist you can use in your Python projects.

1. .env is in .gitignore (never commit real credentials)
2. .env.example exists and is committed with empty values
3. Code reads secrets only via environment variables (os.getenv, a settings class, etc.)
4. The app fails fast with a clear error if a required secret is missing
5. You use different secrets for dev, staging, and prod (never reuse the same key)
6. CI and deployments use encrypted secrets (GitHub Actions secrets, AWS Parameter Store, etc.)
7. Push protection and or secret scanning is enabled on your repos
8. You have a rotation policy (rotate keys immediately if leaked and regularly otherwise)

Kanwal Mehreen is a machine learning engineer and a technical writer with a profound passion for data science and the intersection of AI with medicine. She co-authored the ebook "Maximizing Productivity with ChatGPT". As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She's also recognized as a Teradata Diversity in Tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having founded FEMCodes to empower women in STEM fields.