As a developer, you’ve likely used environment variables in the command line or shell scripts, but have you used them as a way of configuring your Python applications?
This guide will show you all the code necessary for getting, setting, and loading environment variables in Python, including how to use them for supplying application config and secrets.
Not familiar with environment variables? Check out our ultimate guide for using environment variables in Linux and Mac.
Before digging into how to use environment variables in Python, it's important to understand why they're arguably the best way to configure applications. The main benefits are:
Environment variables have the additional benefit of abstracting from your application how config and secrets are supplied.
Finally, environment variables enable your application to run anywhere, whether it's for local development on macOS, a container in a Kubernetes Pod, or platforms such as Heroku or Vercel.
Here are some examples of using environment variables to configure a Python script or application:
When a Python process is created, the available environment variables populate the os.environ object which acts like a Python dictionary. This means that:
Now that you know how environment variables in Python are populated, let's look at how to access them.
Environment variables in Python are accessed using the os.environ object.
The os.environ object seems like a dictionary but is different as values may only be strings, plus it's not serializable to JSON.
You've got a few options when it comes to referencing the os.environ object:
1# 1. Standard way
2import os
3# os.environ['VAR_NAME']
4
5# 2. Import just the environ object
6from os import environ
7# environ['VAR_NAME']
8
9# 3. Rename the `environ` to env object for more concise code
10from os import environ as env
11# env['VAR_NAME']
I personally prefer version 3 as it's more succinct, but will stick to using os.environ for this article.
Accessing a specific environment variable in Python can be done in one of three ways, depending upon what should happen if an environment variable does not exist.
Let's explore with some examples.
If your app should crash when an environment variable is not set, then access it directly:
1print(os.environ['HOME']
2# >> '/home/dev'
3
4print(os.environ['DOES_NOT_EXIST']
5# >> Will raise a KeyError exception
For example, an application should fail to start if a required environment variable is not set, and a default value can't be provided, e.g. a database password.
If instead of the default KeyError exception being raised (which doesn't communicate why your app failed to start), you could capture the exception and print out a helpful message:
1import os
2import sys
3
4# Ensure all required environment variables are set
5try:
6 os.environ['API_KEY']
7except KeyError:
8 print('[error]: `API_KEY` environment variable required')
9 sys.exit(1)
You can have a default value returned if an environment variable doesn't exist by using the os.environ.get method and supplying the default value as the second parameter:
1# If HOSTNAME doesn't exist, presume local development and return localhost
2print(os.environ.get('HOSTNAME', 'localhost')
If the variable doesn't exist and you use os.environ.get without a default value, None is returned
1assert os.environ.get('NO_VAR_EXISTS') == None
You may need to check if an environment variable exists, but don't necessarily care about its value. For example, your application can be put in a "Debug mode" if the DEBUG environment variable is set.
You can check for just the existence of an environment variable:
1if 'DEBUG' in os.environ:
2 print('[info]: app is running in debug mode')
Or check to see it matches a specific value:
1if os.environ.get('DEBUG') == 'True':
2 print('[info]: app is running in debug mode')
Setting an environment variable in Python is the same as setting a key on a dictionary:
1os.environ['TESTING'] = 'true'
What makes os.environ different to a standard dictionary, is that only string values are allowed:
1os.environ['TESTING'] = True
2# >> TypeError: str expected, not bool
In most cases, your application will only need to get environment variables, but there are use cases for setting them as well.
For example, constructing a DB_URL environment variable on application start-up using DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME environment variables:
1os.environ['DB_URL'] = 'psql://{user}:{password}@{host}:{port}/{name}'.format(
2 user=os.environ['DB_USER'],
3 password=os.environ['DB_PASSWORD'],
4 host=os.environ['DB_HOST'],
5 port=os.environ['DB_PORT'],
6 name=os.environ['DB_NAME']
7)
Another example is setting a variable to a default value based on the value of another variable:
1# Set DEBUG and TESTING to 'True' if ENV is 'development'
2if os.environ.get('ENV') == 'development':
3 os.environ.setdefault('DEBUG', 'True') # Only set to True if DEBUG not set
4 os.environ.setdefault('TESTING', 'True') # Only set to True if TESTING not set
If you need to delete a Python environment variable, use the os.environ.pop function:
To extend our DB_URL example above, you may want to delete the other DB_ prefixed fields to ensure the only way the app can connect to the database is via DB_URL:
Another example is deleting an environment variable once it is no longer needed:
1auth_api(os.environ['API_KEY']) # Use API_KEY
2os.environ.pop('API_KEY') # Delete API_KEY as it's no longer needed
To view all environment variables:
The output of this command is difficult to read though because it's printed as one huge dictionary.
A better way, is to create a convenience function that converts os.environ to an actual dictionary so we can serialize it to JSON for pretty-printing:
1import os
2import json
3
4def print_env():
5 print(json.dumps({**{}, **os.environ}, indent=2))
You might be surprised to learn it's best to avoid providing default values as much as possible. Why?
Default values can make debugging a misconfigured application more difficult, as the final config values will likely be a combination of hard-coded default values and environment variables.
Relying purely on environment variables (or as much as possible) means you have a single source of truth for how your application was configured, making troubleshooting easier.
As an application grows in size and complexity, so does the number of environment variables.
Many projects experience growing pains when using environment variables for app config and secrets because there is no clear and consistent strategy for how to manage them, particularly when deploying to multiple environments.
A simple (but not easily scalable) solution is to use a .env file to contain all of the variables for a specific environment.
Then you would use a Python library such as python-dotenv to parse the .env file and populate the os.environ object.
To follow along, create and activate a new virtual environment, then install the python-dotenv library:
1# 1. Create
2python3 -m venv ~/.virtualenvs/doppler-tutorial
3
4# 2. Activate
5source ~/.virtualenvs/doppler-tutorial/bin/activate
6
7# 3. Install dotenv package
8pip install python-dotenv
Now save the below to a file named .env (note how it's the same syntax for setting a variable in the shell):
1API_KEY="357A70FF-BFAA-4C6A-8289-9831DDFB2D3D"
2HOSTNAME="0.0.0.0"
3PORT="8080"
Then save the following to dotenv-test.py:
1# Rename `os.environ` to `env` for nicer code
2from os import environ as env
3
4from dotenv import load_dotenv
5load_dotenv()
6
7print('API_KEY: {}'.format(env['API_KEY']))
8print('HOSTNAME: {}'.format(env['HOSTNAME']))
9print('PORT: {}'.format(env['PORT']))
Then run dotenv-test.py to test the environment variables are being populated:
1python3 dotenv-test.py
2# >> API_KEY: 357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
3# >> HOSTNAME: 0.0.0.0
4# >> PORT: 8080
While .env files are simple and easy to work with at the beginning, they also cause a new set of problems such as:
These are just some of the reasons why we recommend moving away from .env files and using something like Doppler instead.
Doppler provides an access-controlled dashboard to manage environment variables for every environment with an easy-to-use CLI for accessing config and secrets that work for every language, framework, and platform.
Creating a config specific data structure abstracts away how the config values are set, what fields have default values (if any), and provides a single interface for accessing config values instead of os.environ being littered throughout your codebase.
Below is a reasonably full-featured solution that supports:
To try it out, save this code to config.py:
1import os
2from typing import get_type_hints, Union
3from dotenv import load_dotenv
4
5load_dotenv()
6
7class AppConfigError(Exception):
8 pass
9
10def _parse_bool(val: Union[str, bool]) -> bool: # pylint: disable=E1136
11 return val if type(val) == bool else val.lower() in ['true', 'yes', '1']
12
13# AppConfig class with required fields, default values, type checking, and typecasting for int and bool values
14class AppConfig:
15 DEBUG: bool = False
16 ENV: str = 'production'
17 API_KEY: str
18 HOSTNAME: str
19 PORT: int
20
21 """
22 Map environment variables to class fields according to these rules:
23 - Field won't be parsed unless it has a type annotation
24 - Field will be skipped if not in all caps
25 - Class field and environment variable name are the same
26 """
27 def __init__(self, env):
28 for field in self.__annotations__:
29 if not field.isupper():
30 continue
31
32 # Raise AppConfigError if required field not supplied
33 default_value = getattr(self, field, None)
34 if default_value is None and env.get(field) is None:
35 raise AppConfigError('The {} field is required'.format(field))
36
37 # Cast env var value to expected type and raise AppConfigError on failure
38 try:
39 var_type = get_type_hints(AppConfig)[field]
40 if var_type == bool:
41 value = _parse_bool(env.get(field, default_value))
42 else:
43 value = var_type(env.get(field, default_value))
44
45 self.__setattr__(field, value)
46 except ValueError:
47 raise AppConfigError('Unable to cast value of "{}" to type "{}" for "{}" field'.format(
48 env[field],
49 var_type,
50 field
51 )
52 )
53
54 def __repr__(self):
55 return str(self.__dict__)
56
57# Expose Config object for app to import
58Config = AppConfig(os.environ)
The Config object exposed in config.py is then used by app.py below:
1from config import Config
2
3print('ENV: {}'.format(Config.ENV))
4print('DEBUG: {}'.format(Config.DEBUG))
5print('API_KEY: {}'.format(Config.API_KEY))
6print('HOSTNAME: {}'.format(Config.HOSTNAME))
7print('PORT: {}'.format(Config.PORT))
Make sure you have the .env file still saved from earlier, then run:
1python3 app.py
2# >> ENV: production
3# >> DEBUG: False
4# >> API_KEY: 357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
5# >> HOSTNAME: 0.0.0.0
6# >> PORT: 8080
You can view this code on GitHub and if you're after a more full-featured typesafe config solution, then check out the excellent Pydantic library.
Awesome work! Now you know how to use environment variables in Python for application config and secrets.
Although we're a bit biased, we encourage you to try using Doppler as the source of truth for your application environment variables, and it's free to get started with our Community plan (unlimited projects and secrets).
We also have a tutorial for building a random Mandalorion GIF generator in Python that puts into practice the techniques shown in this article.
Hope you enjoyed the post and if you have any questions or feedback, we'd love to chat with you over in our Community forum.
Big thanks to Stevoisiak, Olivier Pilotte, Jacob Kasner, and Alex Hall for their input and review!
If you're new to Python, check out Cory Althoff's Python environment variables primer which covers entry-level concepts in more detail.
The benefits of using environment variables in Python include the ability to deploy applications in any environment without code changes and the safeguarding of secrets like API keys from being exposed in the source code. This approach also abstracts how configurations and secrets are supplied, allowing applications to run anywhere, from local development setups to cloud platforms.
If you can't access an environment variable in your Python code, it might be because the environment variable was not set before the Python process started or there's a typo in the variable name. Python's os.environ object, which stores environment variables, only reflects variables available at the start of the process, and changes made afterward are not visible to it.
To protect your credentials using environment variables with Python, avoid hardcoding them in your source code. Instead, use environment variables to store sensitive information. This method keeps your credentials safe by not including them in your application's source code, reducing the risk of exposing them through version control or other means.
Stay up to date with new platform releases and get to know the team of experts behind them.
Trusted by the world’s best DevOps and security teams. Doppler is the secrets manager developers love.