How to store a Python dictionary as an Environment Variable [closed]

Question

I am trying to store an Environment Variable that Python can read in as a Dictionary. If this is a bad idea, please let me know your suggestions, I am eager to learn. I'm worried that if I store the data as many environment variables it might become hard to manage.

I programmed a Raspberry Pi to unlock a door based off the Caller ID's of incoming phone calls to a Twilio number, it is working great. I want to share my code on Github for feedback but I do not want to share my list of phone numbers with the world so I am trying to save it as an environment variable and then let Python read it in.

The phone numbers are in a Python Dictionary like this.

building_admins = {
    "[phone]": "De Wet",
    "[phone]": "Marysol",
    "[phone]": "Blomerus",
}

I am trying to save it as an Environment Variable like this on Ubuntu 14.04

export BUILDING_ADMINS='{
    "[phone]": "De Wet",
    "[phone]": "Marysol",
    "[phone]": "Blomerus",
}'

1) I can't get Linux to save the environment variable, is there something I could do to make it work?

2) I feel like there must be a better way to do this and I am going in the wrong direction but nothing I seem to find with Googling is solving the problem I am facing. Please point me in the right direction.

Pyrogen

Answer 1

An environment variable is not something a script user would like to set. Use the json module and a file:

import json

with open('numbers') as f:
    numbers = json.load(f)

print numbers['[phone]']    #  De Wet

When pushing to GitHub don't commit the numbers file. Maybe commit an example one and add the real one to your .gitignore.

Personality

Make sure to not upload numbers.json by either ignoring it in the .gitignore of your repo or placing it and reading it from a dotfile in the user's home directory, and therefore outside the repo (e.g. ~/.numbers.json). Also see os.path.expanduser if you choose the latter option. – Slotnick

Answer 2

I don't know if this is what you were looking for, but I ended up here while trying to save a dictionary as a Linux environment variable to consume it on my app.

What I did was saving it as a string like this:

export BUILDING_ADMINS="{'[phone]': 'De Wet','[phone]': 'Marysol','[phone]': 'Blomerus'}'

And then on your python code you read it and transform it into a dictionary using this (taken from: Convert a String representation of a Dictionary to a dictionary?):

import ast
import os

ba_dict = ast.literal_eval(os.environ["BUILDING_ADMINS"])

If you type

type(ba_dict)

You should see your string is now a dict.

<class 'dict'>

Hope this helps someone else!

Anarchism

Answer 3

There is a option named environs, unfortunately, the way of exporting is a little different from the one you want, but I will suggest you because can be an alternative for you or other people (I needed to do this another day).

https://pypi.org/project/environs/

pip install environs

You can export via terminal or even creating a .env file

export BUILDING_ADMINS=[phone]=De Wet,[phone]=Marysol, [phone]=Blomerus

In the code:

from environs import Env

env = Env()
env.read_env() // to allow reading from .env file

building_admins = env.dict('BUILDING_ADMINS', subcast=str)

result:

print(building_admins)

{
    "[phone]": "De Wet",
    "[phone]": "Marysol",
    "[phone]": "Blomerus",
}

Voiceless

Hope it works, because this is exactly what I need:-) – Pietism

Answer 4

If you choose to use the environment, you should serialize the Python dictionary as JSON and dump/load that when setting/getting the environment variable. You can access the environment using os module's environ attribute. You can dump/load JSON using the json module. You might want to watch out for a maximum length on environment variables if there is such a thing.

If I were you I would use a sqlite database, see https://docs.python.org/2/library/sqlite3.html. This would give you persistence, a defined schema, and a nice interface for handling your data.

Nary

Answer 5

I would suggest you to use environ just because of the fact that the library does some extra safety checking and has seen all kinds of use cases.

https://pypi.org/project/environs/

pip install environs

Your config.py

import environ

# Django cookiecutter does it like this.
env = environ.Env()
ROOT_DIR = (
    environ.Path(__file__)
) # let's say your configs sit at the root of the project and it's called .env
env.read_env(str(ROOT_DIR.path(".env")))

# Your env variable looks like a JSON
# There are multiple types available that you can go for if you don't think JSON is correct.
BUILDING_ADMINS = env.json('BUILDING_ADMINS')

>> from configs import *
>> print(BUILDING_ADMINS)
{
    "[phone]": "De Wet",
    "[phone]": "Marysol",
    "[phone]": "Blomerus"
}
>> type(BUILDING_ADMINS)
<class 'dict'>

Commander

Answer 6

Old thread, but here's how I do it for large dictionaries:

1- Make a compressed json object:

{"FOO":"BAR","NAME":"JOHN"}

2- base64 encode the string-dict, so it becomes something like this:

eyJGT08iOiJCQVIiLCJOQU1FIjoiSk9ITiJ9

3- Now you can export it as an env var.

4- base64 decode it in your code and do a json.loads(decoded_string) to get your dictionary :)

Horde

Answer 7

As I didn't want to have to insert a new package environs, I tried all others exposed solutions, but in Python 3.6, none worked for me.

My solution for tranform this below env variable into dict object is :

MY_ENV_VAR_DICT="{'test1':'test1','test2':'test2'}"

Code:

import os

my_dict = eval(os.environ['MY_ENV_VAR_DICT'].replace('"',''))
print(my_dict.items())

Output :

dict_items([('test1', 'test1'), ('test2', 'test2')])

This works fine for me (with no superfluous imports) ! :)

Zippy

Answer 8

It is possible! Just do

import os
os.environ.__dict__["_data"]["yourkey"] = {"key":0} #Your data
print (os.environ.__dict__["_data"]["yourkey"]) #Will print out value!

It's a little complicated but works!

Daggna

While setting os.environ['TEST'] = 'test' does mean that running !echo $TEST in an ipython shell will output the correct value, when you start trying to stuff dictionaries in it will explode. – Columelliform