Argument passing strategy - environment variables vs. command line
Asked Answered
N

4

108

Most of the applications we developers write need to be externally parametrized at startup. We pass file paths, pipe names, TCP/IP addresses etc. So far I've been using command line to pass these to the appplication being launched. I had to parse the command line in main and direct the arguments to where they're needed, which is of course a good design, but is hard to maintain for a large number of arguments. Recently I've decided to use the environment variables mechanism. They are global and accessible from anywhere, which is less elegant from architectural point of view, but limits the amount of code.

These are my first (and possibly quite shallow) impressions on both strategies but I'd like to hear opinions of more experienced developers -- What are the ups and downs of using environment variables and command line arguments to pass arguments to a process? I'd like to take into account the following matters:

  1. design quality (flexibility/maintainability),
  2. memory constraints,
  3. solution portability.

Remarks:

Ad. 1. This is the main aspect I'm interested in.

Ad. 2. This is a bit pragmatic. I know of some limitations on Windows which are currently huge (over 32kB for both command line and environment block). I guess this is not an issue though, since you just should use a file to pass tons of arguments if you need.

Ad. 3. I know almost nothing of Unix so I'm not sure whether both strategies are as similarily usable as on Windows. Elaborate on this if you please.

Nolanolan answered 16/9, 2011 at 10:30 Comment(5)
Would you give more specifics, as in the actual number of parameters? and if there are groupings to them or are they all random? and what language is this for? java, c++, etc... The reason I'm asking for that level of detail is that while it could be a problem to deal with in any language, there may be a language implementation specific solution that you aren't aware of.Baltimore
Just to mention *nix OSs, they have nothing like "global environment variable" and each env var is inherited from the parent process to child process on the fork time. So, "global" is not a pro for env var over command line, at least for Those OSs.Candlewick
Hi, @jamesDrinkard. I'm interested in general approach. If you wanted to pass 20 different labeled string/integral/real-number arguments from a Python script running by an 32-bit interpreter to a 64-bit application written in C++, what method would you use?Nolanolan
Hi, @shr. Thank you for the *nix note. As Raymond pointed out below, for this task such globality isn't a pro at all.Nolanolan
This might be relevant and advocates environmental variables: devcenter.heroku.com/articles/config-varsPersevere
C
99

1) I would recommend avoiding environmental variables as much as possible.

Pros of environmental variables

  • easy to use because they're visible from anywhere. If lots of independent programs need a piece of information, this approach is a whole lot more convenient.

Cons of environmental variables

  • hard to use correctly because they're visible (delete-able, set-able) from anywhere. If I install a new program that relies on environmental variables, are they going to stomp on my existing ones? Did I inadvertently screw up my environmental variables when I was monkeying around yesterday?

My opinion

  • use command-line arguments for those arguments which are most likely to be different for each individual invocation of the program (i.e. n for a program which calculates n!)
  • use config files for arguments which a user might reasonably want to change, but not very often (i.e. display size when the window pops up)
  • use environmental variables sparingly -- preferably only for arguments which are expected not to change (i.e. the location of the Python interpreter)
  • your point They are global and accessible from anywhere, which is less elegant from architectural point of view, but limits the amount of code reminds me of justifications for the use of global variables ;)

My scars from experiencing first-hand the horrors of environmental variable overuse

  • two programs we need at work, which can't run on the same computer at the same time due to environmental clashes
  • multiple versions of programs with the same name but different bugs -- brought an entire workshop to its knees for hours because the location of the program was pulled from the environment, and was (silently, subtly) wrong.

2) Limits

If I were pushing the limits of either what the command line can hold, or what the environment can handle, I would refactor immediately.

I've used JSON in the past for a command-line application which needed a lot of parameters. It was very convenient to be able to use dictionaries and lists, along with strings and numbers. The application only took a couple of command line args, one of which was the location of the JSON file.

Advantages of this approach

  • didn't have to write a lot of (painful) code to interact with a CLI library -- it can be a pain to get many of the common libraries to enforce complicated constraints (by 'complicated' I mean more complex than checking for a specific key or alternation between a set of keys)
  • don't have to worry about the CLI libraries requirements for order of arguments -- just use a JSON object!
  • easy to represent complicated data (answering What won't fit into command line parameters?) such as lists
  • easy to use the data from other applications -- both to create and to parse programmatically
  • easy to accommodate future extensions

Note: I want to distinguish this from the .config-file approach -- this is not for storing user configuration. Maybe I should call this the 'command-line parameter-file' approach, because I use it for a program that needs lots of values that don't fit well on the command line.


3) Solution portability: I don't know a whole lot about the differences between Mac, PC, and Linux with regard to environmental variables and command line arguments, but I can tell you:

  • all three have support for environmental variables
  • they all support command line arguments

Yes, I know -- it wasn't very helpful. I'm sorry. But the key point is that you can expect a reasonable solution to be portable, although you would definitely want to verify this for your programs (for example, are command line args case sensitive on any platforms? on all platforms? I don't know).


One last point:

As Tomasz mentioned, it shouldn't matter to most of the application where the parameters came from.

Couperin answered 28/9, 2011 at 19:6 Comment(6)
Thank you, Matt. This is a kind of opinion I've been looking for. The most important piece of advice of yours is to use environment variables for the execution environment description, which does hardly change, and cmd-file for actual execution simple/complex arguments passing. Very rational, thanks. Note though that you could use 'local' environment variables which only could mess up child processes. It's very similar to command line arguments passing, except what Raymond pointed out under Tomasz's answer.Nolanolan
Very nice answer! Concerning the disadvantage that environment variables can be changed from anywhere: There is also the possibility to set environment variables locally from a start script (e.g. Bash or Batch script) for the application. In that case there can be an system-wide default value, but an application can if necessary change the default to a custom value. What are your thoughts on that?Canning
Are there any pros and cons when considering how to pass in secrets/credentials?Operand
I agree for desktop and CLI applications. For cloud system that have many deploys, env variables are a good alternative and for example recommended in the 12factor guide: 12factor.net/configHajji
Environment variables can be set per-process in every OS I've used. Foo=bar app & Foo=baz app will run two instances of the same program, each being supplied with a different value for Foo. So, those two programs you need at work can be run at the same time despite environment variable clashes - as long as you know how to manage the variables' scopes properly.Exostosis
agreed @Exostosis -- thanks for pointing this out and TBH, my answer hasn't aged well. I wrote it thinking of desktop/CLI programs but as others point out, this advice doesn't make any sense for running in different environments (such as containers). I'll try to think of a good way to edit this answer to make it clear what context I'm thinking of. Again, thanks for pointing this out!Couperin
F
12

You should abstract reading parameters using Strategy pattern. Create an abstraction named ConfigurationSource having readConfig(key) -> value method (or returning some Configuration object/structure) with following implementations:

  • CommandLineConfigurationSource
  • EnvironmentVariableConfigurationSource
  • WindowsFileConfigurationSource - loading from a configuration file from C:/Document and settings...
  • WindowsRegistryConfigurationSource
  • NetworkConfigrationSource
  • UnixFileConfigurationSource - - loading from a configuration file from /home/user/...
  • DefaultConfigurationSource - defaults
  • ...

You can also use Chain of responsibility pattern to chain sources in various configurations like: if command line argument is not supplied, try environment variable and if everything else fails, return defauls.

Ad 1. This approach not only allows you to abstract reading configuration, but you can easily change the underlying mechanism without any affect on client code. Also you can use several sources at once, falling back or gathering configuration from different sources.

Ad 2. Just choose whichever implementation is suitable. Of course some configuration entries won't fit for instance into command line arguments.

Ad 3. If some implementations aren't portable, have two, one silently ignored/skipped when not suitable for a given system.

Fyke answered 16/9, 2011 at 10:50 Comment(6)
Thank you, this is generally a good idea. But it doesn't help with the decision whether to use the environment or the command line. Elaboration on your Ad.2.'s 'some configuration entries won't fit for instance into command line arguments' would be helpful. What won't fit into a string? If it doesn't fit, it should be probably passed indirectly in a sort of file, shouldn't it?Nolanolan
My point is: don't force user to use either command line parameters or environment variables. Be flexible (and yet preserve maintable code). I believe configuration file is the best place to store configuration (it can be arbitrarily long, contain comments, etc.), however sometimes it is useful to override file configuration using command line parameters. What won't fit into command line parameters? If you need to pass several file paths, it will probably work, but nobody likes excessively long command lines.Fyke
Configuration file's the best for arguments -- this is valuable opinion and support for comments is a good reason to use it, thank you. If you use environment variables when launching an app from a batch script, you can have a very readable form using rem and set. If you're spawning a process, you just setenv what you wish before spawnl-ing. It's convenient, readable and flexible. Why would you use .config instead of the environment? That is the question.Nolanolan
Beware that environment variables are inherited. Suppose your program has two parameters ACTION and an optional NOTIFY. Program A sets ACTION=if owner=nobody set owner=bob and NOTIFY=send then runs your program. Your program updates an item, then sees that NOTIFY is set and runs send. The send program sends email to Bob and then runs your program again, setting ACTION=set last_send = today. It doesn't want any notification, so it doesn't set NOTIFY. But it inherited NOTIFY from program A, so your program updates the last-run to today, and then runs send. Infinite loop.Ketchan
Thank you, @Raymond. The environment variables' scope is dangerously wide. Good point.Nolanolan
This must be the canonical answer for java. That's why I don't like java.Clavicembalo
D
7

I think this question has been answered rather well already, but I feel like it deserves a 2018 update. I feel like an unmentioned benefit of environmental variables is that they generally require less boiler plate code to work with. This makes for cleaner more readable code. However a major disadvatnage is that they remove a layers of isolation from different applications running on the same machine. I think this is where Docker really shines. My favorite design pattern is to exclusively use environment variables and run the application inside of a Docker container. This removes the isolation issue.

Disburse answered 9/5, 2018 at 17:22 Comment(0)
H
3

I generally agree with previous answers, but there is another important aspect: usability.

For example, in git you can create a repository with the .git directory outside of that. To specify that, you can use a command line argument --git-dir or an environmental variable GIT_DIR.

Of course, if you change the current directory to another repository or inherit environmental variables in scripts, you get a mistake. But if you need to type several git commands in a detached repository in one terminal session, this is extremely handy: you don't need to repeat the git-dir argument.

Another example is GIT_AUTHOR_NAME. It seems that it even doesn't have a command line partner (however, git commit has an --author argument). GIT_AUTHOR_NAME overrides the user.name and author.name configuration settings.

In general, usage of command line or environmental arguments is equally simple on UNIX: one can use a command line argument

$ command --arg=myarg

or an environmental variable in one line:

$ ARG=myarg command

It is also easy to capture command line arguments in an alias:

alias cfg='git --git-dir=$HOME/.cfg/ --work-tree=$HOME'  # for dotfiles
alias grep='grep --color=auto'

In general most arguments are passed through the command line. I agree with the previous answers that this is more functional and direct, and that environmental variables in scripts are like global variables in programs.

GNU libc says this:

The argv mechanism is typically used to pass command-line arguments specific to the particular program being invoked. The environment, on the other hand, keeps track of information that is shared by many programs, changes infrequently, and that is less frequently used.

Apart from what was said about dangers of environmental variables, there are good use cases of them. GNU make has a very flexible handling of environmental variables (and thus is very integrated with shell):

Every environment variable that make sees when it starts up is transformed into a make variable with the same name and value. However, an explicit assignment in the makefile, or with a command argument, overrides the environment. (-- and there is an option to change this behaviour) ...

Thus, by setting the variable CFLAGS in your environment, you can cause all C compilations in most makefiles to use the compiler switches you prefer. This is safe for variables with standard or conventional meanings because you know that no makefile will use them for other things.

Finally, I would stress that the most important for a program is not programmer, but user experience. Maybe you included that into the design aspect, but internal and external design are pretty different entities.

And a few words about programming aspects. You didn't write what language you use, but let's imagine your tools allow you the best possible argument parsing. In Python I use argparse, which is very flexible and rich. To get the parsed arguments, one can use a command like

args = parser.parse_args()

args can be further split into parsed arguments (say args.my_option), but I can also pass them as a whole to my function. This solution is absolutely not "hard to maintain for a large number of arguments" (if your language allows that). Indeed, if you have many parameters and they are not used during argument parsing, pass them in a container to their final destination and avoid code duplication (which leads to inflexibility).

And the very final comment is that it's much easier to parse environmental variables than command line arguments. An environmental variable is simply a pair, VARIABLE=value. Command line arguments can be much more complicated: they can be positional or keyword arguments, or subcommands (like git push). They can capture zero or several values (recall the command echo and flags like -vvv). See argparse for more examples.

And one more thing. Your worrying about memory is a bit disturbing. Don't write overgeneral programs. A library should be flexible, but a good program is useful without any arguments. If you need to pass a lot, this is probably data, not arguments. How to read data into a program is a much more general question with no single solution for all cases.

Homeward answered 3/2, 2021 at 8:49 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.