Expand environment variables in text

Question

I'm trying to write a function to perform substitutions of environment variables in java. So if I had a string that looked like this:

User ${USERNAME}'s APPDATA path is ${APPDATA}.

I want the result to be:

User msmith's APPDATA path is C:\Users\msmith\AppData\Roaming.

So far my broken implementation looks like this:

public static String expandEnvVars(String text) {        
    Map<String, String> envMap = System.getenv();
    String pattern = "\\$\\{([A-Za-z0-9]+)\\}";
    Pattern expr = Pattern.compile(pattern);
    Matcher matcher = expr.matcher(text);
    if (matcher.matches()) {
        for (int i = 1; i <= matcher.groupCount(); i++) {
            String envValue = envMap.get(matcher.group(i).toUpperCase());
            if (envValue == null) {
                envValue = "";
            } else {
                envValue = envValue.replace("\\", "\\\\");
            }
            Pattern subexpr = Pattern.compile("\\$\\{" + matcher.group(i) + "\\}");
            text = subexpr.matcher(text).replaceAll(envValue);
        }
    }
    return text;
}

Using the above sample text, matcher.matches() returns false. However if my sample text, is ${APPDATA} it works.

Can anyone help?

Constitutionality

Answer 1

You don't want to use matches(). Matches will try to match the entire input string.

Attempts to match the entire region against the pattern.

What you want is while(matcher.find()) {. That will match each instance of your pattern. Check out the documentation for find().

Within each match, group 0 will be the entire matched string (${appdata}) and group 1 will be the appdata part.

Your end result should look something like:

String pattern = "\\$\\{([A-Za-z0-9]+)\\}";
Pattern expr = Pattern.compile(pattern);
Matcher matcher = expr.matcher(text);
while (matcher.find()) {
    String envValue = envMap.get(matcher.group(1).toUpperCase());
    if (envValue == null) {
        envValue = "";
    } else {
        envValue = envValue.replace("\\", "\\\\");
    }
    Pattern subexpr = Pattern.compile(Pattern.quote(matcher.group(0)));
    text = subexpr.matcher(text).replaceAll(envValue);
}

Trajectory

Your example with matcher.group(0) throws because the ${} parts of the returned string must be escaped for this case. So I used "\\$\\{" + matcher.group(1) + "\\}" for expression instead. – Constitutionality

@Michael, thanks for pointing that out. See my edit to get that to work. (hint, Pattern.quote()) – Trajectory

Answer 2

If you don't want to write the code for yourself, the Apache Commons Lang library has a class called StrSubstitutor. It does exactly this.

Retinite

IMO, this is the cleanest solution. Implementation examples at dkbalachandar.wordpress.com/2017/10/13/…. Or, if that link doesn't work, see my answer at #58770880. – Comply

Answer 3

The following alternative has the desired effect without resorting to a library:

• reads the map of environment variables once at startup
• on calling expandEnvVars() takes the text with potential placeholders as an argument
• the method then goes through the map of environment variables, one entry at at time, fetches the entry's key and value
• and tries to replace any occurrence of ${<key>} in the text with <value>, thereby expanding the placeholders to their current values in the environment

    private static Map<String, String> envMap = System.getenv();        
    public static String expandEnvVars(String text) {        
        for (Entry<String, String> entry : envMap.entrySet()) {
            String key = entry.getKey();
            String value = entry.getValue();
            text = text.replaceAll("\\$\\{" + key + "\\}", value);
        }
        return text;
    }

Raffin

+1. This one is more compact than a library. When working under windows with paths I prefer String value = entry.getValue().replace('\\', '/'); because otherwhise the "\" char is interpreted as escape which results in an invalid paths – Profiteer

While the accepted solution works, this one is more efficient since it doesn't need to use regular expressions. – Dina

@Dave C String # replaceAll uses regular expressions under the hood – Barringer

In the future, please explain the solution instead of just dumping a code block. – Bignoniaceous

Depending on the number of set environment variables, this solution may be a lot less efficient than the accepted solution, because it scans the text for each set environment variable separately. The accepted solution scans the text once. – Tarrance

Answer 4

Based on the accepted answer but without nested pattern replacement. Supports also default value and underscores in env-name: ${env-var[:default]}

  static String substituteEnvVars(String text) {
        StringBuffer sb = new StringBuffer();
        String pattern = "\\$\\{([A-Za-z0-9_]+)(?::([^\\}]*))?\\}";
        Pattern expr = Pattern.compile(pattern);
        Matcher matcher = expr.matcher(text);
        while (matcher.find()) {
            final String varname = matcher.group(1);
            String envValue = System.getenv(varname);
            if (envValue == null) {
                envValue = matcher.group(2);
                if (envValue == null)
                    envValue = "";
            }
            matcher.appendReplacement(sb, envValue);
        }
        matcher.appendTail(sb);
        return sb.toString();
    }

In additon variables are exactly substituted once. Text "${VAR}${X}" with VAR=${X} and X=x will be return "${X}x" not "xx".

Chalkstone

@Gray, (?:X) is a non-capturing group, your change wasn't code fix. Found it already in java 1.5 docs (docs.oracle.com/javase/1.5.0/docs/api/java/util/regex/…) – Chalkstone

Sorry @lexx9999. I've been using regex for a decade now and didn't know that. I also tested your code and thought it didn't work but it does now for me. Sigh. Rolled back. – Bignoniaceous

@lexx9999, I'm naïeve; is that :default syntax from somewhere or did you make it up? – Scratchboard

Answer 5

I've taken @lexx9999 code and tweaked it a bit to use a StringBuilder and handle a Map as well as environmental variables.

private final static Pattern ENV_VAR_PATTERN =
    Pattern.compile("\\$\\{([A-Za-z0-9_.-]+)(?::([^\\}]*))?\\}");
/**
 * Handle "hello ${var}", "${var:default}" formats for environ variable expansion.
 */
public static String substituteEnvVars(String text) {
    return substituteEnvVars(text, System.getenv());
}
/**
 * Handle "hello ${var}", "${var:default}", find var in replaceMap replace value.
 */
public static String substituteEnvVars(String text, Map<String, ?> replaceMap) {
    StringBuilder sb = new StringBuilder();
    Matcher matcher = ENV_VAR_PATTERN.matcher(text);
    int index = 0;
    while (matcher.find()) {
        sb.append(text, index, matcher.start());
        String var = matcher.group(1);
        Object obj = replaceMap.get(var);
        String value;
        if (obj != null)
            value = String.valueOf(obj);
        else {
            // if no env variable, see if a default value was specified
            value = matcher.group(2);
            if (value == null)
                value = "";
        }
        sb.append(value);
        index = matcher.end();
    }
    sb.append(text, index, text.length());
    return sb.toString();
}

Bignoniaceous

Answer 6

Unless you're just trying to learn how to build things from scratch, you should not really bother implementing your own template engine - there are multitudes already available.

One very good one is FreeMarker (which has Java API). Here is a tutorial.

Splendid

I just found Apache commons org.apache.commons.exec.util.StringUtils.stringSubstitution(), it does exactly what I want, and we're already using exec. – Constitutionality

Answer 7

Another option for Windows is JNA's Kernel32Util.expandEnvironmentStrings() which calls the win32 function ExpandEnvironmentStringsForUser.

Quoting:

Expands environment-variable strings and replaces them with the values defined for the current user.

Note, variables are escaped by % (percent) signs, not $ (dollar) signs.

Example (after adding JNA to your project):

import com.sun.jna.platform.win32.Kernel32Util;

...

String desktop = Kernel32Util.expandEnvironmentStrings("%USERPROFILE%\\Desktop");

Warning: This solution uses the underlying win32 API, so it will only work for Windows platforms.

Berg