Export multiple environment variables extracted from a single jq invocation

Question

When I use

<some commands that output a json> | jq -r '.Credentials | .AccessKeyId, .SecretKey, .SessionToken'

I get the following output:

ABCDEF
123456
AAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBB

Three distinct lines with various keys.

I would like to export these outputs as exports:

export AWS_ACCESS_KEY_ID=<the first line of the output>
export AWS_SECRET_KEY=<the second line of the output>
export AWS_SESSION_TOKEN=<the third line of the output>

How do I do that (and still remain with oneliner)?

I tried doing the following:

<some commands that output a json> | jq -r '.Credentials | .AccessKeyId, .SecretKey, .SessionToken' | export AWS_ACCESS_KEY_ID=`awk 'NR==1'`

and it works but

<some commands that output a json> | jq -r '.Credentials | .AccessKeyId, .SecretKey, .SessionToken' | export AWS_ACCESS_KEY_ID=`awk 'NR==1'; export AWS_SECRET_KEY=`awk 'NR==2'`

hangs.

I'm using zsh.

Nydia

Answer 1

An option not yet discussed by other answers is using the jq @sh filter to generate code that's directly safe to eval.

eval "$(printf '%s\n' "$json" | jq -r '
  .Credentials | ("export AWS_ACCESS_KEY=\(.AccessKeyId | @sh)",
                  "export AWS_SECRET_KEY=\(.SecretKey | @sh)",
                  "export AWS_SESSION_TOKEN=\(.SessionToken | @sh)")')"

Note that the above could trivially be one line, and was broken up to generate three separate shell commands only for the sake of readability:

eval "$(printf '%s\n' "$json" | jq -r '.Credentials | "export AWS_ACCESS_KEY=\(.AccessKeyId | @sh) AWS_SECRET_KEY=\(.SecretKey | @sh) AWS_SESSION_TOKEN=\(.SessionToken | @sh)"')"

One advantage of this approach, which no other answers currently provide as-written, is that it will correctly handle keys or tokens that contain literal newlines.

Workingman

You can also use jq -rn --argjson x "$json" '$x.Credentials | ...' to avoid the pipeline. – Xylidine

I think ".SecretKey" should be ".SecretAccessKey" – Perfecto

@BenStickley, that would be consistent with some other/common tools and so isn't unlikely, but the OP showed us existing code using .SecretKey and didn't show us their JSON or what tool they're using to generate that JSON, so it seems like we'd be stepping beyond the scope of what's in evidence to make that assertion. – Workingman

@CharlesDuffy, good call out. Agreed. – Perfecto

Answer 2

As suggested by Charles Duffy, you can use something like this:

{ read -r AWS_ACCESS_KEY_ID && read -r AWS_SECRET_KEY && read -r AWS_SESSION_TOKEN; } << EOF
$(<some commands that output a json> | jq -r '.Credentials | .AccessKeyId, .SecretKey, .SessionToken')
EOF
export AWS_ACCESS_KEY_ID AWS_SECRET_KEY AWS_SESSION_TOKEN

Also, as suggested by Charles, you can use a here string, like this:

{ read -r AWS_ACCESS_KEY_ID && read -r AWS_SECRET_KEY && read -r AWS_SESSION_TOKEN; } <<<"$(some commands that output a json | jq -r '.Credentials | .AccessKeyId, .SecretKey, .SessionToken')"
export AWS_ACCESS_KEY_ID AWS_SECRET_KEY AWS_SESSION_TOKEN

And here is a proof of concept:

$ unset a b c

$ { read -r a && read -r b && read -r c; }<< EOF
$(cat t.txt)
EOF

$ echo $a $b $c
ABCDEF 123456 AAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBB

$ unset a b c

$ { read -r a && read -r b && read -r c; }<<<"$(cat t.txt)"

$ echo $a $b $c
ABCDEF 123456 AAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBBAAAAAABBBBB

Pageboy

Doesn't work. Looks like there's an unclosed parentheses? (heredoc>). I'm using zsh - I've added that to the question. – Nydia

read reads only one line by default. You need IFS=$'\n' read -d '' AWS_... to make this work. I'd also use process substitution instead of a heredoc: IFS=$'\n' read -d '' AWS_... < <(json cmd | jq); export AWS_... – Renfroe

That's why tr '\n' ' ' is there. – Pageboy

@blahblah, post the exact command you tried. The solution I posted works. – Pageboy

Why replace the newlines with spaces when you could split on the newlines, and be able to correctly handle inputs containing literal whitespace? Just change read a b c <<EOF to { read a && read b && read c; } <<EOF -- or, better, { IFS= read -r a && IFS= read -r b && IFS= read -r c; } <<EOF to correctly handle inputs with backslashes or leading/trailing whitespace. – Workingman

@CharlesDuffy, as always, you are a 100% right. That was just a quick and dirty hack to achieve the result OP wanted. – Pageboy

@blahblah, since what you got was heredoc> -- any chance you indented the EOF? It needs to be at the beginning of the line, no leading whitespace, to be recognized (which is to say, it should indeed work if used exactly as this answer shows). – Workingman

BTW, if we're only trying to support zsh and bash, one could reasonably switch from a heredoc to a herestring; <<<"$(...) has the same effect as <<EOF $(...) EOF on three separate lines, but a bit more tersely. – Workingman

... and it has the positive collateral effect of being shorter. – Pageboy

Answer 3

This answer is built off Charles Duffy's. It's specifically for assuming a role and it's one line which is easy to alias. Remember to replace AWS_ACCOUNT_ID and ROLE_NAME with yours.

eval "$(printf '%s\n' $(aws sts assume-role --role-arn arn:aws:iam::AWS_ACCOUNT_ID:role/ROLE_NAME --role-session-name AWSCLISession) | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId | @sh) AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey | @sh) AWS_SESSION_TOKEN=\(.SessionToken | @sh)"')"

Perfecto

Answer 4

I'd do it like this:

for i in AWS_ACCESS_KEY_ID AWS_SECRET_KEY AWS_SESSION_TOKEN; do
    read "$i" &&
    export "$i"
done \
< <(json commands |
jq -r '...')

The variables are only exported if something is successfully read. If you want them exported regardless (empty), just remove the "and" operator (&&).

Renfroe

Just removing the && operator will result in an invalid command. You want to replace it with a ; instead, if you want to export the variable regardless its value. – Pageboy

@Pageboy Yes that's what I meant. I will edit the answer to make it clearer. – Renfroe