jq - Discrepancy in newline treatment between bash and zsh and zsh-subshell

Question

I'm trying to format a string as JSON using jq and I noticed differing behaviors on bash vs zsh; specifically when zsh runs jq directly, the outcome is different than when it runs it in subshell: \n input gets output as \\n in first case, vs as\n in the latter.

I'm puzzled and not sure what's going on there:

• Is this a known zsh behavior?
• Is this a jq bug?
• Or does it work as designed and I'm missing something?

BTW: Use newline with jq suggests to use printf %b to obtain \n instead of \\n, which works for bash,. but discrepancy in zsh between the modes is still there.

$ jq --version
jq-1.6

# ---
# Using \n directly

bash-3.2$        jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'
{"test":"A\\nB"}
bash-3.2$  OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'); echo $OUT
{"test":"A\\nB"}


zsh-5.8.1>       jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'
{"test":"A\\nB"}
zsh-5.8.1> OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'); echo $OUT
{"test":"A\nB"}

# -----
# Using `printf %b` to convert `\n` to real newline

bash-3.2$        jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'
{"test":"A\nB"}
bash-3.2$  OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'); echo $OUT
{"test":"A\nB"}


zsh-5.8.1>       jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'
{"test":"A\nB"}
zsh-5.8.1> OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'); echo $OUT
{"test":"A
B"}

Foreleg

echo is perhaps the only place I'm aware of where zsh is more compliant with the POSIX specification on default settings than bash is. (Treating backslash-escape expansion as always-on is explicitly permitted; treating it as always-off is also permitted; turning it on only when -e is present is not okay, as -n is the only thing echo is allowed to treat as a flag). – Mitigate

(because POSIX specifies that echo's output becomes unspecified when any backslash is present in input, one can argue that this makes echo -e's behavior permissible in bash... but I'd argue that this makes it permissible only when there's actually a backslash somewhere in the string, but bash's echo consumes -e as an option even when no backslashes are present in later arguments). – Mitigate

Answer 1

printf behavior is the same between both shells, as are all the shell expansions related to your jq invocation -- but default echo behavior differs.

You can avoid this by switching from echo to printf.

% OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}')
% printf '%s\n' "$OUT"
{"test":"A\nB"}

Mitigate

TIL! Thanks! From my quick test it seems that zsh echo works like bash echo -e when it comes to \n printing. – Foreleg

echo is a mess. For a fun read, see the APPLICATION USAGE and RATIONALE sections of pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html -- the very standards document defining echo recommends using printf instead. – Mitigate