Regression: Exported Bash function lost after going through another process

Question

When moving from Ubuntu 14.04 to 16.04, I've noticed several of my Bash scripts failing due to missing exported functions. I wonder whether this is related to the fixes for the Shellshock bug, even though I simply export -f the functions, and not relying on the Bash-internal function representation. The failure does not occur in a direct Bash subshell, only if there's another process in between. For example, Bash invoking awk / Perl / Vim invoking another Bash. Here's an example with Perl:

Good

$ foo() { echo "foobar"; }
$ export -f foo
$ export -f; foo
foo ()
{
    echo "foobar"
}
declare -fx foo
foobar
$ bash -c "export -f; foo"
foo ()
{
    echo "foobar"
}
declare -fx foo
foobar
$ perl -e 'system("bash -c \"export -f; foo\"")'
foo ()
{
    echo "foobar"
}
declare -fx foo
foobar
$ echo $BASH_VERSION
4.3.11(1)-release

Bad

$ foo() { echo "foobar"; }
$ export -f foo
$ export -f; foo
foo ()
{
    echo "foobar"
}
declare -fx foo
foobar
$ bash -c "export -f; foo"
foo ()
{
    echo "foobar"
}
declare -fx foo
foobar
$ perl -e 'system("bash -c \"export -f; foo\"")'
bash: foo: command not found
$ echo $BASH_VERSION
4.3.42(1)-release

Am I doing something wrong, or is this a bug?

Edit: @chepner pointed out that Bash uses specially-named shell identifiers to store the functions. When going through dash (0.5.8-2.1ubuntu2, working with 0.5.7-4ubuntu1), these identifiers are removed. With ksh, they are kept alive. I checked with

$ dash
$ sudo strings /proc/$$/environ | grep foo # Still passed from Bash to Dash
BASH_FUNC_foo%%=() {  echo "foobar"
$ bash
$ sudo strings /proc/$$/environ | grep foo # But went missing from Dash to Bash
$ exit
$ exit
$ ksh
$ sudo strings /proc/$$/environ | grep foo
BASH_FUNC_foo%%=() {  echo "foobar"
$ bash
$ sudo strings /proc/$$/environ | grep foo # Kept from Ksh to Bash
BASH_FUNC_foo%%=() {  echo "foobar"

Likewise, the behavior of Vim can be changed via :set shell=/bin/bash / :set shell=/bin/ksh

So, is dash to blame?!

Prochronism

I can't reproduce this error using 4.3.42(1)-release on OS X 10.10.5 – Pocket

Assuming system is using dash instead of bash in the new version of Ubuntu, it appears dash does not correctly pass its inherited environment to a child process. Compare env foo-bar=3 dash -c env with env foo-bar=3 bash -c env. In both cases, the shell run by the first call to env receives an environment name (foo-bar) that is not a valid shell identifier. bash pass it on to the env process it runs, but dash does not. – Pocket

@chepner: So, do you imply that the new function name-mangling of Bash uses invalid shell identifiers, and that breaks the function exports? Like my example, the roundtrip bash -> dash -> bash only works in old Ubuntu. – Prochronism

Yeah, if you export foo, the environment string is something like BASH_FUNC_foo%%=() { echo footer; }. (Newline instead of ;, though). dash seems to strip that out of its environment. A process isn't supposed to remove anything from its environment, though, as far as I know. It can ignore it, but it is supposed to pass what it gets on to any of its children. – Pocket

Answer 1

TL;DR: Known dash problem; gray area, might be fixed; better not rely on exports surviving a non-bash parent.

This is caused by a change in dash 0.5.8; cp. dash removes exported bash functions from the environment.

There is no consensus yet whether this will be fixed. POSIX seems to allow the stripping of invalid environment entries, other (more obscure) shells apparently do this as well, but it causes problems in various applications, especially because /bin/sh is symlinked to dash (and therefore the default shell) in Ubuntu.

---

My personal use case is some short utility functions that I've put in my ~/.profile, and which I reference in some shell scripts. One of those runs in an autostarted Conky daemon, and that one misses the functions because the autostart happens through dash. I can work around this. The FPATH autoload mechanism from Korn shell would be nice to have in Bash, too...

Prochronism

Note that pdksh and derivatives does it as well and has been doing it since forever. – Threonine

Answer 2

This is not ideal, but you can define your functions inside of Dash:

$ foo() { echo "foobar"; }
$ dash -c "$(declare -f); foo"
foobar

Biquarterly