Pipe output to bash function

Question

I have as simple function in a bash script and I would like to pipe stdout to it as an input.

jc_hms(){
  printf "$1"
}

I'd like to use it in this manner.

var=`echo "teststring" | jc_hms`

Of course I used redundant functions echo and printf to simplify the question, but you get the idea. Right now I get a "not found" error, which I assume means my parameter delimiting is wrong (the "$1" part). Any suggestions?

Originally the jc_hms function was used like this:

echo `jc_hms "teststring"` > //dev/tts/0

but I'd like to store the results in a variable for further processing first, before sending it to the serial port.

EDIT: So to clarify, I am NOT trying to print stuff to the serial port, I'd like to interface to my bash functions should the "|" pipe character, and I am wondering if this is possible.

EDIT: Alright, here's the full function.

jc_hms(){
  hr=$(($1 / 3600))
  min=$(($1 / 60))
  sec=$(($1 % 60))

  printf "$hs:%02d:%02d" $min $sec
}

I'm using the function to form a string which come this line of code

songplaytime=`echo $songtime | awk '{print S1 }'`
printstring="`jc_hms $songplaytime`"  #store resulting string in printstring

Where $songtime is a string expressed as "playtime totaltime" delimited by a space.

I wish I can just do this in one line, and pipe it after the awk

printstring=`echo $songtime | awk '{print S1 }' | jc_hms`

like so.

Tumbler

Your problem is that "$1" is a command-line argument to the function, not standard input, which is where the text from the pipe will be found. – Spoils

So how would I access the standard input. Example? – Tumbler

I think this might be an XY problem. Can you please update your question to tell us what you're really trying to achieve, rather than how you're trying to achieve it? – Melanimelania

jliu83: if jc_hms is on the receiving end of a pipe, stdin will be presented to the first command inside the function. But please, post what jc_hms really looks like, so we can determine the best solution in your case. – Spoils

please notice that your function is wrong. it should really be like jc_hms() { hr=$(($1 / 3600)); min=$((($1 % 3600) / 60)); sec=$((1 % 60)); printf "$hs:%02d:%02d" $min $sec; } – Osana

Answer 1

To answer your actual question, when a shell function is on the receiving end of a pipe, standard input is inherited by all commands in the function, but only commands that actually read form their standard input consume any data. For commands that run one after the other, later commands can only see what isn't consumed by previous commands. When two commands run in parallel, which commands see which data depends on how the OS schedules the commands.

Since printf is the first and only command in your function, standard input is effectively ignored. There are several ways around that, including using the read built-in to read standard input into a variable which can be passed to printf:

jc_hms () {
    read foo
    hr=$(($foo / 3600))
    min=$(($foo / 60))
    sec=$(($foo % 60))
    printf "%d:%02d:%02d" "$hr" "$min" "$sec"
}

However, since your need for a pipeline seems to depend on your perceived need to use awk, let me suggest the following alternative:

printstring=$( jc_hms $songtime )

Since songtime consists of a space-separated pair of numbers, the shell performs word-splitting on the value of songtime, and jc_hms sees two separate parameters. This requires no change in the definition of jc_hms, and no need to pipe anything into it via standard input.

If you still have a different reason for jc_hms to read standard input, please let us know.

Spoils

I had a problem related to read, try this: echo -e "\ta" |if read str;then echo "$str";fi, it will ignore the tab and print just "a", any tip? – Precis

found fix here: #7314544 – Precis

echo is far more concluent than printf, isn't it? – Institutive

echo was never fully standardized, as there were too many different (and in some cases, contradictory) implementations already in existence. (For example, the POSIX standard is for echo to process sequences like \t and \n, which bash's implementation does not do unless you use the non-standard option -e.)' printf is far more portable. – Spoils

"standard input is read by the first command executed inside the function" -- this isn't true. Rather, any command that reads from stdin will do so, assuming no commands before it has already consumed the stream. You could have 500 lines of "printf" and then, on the last line of the function, cat -- and the function would print out your 500 lines, and then cat would read the contents of stdin and write them to stdout. – Staff

@Staff Agreed, the answer is poorly worded. – Spoils

Answer 2

You can't pipe stuff directly to a bash function like that, however you can use read to pull it in instead:

jc_hms() {
  while read -r data; do
      printf "%s" "$data"
  done
}

should be what you want

Marking

it'll work as you suggested. If I replace the printf with echo "manipulated $data" and run var=$(echo "teststring" | jc_hms); echo $var from the command line I get "manipulated teststring". Edited to $(..) because backticks don't show up in comments, but your original assignment should work – Marking

This does it for me. is there any way to read all the data in one shot with a one-liner? – Griner

It should be printf "%s" "$data" or else the $data will be intrepeted as a format string. – Keikokeil

This does the correct thing, but I've noticed it's significantly slower than simply writing | printf "%s" "$data". Any advice on improving speed? – Vehemence

Answer 3

1) I know this is a pretty old post

2) I like most of the answers here

However, I found this post because I needed to something similar. While everyone agrees stdin is what needs to be used, what the answers here are missing is the actual usage of the /dev/stdin file.

Using the read builtin forces this function to be used with piped input, so it can no longer be used in a typical way. I think utilizing /dev/stdin is a superior way of solving this problem, so I wanted to add my 2 cents for completeness.

My solution:

jc_hms() { 
  declare -i i=${1:-$(</dev/stdin)};
  declare hr=$(($i/3600)) min=$(($i/60%60)) sec=$(($i%60));
  printf "%02d:%02d:%02d\n" $hr $min $sec;
}

In action:

user@hostname:pwd$ jc_hms 7800
02:10:00
user@hostname:pwd$ echo 7800 | jc_hms 
02:10:00

I hope this may help someone.

Happy hacking!

Antitrust

very helpful - I had exactly this need, to be able to process args or stdin in a function. – Cori

Very good! But seems to be a bashism (works under BASH but not with DASH) I put that in a file: #!/bin/sh jc_hms() { i="${1:-$(</dev/stdin)}" printf "%02d:%02d:02d\n" $(($i/3600)) $(($i/60%60)) $(($i%60)) } jc_hms 7800 echo 7800 | jc_hms Then chmod +x test.sh ./test.sh 02:10:00 ./test.sh: 6: ./test.sh: arithmetic expression: expecting primary: "/3600" – Outdare

absolute gold, thanks! on MacOS with bash 4.4 installed, I couldn't use declare -i, but just did my assignment: i=${1:-$(</dev/stdin)}; – Crabb

The only issue with this as it uses ${1}, it doesn't accept multiple arguments if called directly. e.g. jc_hms 7800 4321 will completely ignore the 4321 input, whereas echo 7800 4321 | jc_hms works as expected – Hoffer

@Hoffer I agree, this is made to support only a single argument. Given the original question, I'm not sure why there would be more than 1 arg, however, you just need a simple for loop wrapper to accomplish this: for x; do jc_hms "$x"; done – Antitrust

@Outdare ${1:-default} is indeed a bashism, but you can achieve assignment of default values in a POSIX manner without much trouble. Something like i="$1"; [ -z "$i" ] && i="$(</dev/stdin)" should do the trick. – Irrevocable

@SteenSchütt no… ${1:-default} is POSIX (hence not a bashism). See pubs.opengroup.org/onlinepubs/9699919799/utilities/… The bash stuff there, in fact, was $(</dev/stdin) as default value. – Outdare

Doesn't this need to read the entire /dev/stdin before proceeding to the computation? It doesn't feel very pipe-like... – Densmore

Answer 4

Or, you can also do it in a simple way.

jc_hms() {
    cat
}

Though all answers so far have disregarded the fact that this was not what OP wanted (he stated the function is simplified)

Osana

Thanks for this; now I know I can use this for piping into printf, e.g. xdotool search --onlyvisible --name 'Audacity' | printf "0x%08x\n" `cat` – Berglund

You actually can using this method, refer to my answer below: # echo 12345-1234 | printf 'Zip: %s\n' $(</dev/stdin) Zip: 12345-1234 # echo 1 2 3 4 | printf 'Number: %d\n' $(</dev/stdin) Number: 1 Number: 2 Number: 3 Number: 4 – Antitrust

@TrueFalse: Yes, that works, but it uses the shell to convert back from stdin to arguments for printf. printf never reads from stdin, there is no point piping to it. (What you do is the same as printf 'Number: %d\n' $(echo 1 2 3 4)) – Osana

@Jo So: That's true and the same can be said about the OP's question. Using it in the way that I used here is pointless but its demonstrating the method. You said you cannot pipe into printf; that's not really true, so I was simply making an argument for anyone who may very well wish to pipe to printf. – Antitrust

@TrueFalse: You can not pipe into printf. You pipe into <, which is short for cat, and let the shell perform command substitution (which puts the output of the cat as arguments to printf). – Osana

Well done. This works for a wide range of things if you want to pipe to a subsequent command. Good to know. – Enticement

I was looking for a way to use a pipe as the input to a bash function and your suggestion of using cat solved my problem. Thanks! – Zipangu

Answer 5

I like user.friendly's answer using the Bash built-in conditional unset substitution syntax. Here's a slight tweak to make his answer more generic, such as for cases with an indeterminate parameter count:

function myfunc() {
    declare MY_INPUT=${*:-$(</dev/stdin)}
    for PARAM in $MY_INPUT; do
        # do what needs to be done on each input value
    done
}

Aesthete

Could you explain how ${*:-$(</dev/stdin)} works exactly? – Acidforming

@Acidforming $1 or ${1} is the first positional parameter. $* or ${*} (also $@) are all positional parameters. In Bash parameter substitution, :- sets a default value in case the parameter you are accessing has no value. $( ) is command substitution and executes whatever is inside. < is shorthand for cat, which reads file content. In this case, the file is /dev/stdin. – Antitrust

@Antitrust agree with everything but < is shorthand cat. Bash and other modern shells special case $(< file) — "The command substitution $(cat file) can be replaced by the equivalent but faster $(< file)." - Bash Reference Manual. — $(< file) is a syntactical shortcut for $(cat file). Moreover, $(< file) does not need a separate cat process or a subshell process. Here's an in-depth discussion about $(< file) on comp.unix.shell. – Cheeky

@Antitrust in bash the < operator/keyword is used to redirect input. More specifically — "Redirection of input causes the file whose name results from the expansion of word to be opened for reading on file descriptor n, or the standard input (file descriptor 0) if n is not specified. The general format for redirecting input is: [n]<word" - Bash Reference Manual. – Cheeky

Answer 6

Hmmmm....

songplaytime=`echo $songtime | awk '{print S1 }'`
printstring="`jc_hms $songplaytime`"  #store resulting string in printstring

if you're calling awk anyway, why not use it?

printstring=`TZ=UTC gawk -vT=$songplaytime 'BEGIN{print strftime("%T",T)}'`

I'm assuming you're using Gnu's Awk, which is the best one and also free; this will work in common linux distros which aren't necessarily using the most recent gawk. The most recent versions of gawk will let you specify UTC as a third parameter to the strftime() function.

Bubaline

Answer 7

The proposed solutions require content on stdin or read to be only conditionally called. Otherwise the function will wait for content from the console and require an Enter or Ctrl+D before continuing.

A workaround is to use read with a timeout. e.g. read -t <seconds>

function test ()
{
  # ...
  # process any parameters
  # ...
  read -t 0.001 piped
  if [[ "${piped:-}" ]]; then
    echo $piped
  fi
}

Note, -t 0 did not work for me.
You might have to use a different value for the time-out. Too small a value might result in bugs and a too large time-out delays the script.

Dextrorotation

A better test for empty stdin seems to be [[ ! -t 0 ]] as in this answer: unix.stackexchange.com/a/388462/84777 – Marcheshvan

Answer 8

seems nothing works, but there are work arounds

mentioned work around xargs ref function

$ FUNCS=$(functions hi); seq 3 | xargs -I{} zsh -c "eval $FUNCS; hi {}"

then this doesn't work either because your function could reference another function. so I ended up writing some function that accepts pipe inputs, like this:

somefunc() {
    while read -r data; do
        printf "%s" "$data"
    done
}

Lucubration

Answer 9

I know that this is an old post, but it was the first result brought to me by search engines when i was looking the correct way to do it.

I prefer this way since it can cover all cases:
(a) accept data by argument like $1 inside a function
(b) accept data by pipe
(c) avoid hanging if /dev/stdin is not open - no data in pipe or no argument is given

function somefunc() {
if test -n "$1"; then
   dt="$1"
   #Read from positional argument $1;
elif test ! -t 0; then
   dt="$(</dev/stdin)"
   #Read from stdin if file descriptor /dev/stdin is open
else
   echo "No standard input."
fi
}

source: https://www.baeldung.com/linux/pipe-output-to-function

Solubilize

Answer 10

I have modified answer from 'Der Schley' https://mcmap.net/q/203993/-pipe-output-to-bash-function:

f_myfunc() {
  local data=("${@:-$(</dev/stdin)}")
  for parm in "${data[@]}" ; do
    # do what needs to be done on each input value
  done
}

Dictum