Why does `ack` not produce output when used with `bash` like this?

Question

I'm guessing this has nothing to do with ack but more with bash:

Here we create file.txt containing the string foobar soack can find foobar in it:

> echo foobar > file.txt
> echo 'ack foobar file.txt' > ack.sh
> bash ack.sh
foobar
> bash < ack.sh
foobar

So far so good. But why doesn't ack find anything in it like this?

> cat ack.sh | bash
(no output)

or

> echo 'ack foobar file.txt' | bash
(no output)

Why doesn't ack find foobar in the last two cases?

Adding unbuffer (from expect) in front makes it work, which I don't understand:

> echo 'unbuffer ack foobar file.txt' | bash
foobar

Even stranger:

> cat ack2.sh
echo running
ack foobar file.txt
echo running again
unbuffer ack foobar file.txt

# Behaves as I'd expect
> bash ack2.sh
running
foobar
running again
foobar

# Strange output
> cat ack2.sh | bash
running
unbuffer ack foobar file.txt

What's up with this output? It echos unbuffer ack foobar file.txt but not running again? Huh?

Queenstown

it's an ack issue. Try to replace ack by grep and it works as expected. – Thermocouple

looks like ack isn't happy that stdin isn't a terminal when running cat ack.sh | bash. I'd file file a bug for ack. Or just use grep ;) – Thermocouple

Note that explicitly redirecting standard input of ack also solves the problem: echo 'ack foobar file.txt <&-' | bash or echo 'ack foobar file.txt </dev/null' | bash – Alternate

Answer 1

ack gets confused because stdin is a pipe rather than a terminal. You need to pass the --nofilter option to force ack to treat stdin as a tty.

This:

# ack.sh
ack --nofilter foobar file.txt

works:

$ cat ack.sh | bash
foobar

---

If you ask me, that behaviour is quite unexpected. Probably it is expected when someone understand the concepts of ack which I do not atm. I would expect that ack doesn't look at stdin when filename arguments are passed to it.

---

Why does unbuffer "solve" the problem?

unbuffer, following it's man page, does not attempt to read from stdin:

  Normally, unbuffer does not read from stdin.  This  simplifies  use  of
   unbuffer in some situations.  To use unbuffer in a pipeline, use the -p
   flag. ...

Looks like ack tries to be too! smart about stdin here. If it is empty it does not read from stdin and looks at the filenames passed to it. Again, imo it would be correct to not look at stdin at all if filename arguments are present.

Thermocouple

I would expect that ack doesn't look at stdin when filename arguments are passed to it. That's exactly right. – Cloutier

You agree that it is a bug? – Thermocouple

No, it's intentional. If ack doesn't get a filename or directory, it goes and finds files for you in the tree starting in the current directory. It will only look on stdin if there's something on stdin for it to use. – Cloutier

Yeah, but in this case we pass the filename file.txt. Imo ack must not look at stdin in that case. – Thermocouple

Answer 2

The big mismatch here is that ack was never intended to be used in shell scripts. It's meant as a command line tool for humans. That means that it makes some assumptions and optimizations for humans. For example, by default ack's output is different if it's going to a terminal vs. getting redirected in a pipe. There's also dangers in using ack in a shell script because its behavior can be affected by ackrc files and environment variables. If you're going to be using ack in a script, you should be using the --noenv flag. Better still, for shell scripts I'd use plain ol' grep.

What is the use case that brought up this problem?

Cloutier

Good to have the author here on stackoverflow :) – Thermocouple

Hi @AndyLester, Putting --noenv in my script doesn't change anything. I'm using ack and not grep because I want to --ignore-dir and --ignore-file. I'm piping the output of git diff origin/master | perl -ne <find removed function names> | ack to make sure I've handled all case where they were used. – Helmholtz

grep has --ignore-dir and --ignore-file, but they're called --exclude-dir and --exclude. And if you're piping into ack then the --ignore-dir and --ignore-file have no effect anyway. Maybe post your entire script the ack-users mailing list on Google Groups and we can help you through it there. – Cloutier

And --noenv won't affect anything here. – Cloutier

Answer 3

I agree that this is a bug – ack could look at stdin, however in a NON BLOCKING way. It is a bug to hang over a pipe that's empty…

Mesonephros