Zenity --progress from Handbrake CLI output

Question

My goal is to create a gtk progress bar with the output of HandBrakeCLI using zenity --progress. I've ran into some snags and I'm wondering if someone knows of a better way or can help me with what I'm currently doing.

Normal Output:

HandBrakeCLI -i infile -o outfile --preset=iPad

Displays

Encoding: task 1 of 1, 11.97 % (72.81 fps, avg 86.78 fps, ETA 00h00m43s)

HandBrake piped to tr and cut commands so I only have the percentages that zenity will expect.

HandBrakeCLI -i infile -o outfile --preset=iPad 2>&1 | tr -s '\r' '\n' | cut -b 24-28

Results in what I would expect:

1.05 
1.06 
1.10 
1.10

But, the output is delayed a lot and sometimes won't even display. If I only use my tr expression I get the output above on each line but it's the whole output including "Encoding: task......".

It's like the cut command can't keep up with the std out of Handbrake. I read up on using named pipes, created one and directed HandBrake's output to the pipe then in another terminal tried the tr and cut command via the pipe and it results in the same delay.

Using awk's print substring also results in the same delay.

I can't figure it out. I'm after the zenity --progress indicator because my HandBrake job is called as a MythTV job and I'd like a progress bar to pop up so I know when and an encode is in process.

Nabonidus

Is it indeed that the output is delayed? or zenity truncates the % to only integer part? You may check that using tee /dev/stderr (or /dev/tty) just before zenity. – Trochelminth

The output is delayed with cut but to where it appears that the cut command can't keep up with the data. I did try piping to zenity and could never get an update on the progress bar. I figured it was because cut was having such a hard time keeping up. – Nabonidus

I just tried adding | tee /tmp/pipe1 and in another terminal did tail -f /tmp/pipe1. I get the same delayed output even without using the cut command. Just directing the raw HandBrake output through a pipe and it won't update but maybe every 10 seconds and the updates progressively get behind. – Nabonidus

So basically, I am trying to tell to find which part in the pipeline is doing the buffering or otherwise consuming the time. – Trochelminth

Between the tr and cut commands. tr -s '\r' '\n' | cut -b 24-28 It's fine out of tr but not out of cut. – Nabonidus

Answer 1

You can use the solutions answered here on stackexchange.

For example:

The stdbuf program which is part of the GNU coreutils.

stdbuf -i0 -o0 -e0 command

---

Another example is the expect command unbuffer, e.g.

unbuffer long_running_command | print_progress

unbuffer connects to long_running_command via a pseudoterminal (pty), which makes the system treat it as an interactive process, therefore not using the 4-kiB buffering in the pipeline that is the likely cause of the delay.

For longer pipelines, you may have to unbuffer each command (except the final one), e.g.

unbuffer x | unbuffer -p y | z

Nth

There is something wrong in the last command. I run unbuffer xxd -ps /usr/bin/telnet | unbuffer -p less and I get Missing filename ("less --help" for help). What is the role of the flag -p in your command? I asked this question also here unix.stackexchange.com/q/212364/16920 – Beauty

@Masi: No, there is nothing wrong. Your command is wrong, because you used unbuffer at the last command. See my answer to your question. – Nth

Answer 2

Using UNBUFFER suggested by erik to use with your first post, the following command do the trick :

( HandBrakeCLI -i infile -o outfile --preset=iPad | 
        unbuffer -p grep -i "encoding: task " | 
        unbuffer -p cut -c24-25 ) | 
        zenity --progress --auto-close

Harbird