python progress bar using tqdm not staying on a single line

Question

I am trying to run a script that tries to install modules on a centos7 system via puppet management. I want to implement a progress bar for the installation that happens along while running the script. I am using tqdm module to do this. this is a snap of how i have implemented the module:

from tqdm import tqdm
for i in tqdm(commands):
    res = run_apply(i)

Here run_apply() is the function that actually handles running and applying the puppet configuration.

So far so good, i get a progress bar but it keeps moving down the console as and when execution messages are written to the console. But, i need the progress bar to stay constant at the bottom of the console and get updated dynamically without the run messages interfering with the bar. I want the execution related messages on the console to go on as they want but the progress bar should just stay there at the bottom from start to the end of the execution.

Below is what i am seeing:

        File line: 0.00
          Package: 0.05
          Service: 0.19
             File: 0.23
             Exec: 0.23
         Last run: 1470308227
   Config retrieval: 3.90
            Total: 4.60
Version:
           Config: 1470308220
           Puppet: 3.7.3
now here x
result:  2
 38%|█████████████████████████████████████▋                                                            | 5/13 [00:29<00:51,  6.44s/it]about to:  profiles::install::download_packages
about to run puppet apply --summarize  --detailed-exitcodes --certname puppet -e "include profiles::install::download_packages"
Error: Could not find class profiles::install::download_packages for puppet on node puppet
Error: Could not find class profiles::install::download_packages for puppet on node puppet
now here x
result:  1
 46%|█████████████████████████████████████████████▏                                                    | 6/13 [00:32<00:36,  5.27s/it]about to:  profiles::install::install
about to run puppet apply --summarize  --detailed-exitcodes --certname puppet -e "include profiles::install::install"
Error: Could not find class profiles::install::install for puppet on node puppet
Error: Could not find class profiles::install::install for puppet on node puppet
now here x
result:  1
 54%|████████████████████████████████████████████████████▊                                             | 7/13 [00:34<00:26,  4.45s/it]about to:  stx_network
about to run puppet apply --summarize  --detailed-exitcodes --certname puppet -e "include stx_network"
Notice: Compiled catalog for puppet in environment production in 0.84 seconds
Notice: /Stage[main]/Stx_network/Tidy[purge unused nics]: Tidying File[/etc/sysconfig/network-scripts/ifcfg-lo]
...  

Please let me know how i can achieve what i want.

Eats

Answer 1

For the messages to be printed above the progress bar, you need to signal to tqdm that you are printing messages (else tqdm, nor any other progress bar, can not know that you are outputting messages beside the progress bar).

To do that, you can print your messages using tqdm.write(msg) instead of print(msg). If you don't want to modify run_apply() to use tqdm.write(msg) instead of print(msg), you can redirect all standard output through tqdm from the toplevel script as described here.

Skerl

Answer 2

I discovered this fantastic library: Rich.

No need to fiddle with print or replace them with tqdm.write().
It's as easy as adding from rich.progress import track and using track() instead of tqdm().

So convenient!

Wassail

Answer 3

Try Using : Progressbar

import Progressbar
progress = progressbar.ProgressBar()
for i in progress(range(30)):
    time.sleep(0.1)

It will look like this: 43% (13 of 30) |############################ | Elapsed Time: 0:00:01 ETA: 0:00:01

Tantalous

Here is the link for it : pypi.python.org/… – Tantalous

This unluckily won't solve the issue as the messages will get printed over the progress bar and thus the output will be garbled as in OP. – Skerl

If OP uses print() inside run_apply() (as it appears he does from the sample output), at best the progress bar will create a new line and you'll get multiple displays of the same bar at different states all over the terminal, or at worst the bar will erase the previous print on current line because of the \r control character which is used to clear the previous bar display, but it can erase any display including user's. – Skerl