How to write a polling function in Python?

Question

I am trying to write a function that reads the variable every 1 minute and return the value of each time. The variable name is proc:

proc = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz', '--info=progress2', source12, destination], 
                                    stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()[0]

The progress is stored in proc variable. I want the function to poll the variable every 1 minute and return value. This is done until the variable is upon execution. What's the best way to do it?

Tried using:

def doWork():
    while True:
        proc = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz', '--info=progress2', source12, destination], 
                                    stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()[0]stdout=subprocess.PIPE).communicate()[0]
        data = sort(proc)
        print data 
        time.sleep(10)

No luck though! It prints the entire progress at same time and loop over.

Blairblaire

You can check this, #1039407 – Fagot

Answer 1

The code below will run rsync and read any output from the command the moment it is available. I used the --progress option of rsync to have it print out it's progress. The progress updates sometimes end in a \n and sometimes in a \r. So I read individual characters and then form each line of characters by looking for either of those two characters. Whenever I encounter a new progress line I print it to the screen. You could chose to do anything though such as parsing the percent complete and displaying a graphical progress bar. If you are interested in seeing how to produce a progress bar in terminal check this answer out. I put an example call of the sync function and example output.

code

from subprocess import Popen, PIPE

def sync(src, dest, passwd):
    cmd = ['sshpass', '-p', passwd, 'rsync', '-avz', '--progress', src, dest]
    p = Popen(cmd, stdout=PIPE)
    line = ''
    while True:
        c = p.stdout.read(1)
        if not c:
            break
        if c in ['\n', '\r']:
            print 'rsync progress: %s' % line
            line = ''
        else:
            line += c

sync('/path/big.txt', 'myserver:/path/', 'mypassword')

output

rsync progress: sending incremental file list
rsync progress: big.txt
rsync progress: 
rsync progress:        32768   0%    0.00kB/s    0:00:00  
rsync progress:     65798144  31%   62.72MB/s    0:00:02  
rsync progress:    131596288  62%   62.77MB/s    0:00:01  
rsync progress:    197427200  94%   62.79MB/s    0:00:00  
rsync progress:    209715200 100%   62.80MB/s    0:00:03 (xfer#1, to-check=0/1)
rsync progress: 
rsync progress: sent 204032 bytes  received 31 bytes  45347.33 bytes/sec
rsync progress: total size is 209715200  speedup is 1027.70

Antipus

Thank you so much for your answer. How do I show the same thing in Django(in web browser?) – Blairblaire

@Blairblaire look at https://mcmap.net/q/169370/-how-to-stream-an-httpresponse-with-django/1699750. Check out my answer and the other answers too. – Antipus

That was quite helpful but not too specific to my question. Can you please edit your answer. – Blairblaire

@Blairblaire I recommend post a new question and give more details of exactly what you would like. Namely django, do you want a progressbar? or just text output. The answer above solves the question you posed. Also have a go at it, adapt the code, show us where you get stuck. Generally in stack overflow there isn't a culture of writing the code for you, more trying to help you get through coding issues you face. – Antipus

I will post a new question. Yes, I want the progress bar. – Blairblaire

Here's the link to the new question: #14028345 – Blairblaire