pinging ~ 100,000 servers, is multithreading or multiprocessing better?

Question

I have created a simple script that iterates through a list of servers that I need to both ping, and nslookup. The issue is, pinging can take some time, especially pinging more server than that are seconds in a day.

Im fairly new to programming and I understand that multiprocessing or multithreading could be a solution to make my job run faster.

My plan is to take my server list and either 1. Break it into lists of even size, with the number of lists matching the threads / processes or 2. If one of these options support it, loop through the single list passing each a new server name to a thread or process after it finishes its previous ping and nslookup. This is preferable since it ensures I spend the least time, where as if list 1 has 200 offline servers and list 6 has 2000, it will need to wait for the process using list 6 to finish, even though all others would be free at that point.

1. Which one is superior for this task and why?

2. If possible, how would I make sure that each thread or process has essentially the same runtime

code snippet even though rather simple right now

import subprocess
import time
server_file = open(r"myfilepath", "r")
initial_time = time.time()
for i in range(1000):
    print(server_file.readline()[0:-1]+ ' '+str(subprocess.run('ping '+server_file.readline()[0:-1]).returncode)) #This returns a string with the server name, and return code,
print(time.time()-initial_time)

The issue arises because a failed ping takes over 3 seconds each on average. Also I am aware that not putting the print statement will make it faster, but I wanted to monitor it for a small case. I am pinging something to the effect of 100,000 servers, and this will need to be done routinely, and the list will keep growing

Abscind

multi-threading is good for non-cpu bound tasks and multiprocessing is good for cpu bound tasks. Think about it like this: Does the task require a lot of CPU and need to run in parallel? Use multiprocessing Does the task simply need to run a bunch of concurrent processes that don't meaningfully load the CPU? Use multi-threading. – Confederacy

Note that a ping can be slow so you may want (if the server is dedicated to that) to use a lot more threads than you have CPUs as almost all of them will be waiting at a given time. You have to find the right number by testing but definitely try with a fairly large number (just not so large the OS would spend more time switching threads than executing them either). Assuming a decend dedicated server maybe 1000 thread for a start? – Hydraulics

Answer 1

TLDR; MultiThreading is the solution for you- The threading module uses threads, the multiprocessing module uses processes. The difference is that threads run in the same memory space, while processes have separate memory.

As for question 1-

For IO tasks, like querying a database or loading a webpage the CPU is just doing nothing but waiting for an answer and that's a waste of resources, thus multithreading is the answer (:

As for question 2-

You can just create pool of threads that will manage them to run simultaneously without you needing to break your head.

Negrito

Thanks, Ill look into this and if it works you'll get my vote. Currently my test on the first 1k servers took 3389 seconds to ping and boy is that not fun – Abscind

MultiThreading seems to have been the correct solution, so I upvoted you. Ive added an answer at the bottom ( but I might just edit it into my question) that shows my code that now runs 75 times faster, and I believe I could actually run more cores without any real issue – Abscind

Great! I would appreciate if you could also accept my answer(: – Negrito

Answer 2

For best performance you want neither; with 100,000 active jobs it's best to use asynchronous processing, in a single or possibly a handful of threads or processes (but not exceeding the number of available cores).

With async I/O many networking tasks can be performed in a single thread, easily achieving rates of 100,000 or more due to savings on context switching (that is, you can theoretically ping 100,000 machines in 1 second).

Python supports asynchronous I/O via asyncio (here's a nice intro into asyncio and coroutines).

It is also important to not depend on an external process like ping, because spawning a new process is a very costly operation.

aioping is an example of a native Python ping done using asyncio (note that a ping is actually a pair of ICMP request/reply packets). It should be easy to adapt it to perform multiple pings simultaneously.

Baber

I think that asyncio could be a good suggestion if he had more to do with his code than only show nslookup results. AsyncIO is all about keeping the flow of your code if it doesn't directly depend on a response which isn't the case here. – Negrito

I wouldnt considering it 100,000 jobs, its just a job that needs to be iterated 100,000 times, and therefore can be broken up into multiple tasks of 50,000, 20,000 etc – Abscind

Answer 3

+1 to Yoel's answer. Threading is definitely the way to go.

I was curious how much time it would actually save, so just wrote a script to ping google over and over:

import subprocess  
import os

import threading
from multiprocessing import Process

FNULL = open(os.devnull, 'w')
def ping(host):
    command = ['ping', '-c', '1', host]

    return subprocess.call(command, stdout=FNULL, stderr=subprocess.STDOUT) == 0

def ping_hosts(hosts, i):
    for h in hosts:
        ping(h)
        # print "%d: %s" % (i, str(ping(h)))

hosts = ["www.google.com"] * 1000
num_threads = 5

for i in range(num_threads):
    ping_hosts(hosts, i)

#for i in range(num_threads):
#    p = Process(target=ping_hosts, args=(hosts, i))
#    p.start()

#for i in range(num_threads):
#    t = threading.Thread(target=ping_hosts, args=(hosts, i))
#    t.start()

The results:

# no threading no multiprocessing
$ time python ping_hosts.py  # 5000 in a row
real    0m34.657s
user    0m5.817s
sys 0m11.436s

# multiprocessing
$ time python ping_hosts.py
real    0m8.119s
user    0m6.021s
sys 0m16.365s

# threading
$ time python ping_hosts.py
real    0m8.392s
user    0m7.453s
sys 0m16.376s

Obviously there are flaws in the test, but it's pretty clear that you get a significant boost from adding either library. Note that the savings are about the same. But, as Yoel said, since most of the time you just spend most of your time waiting threading is the way to go. Easy enough to just dump your host names into a queue and have a pool of worker threads churn through it.

Pistoia

Can you help me understand the FNULL and command line? a quick google search tells me it has to do with linux, which Ive never had any experience with. Im on windows and thats all Ive ever used – Abscind

It redirects the output from the ping to an open 'file' that goes 'nowhere'. en.m.wikipedia.org/wiki/Null_device – Pistoia

Thanks very much! Im going to share my code as an answer, but I upvoted you for the direction you gave. – Abscind

Answer 4

Multithreading can run faster than multiprocessing, since pings are not CPU intensive. It is possible to run a large number of threads. Below is my working code

import subprocess
import threading
raw_list = []
def ping(host):
    raw_list.append(host+ ' '+ str((subprocess.run('ping '+host).returncode)))
with open(r'RedactedFilePath', "r") as server_list_file:
    hosts = server_list_file.read()
    hosts_list =hosts.split('\n')
num_threads = 75
number = 0
while number< len(hosts_list):
    print(number)
    for i in range(num_threads):
        t = threading.Thread(target=ping, args=(hosts_list[number+i],))
        t.start()
    t.join()
    number = number +75

there may be a more pythonic way to ping, which may make it faster since it will not be launching subprocesses for every single one

Also, tuning the proper number of threads is machine dependent and further more dependent on what you are doing with the threads.

And of course, there should be a try statement to stop errors when the server list is not directly divisible by the number of threads.

Also, the split function should be used to remove your separator, in my case, my server list is in a text file with each one on a new line, but a csv would be .split(',') and tsv would be ('\t') etc

Abscind