Multi-threaded XML-RPC (python3.7.1)

Question

Server:

  import time                                                                   
  import random                                                                 
  from threading import Thread                                                  
  from xmlrpc.server import SimpleXMLRPCServer                                  

  class ServerThread(Thread):                                                   
      def __init__(self, server_addr):                                          
      ┆   Thread.__init__(self)                                                 
      ┆   self.server = SimpleXMLRPCServer(server_addr)                         
      ┆   self.server.register_function(sleep, 'sleep')                         

      def run(self):                                                            
      ┆   self.server.serve_forever()                                           

  # sleep for random number of seconds                                          
  def sleep():                                                                  
      r = random.randint(2,10)                                                  
      print('sleeping {} seconds'.format(r))                                    
      time.sleep(r)                                                             
      return 'slept {} seconds, exiting'.format(r)                              

  # run server                                                                  
  def run_server(host="localhost", port=8000):                                  
      server_addr = (host, port)                                                
      thread1 = ServerThread(server_addr)                                       
      thread1.start()                                                           
      print("Server thread started. Testing server ...")                        
      print('listening on {} port {}'.format(host, port))                       

  if __name__ == '__main__':                                                    
     run_server()

Client:

import xmlrpc.client

server = xmlrpc.client.ServerProxy("http://localhost:8000/", allow_none=True)

print(server.sleep())
print(server.sleep())
print(server.sleep())
print(server.sleep())

Question:

I can't create multiple ServerThread instances all listening to the same port, throws exception.

I would like to see all 4 threads executing in parallel.

What am I missing? Does a lecture on GIL follow?

Griffiths

Answer 1

The problem you've uncovered is that it's only possible to listen once on a particular port. This means that instead of starting multiple threads to listen, you need to have a single listener which delegates the response to different threads.

Python provides a ready made multithreaded server which can be used in combination with the built in server classes like SimpleXMLRPCServer.

The following code implemented a multithreaded sleep server, and see below for a multithreaded client and outputs, noting the different ordering of sleep times between the server and the client.

Multithreaded server:

import random
import time
from socketserver import ThreadingMixIn
from xmlrpc.server import SimpleXMLRPCServer


class SimpleThreadedXMLRPCServer(ThreadingMixIn, SimpleXMLRPCServer):
    pass


# sleep for random number of seconds
def sleep():
    r = random.randint(2, 10)
    print('sleeping {} seconds'.format(r))
    time.sleep(r)
    return 'slept {} seconds, exiting'.format(r)


# run server
def run_server(host="localhost", port=8000):
    server_addr = (host, port)
    server = SimpleThreadedXMLRPCServer(server_addr)
    server.register_function(sleep, 'sleep')

    print("Server thread started. Testing server ...")
    print('listening on {} port {}'.format(host, port))

    server.serve_forever()


if __name__ == '__main__':
    run_server()

Multithreaded client:

import xmlrpc.client
from concurrent.futures import ThreadPoolExecutor, as_completed

def submit_sleep():
   server = xmlrpc.client.ServerProxy("http://localhost:8000/", allow_none=True)
   return server.sleep()

with ThreadPoolExecutor() as executor:
    sleeps = {executor.submit(submit_sleep) for _ in range(4)}
    for future in as_completed(sleeps):
        sleep_time = future.result()
        print(sleep_time)

Server output:

Server thread started. Testing server ...
listening on localhost port 8000
sleeping 3 seconds
sleeping 9 seconds
sleeping 3 seconds
sleeping 10 seconds
127.0.0.1 - - [05/Dec/2018 14:32:02] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [05/Dec/2018 14:32:02] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [05/Dec/2018 14:32:08] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [05/Dec/2018 14:32:09] "POST / HTTP/1.1" 200 -

Client output:

slept 3 seconds, exiting
slept 3 seconds, exiting
slept 9 seconds, exiting
slept 10 seconds, exiting

Bellda