Parallel numerical integration using python

Question

I would like to numerically integration a function using multiple cpus in python. I would like to do something like:

from scipy.integrate import quad
import multiprocessing
def FanDDW(arguments):
  wtq,eigq_files,DDB_files,EIGR2D_files,FAN_files = arguments
  ...
  return tot_corr

# Numerical integration
def integration(frequency):
# Parallelize the work over cpus
  pool = multiprocessing.Pool(processes=nb_cpus)
  total = pool.map(FanDDW, zip(wtq,eigq_files,DDB_files,EIGR2D_files,FAN_files))
  FanDDW_corr = sum(total)
  return quad(FanDDW, -Inf, Inf, args=(zip(wtq,eigq_files,DDB_files,EIGR2D_files,FAN_files)))[0]

vec_functionint = vectorize(integration)
vec_functionint(3,arange(1.0,4.0,0.5))

Also "frequency" is a global variable (external to FanDDW(arguments)). It is a vector containing the position where the function must be evaluated. I guess that quad should choose frequency in a clever way. How to pass it to FanDDW knowing that it should NOT be distributed among CPUs and that pool.map does exactly that (it is the reason why I did put it as a global variable and did not pass it to the definition as argument).

Thank you for any help.

Samuel.

Uninterrupted

Answer 1

All classical quadrature rules have the form

The computation of the f(x_i) is typically the most costly, so if you want to use multiple CPUs, you'll have to think about how to design your f. The sum can be expressed as a scalar product <w, f(x_i)>, and when using numpy.dot for it, it uses threading on most architectures.

quadpy (a project of mine) calls your integrand with all points and once, so in f you have to chance to get fancy with the computations.

import quadpy


def f(x):
    print(x.shape)  # (1, 50)
    return x[0] ** 2


scheme = quadpy.e1r2.gauss_hermite(50)
val = scheme.integrate(f)

print(val)  # 0.886226925452758

Apc