ValueError: The input contains nan values - from lmfit model despite the input not containing NaNs
Asked Answered
D

1

8

I'm trying to build a model using lmfit (link to docs) and I can't seems to find out why I keep getting a ValueError: The input contains nan values when I try to fit the model.

from lmfit import minimize, Minimizer, Parameters, Parameter, report_fit, Model
import numpy as np

def cde(t, Qi, at, vw, R, rhob_cb, al, d, r):
    # t (time), is the independent variable
    return Qi / (8 * np.pi * ((at * vw)/R) * t * rhob_cb * (np.sqrt(np.pi * ((al * vw)/R * t))))  * \
        np.exp(- (R * (d - (t * vw)/ R)**2) / (4 * (al * vw) * t) - (R * r**2)/ (4 * (at * vw) * t))

model_cde =  Model(cde)


# Allowed to vary
model_cde.set_param_hint('vw', value =10**-4, min=0.000001)
model_cde.set_param_hint('d', value = -0.038, min = 0.0001)
model_cde.set_param_hint('r', value = 5.637e-10)
model_cde.set_param_hint('at', value =0.1)
model_cde.set_param_hint('al', value =0.15)

# Fixed
model_cde.set_param_hint('Qi', value = 1000, vary = False)
model_cde.set_param_hint('R', value =1.7, vary = False)
model_cde.set_param_hint('rhob_cb', value =3000, vary = False)

# test data
data = [ 1.37,  1.51,  1.65,  1.79,  1.91,  2.02,  2.12,  2.2 ,
        2.27,  2.32,  2.36,  2.38,  2.4 ,  2.41,  2.42,  2.41,  2.4 ,
        2.39,  2.37,  2.35,  2.33,  2.31,  2.29,  2.26,  2.23,  2.2 ,
        2.17,  2.14,  2.11,  2.08,  2.06,  2.02,  1.99,  1.97,  1.94,
        1.91,  1.88,  1.85,  1.83,  1.8 ,  1.78,  1.75,  1.72,  1.7 ,
        1.68,  1.65,  1.63,  1.61,  1.58]

time = list(range(5,250,5))

model_cde.fit(data, t= time)

Produces the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-785fcc6a994b> in <module>()
----> 1 model_cde.fit(data, t= time)

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/model.py in fit(self, data, params, weights, method, iter_cb, scale_covar, verbose, fit_kws, **kwargs)
    539                              scale_covar=scale_covar, fcn_kws=kwargs,
    540                              **fit_kws)
--> 541         output.fit(data=data, weights=weights)
    542         output.components = self.components
    543         return output

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/model.py in fit(self, data, params, weights, method, **kwargs)
    745         self.init_fit    = self.model.eval(params=self.params, **self.userkws)
    746 
--> 747         _ret = self.minimize(method=self.method)
    748 
    749         for attr in dir(_ret):

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/minimizer.py in minimize(self, method, params, **kws)
   1240                     val.lower().startswith(user_method)):
   1241                     kwargs['method'] = val
-> 1242         return function(**kwargs)
   1243 
   1244 

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/minimizer.py in leastsq(self, params, **kws)
   1070         np.seterr(all='ignore')
   1071 
-> 1072         lsout = scipy_leastsq(self.__residual, vars, **lskws)
   1073         _best, _cov, infodict, errmsg, ier = lsout
   1074         result.aborted = self._abort

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/scipy/optimize/minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
    385             maxfev = 200*(n + 1)
    386         retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
--> 387                                  gtol, maxfev, epsfcn, factor, diag)
    388     else:
    389         if col_deriv:

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/minimizer.py in __residual(self, fvars, apply_bounds_transformation)
    369 
    370         out = self.userfcn(params, *self.userargs, **self.userkws)
--> 371         out = _nan_policy(out, nan_policy=self.nan_policy)
    372 
    373         if callable(self.iter_cb):

/home/bprodz/.virtualenvs/phd_dev/lib/python3.5/site-packages/lmfit/minimizer.py in _nan_policy(a, nan_policy, handle_inf)
   1430 
   1431         if contains_nan:
-> 1432             raise ValueError("The input contains nan values")
   1433         return a
   1434 

ValueError: The input contains nan values

However the results of the following checks for NaNs confirms that there were no NaN values in my data:

print(np.any(np.isnan(data)), np.any(np.isnan(time)))
False False

So far I've tried converting 1 and/or both of data and time from lists to numpy ndarrays, removing the 0th time step (in case there was a dividing by 0 error), explicitly specifying the t as being independent and allowing all variables to vary. However these all throw the same error.

Does anyone have ideas what is causing this error to be thrown? Thanks.

Delouse answered 18/8, 2016 at 20:54 Comment(6)
I have no idea why it's happening, but if you use time = np.arange(.5,25.,.5) it seems to work. There must be some sort of illegal operation in the model with the time values you picked.Reality
@Reality Did you change anything else? I'm still getting the same error.. Cheers.Delouse
That's odd. I copy and pasted your code exactly up until where you assigned the time variable. Did you try starting in a fresh session? Maybe some variables are being shadowed.Reality
@Reality I started a new session and now it's working, thanks. (jupyter notebook's restart kernel didn't seem to clear everything) I also tried np.arange(5., 25., 5.) incase it was something to do with ints/floats but it threw the same error as previously. I'll think I'll keep trying to find a solution...failing that perhaps I'll need to change the units throughout my data..Delouse
@Reality You hit the nail on your head with your first comment. It seems that there's a mistake in my model somewhere which is causing some negative values to be passed to np.sqrt which evaluates them to nan by default.Delouse
Mine did the same thing until I added initial guesses..then worked like magic. Seems like a better error could be thrown?Manning
D
3

I tried to fit my model using scipy.optimize.curve_fit and got the following error:

/home/bprodz/.virtualenvs/phd_dev/lib/python3.4/site-packages/ipykernel/__main__.py:3: RuntimeWarning: invalid value encountered in sqrt
  app.launch_new_instance()

Which suggests the problem is with my model generating some negative numbers for np.sqrt(). The default behaviour for np.sqrt() when given a negative number is to output nan as per this question. NB the np.sqrt can be set to raise an error if given a negative number be setting the following: np.seterr(all='raise') source

TIP I also asked for help in the lmfit google group and received the following helpful advice:

  • Consider breaking long formulas into smaller pieces to make troubleshooting easier
  • Use Model.eval() to test what certain parameters will produce when run through your model function
  • np.ndarray is generally superior to python lists in these (numerical) situations
Delouse answered 19/8, 2016 at 16:37 Comment(1)
Thanks a lot for the useful tips and for following up!Reality

© 2022 - 2024 — McMap. All rights reserved.