Selected KDE bandwidth is 0. Cannot estimate density

Question

import pandas as pd
import seaborn as sns

ser_test = pd.Series([1,0,1,4,6,0,6,5,1,3,2,5,1])
sns.kdeplot(ser_test, cumulative=True)

The above code generates the following CDF graph:

But when the elements of the series are modified to:

ser_test = pd.Series([1,0,1,1,6,0,6,1,1,0,2,1,1])
sns.kdeplot(ser_test, cumulative=True)

I get the following error:

ValueError: could not convert string to float: 'scott'

RuntimeError: Selected KDE bandwidth is 0. Cannot estimate density.

What does this error mean and how can I resolve it to generate a CDF (even if it is very skewed).

Edit: I am using seaborn version 0.9.0

The complete trace is below:

ValueError: could not convert string to float: 'scott'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-93-7cee594b4526> in <module>
      1 ser_test = pd.Series([1,0,1,1,6,0,6,1,1,0,2,1,1])
----> 2 sns.kdeplot(ser_test, cumulative=True)

~/.local/lib/python3.5/site-packages/seaborn/distributions.py in kdeplot(data, data2, shade, vertical, kernel, bw, gridsize, cut, clip, legend, cumulative, shade_lowest, cbar, cbar_ax, cbar_kws, ax, **kwargs)
    689         ax = _univariate_kdeplot(data, shade, vertical, kernel, bw,
    690                                  gridsize, cut, clip, legend, ax,
--> 691                                  cumulative=cumulative, **kwargs)
    692 
    693     return ax

~/.local/lib/python3.5/site-packages/seaborn/distributions.py in _univariate_kdeplot(data, shade, vertical, kernel, bw, gridsize, cut, clip, legend, ax, cumulative, **kwargs)
    281         x, y = _statsmodels_univariate_kde(data, kernel, bw,
    282                                            gridsize, cut, clip,
--> 283                                            cumulative=cumulative)
    284     else:
    285         # Fall back to scipy if missing statsmodels

~/.local/lib/python3.5/site-packages/seaborn/distributions.py in _statsmodels_univariate_kde(data, kernel, bw, gridsize, cut, clip, cumulative)
    353     fft = kernel == "gau"
    354     kde = smnp.KDEUnivariate(data)
--> 355     kde.fit(kernel, bw, fft, gridsize=gridsize, cut=cut, clip=clip)
    356     if cumulative:
    357         grid, y = kde.support, kde.cdf

~/.local/lib/python3.5/site-packages/statsmodels/nonparametric/kde.py in fit(self, kernel, bw, fft, weights, gridsize, adjust, cut, clip)
    138             density, grid, bw = kdensityfft(endog, kernel=kernel, bw=bw,
    139                     adjust=adjust, weights=weights, gridsize=gridsize,
--> 140                     clip=clip, cut=cut)
    141         else:
    142             density, grid, bw = kdensity(endog, kernel=kernel, bw=bw,

~/.local/lib/python3.5/site-packages/statsmodels/nonparametric/kde.py in kdensityfft(X, kernel, bw, weights, gridsize, adjust, clip, cut, retgrid)
    451         bw = float(bw)
    452     except:
--> 453         bw = bandwidths.select_bandwidth(X, bw, kern) # will cross-val fit this pattern?
    454     bw *= adjust
    455 

~/.local/lib/python3.5/site-packages/statsmodels/nonparametric/bandwidths.py in select_bandwidth(x, bw, kernel)
    172         # eventually this can fall back on another selection criterion.
    173         err = "Selected KDE bandwidth is 0. Cannot estimate density."
--> 174         raise RuntimeError(err)
    175     else:
    176         return bandwidth

RuntimeError: Selected KDE bandwidth is 0. Cannot estimate density.

Luetic

please include the full traceback – Maximilianus

I have included it now. – Luetic

Answer 1

What's going on here is that Seaborn (or rather, the library it relies on to calculate the KDE - scipy or statsmodels) isn't managing to figure out the "bandwidth", a scaling parameter used in the calculation. You can pass it manually. I played with a few values and found 1.5 gave a graph at the same scale as your previous:

sns.kdeplot(ser_test, cumulative=True, bw=1.5)

See also here. Worth installing statsmodels if you don't have it.

Bowe

I have the exact same problem just with sns.jointplot and playing with bw does not help. It helps if I disable statsmodels but then all the rest looks quite bad. Any comments on this? – Laminated

best to make a new question – Bowe

Answer 2

if you don't want to wait for the seaborn git update to get released in a stable version, you can try one of the solutions in the issue page. specifically henrymartin1's suggestion to try manually passing in a small bandwidth inside a try/catch block (suggested by ahartikainen) which grabs the text of this specific error (so other errors still get raised):

try:
    sns.distplot(df)
except RuntimeError as re:
    if str(re).startswith("Selected KDE bandwidth is 0. Cannot estimate density."):
        sns.distplot(df, kde_kws={'bw': 0.1})
    else:
        raise re

This worked for me.

Cleanshaven

I had to use this argument to get the plot to work. Thanks! – Melonie

Answer 3

you have three options to try

first: showing KDE lumps with the default settings

sns.distplot(ser_test, hist = False, rug = True, rug_kws = {'color' : 'r'})

second: KDE with narrow bandwidth to show individual probability lumps

sns.distplot(ser_test, hist = False, rug = True, rug_kws = {'color' : 'r'}, kde_kws = {'bw' : 1})

third: choosing a different, triangular kernel function (lump shape)

sns.distplot(ser_test, hist = False, rug = True, rug_kws = {'color' : 'r'}, kde_kws = {'bw' : 1.5, 'kernel' : 'tri'})

Topnotch

Answer 4

The problem occurs because of statsmodels.

Anyway, to solve the issue for seaborn version starting from 0.10.0, just place diag_kws={'bw': 1} as arg.

Try to figure out the optimal value for bandwidth.

Braziel

Answer 5

pip uninstall statsmodels solved a similar problem with the same error.

Trapezium

that doesn't help if you want to use statsmodels in your package. there's an update on the git repo for seaborn that's expected to fix it here. – Cleanshaven