Confidence intervals for model prediction

Question

I am following along with a statsmodels tutorial

An OLS model is fitted with

formula = 'S ~ C(E) + C(M) + X' 
lm = ols(formula, salary_table).fit()
print lm.summary()

Predicted values are provided through:

lm.predict({'X' : [12], 'M' : [1], 'E' : [2]})

The result is returned as a single value array.

Is there a method to also return confidence intervals for the predicted value (prediction intervals) in statsmodels?

Thanks.

Acerate

Did you google "python statsmodels confidence"? – Marquardt

Yes, I looked at that. It returns the CI of the fitted values in the model results summary. I was looking for CI of predicted values. Essentially I am looking for 95% CI's of Y_hat. – Acerate

OLSResults.conf_int – Swithin

Answer 1

We've been meaning to make this easier to get to. You should be able to use

from statsmodels.sandbox.regression.predstd import wls_prediction_std
prstd, iv_l, iv_u = wls_prediction_std(results)

If you have any problems, please file an issue on github.

Trickish

Thanks @jseabold. I knew it had to be feature somewhere. – Acerate

Is there a way to get prediction intervals also for NEW data predicted by the model? The method quoted here returns prediction intervals just for training data. – Angle

@MarcoMene the wls_prediction_std() accept a second argument, exog which can be NEW data. – Cite

Answer 2

additionally you can try to use the get_prediction method.

values_to_predict = pd.DataFrame({'X' : [12], 'M' : [1], 'E' : [2]})
predictions = result.get_prediction(values_to_predict)
predictions.summary_frame(alpha=0.05)

I found the summary_frame() method buried here and you can find the get_prediction() method here. You can change the significance level of the confidence interval and prediction interval by modifying the "alpha" parameter.

Debus