How to plot decision boundary?

Question

How I might plot the decision boundary which is the weight vector of the form [w1,w2], which basically separates the two classes lets say C1 and C2, using matplotlib?

Is it as simple as plotting a line from (0,0) to the point (w1,w2) (since W is the weight "vector") if so, how do I extend this like in both directions if I need to?

Right now all I am doing is :

 import matplotlib.pyplot as plt
 plt.plot([0,w1],[0,w2])
 plt.show()

Alduino

There are some good answers for this at #22294741 – Commingle

Answer 1

Decision boundary is generally much more complex then just a line, and so (in 2d dimensional case) it is better to use the code for generic case, which will also work well with linear classifiers. The simplest idea is to plot contour plot of the decision function

# X - some data in 2dimensional np.array

x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                     np.arange(y_min, y_max, h))

# here "model" is your model's prediction (classification) function
Z = model(np.c_[xx.ravel(), yy.ravel()]) 

# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, cmap=pl.cm.Paired)
plt.axis('off')

# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=Y, cmap=pl.cm.Paired)

some examples from sklearn documentation

Wiles

so here Z would be the resulting weight vector? – Alduino

no, Z is the matrix of classifications, if your neural network consists of just two weights (and no bias) Z(x,y)=sgn(w1*x+w2*y) – Wiles

so in this case on the line where arguments are being passed to my model, I would actually pass my data that I am analyzing? or should I already have trained my classifier and then pass this data? – Alduino

I don't understand anything from your question. Provided answer shows how to plot current model decision boundary, you can plot the decision boundary of random (just initialized) model, during training or after training is done (it visualizes the current boundary) – Wiles

so if my model had only 2 classes, then z would be a list of let's say +1 or -1 values that my model returned for each input? +1 and -1 values because my output for the perceptron is +1 or -1 depending on whether the dot product of weight vector and feature vector has positive or negative value – Alduino

Ok I got how to incorporate this into my model. This does exactly what I want thank you for all your help. It is much appreciated. – Alduino

what is h in your answer above? – Roughdry

h is just a size of your meshgrid (smaller the h, more detailed the plot, but also it takes much longer to draw) – Wiles

what does this do? Z = model(np.c_[xx.ravel(), yy.ravel()]) – Paulus

I think there are several typos in the code. Sometimes you use pl.cm.Paired when it might be plt.cm.Paired. Moreover, in the last command, what is Y in c=Y? Python gives me an error with that – Johnajohnath

Y is your classification, e.g. result of clf.predict(X). "pl" is just a reference to pyplot, you might have imported it as plt, but it is up to you. – Wiles