Setting the number of output nodes in scikit-learn's MLPClassifier

Question

I'm currently experimenting with scikit-learn's neural net capabilites. Is there are way to set the number of output nodes in its MLPClassifier? I know you can set the number of hidden layers by passing it as parameters like:

clf = MLPClassifier(hidden_layer_sizes=(100,)

Thanks

Zoniazoning

@konpsych Why do you need to specify optional value in outputs? What do you mean by "Can someone train the same network on two different output sizes and different input sizes?". – Surgeonfish

@VivekKumar the only reason I see not to be able to specify the input and output layer when constructing the classifier is that the same network can be trained to classify different things. But is this the case? For example you decide to train the network to recognize digits and you later decide to use the same network to recognize digits+latin characters. Is this possible? – Balanced

No, scikit-learn estimators are not meant to be extended the way you describe. The input features and output classes should be fixed in that. This is for the whole sklearn api and keeps all the internal estimators usable and compatible. Thats the reason why sklearn does not support (nor plan to support) neural networks in more depth. – Surgeonfish

Answer 1

The number of output nodes is dependent on the size of your labels.

An example of the User Guide for Neural Networks:

>>> from sklearn.neural_network import MLPClassifier
>>> X = [[0., 0.], [1., 1.]]
>>> y = [[0, 1], [1, 1]]
>>> clf = MLPClassifier(solver='lbfgs', alpha=1e-5,
...                     hidden_layer_sizes=(15,), random_state=1)
>>> clf.fit(X, y)                         
>>> clf.predict([[1., 2.]])
array([[1, 1]])

Lowering

Ohhh I get it Thanks! – Zoniazoning

Answer 2

The architecture and the units of the input, hidden and output layers in sklearn are decribed as below:

• The number of input units will be the number of features (in general +1 node for bias)
• For multiclass classification the number of output units will be the number of labels
• Try a single hidden layer first
• The more the units in a hidden layer the better, try the same as the number of input features.

---

Some general rules about the hidden layer are the following based on this paper: Approximating Number of Hidden layer neurons in Multiple Hidden Layer BPNN Architecture by Saurabh Karsoliya.

In general:

• The number of hidden layer neurons are 2/3 (or 70% to 90%) of the size of the input layer.
• The number of hidden layer neurons should be less than twice of the number of neurons in input layer.
• The size of the hidden layer neurons is between the input layer size and the output layer size.

---

Keep always in mind that you need to explore and try a lot of different combinations. Also, using GridSearch you could find the "best model and parameters". E.g. you can do a GridSearch in order to determine the "best" size of the hidden layer.

Scruggs

I was wondering if I could set 2 output neurons for binary classification in sklearn. – Alisha

For a multiclassification case with N classes, i have provided the output vector y_train in one-hot-encoded format and i also tried providing y_train with the original integer labels. Both approaches worked (the integer label gave better results) . Is MLPCLassifier still creating the output layer with N nodes for both cases? What is the difference between them, then? – Albaugh