Supervised learning(document classification) using deep learning techniques [closed]

Question

I was reading the papers on deep learning. Most of them refer to unsupervised learning.

They also say the neurons are pre-trained using unsupervised RBM network. Later they are fine tuned using Back propagation algorithm (supervised).

So can we solve supervised learning problems using deep learning??

I am trying to find out if deep learning can be applied for document classification problem. I know there are pretty good classifiers available. But my goal is to find out whether we can use deep learning for this purpose or not.

Renaissance

Answer 1

In short - yes, it can be, and is often used in supervised manner. Exactly as Ben J described - the "depth" of the architecture is used in the unsupervised manner in order to create very abstract representation of your data, which can be later on used as either preprocessing layer (no fine-tuning) or as an initialization for the neural network (with fine-tuning, Hinton-like). In particular you can use such approach for texts.

There is interesting very recent paper by Hinton regarding modeling the text documents with DBMs: http://www.cs.toronto.edu/~rsalakhu/papers/uai13.pdf

There are many resources available online, in particular pylearn library implements such approaches, including "classic" DBN (Deep Belief Network) http://deeplearning.net/tutorial/DBN.html

Furthermore, it is also possible to actually make classification using just stacked RBMs, this model is called "classification RBM". More details can be obtained here: http://machinelearning.org/archive/icml2008/papers/601.pdf

Gloss

Thanks lejlot and Ben. These insights are quiet helpful. – Renaissance

Feel free to accept one of provided answers (whichever better suits your needs) – Gloss

Answer 2

Deep learning is about learning unknown concepts so are typically used in terms of finding patterns in sets of data. This is unsupervised since these patterns are not necessarily known a-priori. In supervised learning, however, the type of pattern you require is easily understood a-priori in the form of training patterns which fit the data you are trying to learn about. These patterns become the basis for fitting your model (e.g. a neural network trained using back-propagation) to your data. There is no true discovery of new concepts and components. So from this perspective, I would say that no, deep learning cannot be applied to solving supervised learning problems.

Having said that, you might be able to use it to find interesting patterns in your data. You could then use these interesting patterns as a basis for training using a standard supervised approach. Perhaps this is what they did above, where you mention

"They also say the neurons are pre-trained using unsupervised RBM network. Later they are fine tuned using Back propagation algorithm (supervised)."

Without having read what you read, perhaps they started with an unsupervised algorithm to find the most interesting data and in doing so performed a form of dimensionality reduction, leading to data that was easier to train than the original data, using a supervised algorithm.

Regulator

Answer 3

Besides the good reference provided above, there is another paper by Yann Le Cunn's group that does text classification just by encoding characters without using any external feature extraction library. It works simply by encoding at the character level. They claim 98% accuracy.

http://arxiv.org/pdf/1502.01710v2.pdf

Burning

Answer 4

Yes, Definately. Actually in industry deep learning is mostly used to solve supervised problems. RBM and such unsupervised things is not widely applied outside academic circle. To know more about deep learning with supervised learning, you can read papers after 2012, start from anything related to ImageNet Challenge.

These are main milestones of deep neural networks:

Teston

Answer 5

Absolutely yes, you can use deep learning for both supervised and unsupervised learning. For example the well-known Network with is named CNN is used to unsupervised Learning. maybe you need learn something about text mining fundamentally to comprehend deep learning model that classify documents. I found this link interesting, I'm hoping that help you too: https://machinelearningmastery.com/best-practices-document-classification-deep-learning/

Trilemma

Answer 6

Yes Deep learning can be used for supervised learning. but i think what you're trying to understand is automatic feature extraction that happens inside the neural network that's what makes deep learning unique, unlike any other machine learning model that require manual human feature extraction

Lapotin

Answer 7

There are three major fields in Machine Learning

unsupervised learning

Unsupervised Learning is a field where you have bunch of data, and you want to get meaningful insight from it. For example you have user behavior data from the ecommerce website, and you want to group them somehow, Unsupervised Learning will help to catch simmilarities between them.

Reinforcement Learning

Reinforcement Learning is a field, where you create an environment and teach the target object how to behave correctly in this environment.

Supervised Learning

Supervised Learning is a field, where you have bunch of input data (features), the output data (target),and you want to find the connection between these two.

In which field does Deep Learning live?

Deep Learning is a universal function approximator, which means that you have some unknown function and you want to know what this function is, Deep Learning will help you out. By definition a function is transformation from X into Y. Therefore, Deep Learning needs both, X and Y to find the function. If the data misses either X or Y, function can’t be defined, hence you can’t use Deep Learning.

Palmerpalmerston

Answer 8

As the basis of deep learning is a neuron, e just have supervised algorithms like the logistic regression in a neuron and it means, YES.

Eveleen