Online logistic regression model

Question

I have seen wolfram and other few websites which claim that every time we use object detection application and label it with the correct object the image represents, their algorithm uses the data to update their object detection model.

I am working on logistic regression and have implemented it successfully for the datasets that are already available to me. I am now looking forward to make this algorithm work on the fly, i.e whenever I use new dataset (it can be a single input dataset), it should be capable of updating its weight and should not start its training from scratch using the entire dataset.

Do we have any online guide or tutorial or any such material that can give me a good start here. thanks!

Whetstone

Answer 1

What you are looking for is called "online learning". A very thorough overview of how to implement an online logistic regression algorithm is here. This is not the only way to do it, but one way - and the paper provides every detail you need.

Note, however, that this is probably not what those websites are referring to. They do not update their models for every new labeled data point they get. First, they likely have a whole review process to filter out the new data they get to try and minimize noise (ie: miss-labeled data).

After that, they likely are waiting until they have accumulated enough new data that building a new model from scratch becomes worth while. There may be a number of other strategies / work going into deciding when to train again and which data to use for training.

Chockablock

I could not relate how do I use the mathematical functions mentioned in the pdf with my problem. Could you suggest some other material. Also other ways of dealing with it. Thank you! – Whetstone

If you can't understand that then there is nothing anyone can do for you. You need to improve your math skills dramatically in order to understand what is going on or how to develop what you are asking. – Chockablock

From the PDF you linked to: "First, they only require the parameters and a single training example to be stored in memory. Algorithms with this property are said to be online algorithms. In order to converge, most online algorithms require multiple passes through the data. The algorithm itself doesn’t change if it’s run over the same data more than once or run over fresh data." This is minibatch algorithm, not online learning. For the definition of online learning, see e.g. Chapter 10 in cs.huji.ac.il/~shais/UnderstandingMachineLearning – Bugger