Can an SVM learn incrementally?

Question

I am using a multi-dimensional SVM classifier (SVM.NET, a wrapper for libSVM) to classify a set of features.

Given an SVM model, is it possible to incorporate new training data without having to recalculate on all previous data? I guess another way of putting it would be: is an SVM mutable?

Doak

I started going through Bishop's book to help answer this but I believe you might get a more insightful answer over at mathoverflow.net – Seve

Answer 1

Actually, it's usually called incremental learning. The question has come up before and is pretty well answered here : A few implementation details for a Support-Vector Machine (SVM).

In brief, it's possible but not easy, you would have to change the library you are using or implement the training algorithm yourself.

I found two possible solutions, SVMHeavy and LaSVM, that supports incremental training. But I haven't used either and don't know anything about them.

Verbality

Thanks, I changed my question title to be more accurate. The question you refer to is a good answer to mine. – Doak

@larsmans Good point. It's also sometimes confused/conflated with active learning. – Verbality

Answer 2

Online and incremental although similar but differ slightly. In online, its generally a single pass(epoch=1) or number of epochs could be configured. Where as, incremental would mean that you already have a model; no matter how it is built, but then model can be mutable by new examples. Also, a combination of online and incremental is often what is required.

Here is a list of tools with some remarks on the online and/or incremental SVM : https://stats.stackexchange.com/questions/30834/is-it-possible-to-append-training-data-to-existing-svm-models/51989#51989

Ancestress