In R, how do I replace a string that contains a certain pattern with another string?

Question

I'm working on a project involving cleaning a list of data on college majors. I find that a lot are misspelled, so I was looking to use the function gsub() to replace the misspelled ones with its correct spelling. For example, say 'biolgy' is misspelled in a list of majors called Major. How can I get R to detect the misspelling and replace it with its correct spelling? I've tried gsub('biol', 'Biology', Major) but that only replaces the first four letters in 'biolgy'. If I do gsub('biolgy', 'Biology', Major), it works for that case alone, but that doesn't detect other forms of misspellings of 'biology'.

Thank you!

Thalamencephalon

Answer 1

You should either define some nifty regular expression, or use agrep from base package. stringr package is another option, I know that people use it, but I'm a very huge fan of regular expressions, so it's a no-no for me.

Anyway, agrep should do the trick:

agrep("biol", "biology")
[1] 1
agrep("biolgy", "biology")
[1] 1

EDIT:

You should also use ignore.case = TRUE, but be prepared to do some bookkeeping "by hand"...

Bingo

Thanks for your response, I played around with agrep just now. I find that it simply returns an integer (which I'm guessing corresponds to the number of strings changed), but does it actually execute the change? So for example, say I had animals = c("mouse", "dog", "cat"). If I wanted to replace mouse with, say "cheese", then can I use agrep("mou", "cheese", animals)? It returns an integer(0) when I do this. Thank you!! – Thalamencephalon

agrep returns vector indices, so you can easily assign a value by using subscripts: animals[agrep("mou", animals)] <- "cheese" – Bingo

Use value = TRUE to return the near-matches, rather than just the indicies. – Feckless

Answer 2

You can set up a vector of all the possible misspellings and then do a loop over a gsub call. Something like:

biologySp = c("biolgy","biologee","bologee","bugs")

for(sp in biologySp){
  Major = gsub(sp,"Biology",Major)
}

If you want to do something smarter, see if there's any fuzzy matching packages on CRAN, or something that uses 'soundex' matching....

The wikipedia page on approx. string matching might be useful, and try searching R-help for some of the key terms.

http://en.wikipedia.org/wiki/Approximate_string_matching

Susan

There's already fuzzy matching in base package: agrep function does that. See my answer below. – Bingo

Answer 3

You could first match the majors against a list of available majors, any not matching would then be the likely missspellings. Then use the agrep function to match these against the known majors again (agrep does approximate matching, so if it is similar to a correct value then you will get a match).

Helotism

And for that for the remaning majors, something like replace(remainingMajor,agrep("biology",remainingMajor),"biology") should do it. – Hairbreadth

(but check before with remainingMajor[agrep("biology",remainingMajor)] to see what you'll be replacing) – Hairbreadth

Answer 4

The vwr package has methods for string matching:

http://ftp.heanet.ie/mirrors/cran.r-project.org/web/packages/vwr/index.html

so your best bet might be to use the string with the minimum Levenshtein distance from the possible subject strings:

> levenshtein.distance("physcs",c("biology","physics","geography"))
  biology   physics geography 
        7         1         9 

If you get identical minima then flip a coin:

> levenshtein.distance("biolsics",c("biology","physics","geography"))
  biology   physics geography 
        4         4         8 

Susan

Answer 5

example 1a) perl/linux regex: 's/oldstring/newstring/'

example 1b) R equivalent of 1a: srcstring=sub(oldstring, newstring, srcstring)

example 2a) perl/linux regex: 's/oldstring//'

example 2b) R equivalent of 2a: srcstring=sub(oldstring, "", srcstring)

Decrial