Remove specific characters from a string in Python

Question

I'm trying to remove specific characters from a string using Python. This is the code I'm using right now. Unfortunately, it appears to do nothing to the string.

for char in line:
    if char in " ?.!/;:":
        line.replace(char,'')

How do I do this properly?

---

_{See Why doesn't calling a string method (such as .replace or .strip) modify (mutate) the string? for the specific debugging question about what is wrong with this approach. Answers here mainly focus on how to solve the problem.}

Comparison

It's been over 5 years, but how about using the filter function and a Lambda Expression: filter(lambda ch: ch not in " ?.!/;:", line). Pretty concise and efficient too, I think. Of course, it returns a new string that you'll have to assign a name to. – Grabble

@JohnRed: Actually it returns an iterator that returns a list of characters but if you'd put this in an answer a few of us would be pleased to up-vote it. – Loosen

@BillBell: see https://mcmap.net/q/63442/-remove-specific-characters-from-a-string-in-python and https://mcmap.net/q/63442/-remove-specific-characters-from-a-string-in-python – Influent

@BillBell: PS: it's an iterator in Python3 and a string, tuple, or list in Python2 – Influent

Related: Why doesn't calling a Python string method do anything unless you assign its output? – Jackquelin

Answer 1

Strings in Python are immutable (can't be changed). Because of this, the effect of line.replace(...) is just to create a new string, rather than changing the old one. You need to rebind (assign) it to line in order to have that variable take the new value, with those characters removed.

Also, the way you are doing it is going to be kind of slow, relatively. It's also likely to be a bit confusing to experienced pythonators, who will see a doubly-nested structure and think for a moment that something more complicated is going on.

Starting in Python 2.6 and newer Python 2.x versions *, you can instead use str.translate, (see Python 3 answer below):

line = line.translate(None, '!@#$')

or regular expression replacement with re.sub

import re
line = re.sub('[!@#$]', '', line)

The characters enclosed in brackets constitute a character class. Any characters in line which are in that class are replaced with the second parameter to sub: an empty string.

Python 3 answer

In Python 3, strings are Unicode. You'll have to translate a little differently. kevpie mentions this in a comment on one of the answers, and it's noted in the documentation for str.translate.

When calling the translate method of a Unicode string, you cannot pass the second parameter that we used above. You also can't pass None as the first parameter. Instead, you pass a translation table (usually a dictionary) as the only parameter. This table maps the ordinal values of characters (i.e. the result of calling ord on them) to the ordinal values of the characters which should replace them, or—usefully to us—None to indicate that they should be deleted.

So to do the above dance with a Unicode string you would call something like

translation_table = dict.fromkeys(map(ord, '!@#$'), None)
unicode_line = unicode_line.translate(translation_table)

Here dict.fromkeys and map are used to succinctly generate a dictionary containing

{ord('!'): None, ord('@'): None, ...}

Even simpler, as another answer puts it, create the translation table in place:

unicode_line = unicode_line.translate({ord(c): None for c in '!@#$'})

Or, as brought up by Joseph Lee, create the same translation table with str.maketrans:

unicode_line = unicode_line.translate(str.maketrans('', '', '!@#$'))

---

* for compatibility with earlier Pythons, you can create a "null" translation table to pass in place of None:

import string
line = line.translate(string.maketrans('', ''), '!@#$')

Here string.maketrans is used to create a translation table, which is just a string containing the characters with ordinal values 0 to 255.

Unconformity

In Python3, line.translate takes only one argument and the first solution will not work – Haskel

In python3, str.translate() does not take the 2nd argument. So, your answer will become line.translate({ord(i):None for i in '!@#$'}) – Anonym

how do you remove " ' " that char using your method? – Aristarchus

Same as any other character. Python lets you use pairs of either single or double quotes. So you just write "'" for the character set. – Unconformity

I'm using Python 2.7.12, translate takes only one argument – Concentrated

@naveen's comment above worked for me. Pythony 2.7.13. In my case I wanted to strip " and ' characters: notes = notes.translate({ord(i):None for i in '\"\''}) – Mallorymallow

In Python 3, you can use unicode_line.translate(str.maketrans('', '', '!@#$')). Or unicode_line.translate(dict.fromkeys(map(ord, '!@#$'))) – Brit

Is there a way to also count the number of replacement/translations that happened? – Verbena

found a way: n_removed = sum([text.count(a) for a in chars_to_remove]) – Verbena

Answer 2

Am I missing the point here, or is it just the following:

string = "ab1cd1ef"
string = string.replace("1", "") 

print(string)
# result: "abcdef"

Put it in a loop:

a = "a!b@c#d$"
b = "!@#$"
for char in b:
    a = a.replace(char, "")

print(a)
# result: "abcd"

Atomizer

This will make a copy of the string in each loop, which might not be desirable. Also it is not very good Python. In Python you would loop like this instead: for char in b: a=a.replace(char,"") – Jacaranda

In order to be more efficient, put all characters in b into a set, initialize an empty list, and then iterate a, if the current character is not in the set, add it to the list. After that is done you can then convert the list back into a string. Linear time and linear (extra) space. – Botryoidal

Sure, this works, but it has a time complexity O, O(n^2) where n = len(string) – Repute

this worked for me without an issue – Danube

Answer 3

>>> line = "abc#@!?efg12;:?"
>>> ''.join( c for c in line if  c not in '?:!/;' )
'abc#@efg12'

Jenaejenda

use another string delimitor such as ''' or " – Brunell

If you have a lot of characters that are forbiden, you can speed up your code by turning it into a set first. blacklist = set('?:!/;') and then ''.join(c for c in line if c not in blacklist) – Underslung

Answer 4

With re.sub regular expression

Since Python 3.5, substitution using regular expressions re.sub became available:

import re
re.sub('\ |\?|\.|\!|\/|\;|\:', '', line)

Example

import re
line = 'Q: Do I write ;/.??? No!!!'
re.sub('\ |\?|\.|\!|\/|\;|\:', '', line)

'QDoIwriteNo'

Explanation

In regular expressions (regex), | is a logical OR and \ escapes spaces and special characters that might be actual regex commands. Whereas sub stands for substitution, in this case with the empty string ''.

Perpetual

@vitaliis If you would like to also remove newlines and returns, replace the first string in re.sub() with '\ |\?|\.|\!|\/|\;|\:|\n|\r' – Perpetual

Answer 5

The asker almost had it. Like most things in Python, the answer is simpler than you think.

>>> line = "H E?.LL!/;O:: "  
>>> for char in ' ?.!/;:':  
...  line = line.replace(char,'')  
...
>>> print line
HELLO

You don't have to do the nested if/for loop thing, but you DO need to check each character individually.

Christoperchristoph

yes i know, probably too late, but should work if you escape it. Like this: line = line.replace('`', '') read on: learnpythonthehardway.org/book/ex10.html – Mintamintage

This probably isn't performant because you're allocating a new string for every character – Minstrel

Answer 6

For the inverse requirement of only allowing certain characters in a string, you can use regular expressions with a set complement operator [^ABCabc]. For example, to remove everything except ascii letters, digits, and the hyphen:

>>> import string
>>> import re
>>>
>>> phrase = '  There were "nine" (9) chick-peas in my pocket!!!      '
>>> allow = string.letters + string.digits + '-'
>>> re.sub('[^%s]' % allow, '', phrase)

'Therewerenine9chick-peasinmypocket'

From the python regular expression documentation:

Characters that are not within a range can be matched by complementing the set. If the first character of the set is '^', all the characters that are not in the set will be matched. For example, [^5] will match any character except '5', and [^^] will match any character except '^'. ^ has no special meaning if it’s not the first character in the set.

Amarillas

Answer 7

line = line.translate(None, " ?.!/;:")

Viniferous

+1 When using unicode it requires setting up a translation to delete instead of a delete string. docs.python.org/library/stdtypes.html#str.translate – Exogamy

This is a great suggestion (ref: docs.python.org/2/library/string.html#string.translate ) The unicode note is good as well. – Ore

TypeError: translate() takes exactly one argument (2 given) – Mollymollycoddle

Answer 8

>>> s = 'a1b2c3'
>>> ''.join(c for c in s if c not in '123')
'abc'

Hogfish

My answer does provide a solution to the original question, but I was also interested (an perhaps the OP as well) in feedback as to why my solution might not be ideal. Should I have created a new question and referenced this one for context? – Hogfish

Answer 9

Strings are immutable in Python. The replace method returns a new string after the replacement. Try:

for char in line:
    if char in " ?.!/;:":
        line = line.replace(char,'')

This is identical to your original code, with the addition of an assignment to line inside the loop.

Note that the string replace() method replaces all of the occurrences of the character in the string, so you can do better by using replace() for each character you want to remove, instead of looping over each character in your string.

Succor

How can you iterate over line and modify it at the same time? – Stagger

@eumiro: The iteration proceeds over the original line. – Succor

good to know! So if I iterate over an array, I iterate over an original array. Iteration over an iterator wouldn't be possible. – Stagger

This is very wasteful. You iterate over every char of line and check whether that char is in the set of chars to remove. If it is, you remove all of its occurrences within line so why keep going through the rest of the chars in line and check all over again for chars that are guaranteed not to be there any longer? Instead I'd propose this: for char in " ?.!/;:": line = line.replace(char, "") This will have as many turns of iteration as there are chars to remove. More legible version here – Interface

Answer 10

I was surprised that no one had yet recommended using the builtin filter function.

    import operator
    import string # only for the example you could use a custom string

    s = "1212edjaq"

Say we want to filter out everything that isn't a number. Using the filter builtin method "...is equivalent to the generator expression (item for item in iterable if function(item))" [Python 3 Builtins: Filter]

    sList = list(s)
    intsList = list(string.digits)
    obj = filter(lambda x: operator.contains(intsList, x), sList)))

In Python 3 this returns

    >>  <filter object @ hex>

To get a printed string,

    nums = "".join(list(obj))
    print(nums)
    >> "1212"

I am not sure how filter ranks in terms of efficiency but it is a good thing to know how to use when doing list comprehensions and such.

UPDATE

Logically, since filter works you could also use list comprehension and from what I have read it is supposed to be more efficient because lambdas are the wall street hedge fund managers of the programming function world. Another plus is that it is a one-liner that doesnt require any imports. For example, using the same string 's' defined above,

      num = "".join([i for i in s if i.isdigit()])

That's it. The return will be a string of all the characters that are digits in the original string.

If you have a specific list of acceptable/unacceptable characters you need only adjust the 'if' part of the list comprehension.

      target_chars = "".join([i for i in s if i in some_list]) 

or alternatively,

      target_chars = "".join([i for i in s if i not in some_list])

Mcdonald

There is no reason to use operator.contains if you're using a lambda anyway. lambda x: operator.contains(intsList, x) should be spelled lambda x: x in intsList, or if you're trying to get the C-level check, intsList.__contains__ (no lambda at all) will do the trick. – Frication

Answer 11

Using filter, you'd just need one line

line = filter(lambda char: char not in " ?.!/;:", line)

This treats the string as an iterable and checks every character if the lambda returns True:

>>> help(filter)
Help on built-in function filter in module __builtin__:

filter(...)
    filter(function or None, sequence) -> list, tuple, or string

    Return those items of sequence for which function(item) is true.  If
    function is None, return the items that are true.  If sequence is a tuple
    or string, return the same type, else return a list.

Influent

Answer 12

Try this one:

def rm_char(original_str, need2rm):
    ''' Remove charecters in "need2rm" from "original_str" '''
    return original_str.translate(str.maketrans('','',need2rm))

This method works well in Python 3

Del

This seems to be the best answer for the question. – Conjecture

Definitely. I generalized it to work in Python 3. – Dasilva

Answer 13

Here's some possible ways to achieve this task:

def attempt1(string):
    return "".join([v for v in string if v not in ("a", "e", "i", "o", "u")])


def attempt2(string):
    for v in ("a", "e", "i", "o", "u"):
        string = string.replace(v, "")
    return string


def attempt3(string):
    import re
    for v in ("a", "e", "i", "o", "u"):
        string = re.sub(v, "", string)
    return string


def attempt4(string):
    return string.replace("a", "").replace("e", "").replace("i", "").replace("o", "").replace("u", "")


for attempt in [attempt1, attempt2, attempt3, attempt4]:
    print(attempt("murcielago"))

PS: Instead using " ?.!/;:" the examples use the vowels... and yeah, "murcielago" is the Spanish word to say bat... funny word as it contains all the vowels :)

PS2: If you're interested on performance you could measure these attempts with a simple code like:

import timeit


K = 1000000
for i in range(1,5):
    t = timeit.Timer(
        f"attempt{i}('murcielago')",
        setup=f"from __main__ import attempt{i}"
    ).repeat(1, K)
    print(f"attempt{i}",min(t))

In my box you'd get:

attempt1 2.2334518376057244
attempt2 1.8806643818474513
attempt3 7.214925774955572
attempt4 1.7271184513757465

So it seems attempt4 is the fastest one for this particular input.

Gand

You are creating a needless list in attempt1 and the tuple can be rewritten to "aeiou" for simplicity sakes (removing [ and ] will turn in into a generator without creating a list). You create tons of throwaway intermediary strings in attemt2, you use multiple applications of regex in attempt3 where you could use r'[aeiou]' in one pass. each one has flaws - its nice to see different ways to do things, but please fix them to be good attempts as well – Furey

@PatrickArtner You're absolutely right... from the dozens ways I've got in mind to achieve this task I've picked up the slower ones (wanted to show the OP some easiest ones)... That said, after you guys closed the other thread I've lost motivation to put more effort on this already answered old thread, so... :) . Thanks for the points though. – Gand

@PatrickArtner Ok... just for just sake added a new one, "attempt4"... haven't measured but I think that one should be the faster one – Gand

@PatrickArtner Edited... attempt4 was the fastest from the little set of attempts. Anyway, I'm not wasting more time with this stuff :) – Gand

Answer 14

Here's my Python 2/3 compatible version. Since the translate api has changed.

def remove(str_, chars):
    """Removes each char in `chars` from `str_`.

    Args:
        str_: String to remove characters from
        chars: String of to-be removed characters

    Returns:
        A copy of str_ with `chars` removed

    Example:
            remove("What?!?: darn;", " ?.!:;") => 'Whatdarn'
    """
    try:
        # Python2.x
        return str_.translate(None, chars)
    except TypeError:
        # Python 3.x
        table = {ord(char): None for char in chars}
        return str_.translate(table)

Pappas

I'd use dict.fromkeys(map(ord, '!@#$')) to create the map. – Brit

map is generally less readable than a list/dict/set/generator comprehension. So much so that Guido wanted to remove it from the language. Using fromkeys is also a bit clever and requires a doc check. – Pappas

@MartijnPieters: For Python 3, it should just be str.maketrans('', '', chars), which handles the ord conversion and dict construction all in one go (not to mention being rather more obvious in intent, since it's designed to pair with str.translate). – Frication

@Frication ... as already answered by Joseph Lee – Dasilva

@Wolf: I was responding specifically to the Martijn's point, not answering independently. Easier than expecting people to see an improvement on a comment in an unrelated answer. – Frication

@Frication Definitely. I agree that it's not as easy for Python newcomers to have the combination dict.fromkeys+map at hand if there is a str.maketrans prepared for the same purpose. I only wanted to point out that this is already contained in the set of answers since 2017. I mean you are of course right to point out that it's much easier to keep the translate+maketrans combination in mind, because also the naming reflects the relationship ... – Dasilva

Answer 15

#!/usr/bin/python
import re

strs = "how^ much for{} the maple syrup? $20.99? That's[] ricidulous!!!"
print strs
nstr = re.sub(r'[?|$|.|!|a|b]',r' ',strs)#i have taken special character to remove but any #character can be added here
print nstr
nestr = re.sub(r'[^a-zA-Z0-9 ]',r'',nstr)#for removing special character
print nestr

Karsten

Do you mean speech marks? re has backslash to escape the code and consider ' as a string. docs.python.org/2/library/re.html – Walkin

Answer 16

You can also use a function in order to substitute different kind of regular expression or other pattern with the use of a list. With that, you can mixed regular expression, character class, and really basic text pattern. It's really useful when you need to substitute a lot of elements like HTML ones.

*NB: works with Python 3.x

import re  # Regular expression library


def string_cleanup(x, notwanted):
    for item in notwanted:
        x = re.sub(item, '', x)
    return x

line = "<title>My example: <strong>A text %very% $clean!!</strong></title>"
print("Uncleaned: ", line)

# Get rid of html elements
html_elements = ["<title>", "</title>", "<strong>", "</strong>"]
line = string_cleanup(line, html_elements)
print("1st clean: ", line)

# Get rid of special characters
special_chars = ["[!@#$]", "%"]
line = string_cleanup(line, special_chars)
print("2nd clean: ", line)

In the function string_cleanup, it takes your string x and your list notwanted as arguments. For each item in that list of elements or pattern, if a substitute is needed it will be done.

The output:

Uncleaned:  <title>My example: <strong>A text %very% $clean!!</strong></title>
1st clean:  My example: A text %very% $clean!!
2nd clean:  My example: A text very clean

Haematoxylon

Answer 17

My method I'd use probably wouldn't work as efficiently, but it is massively simple. I can remove multiple characters at different positions all at once, using slicing and formatting. Here's an example:

words = "things"
removed = "%s%s" % (words[:3], words[-1:])

This will result in 'removed' holding the word 'this'.

Formatting can be very helpful for printing variables midway through a print string. It can insert any data type using a % followed by the variable's data type; all data types can use %s, and floats (aka decimals) and integers can use %d.

Slicing can be used for intricate control over strings. When I put words[:3], it allows me to select all the characters in the string from the beginning (the colon is before the number, this will mean 'from the beginning to') to the 4th character (it includes the 4th character). The reason 3 equals till the 4th position is because Python starts at 0. Then, when I put word[-1:], it means the 2nd last character to the end (the colon is behind the number). Putting -1 will make Python count from the last character, rather than the first. Again, Python will start at 0. So, word[-1:] basically means 'from the second last character to the end of the string.

So, by cutting off the characters before the character I want to remove and the characters after and sandwiching them together, I can remove the unwanted character. Think of it like a sausage. In the middle it's dirty, so I want to get rid of it. I simply cut off the two ends I want then put them together without the unwanted part in the middle.

If I want to remove multiple consecutive characters, I simply shift the numbers around in the [] (slicing part). Or if I want to remove multiple characters from different positions, I can simply sandwich together multiple slices at once.

Examples:

 words = "control"
 removed = "%s%s" % (words[:2], words[-2:])

removed equals 'cool'.

words = "impacts"
removed = "%s%s%s" % (words[1], words[3:5], words[-1])

removed equals 'macs'.

In this case, [3:5] means character at position 3 through character at position 5 (excluding the character at the final position).

Remember, Python starts counting at 0, so you will need to as well.

Principium

Answer 18

In Python 3.5

e.g.,

os.rename(file_name, file_name.translate({ord(c): None for c in '0123456789'}))

To remove all the number from the string

Charin

Answer 19

How about this:

def text_cleanup(text):
    new = ""
    for i in text:
        if i not in " ?.!/;:":
            new += i
    return new

Reine

Could you elaborate more your answer adding a little more description about the solution you provide? – Zealot

Adding to a list, then using join would be more efficient than concatenation – Minstrel

Answer 20

Below one.. with out using regular expression concept..

ipstring ="text with symbols!@#$^&*( ends here"
opstring=''
for i in ipstring:
    if i.isalnum()==1 or i==' ':
        opstring+=i
    pass
print opstring

Prosperity

Answer 21

Even the below approach works

line = "a,b,c,d,e"
alpha = list(line)
        while ',' in alpha:
            alpha.remove(',')
finalString = ''.join(alpha)
print(finalString)

output: abcde

Atonement

Answer 22

Recursive split: s=string ; chars=chars to remove

def strip(s,chars):
if len(s)==1:
    return "" if s in chars else s
return strip(s[0:int(len(s)/2)],chars) +  strip(s[int(len(s)/2):len(s)],chars)

example:

print(strip("Hello!","lo"))    #He!

Armyn

Answer 23

You could use the re module's regular expression replacement. Using the ^ expression allows you to pick exactly what you want from your string.

    import re
    text = "This is absurd!"
    text = re.sub("[^a-zA-Z]","",text) # Keeps only Alphabets
    print(text)

Output to this would be "Thisisabsurd". Only things specified after the ^ symbol will appear.

Hammack

Answer 24

# for each file on a directory, rename filename

   file_list = os.listdir (r"D:\Dev\Python")

   for file_name in file_list:

       os.rename(file_name, re.sub(r'\d+','',file_name))

Impenetrability

Answer 25

The string method replace does not modify the original string. It leaves the original alone and returns a modified copy.

What you want is something like: line = line.replace(char,'')

def replace_all(line, )for char in line:
    if char in " ?.!/;:":
        line = line.replace(char,'')
    return line

However, creating a new string each and every time that a character is removed is very inefficient. I recommend the following instead:

def replace_all(line, baddies, *):
    """
    The following is documentation on how to use the class,
    without reference to the implementation details:

    For implementation notes, please see comments begining with `#`
    in the source file.

    [*crickets chirp*]

    """

    is_bad = lambda ch, baddies=baddies: return ch in baddies
    filter_baddies = lambda ch, *, is_bad=is_bad: "" if is_bad(ch) else ch
    mahp = replace_all.map(filter_baddies, line)
    return replace_all.join('', join(mahp))

    # -------------------------------------------------
    # WHY `baddies=baddies`?!?
    #     `is_bad=is_bad`
    # -------------------------------------------------
    # Default arguments to a lambda function are evaluated
    # at the same time as when a lambda function is
    # **defined**.
    #
    # global variables of a lambda function
    # are evaluated when the lambda function is
    # **called**
    #
    # The following prints "as yellow as snow"
    #
    #     fleece_color = "white"
    #     little_lamb = lambda end: return "as " + fleece_color + end
    #
    #     # sometime later...
    #
    #     fleece_color = "yellow"
    #     print(little_lamb(" as snow"))
    # --------------------------------------------------
replace_all.map = map
replace_all.join = str.join

Treatise

Answer 26

If you want your string to be just allowed characters by using ASCII codes, you can use this piece of code:

for char in s:
    if ord(char) < 96 or ord(char) > 123:
        s = s.replace(char, "")

It will remove all the characters beyond a....z even upper cases.

Thoughtless

Answer 27

Tidbits of the following have been explained already in this thread. I am just putting it together as an answer.

"".join(filter(lambda x: x not in " ?.!/;:", line))   

Total