Replace German characters (umlauts, accents) with english equivalents

Question

Replace German characters (umlauts, accents) with english equivalents

I need to remove any german specific characters from various fields of text for processing into another system which wont accept them as valid.

So the characters I am aware of are:

ß ä ö ü Ä Ö Ü

At the moment I have a bit of a manual way of replacing them:

myGermanString.Replace("ä","a").Replace("ö","o").Replace("ü","u").....

But I was hoping there was a simpler / more efficient way of doing it. Since I'll be doing it on thousands of strings per run, 99% of which will not contain these chars.

Maybe a method involving some sort of CultureInfo?

(for example, according to MS, the following returns the strings are equal

String.Compare("Straße", "Strasse", StringComparison.CurrentCulture);

so there must be some sort of conversion table already existing?)

Mcloughlin

possible duplicate of How do I remove diacritics (accents) from a string in .NET? – Easing

Answer 1

The process is known as removing "diacritics" - see Removing diacritics (accents) from strings which uses the following code:

public static String RemoveDiacritics(String s)
{
  String normalizedString = s.Normalize(NormalizationForm.FormD);
  StringBuilder stringBuilder = new StringBuilder();

  for (int i = 0; i < normalizedString.Length; i++)
  {
    Char c = normalizedString[i];
    if (CharUnicodeInfo.GetUnicodeCategory(c) != UnicodeCategory.NonSpacingMark)
      stringBuilder.Append(c);
  }

  return stringBuilder.ToString();
}

Cysticercus

Can you summarise the post here. It helps to keep the information in one place and helps guard against link rot. – Soulier

What this doesnt work for is the 'ß' character - which is just returned as it was. – Mcloughlin

@Mcloughlin I believe you'll have to do a hard replace of the German characters in order to achieve the desired effect. This may be the preferable method since one-character German letters with umlauts can be mapped to two-character non-umlaut versions. See the answers to the question linked in Joe's answer for solutions. – Adjure

Answer 2

Taking inspiration from @Barry Kaye's answer I extended the function a little (and made it a String Extension. The reason for this is that we need to convert german umlauts into combinations of ascii chars eg. ä = ae.

It still uses string builder so it should be plenty fast.

You can call it like myStringVariable.RemoveDiacritics();

using System.Collections.Generic;
using System.Globalization;
using System.Text;
using System.Text.RegularExpressions;

namespace Core.Extensions
{
    public static class StringExtensions
    {
        public static IReadOnlyDictionary<string, string> SPECIAL_DIACRITICS = new Dictionary<string, string>
                                                                   {
                                                                        { "ä".Normalize(NormalizationForm.FormD), "ae".Normalize(NormalizationForm.FormD) },
                                                                        { "Ä".Normalize(NormalizationForm.FormD), "Ae".Normalize(NormalizationForm.FormD) },
                                                                        { "ö".Normalize(NormalizationForm.FormD), "oe".Normalize(NormalizationForm.FormD) },
                                                                        { "Ö".Normalize(NormalizationForm.FormD), "Oe".Normalize(NormalizationForm.FormD) },
                                                                        { "ü".Normalize(NormalizationForm.FormD), "ue".Normalize(NormalizationForm.FormD) },
                                                                        { "Ü".Normalize(NormalizationForm.FormD), "Ue".Normalize(NormalizationForm.FormD) },
                                                                        { "ß".Normalize(NormalizationForm.FormD), "ss".Normalize(NormalizationForm.FormD) },
                                                                   };

        public static string RemoveDiacritics(this string s)
        {
            if (string.IsNullOrWhiteSpace(s))
                return s;

            var stringBuilder = new StringBuilder(s.Normalize(NormalizationForm.FormD));

            // Replace certain special chars with special combinations of ascii chars (eg. german umlauts and german double s)
            foreach (KeyValuePair<string, string> keyValuePair in SPECIAL_DIACRITICS)
                stringBuilder.Replace(keyValuePair.Key, keyValuePair.Value);

            // Remove other diacritic chars eg. non spacing marks https://www.compart.com/en/unicode/category/Mn
            for (int i = 0; i < stringBuilder.Length; i++)
            {
                char c = stringBuilder[i];

                if (CharUnicodeInfo.GetUnicodeCategory(c) == UnicodeCategory.NonSpacingMark)
                    stringBuilder.Remove(i, 1);
            }

            return stringBuilder.ToString();
        }
    }
}

Alluvium