.Net regex: what is the word character \w?

Question

Simple question:
What is the pattern for the word character \w in c#, .net?

My first thought was that it matches [A-Za-z0-9_] and the documentation tells me:

Character class    Description          Pattern     Matches
\w                 Matches any          \w          "I", "D", "A", "1", "3"
                   word character.                  in "ID A1.3"

which is not very helpful.
And \w seems to match äöü, too. What else? Is there a better (exact) definition available?

Weigel

Answer 1

From the documentation:

Word Character: \w

\w matches any word character. A word character is a member of any of the Unicode categories listed in the following table.

• Ll (Letter, Lowercase)
• Lu (Letter, Uppercase)
• Lt (Letter, Titlecase)
• Lo (Letter, Other)
• Lm (Letter, Modifier)
• Nd (Number, Decimal Digit)
• Pc (Punctuation, Connector)
  • This category includes ten characters, the most commonly used of which is the LOWLINE character (_), u+005F.

If ECMAScript-compliant behavior is specified, \w is equivalent to [a-zA-Z_0-9].

See also

• Unicode Character Database
• Unicode Characters in the 'Punctuation, Connector' Category

Argueta

So if I used \w+ this would potentially match any word no matter how crazy as long as it's contents are either lower,upper case letters, numbers 1-9 and a few (10) special characters (like the _underscore). And would be shorthand for writing something like [a-zA-Z1-9_]+ – Leesa

Answer 2

Basically it matches everything that can be considered the intuitive definition of letter in various scripts – plus the underscore and a few other oddballs.

You can find a complete list (at least for the BMP) with the following tiny PowerShell snippet:

0..65535 | ?{([char]$_) -match '\w'} | %{ "$_`: " + [char]$_ }

Chante

In general, I would use -cmatch to avoid some unexpected results of ascii/nonascii pairings. – Eadith

Answer 3

So after some research using '\w' in .NET is equivalent to:

public static class Extensions { 
    /// <summary>
    /// The word categories.
    /// </summary>
    [NotNull]
    private static readonly HashSet<UnicodeCategory> _wordCategories = new HashCollection<UnicodeCategory>(
                new[]
                {
            UnicodeCategory.DecimalDigitNumber,
            UnicodeCategory.UppercaseLetter,
            UnicodeCategory.ConnectorPunctuation,
            UnicodeCategory.LowercaseLetter,
            UnicodeCategory.OtherLetter,
            UnicodeCategory.TitlecaseLetter,
            UnicodeCategory.ModifierLetter,
            UnicodeCategory.NonSpacingMark,
                });

    /// <summary>
    /// Determines whether the specified character is a word character (equivalent to '\w').
    /// </summary>
    /// <param name="c">The c.</param>
    public static bool IsWord(this char c) => _wordCategories.Contains(char.GetUnicodeCategory(c));
}

I've written this as an extension method to be easy to use on any character c just invoke c.IsWord() which will return true if the character is a word character. This should be significantly quicker than using a Regex.

Interestingly, this doesn't appear to match the .NET specification, in fact '\w' match 938 'NonSpacingMark' characters, which are not mentioned.

In total this matches 49,760 of the 65,535 characters, so the simple regex's often shown on the web are incomplete.

Therapeutic

Thanks for this, you saved me a lot of time. I think you may also need to add "UnicodeCategory.SpacingCombiningMark" to the word categories too. I was working with Bengali (which I do not speak) and this class of character is an important word character. – Periodical