Why does Apache Commons consider '१२३' numeric?

Question

According to Apache Commons Lang's documentation for StringUtils.isNumeric(), the String '१२३' is numeric.

Since I believed this might be a mistake in the documentation, I ran tests to verify the statement. I found that according to Apache Commons it is numeric.

Why is this String numeric? What do those characters represent?

Fireworm

Perhaps they represent digits in some language. Not all languages use the symbols 0 to 9 to represent digits. – Embrocate

these are 1, 2 and 3 in hindi – Reginiaregiomontanus

You can get the integer value by Integer.parseInt("१२३"). – Cartilaginous

"ⅯⅭⅯⅬⅩⅩⅩⅤ".isnumeric() is also True (in Python, but presumably in Java too), as is "⅕".isnumeric() – Neap

See also: https://mcmap.net/q/205420/-why-is-39-39-isnumeric-false/974555 – Neap

step 1: realize that those characters are not in your alphabet. step 2: realize that it is probably a different language. step 3: let google translate figure it out: translate.google.com/#auto/en/%E0%A5%A7%E0%A5%A8%E0%A5%A9 – Leonilaleonine

@gerrit: But what about "π".isnumeric() or "e".isnumeric()? – Tillion

@Tillion Those are not numbers, those are letters that are popular to represent particular constants. Note the difference between ⅯⅭ and MC. – Neap

That's why \d is less efficient than [0-9]. Should I use \d or [0-9] to match digits in a Perl regex? – Salsify

@LưuVĩnhPhúc Not in Java. In Java, \d is a synonym for [0-9]. It won't match the Devanagari digits. – Brochu

@Reginiaregiomontanus It's not Hindi. It is Devanagari numeral. – Erv

If you're using Firefox, get the Identify Characters extension! – Haldi

@Sujan, isn't Hindi customarily written in Devanāgarī? – Haldi

@AntonSherwood Yes, and so are Marathi, Bhojpuri, Awadhi, Magahi, Maithili, Nepali, Pali, Konkani, Bodo, Sindhi and Sanskrit and many more. Devanagari is a script, like Latin, Hindi, Marathi are languages like English. – Intermittent

@AshishPatil So how can Sujan say it's not Hindi (rather than “it's not necessarily Hindi”)? – Haldi

Welcome to Unicode :) – Dichogamy

its basically sanskrit, 0 was invented in this language, if you do a simple google search on sanskrit numbers you will get this check this for reference (2indya.com/2011/06/22/sanskrit-counting-1-to-100) – Astronomy

Why did you think the documentation was in error? – Postnasal

GNU Calculator (Linux graphic app) also recognize it as numeric (however the result is showed int Arabic Numerals): १२३+0=123; १२३+100=223; १२३+0=123; १२३+123=246 – Waugh

Answer 1

Because that "CharSequence contains only Unicode digits" (quoting your linked documentation).

All of the characters return true for Character.isDigit:

Some Unicode character ranges that contain digits:

• '\u0030' through '\u0039', ISO-LATIN-1 digits ('0' through '9')
• '\u0660' through '\u0669', Arabic-Indic digits
• '\u06F0' through '\u06F9', Extended Arabic-Indic digits
• '\u0966' through '\u096F', Devanagari digits
• '\uFF10' through '\uFF19', Fullwidth digits

Many other character ranges contain digits as well.

१२३ are Devanagari digits:

• १ is DEVANAGARI DIGIT ONE, \u0967
• २ is DEVANAGARI DIGIT TWO, \u0968
• ३ is DEVANAGARI DIGIT THREE, \u0969

Kopeisk

@Joker_vD well, you've not specified which overload, so yes, sure: Integer.parseInt("222", 2). – Kopeisk

@Joker_vD It's not even hard; there are many unsupported languages. Even if so, there's the Chinise 亿, which represents 10^8 -> this to the power of 3 would cause an overflow. List of numeral systems – Wholewheat

Integer.parseInt() will probably fail if digits are not meant to be consecutive (like the japanese numbers 1, 2, 3, ...) – Uncomfortable

@CedricReichenbach: The key distinction there is that while 亿 is numeric (by the standards of having one of the non-None values of Numeric_Type, in this case Numeric_Type=Numeric), it's not any sort of digit. (Even if it were, you wouldn't take it to the power of 3; you would raise the radix to various powers, not the digits.) parseInt requires digits, and perhaps confusingly, the isNumeric method in this question tests for decimal digit characters (General_Category=Decimal_Number) instead of any broader category of numeric characters. – Vigilant

The complete set of Devangari digits is ०१२३४५६७८९. – Tillion

What did Joker_vD say? – Signory

@Signory (s)he asked if there was a way to get Integer.parseInt() to throw an exception for a 3-character numeric input string. – Kopeisk

Answer 2

The symbol १२३ is the same as 123 for the Nepali language or any other language using the Devanagari script such as Hindi, Gujarati, and so on, and is therefore is a number for Apache Commons.

Nd

That thing almost looks like "123" in Arabic numerals. – Weisler

Arabs got their numerals from Indians. – Epileptoid

@rahul Arabic numbers are 1-9, not ١-٩ as commonly thought. – Phlegm

Answer 3

You can use Character#getType to check the character's general category:

System.out.println(Character.DECIMAL_DIGIT_NUMBER == Character.getType('१'));

This will print true, which is an "evidence" that '१' is a digit number.

Now let's examine the unicode value of the '१' character:

System.out.println(Integer.toHexString('१'));
// 967

This number is on the range of Devanagari digits - which is: \u0966 through \u096F.

Also try:

Character.UnicodeBlock block = Character.UnicodeBlock.of('१');
System.out.println(block.toString());
// DEVANAGARI

Devanagari is:

is an abugida (alphasyllabary) alphabet of India and Nepal

"१२३" is a "123" (Basic Latin unicode).

Reading:

• More details on the '१' character
• StringUtils#isNumeric implementation

Phlegm

It's more significant that they're of type DECIMAL_DIGIT_NUMBER than that they're in the DEVANAGARI block. There are non-digit letters in that block too. – Kopeisk

Answer 4

If you ever want to know what properties a particular "character" has (and there are quite a few), go directly to the source: Unicode.org. They have research tools that can show you most anything you would care to know.

• If you want to see all of the properties of a specific character, try the following:

  http://unicode.org/cldr/utility/character.jsp?a=१

  or:

  http://unicode.org/cldr/utility/character.jsp?a=%E0%A5%A7

• If you want to see all characters classified as "decimal digits" (i.e. with number values of 0 through 9), try the following:

  http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Numeric_Type=Decimal:]
  ^{( 550 Code Points -- currently / as of Unicode 9.0 )}

• If you want to see all characters classified as "non-decimal digit numbers" (i.e. fractions, circled, etc), try the following:

  http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Numeric_Type=Numeric:]
  ^{( 836 Code Points -- currently / as of Unicode 9.0 )}

• If you want to see all characters classified as "decimal digits" (i.e. with number values of 0 through 9), but only up through Unicode 6.0 (which .NET uses), try the following:

  http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Numeric_Type=Decimal:]%26[:Age=6.0:]
  ^{( 420 Code Points -- and shouldn't change )}

• If you want to see all characters classified as "decimal digits" (i.e. with number values of 0 through 9), but only up through Unicode 6.0 (which .NET uses), and only in the Base-Multilingual Plane / no Supplementary Characters (i.e. nothing above Code Point 65535 / U+0xFFFF), try the following:

  http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Numeric_Type=Decimal:]%26[:Age=6.0:]%26[:bmp=Yes:]
  ^{( 350 Code Points -- and shouldn't change )}

KEEP IN MIND: The Unicode Consortium produces a specification, not software. This means that it is up to each software vendor to implement the specification as accurately as they can. So just like HTML, JavaScript, CSS, SQL, etc, there is variation between different platforms, languages, and so on. For example, I found a bug in Microsoft's .NET Framework whereby circled Latin letters A-Z and a-z -- Code Points 0x24B6 through 0x24E9 -- do not properly register as being char.IsLetter = true (bug report here). And that leads to unexpected behavior in related functionality, such as when calling the TextInfo.ToTitleCase() method (bug report here).

Enesco

Great references! (Though they do make me wonder if Unicode has gone over the top!) – Dioptric

If you want to have this sort of reference available locally, you could install uniprops. – Unclinch

@Unclinch Thanks for mentioning that. Interesting utility. It does cover some of the functionality shown in the first 3 links (the original set), but I just updated my answer to include some additional links that show more advanced queries that can be done on Unicode.org that I don't see possible via uniprops. Also, it appears that uniprops is one version behind as Unicode released version 9.0 this past June. – Enesco

Answer 5

Symbols '१२३' are actually derived from Hindi language(Basically from Sanskrit language i.e Devanagiri) which represent numeric values just like:

१ represent 1

२ represent 2

and like wise

Parakeet

CORRECTION: Symbols '१२३' are actually derived from Sanskrit language (i.e., Devanagiri script as other posters have noted) – Newspaper

I was surprised to learn how recently Devanāgarī took its present form – many centuries after Sanskrit was codified! So I'm skeptical of the claim that the digits belong more to Sanskrit than to Indian culture in general. – Haldi