How to handle RTL languages on pre 4.2 versions of Android?

Question

Background

TextView always had issues with RTL (Right-To-Left) languages. Since I know only how to read Hebrew (in addition to English), I will talk about its issues:

• Text alignment (and I'm not talking about gravity) . As an RTL language, Hebrew puts words from right to left (compared to English which is the opposite).

  For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".Hello world" . This could be easily fixed if you had it in a single sentence, but it's harder when there are multiple punctuations characters.

• Vowels positions. Hebrew doesn't require vowels in order to read text, but sometimes it's very hard to read without them (especially the bible). For vowels, Hebrew has what is called "NIKUD", which are actually like dots inside the letters. The problem in Android was that they were usually positioned in the wrong location .

  For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".eHlol owrld" . Even if you try to fix it (put the vowels always one character after the current one), the position in the letter wasn't correct (imagine that the "e" in "Hello" would be like above the "H", for example) .

Only on version 4.2 (read here, under "Native RTL support") , Google has fixed all of the Hebrew related issues (or at least it seems so).

The problem

the problems with Hebrew has caused each Israeli carrier and each custom ROM maker have its own solution of how to fix the different issues, which makes it practically impossible to handle RTL text on pre 4.2 devices.

Things can get even more frustrating in case the text include both Hebrew and English letters.

What I've tried

I've read many websites talking about those problems, and I've tried many variants of the solutions, none has solved the problem on all devices:

• Some suggest to put the character '\u200F' (or '\u202D') at the end/start/both of the text.

• Some suggest using Html.fromHtml() method and put something special there.

• Some even suggest to use the WebView instead (and maybe use WebSettings.setDefaultTextEncodingName() ).

The question

Is there a definite solution for this problem?

I would assume the best thing is that because Android 4.2 solves this, and Android is open source, we should have its TextView imported into a library that we can use, but Google hasn't provided such a library yet.

Unnamed

it's the same with Arabic and Persian – Otisotitis

@Delta7 Yes, as they are RTL languages too. – Unnamed

Answer 1

Sadly, I don't think there's a good solution ("good" meaning "does the job and is readily available"). For our own Android apps that support Hebrew, we use a custom rendering mechanism that we developed over many years. The rendering mechanism does everything: proprietary fonts; bidirectional (bidi) analysis; glyph placement; line break analysis; text flow; etc. Some of the problems trying to use native Android text handling capabilities (especially pre-4.2) are:

1. Really crappy fonts. However, you can package third-party fonts like DejaVu that are pretty good. The right font can do wonders with positioning of nekudot—and te'amim¹, if you need that. (I agree with you about how important correct pointing placement is; reading Hebrew text with misplaced nekudot is like reading a screen-full of CAPTCHAs.)

2. Buggy bidi analysis. What makes it worse is that the bugs seem to be different for different versions of Android. Modifying the text to include strategically placed bidi formatting codes (RTL mark; LTR mark; etc.) can overcome many of these bugs (see the discussion here, which isn't specific to Android). However, it's a nuisance to do this and, because of the inconsistencies among Android versions, it is difficult to predict in advance what help the framework is going to need.

3. No (or poorly thought out) framework-level awareness of right-to-left issues. For instance, good luck getting the scroll bar to display on the left side of a Hebrew TextView. For our apps, we had to build an entire scroll-bar system just to get this to work how we wanted. (Good think Android is open source!)

4. Poor line and word break analysis. At least one early version of Android on which we tested thought that each nikud mark was a word boundary. When it comes to line breaks, the system often doesn't know how to handle Hebrew punctuation like maqaf, gershayim, or sof pasuk.

5. Some of the newer Unicode characters (like HOLAM HASER FOR VAV—U+05BA—new to Unicode 5.0) are not recognized as Hebrew script by the system.

My recommendation is that, unless you are prepared to build a top-to-bottom text handling system yourself, you give up on high-quality text display on pre-4.2 versions of Android, particularly if you need to support nekudot and te'amim. Also, plan to use the techniques I mentioned in the first two points above.

¹ biblical cantillation marks

Dipper

You actually say the exact problem I faced. About the RTL mark, I still don't get how to use it, as i've tried multiple combinations and none has worked, at least not on the devices i've tried. Do you have a library that you've created? I think the best solution is to use the android 4.2 code for the textView, but i also think that since it's so complex, such a thing could be very hard to accomplish . – Unnamed

@androiddeveloper - Let's let upper-case characters be RTL. Suppose the logical sequence is "ABCDE."; you want it to display ".EDCBA" (with the period at the left) but it's coming out "EDCBA." (because Android always uses a left-to-right base flow). If you were to add a right-to-left character after the period it would come out correct: "F.EDCBA". However, you don't want an F. Use an RTL mark instead. It's normally invisible, but let's use # for this explanation. So the logical characters are (in left-to-right order): ABCDE.# and the display should then be ".EDCBA", even on Android. – Dipper

@androiddeveloper - Regarding our library, I'm sorry, but we don't make it available. – Dipper

Answer 2

As of August 2013, Android has posted API documentation for a Bi Directional Formatter which might suit your needs. This is contained in the Android Support v4 library which I believe should run in versions prior to Android 4.2.

Refer to: http://developer.android.com/reference/android/support/v4/text/BidiFormatter.html

Koweit

can you please show how i can use it? are there any samples? i don't understand why they didn't make a textView compatibility instead (or in addition)... – Unnamed

In my tests, the BidiFormatter didn't fixed RTL issue for Persian language on older devices – Autonomic