Japanese characters looking like Chinese on Android

Question

PREAMBLE: since API 17 (Android 4.2), there's a method TextView.setTextLocale() that explicitly solves this problem for TextViews and derived classes. Assign a Japanese locale (Locale.JAPAN), and Unihan characters will look Japanese.

---

I have an application on Android that displays Japanese text in WebViews and TextViews. There are some Chinese characters (kanji) that look, by convention, differently in China and in Japan, but share the same Unicode codepoint. Normally, the browser would rely upon the lang tag to choose the correct glyph. On Android, they all default to their Chinese shapes, and I want Japanese shapes.

The problem is well explained in this article. This article also serves as a perfect illustration of the problem - when watched on Android (up to 2.2), the characters in the "Examples of language-dependent characters" all look the same, and Chinese.

Using the lang="ja" attribute does not help. Switching the whole system locale to Japanese does not help either.

I'm wondering about Android phones that are sold in Japan. Do characters like 直, 今, 化 look Chinese-style on those, too? I'm assuming not.

So the questions are: are there official localized images of Android out there? Can I get one to run on the emulator? Is the DroidSansFallback font still the only CJK-enabled font on those? And if it is, is it the same as on the vanilla USA Android?

I'm kind of hoping that the Japanese glyphs are hidden somewhere deep in the font (Unicode private area or something). If so, I could leverage them...

EDIT: located DroidSansJapanese.ttf, installed it on the emulator by copying into /system/fonts, restarted. It made no difference on the look of the Unihan article. Even the hint area of the Japanese text input (which should know better) displays as if Chinese.

How do I know the typeface name of the DroidSansJapanese.ttf? I have a feeling it's still Droid Sans, same as in the built-in DroidSansFallback font. But if they contain the same typeface, what governs which one should take precedence? One would think - system locale, but apparently not. Fonts in Android are installed just by copying, right?

Referee

FYI those characters look chinese-style in your post. :) – Prevenient

Those character that you posted is Chinese word... some of the Japanese word also will have some Chinese word too – Tony

I'm in japan and I have no problems with it. We(company) have an american(?) version of the galaxy s there and no problem with displaying japanese characters. One thing that would bring me problems on webviews is not setting the UTF-8 META tag: <META http-equiv="Content-Type" content="text/html; charset=UTF-8"> Can you post a link or a sample of the page you're having problems with? What device are you having problems with? I can try to answer if you give more detailed info. here you're just saying that android can't display japanese which I don't think is true. – Octamerous

@DallaRosa: see en.wikipedia.org/wiki/Han_unification under "Examples of language-dependent characters". Characters in the "Japanese" column still look Chinese-style. Trying on Android emulator 1.5 and 2.2. – Referee

I'm gonna try them later on the emulator but at least on my Xperia Arc they all look japanese to me. – Octamerous

@Mister Smith: yours was the crucial clue, the one about DroidSansJapanese.ttf. Make an answer, I'll accept. – Referee

OK, removing comment, posting answer. But please could you explain the mp3 hack? I think your solution on that post (the chunking one) is quite more elegant than the hack. – Hathaway

Tx. See the answer edit. – Referee

I've had a galaxy and an xperia sold in Japan (docomo) and I had japanese characters (andro 4 to 6). Now I bought a parallel market zenfone3 and I have the problem of Chinese characters. – Cerracchio

Answer 1

There are fonts with full Japanese support. I've heard some people talking about DroidSansJapanese.tff and TakaoPGothic.

Hathaway

Answer 2

Found a somewhat cludgey solution.

The DroidSansJapanese.ttf font exists and is available for download, for example, here. I downloaded it, renamed it to DroidSansJapanese.mp3 to avoid the 1MB compressed asset limit, and placed it under assets/web. I then introduced the following CSS statement:

@font-face
{
font-family: "DroidJP";
src:url('DroidSansJapanese.mp3')
}

Then I added 'DroidJP' to the font-family of every relevant CSS style. The way I load my HTML, the assets/web folder is already designated as the base for loading linked content.

Upon careful examination, I found several places in the app where Japanese was in TextViews. For those, I've loaded the typeface like this:

Typeface s_JFont =
    Typeface.createFromAsset(Ctxt.getAssets(), "web/DroidSansJapanese.mp3");

and called setTypeface() on every relevant TextView. Now for PROFIT!

This expanded my APK by about 1 MB. There was an alternative way, where I'd store the font in the assets in compressed form - that'd save me about 500 KB of APK size, but then I'd have to expand it to phone memory on the first run, worry about data folder path, and lose compatibility with Android 1.5.

Some credit is due: here and here. Does not work on Android 2.1 (the WebViews, not the TextViews) - that's a known bug.

Now, the question remains - how do I identify devices where the default font already contains the Japanese shapes?

EDIT re: the mp3 hack. I've implemented the chunked solution at first, but then decided to go with font in the assets. The chunked approach has one upside - smaller APK size - and the following downsides:

• Consumes more phone memory - you end up storing both compressed font in assets and uncompressed in the data directory
• Is incompatible with Android 1.5 on the TextView side - the method Typeface.createFromFile() was introduced in API level 4
• Complicates HTML generation - the CSS with the @font-face declaration needs to be parametrised, since the data path is a variable
• Slows down the first app startup - you spend time combining the chunks

Storing the font as a compressed asset is not an option either - the font then doesn't show up in WebView, and you can clearly see in LogCat the message about "Data exceeds UNCOMPRESS_DATA_MAX".

Referee

I don't understand the .mp3 hack. Theoretically, the Android Asset Packaging Tool doesn't compress (or doesn't do very well) mp3 files. If you open the Package.cpp Android source file, which is located in kernel's \tools\aapt package, just at the top of the file you can find a char array named kNoCompressExt[] where mp3 is listed. – Hathaway

If you store an compressed asset over 1 MB, trying to open it via AssetManager.open() will throw an error. Workarounds listed here. – Referee

Exactly, read answer #2 on that post. If you rename it to mp3, you are telling the AAPT to not to compress it since it is already compressed. – Hathaway

Answer 3

Using the following in your onCreate/OnCreateView:

AssetManager am = getContext().getApplicationContext().getAssets();
mDroidSansJapaneseTypeface = Typeface.createFromAsset(am, String.format(Locale.JAPAN, "fonts/%s", "DroidSansJapanese.ttf")); //I put the font file in the assets/fonts/ folder

And then for your textviews:

myTextView.setTypeface(mDroidSansJapaneseTypeface);
myTextView.setTextLocale(Locale.JAPAN);

Be aware that not 100% of all fonts will be displayed in Android. I haven't seen too many problems for Japanese characters, but other CJK characters may appear as blank boxes, no matter what ttf file you use.

Bacchus