Custom font gives incorrect glyph for certain letter combinations in Android

Question

I'm using a custom font in my Android project. For some reason when the text includes the letters IJ together, it gives me the following glyph:

This appears to be the glyph located at \uE2C5 of the PUA region of the font.

The individual I and J glyphs both exist in the font and I can get them to appear if I set the text to I J.

It's not an OpenType font (I don't think Android even supports OpenType rendering in custom fonts), so there shouldn't be anything like this happening. What is going on here?

The problem is reproducible with the following simple project:

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        TextView textView = (TextView) findViewById(R.id.textview);
        Typeface tf = Typeface.createFromAsset(this.getAssets(), "MenksoftHawang.ttf");
        textView.setTypeface(tf);

        textView.setText("IJ");
    }
}

The font can be downloaded from here. Put it in the assets folder in the project.

Southeastwardly

Answer 1

I can't really speak to the reason that you are seeing what you are seeing, but the letter combination of "IJ" seems to have a history of special treatment. From the Wikipedia page on Typographic ligature.

Dutch ij, however, is somewhat more ambiguous. Depending on the standard used, it can be considered a digraph, a ligature or a letter in itself, and its upper case and lower case forms are often available as a single glyph with a distinctive ligature in several professional fonts (e.g. Zapfino). Sans serif uppercase IJ glyphs, popular in the Netherlands, typically use a ligature resembling a U with a broken left-hand stroke.

Also from Wikipedia:

Glyph substitution and composition

Some compatibility characters are completely dispensable for text processing and display software that conforms to the Unicode standard. These include:

Ligatures Ligatures such as ‘ffi’ in the Latin script were often encoded as a separate character in legacy character sets. Unicode’s approach to ligatures is to treat them as rich text and, if turned on, handled through glyph substitution.

Given this, my guess is that the font you are using has taken advantage of this type of usage to do its own thing.

Update Using setFontFeatureSettings() works to disable the display of the glyph as shown in the following code. Unfortunately, this is only available in API 21+. There is a caveat that there may be other side effects that you may be become aware of.

For APIs below 21, you can use a font editing program such as FontForge and delete the glyph at U+E2C5. I have done this and it does eliminate the glyph. This may be the best way to go as long as you think that you can identify all letter combinations that can result in a glyph.

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        String features = "\"liga\"=0";

        TextView textView = (TextView) findViewById(R.id.textview);
        Typeface tf = Typeface.createFromAsset(this.getAssets(), "MenksoftHawang" +
                ".ttf");
        textView.setTypeface(tf);
        textView.setFontFeatureSettings(features);

        textView.setText("IJ ij");
    }
}

Ingredient

I wasn't aware of that. Both ij and IJ produce other characters. And fi makes a blank space. This sounds like it could be the reason. Any idea of where I would look in the font file for that? – Southeastwardly

@Southeastwardly Do you want to change the glyph or just disable it so "IJ", "ij" show as two separate letters? There are font editors available to edit TrueType fonts. From a cursory look, glyphs may be handled differently in these editors, so you would want to make sure that the editor is "glyph aware." I haven't done any of this type of editing, so I can't recommend one or another. I have seen others mention azizbekian's solution of changing spacing then "kerning it out" as a work-around. – Ingredient

I want to disable it so that they show as two different letters. I've done a little font editing with FontForge before but nothing with ligatures. I was under the impression that only OpenType fonts did that (and that OpenType rendering wasn't wasn't supported in Android before version 6.0). – Southeastwardly

@Southeastwardly Good question about TTF and ligature support. There may be some special circumstances inherited from Apple? Search for "line layout manager" in this Wikipedia article. (Sorry for all the Wikipedia references.) Second was Line Layout Manager, where particular sequences of characters can be coded to flip to different designs in certain circumstances, useful for example to offer ligatures for "fi", "ffi", "ct", etc. while maintaining the backing store of characters necessary for spell-checkers and text searching. – Ingredient

@Southeastwardly One more thing. Take a look at setFontFeatureSettings() that is part of TextView. This supports CSS font features settings. There are some codes there that seem to address turning ligatures on/off. It looks promising, but it may be a rabbit hole. – Ingredient

@Southeastwardly See my addition to the update above. I didn't want to leave a partial solution. – Ingredient

I will study the links you provided. I will also do some more research to get a complete list of ligatures. Unfortunately, deleting the glyph at that location is not an option since it is needed for displaying Mongolian text. I'll also look into other ways to edit out ligatures in fonts. – Southeastwardly

Sounds good. I am certain that ligatures is the issue. FontForge, and probably other font editing programs, will let you remove the ligature but not delete the character itself. It will also give you a complete list of ligatures in the font. Once, and if, you are satisfied with this solution, I will clean up the answer so it is more to the point. – Ingredient

I'm accepting this answer, but also see my summarized answer below. – Southeastwardly

@Southeastwardly Excellent summary! – Ingredient

Answer 2

I'm not sure about exact cause of the issue, I assume it might be also a font issue.

But as a workaround you can apply tiny letter spacing, which will show letters as expected.

textView1.setText("IJ");

textView2.setLetterSpacing(0.04f);
textView2.setText("IJ");

Result:

Tindle

Interesting! It makes me very curious why this is happening. And why setting letter spacing prevents it. – Southeastwardly

Note: set letter spacing is only available from API 21. – Southeastwardly

Answer 3

There appears to be three solutions to this problem:

1. Programmatically disable the ligatures

You can disable the ligatures programmatically with setFontFeatureSettings (as suggested here) or setLetterSpacing (as suggested here). One disadvantage of both of these methods is that they are only available from API 21.

2. Edit the ligatures out of the font

You can use font editing software to fix or delete the ligature errors in the font. See this Q&A for how to do it in FontForge. A potential problem here is that the font copyright owners may not allow third parties to edit their fonts.

3. Use a different font

Different fonts are often available and that is true in this case. The same company that had the problematic TrueType font in the question also has an OpenType version of that font. The OpenType version does not seem to have the ligature errors. A disadvantage, though, is that it is significantly larger in size, which will make that app download size larger.

Notes

• I had previously thought that there was no ligature rendering prior to Android 6.0. However, TrueType fonts do apparently support some basic ligatures and those are rendered in versions of Android prior to 6.0.
• The following image from FontForge shows all the ligatures defined in the problematic font referenced in the question.

Southeastwardly