How to clear font cache filled with emoji characters?

Question

I am developing keyboard extension for iPhone. There is an emoji screen smilar to Apples own emoji keyboard that shows some 800 emoji characters in UICollectionView.

When this emoji UIScrollView is scrolled the memory usage increases and does not drop down. I am reusing cells correctly and when testing with single emoji character displayed 800 times the memory does not increase during scrolling.

Using instruments I found that there is no memory leak in my code but it seems that the emoji glyphs are cached and can take around 10-30MB of memory depending on font size (reseach shows they are actually PNGs). Keyboard extensions can use little memory before they are killed. Is there a way to clear that font cache?

---

Edit

Adding code example to reproduce the problem:

let data = Array("😄😊☺️😉😍😘😚😗😙😜😝😛😳😁😔😌😒😞😣😢😂😭😪😥😰😅😓😩😫😨😱😠😡😤😖😆😋😷😎😴😵😲😟😦😧😈👿😮😬😐😕😯😶😇😏😑👲👳👮👷💂👶👦👧👨👩👴👵👱👼👸😺😸😻😽😼🙀😿😹😾👹👺🙈🙉🙊💀👽💩🔥✨🌟💫💥💢💦💧💤💨👂👀👃👅👄👍👎👌👊✊✌️👋✋👐👆👇👉👈🙌🙏☝️👏💪🚶🏃💃👫👪👬👭💏💑👯🙆🙅💁🙋💆💇💅👰🙎🙍🙇🐶🐺🐱🐭🐹🐰🐸🐯🐨🐻🐷🐽🐮🐗🐵🐒🐴🐑🐘🐼🐧🐦🐤🐥🐣🐔🐍🐢🐛🐝🐜🐞🐌🐙🐚🐠🐟🐬🐳🐋🐄🐏🐀🐃🐅🐇🐉🐎🐐🐓🐕🐖🐁🐂🐲🐡🐊🐫🐪🐆🐈🐩🐾💐🌸🌷🍀🌹🌻🌺🍁🍃🍂🌿🌾🍄🌵🌴🌲🌳🌰🌱🌼🌐🌞🌝🌚🌑🌒🌓🌔🌕🌖🌗🌘🌜🌛🌙🌍🌎🌏🌋🌌🌠⭐️☀️⛅️☁️⚡️☔️❄️⛄️🌀🌁🌈🌊☕️🍵🍶🍼🍺🍻🍸🍹🍷🍴🍕🍔🍟🍗🍖🍝🍛🍤🍱🍣🍥🍙🍘🍚🍜🍲🍢🍡🍳🍞🍩🍮🍦🍨🍧🎂🍰🍪🍫🍬🍭🍯🍎🍏🍊🍋🍒🍇🍉🍓🍑🍈🍌🍐🍍🍠🍆🍅🌽🎍💝🎎🎒🎓🎏🎆🎇🎐🎑🎃👻🎅🎄🎁🎋🎉🎊🎈🎌🔮💛💙💜💚❤️💔💗💓💕💖💞💘💌💋💍💎👑👒👟👞👡👠👢👕👔👚👗🎽👖👘👙💼👜👝👛👓🎀🌂💄📚📖🔬🔭📰🎨🎬🎩🎪🎭🎤🎧🎼🎵🎶🎹🎻🎺🎷🎸👾🎮🃏🎴🀄️🎲🎯🏈🏀⚽️⚾️🎾🎱🏉🎳⛳️🚵🚴🏁🏇🏆🎿🏂🏊🏄🎣").map {String($0)}

class CollectionViewTestController: UICollectionViewController {
    override func viewDidLoad() {
        collectionView?.registerClass(Cell.self, forCellWithReuseIdentifier: cellId)
    }

    override func collectionView(collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        return data.count
    }

    override func collectionView(collectionView: UICollectionView, cellForItemAtIndexPath indexPath: NSIndexPath) -> UICollectionViewCell {
        let cell = collectionView.dequeueReusableCellWithReuseIdentifier(cellId, forIndexPath: indexPath) as! Cell
        if cell.label.superview == nil {
            cell.label.frame = cell.contentView.bounds
            cell.contentView.addSubview(cell.label)
            cell.label.font = UIFont.systemFontOfSize(34)
        }
        cell.label.text = data[indexPath.item]
        return cell
    }

    override func numberOfSectionsInCollectionView(collectionView: UICollectionView) -> Int {
        return 1
    }
}

class Cell: UICollectionViewCell {
    private let label = UILabel()
}

After running and scrolling the UICollectionView I get memory usage graph like this:

Sneed

Answer 1

I ran into the same issue and fixed it by dumping the .png from /System/Library/Fonts/Apple Color Emoji.ttf and using UIImage(contentsOfFile: String) instead of a String.

I used https://github.com/github/gemoji to extract the .png files, renamed the files with @3x suffix.

func emojiToHex(emoji: String) -> String {
    let data = emoji.dataUsingEncoding(NSUTF32LittleEndianStringEncoding)
    var unicode: UInt32 = 0
    data!.getBytes(&unicode, length:sizeof(UInt32))
    return NSString(format: "%x", unicode) as! String
}

let path = NSBundle.mainBundle().pathForResource(emojiToHex(char) + "@3x", ofType: "png")
UIImage(contentsOfFile: path!)

UIImage(contentsOfFile: path!) is properly released so the memory should stay at a low level. So far my keyboard extension hasn't crashed yet.

If the UIScrollView contains a lot of emoji, consider using UICollectionView that retains only 3 or 4 pages in cache and releases the other unseen pages.

Inchon

Yes, this is my backup plan. It has some dowsides trough: Those .PNGs will increase size of app download size by around 15 MB, I will have some 800 extra files in my project, some extra work that I believe should not be neccessary... – Sneed

With proper png compression I think you can reduce the additional size to around 4MB. I agree that it shouldn't be necessary, unfortunately unless Apple fixes this issue I can't see any other option... – Inchon

gemoji did not work very well for me so I created my own tool for making those .png files. But well they have 24 MB! Can you suggest how to reduce this extra size? I know literaly nothing about .png compression... – Sneed

I use Adobe Fireworks batch jobs to compress my images, but I guess any other image software can do the same. You can limit the images to 256 colors to reduce the size. I personally see no difference with the original emojis. – Inchon

Are there any legal issues with that, as Apple Color Emoji is AFAIK a completely unlicensed font? – Martino

Thank you but it will not work for sequenced emoji, see my answer – Greenbelt

Apple just rejected a keyboard app of mine that used PNGs of the emojis because of legal issues. Therefore, I don't believe using the PNGs of the Apple emojis will work. :( – Anybody

I just have to voice in here. Apple will not like this the least bit as they are very picky about their emojis. Either use openmoji or find out how to use labels. Based on my test @Cloov is right about the memory usage. The memory used for the glyps are not counted as "your memory". – Peignoir

Answer 2

I had the same issue and tried many things to release the memory, but no luck. I just changed the code based on Matthew's suggestion. It works, no more memory problem for me including iPhone 6 Plus.

The code change is minimal. Find the change in the UILabel subclass below. If you ask me the challenge is to get the emoji images. I could not figure how gemoji (https://github.com/github/gemoji) works out yet.

    //self.text = title //what it used to be
    let hex = emojiToHex(title)  // this is not the one Matthew provides. That one return strange values starting with "/" for some emojis. 
    let bundlePath = NSBundle.mainBundle().pathForResource(hex, ofType: "png")

    // if you don't happened to have the image
    if bundlePath == nil
    {
        self.text = title
        return
    }
    // if you do have the image 
    else
    {
        var image = UIImage(contentsOfFile: bundlePath!)

        //(In my case source images 64 x 64 px) showing it with scale 2 is pretty much same as showing the emoji with font size 32.
        var cgImage = image!.CGImage
        image = UIImage( CGImage : cgImage, scale : 2, orientation: UIImageOrientation.Up  )!
        let imageV = UIImageView(image : image)

        //center
        let x = (self.bounds.width - imageV.bounds.width) / 2
        let y = (self.bounds.height - imageV.bounds.height) / 2
        imageV.frame = CGRectMake( x, y, imageV.bounds.width, imageV.bounds.height)
        self.addSubview(imageV)
    }

---

The emojiToHex() method Matthew provides returns strange values starting with "/" for some emojis. The solution at the given link work with no problems so far. Convert emoji to hex value using Swift

func emojiToHex(emoji: String) -> String
{
    let uni = emoji.unicodeScalars // Unicode scalar values of the string
    let unicode = uni[uni.startIndex].value // First element as an UInt32

    return String(unicode, radix: 16, uppercase: true)
}

---------- AFTER SOME TIME----

It turned out this emojiToHex method does not work for every emoji. So I end up downloading all emojis by gemoji and map each and every emoji image file (file names are like 1.png, 2.png, etc) with the emoji itself in a dictionary object. Using the following method instead now.

func getImageFileNo(s: String) -> Int
{
        if Array(emo.keys).contains(s)
        {
             return emo[s]!
        }   
        return -1
}

Passion

how about the flag like this? 🇾🇪 , i see the emojiToHex return missing hex value. this flag must be 1f1fe-1f1ea , but it return only 1f1fe? – Liederkranz

I had simmiliar issues with emojiToHex later. Then I happened to figure out emoji download emojis and manually map each image file to the emoji. Updating the post now. – Lepley

Answer 3

I am guessing that you are loading the images using [UIImage imageNamed:], or something that derives from it. That will cache the images in the system cache.

You need to load them using [UIImage imageWithContentsOfFile:] instead. That will bypass the cache.

(And if that's not the problem, then you'll need to include some code in your question so that we can see what's happening.)

Antoniaantonie

I am not loading any images in my code directly. I am just adding labels to UICollectionViewCells and the text of those lables are emojis unicode characters like this: 😀😊☺️. Internally, they are probably PNGs, but I cannot afftect the way they are loaded. If UIKit uses [UIImage imageNamed:] internally to load them I cannot change it. Please see my edit. – Sneed

Ah OK, I see. I think you're right, there's a glyph cache getting in your way somewhere. It looks like rendering with NSTextStorage / NSLayoutManager might be an option, so that you can control the storage yourself, but I've never done that before (I'm just guessing, looking at the docs). – Antoniaantonie

I tried your suggestion with no success. I am now drawing the emoji directly with NSLayoutManager, character codes are stored in NSTextStorage. Even if I make sure that all classes used for rendering are released, the memory is not freed. I can only guess that the glyphs are cached even deeper, perhaps in NSGlyphGenerator but that class is private API on iOS. – Sneed

Answer 4

I've been around the houses on this too, and I've come to the following conclusion after numerous tests:

While the font cache does contribute to your extension's memory footprint and the total usage in the Xcode Debug Navigator and Memory Report, it isn't treated in quite the same way as the rest of your budget.

Some people cite 50 MB as the extension limit, and on Apple docs I think I've seen either 30 or 32 MB cited. We see memory warnings at various points between 30 and 40 MB, and this is too inconsistent to be happy with any particular value, but one thing that does seem to be concrete is a memory exception that occurs at 53 MB, which is logged out by Xcode as exactly that number. If I take a blank keyboard extension and populate it with even 40 MB of image views, this is one thing, but if I use 30 MB and then 20 MB of font glyph usage, I find that my keyboard isn't shut down.

From my observations, the font cache looks to get cleaned up, but not as often as you might feel necessary (especially if you're becoming nervous when that unhelpful combined memory value exceeds 30 or 32 MB).

If you budget your own memory usage at, say, 30 MB, you should be safe, provided that you don't introduce a scenario where 23 MB (i.e. 53-30) of font glyphs are all required in one swoop. This will be influenced by how dense your emoji grid is, and possibly even the font size used. It's common understanding here that if you were to scroll from one end of your emoji collection view to the other, you'll have passed through more than 23 MB of font glyphs, but if the rest of your memory footprint is reasonable (i.e. 30 MB or below), the font cache should get a chance to clean up.

In my testing, I attempted to automate bombardment of the extension with far more font glyphs, and I think I was able to beat the font cache cleanup process, resulting in a crash.

Therefore, given the use case of a UICollectionView and how fast it can be scrolled, it may be possible to crash the application if you really pushed the 30 MB memory budget and also scrolled very quickly. You might allow yourself to hit this 53 MB hard limit.

Given all of the above - with a fully fledged keyboard extension, as long as I keep to approximately 30 MB of my own (non-font-glyph) footprint I haven't encountered a crash, even when rapidly changing emoji categories and scrolling fast. I do, however, encounter system memory warnings this way, which is the thing that re-instills doubt for me.

Another problem with this approach versus using UIImage(contentsOfFile) is that it's harder to use the memory report's overall memory footprint to scrutinise your application besides what the font cache is doing. Perhaps there's a way to separate these out, but I don't know of one.

Fraction

I believe this is the right answer. Based on my tests I've observed the same thing. Inspecting with Instruments also reveals the memory consumption lies at malloc of glyp. As a last note: Apple doesn't like people using their emojis so embedding them as images is a sure way to get rejected. – Peignoir

Answer 5

Many emojis are represented by sequences that contain more than one unicode scalar. Matthew's answer works well with basic emojis but it returns only first scalar from the sequences of emojis like country flags.

The code below will get full sequences and create a string that matches gemoji exported file names.

Some simple smiley emojis also have the fe0f selector. But gemoji doesn't add this selector to file names on exporting, so it should be removed.

func emojiToHex(_ emoji: String) -> String
{
    var name = ""

    for item in emoji.unicodeScalars {
        name += String(item.value, radix: 16, uppercase: false)

        if item != emoji.unicodeScalars.last {
            name += "-"
        }
    }

    name = name.replacingOccurrences(of: "-fe0f", with: "")
    return name
}

Greenbelt

Answer 6

In my case, the plain CATextLayer helped to reduce the memory usage of my app. When I used the UILabel to render Emojis the keyboard extension memory was increasing from ~16MB to ~76MB. After the replacement of the UILabel with the CATextLayer, the keyboard extension memory increasing from ~16MB to only ~26MB.

Previous UICollectionViewCell subclass setup:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) UILabel *textLabel;
    _textLabel = [UILabel new];
    self.textLabel.font = [UIFont fontWithName:@"HelveticaNeue" size:28];
    self.textLabel.textAlignment = NSTextAlignmentCenter;
    [self addSubview:self.textLabel];
    // some auto layout logic using Masonry
    [self.textLabel mas_makeConstraints:^(MASConstraintMaker *make) {
        make.size.equalTo(self);
        make.center.equalTo(self);
    }];

    return self;
}

My UICollectionViewCell subclass setup with the CATextLayer:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) CATextLayer *textLayer;
    _textLayer = [CATextLayer new];
    self.textLayer.frame = CGRectMake(0, 0, 33, 33);
    self.textLayer.font = CFBridgingRetain([UIFont fontWithName:@"HelveticaNeue" size:28].fontName);
    self.textLayer.fontSize = 28;
    self.textLayer.alignmentMode = kCAAlignmentCenter;
    [self.layer addSublayer:self.textLayer];

    return self;
}

Update

Sorry guys forgot to add the self.textLayer.contentsScale = [[UIScreen mainScreen] scale]; to get clear text. That unfortunately increased usage of memory from ~16MB to ~44MB, but still better than the UILabel solution.

Final UICollectionViewCell subclass setup with the CATextLayer:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    [self.layer setRasterizationScale:[[UIScreen mainScreen] scale]];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) CATextLayer *textLayer;
    _textLayer = [CATextLayer new];
    self.textLayer.frame = CGRectMake(0, 0, 33, 33);
    self.textLayer.font = CFBridgingRetain([UIFont fontWithName:@"HelveticaNeue" size:28].fontName);
    self.textLayer.fontSize = 28;
    self.textLayer.alignmentMode = kCAAlignmentCenter;
    NSDictionary *newActions = @{
        @"onOrderIn": [NSNull null],
        @"onOrderOut": [NSNull null],
        @"sublayers": [NSNull null],
        @"contents": [NSNull null],
        @"bounds": [NSNull null]
    };
    self.textLayer.actions = newActions;
    [self.layer addSublayer:self.textLayer];

    [self.layer setShouldRasterize:YES];

    return self;
}

Permanence

Using CATextLayer doesn't seem to have any effect at all. – Taveda

@Taveda why you so sure? did you validate that? on my side, the app benefits from using the CATextLayer. – Permanence