replace emoji unicode symbol using regexp in javascript

Question

As you all know emoji symbols are coded up to 3 or 4 bytes, so it may occupy 2 symbols in my string. For example '😁wew😁'.length = 7 I want to find those symbols in my text and replace them to the value that is dependent from its code. Reading SO, I came up to XRegExp library with unicode plugin, but have not found the way how to make it work.

var str = '😁wew😁';// \u1F601 symbol
var reg = XRegExp('[\u1F601-\u1F64F]', 'g'); //  /[ὠ1-ὤF]/g -doesn't make a lot of sense  
//var reg = XRegExp('[\uD83D\uDE01-\uD83D\uDE4F]', 'g'); //Range out of order in character class
//var reg = XRegExp('\\p{L}', 'g'); //doesn't match my symbols
console.log(XRegExp.replace(str, reg, function(match){
   return encodeURIComponent(match);// here I want to have smth like that %F0%9F%98%84 to be able to map anything I want to this value and replace to it
}));

jsfiddle

I really don't want to bruteforce the string looking for the sequence of characters from my range. Could someone help me to find the way to do that with regexp's.

EDITED Just came up with an idea of enumerating all the emoji symbols. Better than brutforce but still looking for the better idea

var reg = XRegExp('\uD83D\uDE01|\uD83D\uDE4F|...','g');

Aggravate

Why are you trying to match the bytes rather than the codepoints? The example you have using '[\u1F601-\u1F64F]' is the correct way to match these points (although the block is U+1F300-U+1F5FF). – Modena

Not only bytes, I tried many ways, but maybe I did it wrong. What would be the regexp with those codepoints? XRegExp('[\u1F300-\u1F5FF]', 'g');? – Aggravate

@一二三 Javascript does not support characters beyond U+FFFF natively. \u1F601 in a Javascript string encodes two characters, U+1F60 followed by ASCII '1'. There's no way to use U+1F601 in a character class. – Exaggerate

Regex /[\uD800-\uDBFF][\uDC00-\uDFFF]/g solved my problem. It includes not only emojis but also special characters. Referred #3745221 – Decibel

tempted to close as dup of https://mcmap.net/q/235982/-how-to-detect-emoji-using-javascript/11107541 – Hagioscope

Answer 1

The \u.... notation has four hex digits, no less, no more, so it can only represent code points up to U+FFFF. Unicode characters above that are represented as pairs of surrogate code points.

So some indirect approach is needed. Cf. to JavaScript strings outside of the BMP.

For example, you could look for code points in the range [\uD800-\uDBFF] (high surrogates), and when you find one, check that the next code point in the string is in the range [\uDC00-\uDFFF] (if not, there is a serious data error), interpret the two as a Unicode character, and replace them by whatever you wish to put there. This looks like a job for a simple loop through the string, rather than a regular expression.

Insist

Thx. But that is almost what I came to in my edited version of the question. I really want to avoid loops, cause I'm working with my string each time it is changed. But you pushed me to idea to use XRegExp('[\uD800-\uDBFF][\uDC00-\uDFFF]','g') That would be pretty enough for me, I guess. – Aggravate

Answer 2

This is somewhat old, but I was looking into this problem and it seems Bradley Momberger has posted a nice solution to it here: http://airhadoken.github.io/2015/04/22/javascript-string-handling-emoji.html

The regex he proposes is:

/[\uD800-\uDFFF]./ // This matches emoji

This regex matches the head surrogate, which is used by emojis, and the charracter following the head surrogate (which is assumed to be the tail surrogate). Thus, all emojis should be matched correctly and with

.replace(/[\uD800-\uDFFF]./g,'')

you should be able to remove all emojis.

Edit: Better regex found. The above regex misses some emojis.

But there is a reddit post with a version, for which i cannot find an emoji, that is excepted from the rule. The reddit is here: https://www.reddit.com/r/tasker/comments/4vhf2f/how_to_regex_emojis_in_tasker_for_search_match_or/ And the regex is:

/[\uD83C-\uDBFF\uDC00-\uDFFF]+/

To match all occurences, use the g modifier:

/[\uD83C-\uDBFF\uDC00-\uDFFF]+/g

Second Edit: As CodeToad pointed out correctly, ✨ is not recognized by the above Regex, because it's in the dingbats block (thanks to air_hadoken).

The lodash library came up with an excellent Emoji Regex block:

(?:[\u2700-\u27bf]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff])[\ufe0e\ufe0f]?(?:[\u0300-\u036f\ufe20-\ufe23\u20d0-\u20f0]|\ud83c[\udffb-\udfff])?(?:\u200d(?:[^\ud800-\udfff]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff])[\ufe0e\ufe0f]?(?:[\u0300-\u036f\ufe20-\ufe23\u20d0-\u20f0]|\ud83c[\udffb-\udfff])?)*

Kevin Scott nicely put together, what this regex covers in his Blog Post. Spoiler: it includes dingbats 🎉

Sava

this is the best one I tested so far. though it misses this emoji :✨ – Maltase

@Maltase ✨ ("sparkles") is from the dingbats block, which can be represented in UTF-16 without a surrogate pair. If you wanted to catch those as well, you'd need to check for /[\u2700-\u27BF][\uFE0E-\uFE0F]?/ (the latter range is for a possible variant selector) – Ec

Edited to new Regex from lodash, which also includes the dingbats block. – Sava

Answer 3

maybe you can take a look of this article: http://crocodillon.com/blog/parsing-emoji-unicode-in-javascript

the emoji unicode from \u1F601 to \u1F64F

translate to javascript's utf-16 is \ud83d\ude00 to \ud83d\ude4f

the first char is always \ud83d.

so the reg is out:

/\ud83d[\ude00-\ude4f]/g

hope this can make some help

Extravert

That's work nicely at the end of 2021, thanks – Insula

Answer 4

1. /\ud83d[\ude00-\ude4f]/g

not including all emojis like : 👿 👹 👺 💀 👻 👽 🤖 💩, see http://getemoji.com/ and try your regex https://regex101.com/

2. /[\uD83C-\uDBFF\uDC00-\uDFFF]+/g

not including all emojis like : ⛑ ☕️ ☁️☄️ ☀️☃️ ⛄️ ❄️ ☹️☺️⛩⛱™️ ©️ ®️ 〰️ ➰ ➿

3. Even this regex does not allow you to remove all emojis... 🖥 🖨 🖱 🖲 🕹 🗜 :

https://github.com/nizaroni/emoji-strip/blob/master/dist/emoji-strip.js#L79

Then, can you say why you think these regex is bad to remove all exotic characters and emojis ?

/[\u1000-\uFFFF]+/g

Landwehr

Answer 5

To remove all possible emojis:

new RegExp('[\u1000-\uFFFF]+', 'g');

Landwehr

Can you say why -1 ? – Landwehr

This regex is usefull to remove all exotic Characters as emojis, including those of foreign languages. I'm french developper, I would only have UTF-8 characteres for web txt. What characters could this regex remove in excess? – Landwehr

Answer 6

Below regex pattern worked for me in java.

"[\ud83c\udc00-\ud83c\udfff]|[\ud83d\udc00-\ud83d\udfff]|[\u2600-\u27ff]"

As java String uses UTF-16 encoding and as emoji's are above 0xFFFF as well, this regex pattern consider surrogate pairs to identify emojis.

Lehrer

Answer 7

For fun : Solution to remove special characters without using regexp

const str = "abcdefgehijkz Раз, два три! 1234567809 -ab A Z & é è Ö â 😀 😁 😂 🤣 😃 😄 😅 😆 😉 😊 😋 😎 😍 😘 🥰 😗 😙 😚 ☺️ 🙂 🤗 🤩 🤔 🤨 😐 😑 😶 🙄 😏 😣 😥 😮 🤐 😯 😪 😫 😴 😌 😛 😜 😝 🤤 😒 😓 😔 😕 🙃 🤑 😲 ☹️ 🙁 😖 😞 😟 😤 😢 😭 😦 😧 😨 😩 🤯 😬 😰 😱 🥵 🥶 😳 🤪 😵 😡 😠 🤬 😷 🤒 🤕 🤢 🤮 🤧 😇 🤠 🤡 🥳 🥴 🥺 🤥 🤫 🤭 🧐 🤓 😈 👿 👹 👺 💀 👻 👽 🤖 💩 😺 😸 😹 😻 😼 😽 🙀 😿 😾-axxb-"


/********* with regExp ***********/
let startTime = new Date().getTime();
let resp = str.replace(new RegExp('[\u00FF-\uFFFF]+','g'), '');
console.log(resp);
console.log(new Date().getTime() - startTime);


/********* without regExp ***********/
startTime = new Date().getTime();
resp = Array.from(str, x => {
  let theUnicode = x.charCodeAt(0).toString(16);
  while (theUnicode.length < 4) {
    theUnicode = '0' + theUnicode;
  }
  if (theUnicode < '00ff') { 
    return x;
  }
}).join('');
console.log(resp);
console.log(new Date().getTime() - startTime);

Landwehr

Answer 8

emoji's in range of U+1F600 to U+1F64F

you can use this line in your script for sending with Json:

text.replace(/[\u1F60-\u1F64]|[\u2702-\u27B0]|[\u1F68-\u1F6C]|[\u1F30-\u1F70]{\u2600-\u26ff]/g, "");

Felicitous

Answer 9

May be you should use replace in such way?

reg = str.replace(new RegExp('😊','g'),'');

Try out https://github.com/iLeonidze/emoji.js

Adversaria

The correct solution should handle the character range of emojis, not just one. – Tuesday