Call to C++ JNI NewStringUTF crashes android app when using many different kinds of emoji and languages (beyond ascii, but still valid modified utf-8)
Asked Answered
M

1

2

I am trying to solve a Cocos2d-x Keyboard input crash on Android 5.x when I create CCImage from the text with many emoji found on the keyboard (some work though, but most don't.) On Android 4.x several of the devices just display mangled text/extra characters. The source of the crash is the JNI's NewStringUTF() call. It simply does not support all of the 2, 3 and 4 byte utf-8 characters in Android 5/Lollipop.

This crash happens on cocos2d-x v2.2.6 (and confirmed on 3.x) using NDK 10e with Toolchain 4.8 (not sure if any of that makes too much of a difference, we were using 9d prior to moving to Android Studio and I am certain we had this issue, but there was much less usage to lollipop.)

If you never push any of non-modified utf-8 symbols (i.e. stick to ascii) you'll probably never see the issue.

Log Cat:
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]     string: 'πŸ‘ŠπŸ‘Š'
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]     in call to NewStringUTF
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]     from void org.cocos2dx.lib.Cocos2dxHelper.nativeSetEditTextDialogResult(byte[])
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65] "GLThread 45716" prio=5 tid=14 Runnable
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   | group="main" sCount=0 dsCount=0 obj=0x12c0c6c0 self=0xf442bc00
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   | sysTid=10959 nice=0 cgrp=default sched=0/0 handle=0xf450c380
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   | state=R schedstat=( 0 0 0 ) utm=1164 stm=188 core=2 HZ=100
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   | stack=0xeed4e000-0xeed50000 stackSize=1036KB
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   | held mutexes= "mutator lock"(shared held)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #00 pc 00004e64  /system/lib/libbacktrace_libc++.so (UnwindCurrent::Unwind(unsigned int, ucontext*)+23)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #01 pc 00003665  /system/lib/libbacktrace_libc++.so (Backtrace::Unwind(unsigned int, ucontext*)+8)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #02 pc 00271461  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, int, char const*, art::mirror::ArtMethod*)+84)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #03 pc 002534d7  /system/lib/libart.so (art::Thread::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) const+158)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #04 pc 000b7f5b  /system/lib/libart.so (art::JniAbort(char const*, char const*)+610)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #05 pc 000b8681  /system/lib/libart.so (art::JniAbortF(char const*, char const*, ...)+68)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #06 pc 000bac4f  /system/lib/libart.so (art::ScopedCheck::Check(bool, char const*, ...) (.constprop.129)+922)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #07 pc 000c474d  /system/lib/libart.so (art::CheckJNI::NewStringUTF(_JNIEnv*, char const*)+44)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #08 pc 002a6324  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (_JNIEnv::NewStringUTF(char const*)+40)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #09 pc 0076eb6c  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::BitmapDC::getBitmapFromJavaShadowStroke(char const*, int, int, cocos2d::CCImage::ETextAlign, char const*, float, float, float, float, bool, float, float, float, float, bool, float, float, float, float)+312)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #10 pc 0076f12c  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::CCImage::initWithStringShadowStroke(char const*, int, int, cocos2d::CCImage::ETextAlign, char const*, int, float, float, float, bool, float, float, float, float, bool, float, float, float, float)+216)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #11 pc 007aeb14  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::CCTexture2D::initWithString(char const*, cocos2d::_ccFontDefinition*)+1188)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #12 pc 0072cd6c  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::CCLabelTTF::updateTexture()+120)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #13 pc 0072c804  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::CCLabelTTF::setString(char const*)+260)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #14 pc 00523140  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (cocos2d::extension::CCEditBoxImplAndroid::setText(char const*)+344)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #15 pc 00523474  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (???)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #16 pc 0076fb2c  /data/app/com.appsomniacs.da2.debug-1/lib/arm/libcocos2dcpp.so (Java_org_cocos2dx_lib_Cocos2dxHelper_nativeSetEditTextDialogResult+208)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   native: #17 pc 001dfeb1  /data/dalvik-cache/arm/data@[email protected]@[email protected] (Java_org_cocos2dx_lib_Cocos2dxHelper_nativeSetEditTextDialogResult___3B+100)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   at org.cocos2dx.lib.Cocos2dxHelper.nativeSetEditTextDialogResult(Native method)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   at org.cocos2dx.lib.Cocos2dxHelper.access$000(Cocos2dxHelper.java:41)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   at org.cocos2dx.lib.Cocos2dxHelper$1.run(Cocos2dxHelper.java:267)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1501)
    12-11 01:02:17.460 10451-10959/com.appsomniacs.da2.debug A/art: sart/runtime/check_jni.cc:65]   at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1278)

Stack Trace (from a different test, but same crash)

    Build fingerprint: 'samsung/zerofltetmo/zerofltetmo:5.1.1/LMY47X/G920TUVU3DOJ7:user/release-keys'
            Revision: '11'
            ABI: 'arm'
            pid: 18460, tid: 18534, name: GLThread 28670  >>> com.appsomniacs.da2 <<<
            signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
                r0 00000000  r1 00004866  r2 00000006  r3 00000000
                r4 f33a4db8  r5 00000006  r6 00000000  r7 0000010c
                r8 88476100  r9 f442c800  sl 00000000  fp 12f61070
                ip 00004866  sp f33a4960  lr f6f01cf9  pc f6f25c30  cpsr 600b0010

            backtrace:
                #00 pc 0003bc30  /system/lib/libc.so (tgkill+12)
                #01 pc 00017cf5  /system/lib/libc.so (pthread_kill+52)
                #02 pc 00018907  /system/lib/libc.so (raise+10)
                #03 pc 000151a5  /system/lib/libc.so (__libc_android_abort+36)
                #04 pc 00012fec  /system/lib/libc.so (abort+4)
                #05 pc 0075a275  /data/app/com.appsomniacs.da2-2/lib/arm/libcocos2dcpp.so (__gnu_cxx::__verbose_terminate_handler()+220)
                #06 pc 0072a10b  /data/app/com.appsomniacs.da2-

2/lib/arm/libcocos2dcpp.so (__cxxabiv1::__terminate(void (*)())+2)
            #07 pc 0072a13b  /data/app/com.appsomniacs.da2-2/lib/arm/libcocos2dcpp.so (std::terminate()+10)
            #08 pc 0072a4ab  /data/app/com.appsomniacs.da2-2/lib/arm/libcocos2dcpp.so (__cxa_pure_virtual+14)
            #09 pc 0041ecc5  /data/app/com.appsomniacs.da2-2/lib/arm/libcocos2dcpp.so
            #10 pc 005977ab  /data/app/com.appsomniacs.da2-2/lib/arm/libcocos2dcpp.so (Java_org_cocos2dx_lib_Cocos2dxHelper_nativeSetEditTextDialogResult+110)
            #11 pc 001dfe39  /data/dalvik-cache/arm/data@[email protected]@[email protected]
Mazzola answered 16/12, 2015 at 22:42 Comment(3)
"If you never push any of non-modified utf-8 symbols (i.e. stick to ascii) you'll probably never see the issue." Yes, as stated in Android JNI Tips. – Swoosh
Tell that to my chatty users! "don't use that symbol... I know it works in iOS but it crashes every Android user..." They thought that was cool... :/ I started stripping out characters, then I realized almost EVERY language keyboard was going to do this too... I had to find a better way. Any suggestions Tom? – Mazzola
I think it comes down to avoiding NewStringUTF, especially with user-provided data. – Swoosh
M
7

We found a solution by sending the contents of the std::string in jbyte array and let the java side be java and return the jstring we could use on the C++ jni side. For us these strings are coming from the users keyboard and I have 170k crashes in one week that says they use emoji in character names and chat like crazy... and naming their avatars in itself caused crashes too. So any lobby a Android 5.x user joined would cause them to crash as soon as their client tried to render the other players names with the characters in question. In Android 4.x this was not an issue in that it just printed some garbage characters.

In your c++ side you can do something like this to achieve this function:

jstring JniHelper::getjString(const char *input) {
    JniMethodInfo minfo; // JniHelper

    bool hasMethod = JniHelper::getStaticMethodInfo (minfo, APPTAG_JNI_PACKAGE_NAME, "convertCStringToJniSafeString", "([B)Ljava/lang/String;");
    if (!hasMethod)
    {
        return minfo.env->NewStringUTF(""); // TODO Tune your response to fit your needs...
    }
    else
    {
        string nativeString = std::string(input); // has a bit of a code smell, there is probably a better way.
        // cite: https://mcmap.net/q/950110/-c-std-string-to-jstring-with-a-fixed-length
        jbyteArray array = minfo.env->NewByteArray(nativeString.length());
        minfo.env->SetByteArrayRegion(array,0,nativeString.length(),(jbyte*)nativeString.c_str());

        // cite: http://discuss.cocos2d-x.org/t/jni-return-string/9982/3
        jstring str = (jstring)minfo.env->CallStaticObjectMethod(minfo.classID, minfo.methodID, array);
        minfo.env->DeleteLocalRef(array);
        return str;
    }
}

On the java side turn that into a java string and return it in the same method:

public static String convertCStringToJniSafeString(byte[] input) {
    try {
        String nativeString = new String(input, "UTF-8"); // please debate what the safest charset should be?
        return nativeString;
    } catch (UnsupportedEncodingException e) {
        // TODO Simplistic Error handling, tune to your needs.
        Log.e(APPTAG, "Couldn't convert the jbyteArray to jstring");
        return ""; //JSTRING_CONVERT_FAIL
    }
}

In our case we wanted a jstring (which is inputed as modified UTF8) to feed to the rendering side and store for retrieval later, returning an empty string was just a choice we made to clue us in conversion failed.

I hope this helps someone find a way... maybe even the right way...

Mazzola answered 16/12, 2015 at 22:42 Comment(5)
jstring is not modified UTF-8; It's a JNI reference to a standard JVM (Java) string, which is a counted sequence of UTF-16 code units, one or two of which encode a Unicode codepoint. – Swoosh
Suggestions as to a better way is quite welcome. This is but 'a' solution that seems to get me through the worst of it. I started a separate project to unit test this solution but it is far from done and only started to test 5 keyboards. I tried to sample 2, 3, and 4 byte symbols. I expect in time my chosen framework (open source project) cocos2d-x will figure out a better way. This was done on 2 and and 3 still crashes. So I will push up something when I feel confident its close to good enough to include it. – Mazzola
This is the way to do it, either on the Java side, as you have done, or on the JNI side, which is more tedious. – Swoosh
Tom, thanks for your feedback. I looked at this problem for weeks and I am still worried its missing something, but I am still confident I got most of the edges (still needs some kind of unit test). If it drops crash rates by 90% (100% so far btw) while a real long term solution is developed I am "ok" with that. I stripped out the overuse of the UTF-8 talk. The use of getNewStringUTF() by the cocos2d-x muddies the waters as to what's really the main issue. – Mazzola
please show hot to return byte array in jni function – Main

© 2022 - 2024 β€” McMap. All rights reserved.