I'm working on an app that instantiates several MediaCodec instances for decoding streaming audio and video data to a SurfaceTexture on an Nvidia TX1 development kit. After a few minutes of just using one audio and one video decoder, I get an error about overflowing the weak global reference table:
A/art: art/runtime/indirect_reference_table.cc:118] JNI ERROR (app bug): weak global reference table overflow (max=51200 holes= 0 sizecheck=51200)
A/art: art/runtime/indirect_reference_table.cc:118] weak global reference table dump:
A/art: art/runtime/indirect_reference_table.cc:118] Last 100 entries (of 51200):
A/art: art/runtime/indirect_reference_table.cc:118] 51199: 0x12c00460 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51198: 0x12c00400 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51197: 0x12c003a0 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51196: 0x12c00340 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51195: 0x12c002e0 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51194: 0x12c00280 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51193: 0x12c001c0 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51192: 0x12c009a0 java.lang.Thread
A/art: art/runtime/indirect_reference_table.cc:118] 51191: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51190: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51189: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51188: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51187: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51186: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51185: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51184: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51183: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51182: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51181: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51180: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51179: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51178: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51177: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51176: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51175: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51174: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51173: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51172: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51171: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51170: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51169: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51168: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51167: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51166: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51165: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51164: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51163: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51162: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51161: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51160: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51159: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51158: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51157: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51156: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51155: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51154: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51153: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51152: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51151: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51150: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51149: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51148: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51147: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51146: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51145: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51144: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51143: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51142: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51141: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51140: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51139: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51138: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51137: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51136: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51135: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51134: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51133: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51132: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51131: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51130: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51129: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51128: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51127: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51126: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51125: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51124: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51123: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51122: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51121: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51120: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51119: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51118: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51117: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51116: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51115: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51114: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51113: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51112: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51111: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51110: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51109: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51108: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51107: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51106: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51105: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51104: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51103: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51102: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51101: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] 51100: cleared jweak
A/art: art/runtime/indirect_reference_table.cc:118] Summary:
A/art: art/runtime/indirect_reference_table.cc:118] 6497 of java.lang.Class (6497 unique instances)
A/art: art/runtime/indirect_reference_table.cc:118] 1 of java.lang.ThreadGroup
A/art: art/runtime/indirect_reference_table.cc:118] 2 of byte[] (100 elements) (2 unique instances)
A/art: art/runtime/indirect_reference_table.cc:118] 4 of byte[] (576 elements) (4 unique instances)
A/art: art/runtime/indirect_reference_table.cc:118] 5 of byte[] (18432 elements) (5 unique instances)
A/art: art/runtime/indirect_reference_table.cc:118] 1 of byte[] (36608 elements)
A/art: art/runtime/indirect_reference_table.cc:118] 18 of java.lang.Thread (18 unique instances)
A/art: art/runtime/indirect_reference_table.cc:118] 1 of android.media.MediaCodec
A/art: art/runtime/indirect_reference_table.cc:118]
I can recompile the Android image, so I've expanded the list of displayed entries from 10 to 100.
For the record, I am not using weak references in my code, and I am not using JNI directly in my app.
It would seem like this is a bug in something under my code. When the app crashes, it's in some JNISurfaceTexture thread. Apparently causing a thread to sleep makes another entry in this table, which seems to be the straw that breaks the runtime's back.
It looks like most of the entries are "cleared jweak" which would indicate to me that they're not actually in use. The summary info also doesn't add up to anywhere close to the 51200 entries that are supposedly in the table.
I've traced the weak global references and gotten a stack trace:
D/StackDump: #00 pc 00000000002758c0 /system/lib64/libart.so (art::IndirectReferenceTable::Add(unsigned int, art::mirror::Object*)+300)
D/StackDump: #01 pc 000000000030c9c0 /system/lib64/libart.so (art::JavaVMExt::AddWeakGlobalRef(art::Thread*, art::mirror::Object*)+68)
D/StackDump: #02 pc 00000000003616cc /system/lib64/libart.so (art::JNI::NewWeakGlobalRef(_JNIEnv*, _jobject*)+320)
D/StackDump: #03 pc 0000000000152530 /system/lib64/libart.so (art::CheckJNI::NewRef(char const*, _JNIEnv*, _jobject*, art::IndirectRefKind)+704)
D/StackDump: #04 pc 0000000000327ad0 /system/lib64/libart.so (unsigned long art::ObjectRegistry::InternalAdd<art::mirror::Object>(art::Handle<art::mirror::Object>)+880)
D/StackDump: #05 pc 0000000000327dd0 /system/lib64/libart.so (art::ObjectRegistry::Add(art::mirror::Object*)+128)
D/StackDump: #06 pc 00000000001aec5c /system/lib64/libart.so (art::Dbg::GetThreadId(art::Thread*)+364)
D/StackDump: #07 pc 0000000000317268 /system/lib64/libart.so (art::JDWP::JdwpState::PostThreadChange(art::Thread*, bool)+696)
D/StackDump: #08 pc 00000000001b5a44 /system/lib64/libart.so (art::Dbg::PostThreadStart(art::Thread*)+44)
D/StackDump: #09 pc 000000000045f6d4 /system/lib64/libart.so (art::Thread::Attach(char const*, bool, _jobject*, bool)+744)
D/StackDump: #10 pc 0000000000433440 /system/lib64/libart.so (art::Runtime::AttachCurrentThread(char const*, bool, _jobject*, bool)+24)
D/StackDump: #11 pc 000000000030d6a4 /system/lib64/libart.so (art::JII::AttachCurrentThread(_JavaVM*, _JNIEnv**, void*)+208)
D/StackDump: #12 pc 00000000001471bc /system/lib64/libart.so (art::CheckJII::AttachCurrentThread(_JavaVM*, _JNIEnv**, void*)+124)
D/StackDump: #13 pc 00000000000fcd3c /system/lib64/libandroid_runtime.so (android::JNISurfaceTextureContext::getJNIEnv(bool*)+112)
D/StackDump: #14 pc 00000000000fced0 /system/lib64/libandroid_runtime.so (android::JNISurfaceTextureContext::onFrameAvailable(android::BufferItem const&)+28)
D/StackDump: #15 pc 000000000005cfb8 /system/lib64/libgui.so (android::ConsumerBase::onFrameAvailable(android::BufferItem const&)+212)
D/StackDump: #16 pc 00000000000528c0 /system/lib64/libgui.so (android::BufferQueue::ProxyConsumerListener::onFrameAvailable(android::BufferItem const&)+84)
D/StackDump: #17 pc 000000000005a440 /system/lib64/libgui.so (android::BufferQueueProducer::queueBuffer(int, android::IGraphicBufferProducer::QueueBufferInput const&, android::IGraphicBufferProducer::QueueBufferOutput*)+2440)
D/StackDump: #18 pc 00000000000730ac /system/lib64/libgui.so (android::Surface::queueBuffer(ANativeWindowBuffer*, int)+892)
D/StackDump: #19 pc 000000000009a818 /system/lib64/libstagefright.so (android::ACodec::BaseState::onOutputBufferDrained(android::sp<android::AMessage> const&)+892)
D/StackDump: #20 pc 00000000000987bc /system/lib64/libstagefright.so (android::ACodec::BaseState::onMessageReceived(android::sp<android::AMessage> const&)+908)
D/StackDump: #21 pc 000000000000dc08 /system/lib64/libstagefright_foundation.so (android::AHierarchicalStateMachine::handleMessage(android::sp<android::AMessage> const&)+124)
D/StackDump: #22 pc 000000000000d998 /system/lib64/libstagefright_foundation.so (android::AHandler::deliverMessage(android::sp<android::AMessage> const&)+32)
D/StackDump: #23 pc 00000000000114c8 /system/lib64/libstagefright_foundation.so (android::AMessage::deliver()+92)
D/StackDump: #24 pc 000000000000ea0c /system/lib64/libstagefright_foundation.so (android::ALooper::loop()+404)
D/StackDump: #25 pc 0000000000015bf8 /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+208)
D/StackDump: #26 pc 0000000000015448 /system/lib64/libutils.so
D/StackDump: #27 pc 0000000000065ce4 /system/lib64/libc.so (__pthread_start(void*)+52)
D/StackDump: #28 pc 000000000001ebc4 /system/lib64/libc.so (__start_thread+16)
At first, I thought perhaps this was tied to using setSurfaceTextureListener() on my SurfaceTexture that the MediaCodec is writing to. However, removing this from the program didn't help. It seems like the debugger might be involved?
Is there anything I can do to fix this from the application layer? Am I perhaps starving out garbage collection? Or is this an Android/Nvidia problem?