wowza java high cpu usage

Question

I have problem about cpu usage in wowza.

this is doubtful thread. this thread is took high cpu.

SocketAcceptorIoProcessor-1.0 prio=10 tid=0x0000002a9fb53000 nid=0x2428 runnable         [0x000000004f017000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00000007d82ba4e8> (a sun.nio.ch.Util$2)
- locked <0x00000007d82ba4d8> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000007d82ad450> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)

this thread took high cpu. is this jdk bug or else?

this is my environment.

CentOS release 5.4 (Final)

WowzaMediaServer-3.1.2

java version 1.6.0_23 Java(TM) SE Runtime Environment (build 1.6.0_23-b05) Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)

Cantrell

Answer 1

I would first check it really is consuming CPU. If you are profiling the application it can appear to be spending allot of time in this method, when the method is actually blocking. Most profilers have problems telling the difference between a method which is spending allot of time in native code (consumes lots of CPU) and one which is blocking (consumes very little CPU)

Older versions of Java had bugs which could result in the selector spinning. i.e. it keeps returning no selectors without blocking in a busy loop even though it still functions correctly otherwise. I believe newer version of Java have this fixed. I would try Java 6 update 35 or Java 7 update 7 to see if it fixes your problem.

I prefer to use blocking NIO if there is less than a few hundred connections as it much simpler IMHO.

Forum

I tried 2 jdk version jdk1.7.0_9 and jdk1.6.0_37. but high cpu usage still occurred. there are a lot of this thread and take many time and cpu. – Cantrell

So it is high but doesn't consume 100% of one cpu? – Forum

we have 4 cpu and each take 10-20% cpu usage. and then total cpu is 80-90%. – Cantrell

I don't think it's an obvious bug then. I would try profiling the application with a commercial profiler, or VisualVM if you don't have one. You could be consuming more CPU than you should but I suspect epollWait is not the cause. – Forum

Thank you for advice. I just tried to test wowza3 in centos 6.3 in Virtual Machine. it seems low cpu usage. only centos 5.x is high cpu usage. I don't make sense but I will keep watch cpu usage in centos 6.3. – Cantrell

I know that some of the system calls in Centos 5.x are inefficient. E.g. System.currentTimeMillis() took 300 ns on one machine under 5.x and took 40 ns under 6.x The same model running Ubuntu took 18 ns. – Forum

that's quite difference. I wonder slow system call cause problem.. Thank you so much for what you have done for me. – Cantrell

Similar problem here, upgrading to 7u45 did not help. – Ladida

Answer 2

I had the same problem, I fixed it with stopping ntp service and set DATE manually and restart ntp service:

        /etc/init.d/ntp stop
        date -s "$(date)"
        /etc/init.d/ntp start

Simmon