Throttling CPU from within Java

Question

I have seen many questions in this (and others) forum with the same title, but none of them seemed to address exactly my problem. This is it: I have got a JVM that eats all the CPU on the machine that hosts it. I would like to throttle it, however I cannot rely on any throttling tool/technique external to Java as I cannot make assumptions as to where this Vm will be run. Thus, for instance, I cannot use processor affinity because if the VM runs on a Mac the OS won't make process affinity available.

What I would need is an indication as to whether means exist within Java to ensure the thread does not take the full CPU.

I would like to point straightaway that I cannot use techniques based on alternating process executions and pauses, as suggested in some forums, because the thread needs to generate values continuously.

Ideally I'd like some mean for, for instance, setting some VM or thread priority, or cap in some way the percentage of CPU consumed.

Any help would be much appreciated.

Behn

You want to know if there is some API within Java that instructs the JVM to allow other threads / applications to run? The closest you'll get to that is Thread.yield() or setPriority() - but those APIs don't have any guarantees. – Acaricide

Thread.yield() will not solve his problem. It won't slow the JVM at all @noahz. – Welton

What's your definition of "slow the JVM down?" You realize that the JVM ultimately compiles your bytecode to machine instructions, right? Unless your code has instructions to make it run slowly (i.e. Thread.sleep()), you can't make it "run more slowly" via JVM settings. – Acaricide

Thanks all. It is not slowing the JVM down what I want, I would rather want a way to set a priority that is recognised outside the JVM. When you set the priority of a thread, that priority has validity within the JVM only and not outside. Ideally I would need something similar but recognised at system process level. Hope I manage to explain myself properly :-) – Behn

Related: #1202684 – Bord

I would have thought that some kind of intermittent pausing has to be involved, unless perhaps you want to specifically restrict the number of CPU cores allowed to run your program. Why do you want to throttle the program - for power saving or for freeing up CPU power for other applications? – Salinometer

You can look at the number of available processors and limit your thread pool sizes based on this number. – Camilacamile

@assylias: I had looked at that question, however the replies on it tend mostly to address the issue related to memory, rather than CPU. besides, the methods suggested are based on pausing the application, which is not what I wanted. In fact, in my question I placed a disclaimer saying that I am not interested in techniques based on executions and pauses just for this purpose :-) – Behn

@Jimpanzee: I would like to find a way to ensure the JVM takes as much CPU as needed only when CPU is actually available, leaving it to more important system processes when it needs to be reclaimed. – Behn

The only thing I can think of is attempting to monitor the system CPU usage from within your program, then using it to determine the number of milliseconds to Thread.sleep() in your loop, if at all. – Salinometer

Answer 1

Really interesting thread. I found out Java does not provide means for doing what I want to do, and the only way to do this is from outside the JVM. I ended up using nice to alter the scheduling priority in my test (Linux) environment, will still need to find something similar for WIn-based OSs.

Everyone's intervention has been much appreciated.

Behn

Answer 2

What I would need is an indication as to whether means exist within Java to ensure the thread does not take the full CPU.

There is no way that I know of to do this within Java except for tuning your application to use less CPU.

• You could put some Thread.sleep(...); calls in your calculation methods. A profiler would help with showing you the hot loops/methods/etc..
• Forking fewer threads would also affect the CPU used. Moving to fixed sized thread-pools or lowering the number of threads in your pools.
• It may not be CPU that is the problem but other resources. Watch your IO bandwidth for example. Slowing down your network or disk reads/writes might restore your server to proper operation.

From outside of the JVM you could use the ~unix nice command to affect the priority of the running JVM to not dominate the system. This will give it CPU if available but will let other applications get more of the CPU.

Welton

Thanks Gray. I know for a fact that my problem is with CPU and no other resources. Also, my application is single-threaded, it is a simple thread that generates and sends messages. Everything is done within the main thread, there are no threadpools nor forks of any kind. – Behn

@Behn So your application runs on a single core machine? – Bord

As my post mentions @Diferdin, I would consider to use nice to lower the OS priority of the process. This will give more resources to other system applications if there are anything. So it will consume CPU but not dominate. – Welton

Just FYI - even if your application is single threaded, unless you use a tool like cpuset the OS could move it around to different cores. – Acaricide

@Gray: thanks! Have you got a pointer where I can get some info as to what nice is? (I actually thought in your original reply it was a typo! :-) ) – Behn

@noahz: I believe what you mention is called Processor Affinity and I mention it in my email. It is exactly why I cannot/do not want to use means external to Java, I can make no assumptions as to where this class will be run and some OSs (e.g. MacOS) will not allow you to tune affinity – Behn

@Behn In that case, you probably cannot make any assumptions about what JVM is used, as they each have different features. The only guarantees you get are what's in the Java SDK (if you are using Java Standard Edition), the Java Language Specification, and the Java Virtual Machine Specification (which describes byte code, etc. not proprietary JVM vendor features). So, as others have pointed out, you are going to have to take some responsibility in your code and "throttle" it yourself. – Acaricide

@noahz: not entirely true :-) We can recommend a VM type and/or version, but we need to guarantee whatever we recommend works on all OSs... – Behn

Hi, I accepted my own answer, at the bottom of this thread, as all others did not fully address my issue. – Behn

Oh sorry @Diferdin. Didn't see that. Accepting your own is fine although I don't see any information in yours that is not in my answer. I recommended the nice for example. But whatever. – Welton

@Welton let me rephrase. I ended up using nice because there was nothing really like I wanted. So yes -- I took nice from your answer, but I wanted to make it clear that was not my first choice. Thanks in any way! :-) – Behn

Answer 3

I take it you want something more reliable than setting the threads' priorities?

If you want throttled execution of some code that is constantly generating values, you need to look into chunking up the work the thread(s) do, and coding in your own timer. For example, the java.util.Timer allows for scheduling execution at a fixed rate.

Any other technique will still consume as much CPU as is available (1 core per thread, assuming no locks preventing concurrent execution) when the scheduler doesn't have other tasks to prioritize ahead of yours.

Imhoff

+1 I agree with this approach completely. Of course the OP needs to figure out what part of the code is consuming so much CPU, so I would also recommend a profiler of some sort. Also, since its probably not possible to get the CPU load, the approach will need to be tuneable so that it can be tweaked on different systems. – Cooe

With the OP's follow up comments, I think the heart of his question is: "the JVM can be setup up as as sandbox for various things, including file system, network, etc. Can the JVM also provide a sandbox that limits even a poorly written program's impact on the host system's processor(s). Ideally, this needs to be controllable in a non-JVM-implementation-specific way, for example JMX." – Imhoff

I Agree to some extent; I would love to have a priority-based mechanism, however the standard Java thread priority mechanism has validity only within the JVM, whereas I would need something similar but with validity at OS/kernel level. What I ideally would like is a way to tell the JVM to use full CPU when available, and relieve some when other, possibly more important, system processes require it. – Behn

@Diferdin, that should be how a regular program (with no special code for managing priority etc) running on a JVM behaves. If the JVM doesn't exhibit this behavior, I would argue your OS isn't doing its job. – Imhoff

Answer 4

The detail is simply that you said "must generate values continuously", and if that, to the extreme, is true, then CPU saturation is actually the goal.

But, if you define "continuously" as X values per second, then there is room to work.

Because then you can run your process at 100% CPU, measure the number of values over time, and if you find that it's generates more values than necessary (more than X/sec), then you can now insert pauses in to the process as appropriate until the value rate reaches your desired goal.

The plan being to continually monitor and adjust the pauses to maintain your value rate over time. Then your process will take as much CPU as necessary to meet your values/sec goal.

Addenda:

If you have a benchmark of values/sec that you are happy with, then interjecting the sleeps will give "all the priority necessary" to the other applications, but still maintain your throughput. If, on the other hand, you don't have any solid requirement, that is the requirement is "run as fast as possible when nothing else is running, with no actual requirement for ANY results if some other process dominates the CPU", then that's truly a kernel issue of the host OS, and not something the JVM has any direct, portable mechanism to address.

On Unix systems, you have the nice(1) command to adjust process (not thread) priority, and Windows has their own mechanism. With these commands, you can knock the priority of your Java process to just above "idle" (the default "process" that always runs when nothing else is running). But it's platform specific, as this is an inherently platform specific problem. This may well be managed through platform specific startup scripts that launch your Java program (or even a Java launcher that detects the platform and "does the right thing" before executing your actual code).

Most systems will allow you to lower your own process priorities, but few will let you raise unless you're an admin/superuser or have whatever the appropriate role is for your host OS.

Schlosser

Thanks, and if that is the case then let me rephrase: I would like to JVM to use as much CPU as available, but releasing some as and when other, more important, system processes do require it – Behn

Answer 5

Check to see if you have any "tight loops" in your code.

while (true) {
  if (object.checkSomething()) {
    ...
  }
}

If you do, then you are burning the CPU cycles on millions of checks that are probably not that time critical. The JVM will oblige (because it doesn't know if the check is "important" or not) and you'll get 100% CPU.

If you find such loops, rewrite them like so

while (true) {
  if (object.checkSomething()) {
    ...
  }
  try {
    Thread.sleep(100);
  } catch (InterruptedException e) {
    // purposefully do nothing
  }
}

and the sleeping will voluntarily release the CPU within the loop, preventing it from running too quickly (and checking the condition too many times).

Circumscription

Good point, but it would probably be easier and safer to use a profiler of some sort. You dont want to end up "optimizing" something that doesnt need to be optimized, possibly making things worse. Although the answer is for a different language and technology, I think the concepts are the same: #10800872 – Cooe

Removing tight loops is not an optimization. Removing tight loops is correcting bad design. Consider it a performance bug, brought about by people who only think of bugs as things that affect their code from completing. – Circumscription

I agree with you about the definition of bugs and correcting bad design, but a good profiler should always be preferable. – Cooe