Multiple JVMs vs single app server

Question

I'm dealing with a system that runs a Java application per customer in its own JVM. We've got about a half dozen dedicated servers that are running close to 100 JVMs total now and sets of custom scripts for managing these JVMs. This setup is really showing its age at this point: managing that many JVMs is becoming a monitoring/management nightmare and we are constantly dealing with heap sizing issues. We'd like to move to a more modern approach and just run a bunch of applications in a single app server per physical machine. However, keeping the applications separate does have distinct advantages in terms of isolation (e.g. out of memory errors only affect one customer). Each customer's software stack has memory requirements that vary widely.

My question: is there a way to have the best of both worlds here and run multiple applications in one JVM (app server) and still maintain some level of isolation? Or is it just a modern fact of life that you need to manage memory requirements of a set of applications these days? Are there other solutions here besides an app server or Java EE container (e.g. Wildfly or Spring) that I'm missing here? It seems like this system is a holdout from another era!

Lumbar

The modern approach is increasingly to use per-app JVMs on virtualized hosts. For the scale you need, I strongly suggest looking at Cloud Foundry to abstract away most of the management you're talking about. – Lawrencelawrencium

Similar to chrylis' comment, the modern approach is NOT to use app servers. Multi-tenancy is also dead, because why have the hassle of that when you can use VMs or containers like Docker to give you real separation? – Wino

I think you're focusing on the wrong problem: why are you constantly having heap size issues? Are your apps leaking memory? Do you have unbounded consumption of memory? Does your app not settle down to a steady use of memory for a given workload? – Wino

@Lumbar can you add some details on what exactly are those management constraints. Just interested it knowing what it takes to manage 100 JVMs ! – Simonsimona

Answer 1

Checkout 'multi-tenant' JVM's.

IBM's JRE has it already: http://www.ibm.com/developerworks/library/j-multitenant-java/

Waratek has implemented it on top of the Oracle JRE, and they created ElastiCat, a Tomcat fork that isolates different applications in the same container: http://www.elasticat.com/faq/

Multi-tenancy is rumoured to appear in the official Oracle Java 9 JVM, too.

=======================================================

Update: Java 9 is out, but no word from Oracle about multi-tenancy. It seems they prefer having multiple JVM's these days, even multiple Containers (e.g. docker).

Fistic

Answer 2

There's pros and cons of either approach:

Shared JVM

• Lower overhead - JVM memory footprint (core libraries etc.) only needs to be loaded once.
• Better memory usage. Java processes will consume OS memory for heap space that may not currently be in use.

Separate JVM

• Insulation from 'greedy' or 'leaky' applications.
• Better security from malicious code.
• Easier updates, updating one app without bringing down the other.

Overall, I wouldn't set a blanket policy. Look for small / micro-services or other low-usage apps that may be good candidates to share first and expand from there.

Duffy

Thanks Mikaveli. I'm well aware of the tradeoffs here. What I'm dealing with is ~100 instances of the same application, each running in its own JVM (one instance per customer). This gives us great flexibility (can run different version of the app depending on customer) and isolation (application crash does not affect other customers). However, this just feels old and clumsy, a pain to manage/monitor, and increasingly harder to scale. There has to be a more modern way of dealing with these app deployments... – Lumbar

I'd suggest moving anything appropriate to a modern application server (JBoss or Glassfish), depending on whether they're mainly web apps or if they'd need some rework. An app server takes on a lot of the pain of managing and monitoring, so it might be worth the pain of the transition. – Duffy

Glassfish is dead - I wouldn't recommend anyone moving to it. – Wino

Answer 3

Have a look Spring Boot or Fabric8 for a modern take on running Java in a manageable way

Wino

These both go for creating separate virtual machines. You solve the monitoring/managing issues, but you still need to do the memory management manually. – Fistic

I suspect the core problem is not the management of the apps and their memory, but why is the app so unstable that it needs constant tweeking of memory? – Wino

Good point SteveD. The issue isn't that the app is unstable. This is not a webapp, but a rather large application with many different custom options and add-ons that vary based on the customer's needs and size. So, there is no set "one size fits all" way of apportioning memory for the app. While the app is generally quite stable, it is a fact of life that sometimes there are problems; having the apps isolated has been a huge win when something does go wrong. Being able to dedicate chunks of memory per customer is thus both a blessing and a curse. – Lumbar

I'm afraid your suffering from diminished expectations. If highly variable workloads are a feature/constraint of your system, your developers need to come up with an architecture to handle it and not dump the problem on the ops guys. Your system can't do any back pressure and just fails with OOMs if the workload is higher than the current max memory allows? – Wino

Answer 4

Another important reason to have multiple JVM instead of one is when facing numa groups. You cannot distribute your threads within one JVM appropriate over numa groups as you can with multiple jvm processes. At least i never found a way to do so.

We have here machines with two cpu's, each having 18 cores, This gives two numa groups and we cannot enforce 34 Threads to be spread upon both cpu if only one JVM is used. This is obviously because it assumes that all threads of the same JVM process need to have fast access to the same memory which is not the case.

Having 34 Processes the system assumes they have no need for shared memory and thus spread them over both cpus.

If someone knows a better way of doing this i would be very glad to hear it.

Sec