Understanding the Java Virtual Machine Heap for High Performance Applications


Diffusion 5.9

Diffusion 6.0

Diffusion 6.1

Diffusion 6.2

Diffusion 6.3

Diffusion Cloud


It is important to understand the impact of memory use and garbage collection upon performance in the Java Virtual Machine (JVM) when used in high-performance applications, such as a Diffusion based system.

A full-scale Diffusion deployment can deal with millions of topics, with thousands of subscribers. A deployment of this type will inevitably place high memory requirements on the JVM (in the range of gigabytes). Memory, in a rapidly changing, scalable environment, needs to be reclaimed effectively, and the modern garbage collectors used in Java are optimised for this. However, garbage collecting events have an impact on performance. In systems which require high availability and responsiveness to data changes, this impact should be minimised. Understanding the behaviour of the JVM is one of the first steps to find problems such as memory leaks or performance and scalability issues, and the memory profile provides invaluable data for it. The final part of this document introduces tools which can be used to monitor and analyse a memory profile of JVM.

Java Virtual Machine heap area

The memory structure in the Java Virtual Machine consists of several data areas which reside in native memory and which have different roles (Fig.1)

Fig.1 JVM memory structure, containing heap and non-heap areas.

The focus of this article is on the heap area where memory for all class instances and arrays is allocated. The heap is created at JVM start-up and shared by all JVM threads. During execution of a Java program, threads allocate memory for newly-created objects on the heap, but do not reclaim it when these objects are not required anymore. With time the heap, which has a limited amount of memory space, is filled with unreachable objects: objects without reachable reference, eligible for reclamation (1). This memory needs to be reclaimed, otherwise the heap will be exhausted, filled with unreachable objects with no space for freshly created objects. In languages like C or C++, developers are responsible for memory management, but in Java this process is automated and it is called garbage collection (GC). The JVM garbage collector divides the heap into smaller parts called generations. These are: Young Generation and Old or Tenured Generation. Different garbage collection algorithms are used for managing different generations.

Fig.2 Structure of heap in HotSpot Virtual Machine.

Young Generation

The Young Generation is an area where all new objects are allocated and aged (2). When the young generation becomes full, a minor garbage collection is performed which moves still referenced objects (survivors) from it, to allow for threads to create more new objects. At the end of the process, only unreachable objects are left in the garbage collected young generation area. That means the area can be reclaimed and used for the allocation of new objects. The minor garbage collections are stop-the-world events: program execution stops in all threads, excepting threads needed by GC, until collection is completed. However, minor collection is very fast, so for most applications, this pause has negligible impact on latency. Fig.2 represents the heap structure of Java HotSpot Virtual Machine, where the Young Generation is further divided into more areas, called Eden and Survivors. These areas are used to improve the minor garbage collection process. In this architecture, new objects are created in Eden space. During the first minor GC, surviving objects are moved to one of the Survivor areas, instead of directly to the old generation. The following minor GC will collect garbage from Eden and the initial Survivor (with age incremented by 1) and move them to another Survivor space. This step empties the initial Survivor space, which can be then reused. The process is repeated when Eden space is full and only objects of an age exceeding a certain threshold are moved to the Old Generation space.

Old Generation

Objects that survive long enough are moved to the Old Generation space. As these objects collect here, eventually they need to be collected by a major garbage collection event. The JVM offers several types of GC with different characteristics, which are suitable for different platforms and architectures:

  • Serial GC – default for client machines, as it uses a single CPU.
  • Parallel GC – default for server-class machines with more than 2 cores and a large amount of memory. It is similar to Serial GC, but uses multiple threads to speed up the process and shorten collection pauses. Can be used for minor and major collection.
  • Concurrent Mark and Sweep (CMS) GC – this attempts to minimise the stop-the-world pauses by doing most of the work concurrently with the application threads.
  • Garbage-First (G1) GC – a newer GC type, this is the default used when starting Diffusion 6.0. This collector drastically changes how the heap is managed. It divides the heap into small, equal-sized regions, any of which can be used as Eden, Survivor or Old space (3). This provides greater flexibility, as the area assigned to the three different types of space can now vary depending on the demands of the application.

Memory utilisation in JVM

Healthy application

In a healthy JVM application, it is normal to observe increases of memory use over time until a major garbage collection is performed, which clears the heap from dereferenced (dead) objects. This creates a saw-tooth like graph of heap use, as shown in Fig. 3 below:

Fig. 3 Memory utilisation graph for Java application without memory leak.

The horizontal axis represents time and the vertical access represents heap memory used. The arrows are:

  • Blue – Allocation rate. The rate in which running application allocates new object on the heap. The steeper it is, the more objects are allocated in the same amount of time.
  • Black – A GC event. When garbage is collected during a minor or major GC event, memory is freed from unreachable objects and can be allocated again.
  • Green – The baseline trend after GC events. It represents heap utilisation with live (reachable) objects.

The application from Fig.3 has healthy memory use, as the baseline heap usage trend (green arrow) stays on the same level.

Application with memory leak

Fig.4: Memory utilisation graph for Java application with memory leak.

This graph uses the same axes as the previous figure, but this time, the application has a memory leak. After each of the numbered GC events, memory is not fully reclaimed from the heap, so the base line of the heap usage (green arrow) increases over time. Differences between peaks and troughs get smaller toward the end of the graph as the GC was able to reclaim less and less memory. Finally, maximum heap size (thick horizontal red line) is reached and the program exits with an OutOfMemoryError exception. With a large enough heap, garbage collection events might take a long time to be executed. In such a scenario, a steady increase of memory usage will be observed and could be interpreted as memory leak, when it is in fact the natural behaviour of the JVM, and eventually a major GC event will occur which will clean the heap of dead objects.

Measuring memory utilisation with Java Mission Control

There are many tools available to monitor and record memory use in the JVM. The Oracle JDK comes with several monitoring tools. Java Mission Control (JMC) and Java Flight Recorder (JFR) are two of them. JMC and JFR monitor a whole range of JVM parameters. They are free to use for development (an Oracle licence is currently required to use them in production environment) and have a minimal impact on system performance. Java Mission Control provides a control console. Java Flight Recorder records a JVM profile over the specified time period into a file which can be analysed later.  The Java Flight Recording Knowledge Base article provides information on enabling these tools for the JVM and how to record a JVM profile. Long running Java processes such as Diffusion might require observation for a long period of time, during which many GC events may take place. With a large heap size, these events might take a significant time to occur, so triggering GC manually can be used to check the trend of memory use. There are several ways to do this. One way, using JMC, is by clicking the Trash icon in the Memory tab of the Monitor console (Fig.5).

Fig. 5 Memory tab in JMC Monitoring Console with highlighted icon to enforce garbage collection.

When there is an indication of a memory leak in an application, the next step is to find where it occurs. JFR provides heap objects statistics which can help, but a heap dump can provide more detailed information about heap object allocations. The Heap Dumps article discusses in detail how this can be done.

References:

Oracle’s specification for The Structure of the Java Virtual Machine Oracle tutorial on Java Garbage Collection Basics Oracle guide to Getting Started with the G1 Garbage Collector Details on The Parallel Collector in the HotSpot Virtual Machine Garbage Collection Tuning Guide Details on The Concurrent Mark Sweep Collector in the HotSpot Virtual Machine Garbage Collection Tuning Guide Details on The G1 Collector in the HotSpot Virtual Machine Garbage Collection Tuning Guide Garbage Collection discussed in the context of tuning JVMs, including a comparison of different GC types Java Mission Control description in the Java Platform Troubleshooting Guide