Tuning the compaction and thread local area (TLA) size.

a.      Compaction is used to reduce heap fragmentation, i.e., move objects to form contiguous ‘live regions’ of the heap space. Note when a full compaction is run the stopping of a concurrently running application is inevitable. We can say that compaction is one of the worst garbage collection bottlenecks. If anything is known about fragmentation and object sizes it might be beneficial to tune compaction.

b.      The compaction algorithm divides the heap into a number of equally large parts. Each of these is subject to separate compaction that can stop the world. The default is 4096 parts. When compaction is too disruptive it might help to increase the number of heap parts. When compaction fails to keep up with fragmentation it might help to decrease the number of heap parts. Note that, for strategies other than throughput the compaction areas are sized dynamically

c.       The proper tuning of compaction involves two parameters:

                                                              i.      Compaction ratio (-XX:compaction:percentage:<percentage> or -XX:compaction:internalPercentage=<percentage>,externalPercentage=<percentage>) – the percentage of the heap that the garbage collector compacts at each garbage collection. While the JVM is compacting the heap, all threads that want to access objects need to wait because the JVM is moving the objects around.

                                                            ii.      Maximum references (-XX:compaction:maxReferences:<value>) – the maximum number of references to objects in the compaction area. If the number of references exceeds this limit, the compaction is canceled. When compaction has moved objects, the references to these objects must be updated. The pause time introduced by this updating is proprotional to the number of references that have been updated

d.      steps to tune compaction are as follows:

                                                              i.      Set the compaction ratio to 1 and gradually increase the ratio until the pause time becomes too long.

                                                            ii.      If the garbage collection time is too long when the compaction ration is 1, the maximum number of references must be adjusted:

                                                          iii.      Set the maximum references to 10000 and gradually increase the references until the pause time becomes too long. To monitor compaction behavior, we can add the option -Xverbose:compaction to the command-line. If many compactions are skipped, the maximum references must be increased; if, on the other hand, the compaction pause times are too long the maximum references must be decreased.

                                                           iv.      As we are using a dynamic garbage collection that optimizes the pause time (-Xgc:pausetime), we do not need to tune the compaction manually in this case setting the -XpauseTarget is sufficient.

e.      Each thread allocates objects in a TLA that is promoted to the heap when full. The TLA size can be adjusted by using -XXtlaSize:min=<size>,preferred=<size>,wasteLimit=<size>, in which,

                                                              i.      min – sets the minimum size of a TLA.

                                                            ii.      preferred – sets the preferred size of a TLA. The system will try to get TLAs of the preferred size if possible, but accepts TLAs of minimum size as well.

                                                          iii.      wasteLimit – sets the waste limit for TLAs. This is the maximum amount of free memory that a TLA is allowed to have when a thread requires a new TLA

Read more...

Garbage collector(s)

1.  Garbage collector can be generational or single spaced with a parallel or a concurrent mark and a parallel sweep or a concurrent sweep:
2. Generational garbage collection – In this case the heap is divided into two sections: an old generation and a young generation (nursery). Initially objects are allocated in the nursery and when it is full, live objects are promoted to the old generation. For this operation the collector has to stop the Java threads.
3. Single-spaced garbage collection – All objects live in a single space on the heap, regardless of their age.
4. Concurrent mark and sweep algorithm – The concurrent garbage collection algorithm does its marking and sweeping concurrently with all other processing, i.e., the Java threads are not stopped during garbage collection.
5. allel garbage collection mark and sweep algorithm – The parallel garbage collection algorithm stops Java threads when the heap is full and uses all available CPUs to perform a mark and sweep of the heap. Typically a parallel collection has longer pause times than a concurrent collection, but maximizes the application throughput.

We can select a specific garbage collector by using -Xgc:<option>, with option being one of the following:

• singlecon – Single-space, concurrent garbage collection. This mode reduces application throughput but keeps pause times to a minimum.
• gencon – Generational, concurrent garbage collection. The gencon mode is better than the singlecon mode for applications that allocate lots of short-lived objects. This mode reduces application throughput but keeps pause times to a minimum.
• singlepar – Single-space, parallel garbage collection. This mode increases pause times when compared with the concurrent mode but maximizes throughput.
• genpar – Generational, parallel garbage collection. This mode is generally better than the singlepar mode for applications that allocate lots of short-lived objects. In this mode, a higher number of garbage collections are performed than in the singlepar mode, but the individual pause times are shorter, resulting in lower fragmentation in the old generation space.
• genconpar – Generational garbage collection. Sets the garbage collection mode to generational with a concurrent mark algorithm and a parallel sweep algorithm.
• genparcon – Generational garbage collection. Sets the garbage collection mode to generational with a parallel mark algorithm and a concurrent sweep algorithm.
• singleconpar – Single-space garbage collection. Sets the garbage collection mode to single-spaced with a concurrent mark algorithm and a parallel sweep algorithm.
• singleparcon – Single-space garbage collection. Sets the garbage collection mode to single-spaced with a parallel mark and a concurrent sweep algorithm.
• throughput – The garbage collector is optimized for application throughput. This means that the garbage collector works as effectively as possible, giving as much CPU resources to the Java threads as possible. This might, however, cause non-deterministic pauses when the garbage collector stops all Java threads for garbage collection.
• pausetime – The garbage collector is optimized for short pauses. This means that the garbage collection works concurrently with the Java application when necessary, in order to avoid pausing the Java threads. This inflicts a slight performance overhead to the application, as the concurrent garbage collector demands more system resources (CPU time and memory) than the parallel garbage collector that is used for optimal throughput. The target pause time is by default 200 milliseconds, which can be changed by using -XpauseTarget.
• deterministic – Optimizes the garbage collector for very short and deterministic pause times. The garbage collector tries to keep the garbage collection pauses below a given pause target. The performance depends on the application and the hardware. Running on slower hardware, with a different heap size or with a large amount of live data can break the deterministic behavior or cause performance degradation over time; faster hardware or a less amount of live data might allows us to set a lower pause target. The pause target for the deterministic mode is by default 30 milliseconds, which can be changed by using -XpauseTarget

Read more...

EhCache and Hibernate issue

Session factory issue in setting up hibernate distributed ehcache with terracotta.

If you want the cache (or the caches) to be available from different JVMs you will need Terracotta; ehCache alone cannot do that. 

EhCache is not a distributed solution. So you need to use it with Terracotta to obtain the result you want. 

You can use Hazelcast, but also Infinispan to obtain this result

a.      Add the following jar files

<dependency>

  <groupId>net.sf.ehcache</groupId>

  <artifactId>ehcache-core-ee</artifactId>

  <version>2.5.2</version>

  <scope>compile</scope>

</dependency>

<dependency>

  <groupId>net.sf.ehcache</groupId>

  <artifactId>ehcache-terracotta-ee</artifactId>

  <version>2.5.2</version>

  <scope>compile</scope>

</dependency>

<dependency>

  <groupId>org.terracotta</groupId>

  <artifactId>terracotta-toolkit-1.5-runtime-ee</artifactId>

  <version>4.2.0</version>

  <scope>compile</scope>

</dependency>

Read more...

Tomcat locks Drools jar files

I was deploying my application with Tomcat and Drools 5. But most of the time, it seems that Tomcat locks the Drools jar files.

I was trying to unlock it but every time I had to do a complete Tomcat shut down wait for 5-10 mins and bring it up.

I was searching for an appropriate answer but the closest one was the one from the threads in Stack Overflow.

It solved my problem immediately.

"Try using antiResourceLocking='true' in your$tomcat_home/conf/context.xml. This should at least help to release the locks"

Hopefully some one gets this benefit from my forum.

Read more...