This post highlights three performance tuning techniques for Red Hat Enterprise Linux (6.2) and OpenJDK (1.6): TCP send / receive windows, large page memory, and i-CMS.

These techniques are particularly applicable to data grids or to any distributed system that stores a lot of data.

I just finished a technical white paper on JBoss Data Grid (JDG) performance factors, and these techniques were necessary for JDG to perform at its best.

Networking

Set the maximum size of the TCP send / receive windows of the accordingly.

Example: 640K Send Window / 25MB Receive Window

echo 655360 > /proc/sys/net/core/wmem_max
echo 26214400 > /proc/sys/net/core/rmem_max

Memory

Configure large page memory. More information about large page memory can be found here.

Example: A server with 16GB of memory with 12.5GB of it reserved for large page memory.

echo 17179869184 > /proc/sys/kernel/shmmax
echo 6400 > /proc/sys/vm/nr_hugepage
echo 500 > /proc/sys/vm/hugetlb_shm_group

Update the resource limits.

/etc/security/limits.conf

jboss      soft   memlock      unlimited
jboss      hard   memlock      unlimited

Set the JVM options.

Note: The heap size (12GB) is less than the amount of memory reserved for large page memory (12.5GB) as the heap is not the only memory used by the JVM.

-Xms12G -Xmx12G
-XX:+UseLargePages

Garbage Collection

Use the Concurrent Mark Sweep (CMS) garbage collector in Incremental mode (i-CMS).

-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode