I had quite an interesting bug hunting last week in one of our extern engagements. In summer a memory leak was discovered in a new used third party library. Back then, we started with testing using Java profilers (Java SDK ships a good enough profiler - visual vm). Surprisingly, we found nothing; the heap didn’t grow, number of threads remained the same, … in short, the usual suspects seemed innocent. Due to some other projects, we hadn’t enough time to do more testing, and so we just informed the company that sells the library about the issue. After some weeks, the company’s response was a we-dont-have-a-memory-leak-your-testing-is-wrong response (they suggested memory consumption is probably because of a different virtual machine and I have to admit that the task manager was also used to view the memory growth, which isn’t the tool to convince any real developer that there is a memory leak). However, the problem became urgent (like software problems tend to do) when the application was stress tested in real environment for a real customer. This project will be the first shipping with the new library. The application failed after approximately 6 hours due to a java.lang.OutOfMemoryError … Out of swap space?.

In short this error means that the jvm cannot get more memory from the operating system. For us this meant there has to be native memory leak in the jvm or in the OS side. Since the library is for communication and uses heavily sockets, our first guess was a socket problem. Investigating the code of the library (gladly we have the source files) brought no results. During my research (what a fancy word for googling) I stumbled across this forum thread. It describes a native memory leak that occurs when a ClassCastException is thrown! Further, the bug is still not fixed. I remembered that there are a lot of casts in the third party library, so I started testing how fast the memory grows. The following code pushes the memory usage to 340 MB:

[sourcecode language='java']package com.hemju;

/**
* Example of the {@link ClassCastException} memory leak.
*
* @author Helmut Juskewycz, hjuskewycz - at - hemju.com
*/
public class CEMemoryLeak {

public static void main(String argv[]) throws Exception {
Object sObject = new String(”Hello Leak”);
Integer intObject = new Integer(”5″);

for (int i = 0; i < 5000000; i++) {
try {
intObject = (Integer) sObject;
} catch (ClassCastException ex) {
}
}
Thread.sleep(60000);
}
}

[/sourcecode]

Our application crashed with about 600-800 MB memory, hence, over 10 000 000 Class Cast Exceptions have to be thrown and caught! A lot of exceptions, nevertheless it was possible because we tested the application under heavy load. Phaser,