Profiling a Java application with VisualVM is easy enough to do, however if you're on a remote system which cannot accept a JMX connection then you will need to profile the application from the command line.
For this purpose I'm interested in profiling a running server application, rather than an application which is started and stopped.
There are a number of options:
hprof
This utility is provided with the JVM though appears to be somewhat out of date and has probably been superseded by other methods.
brendangregg.com provides a good article on how to use it and the problems with using it to diagnose performance problems. It does appear to have problems with the way it actually measures CPU usage.
Example profiling command:
java -agentlib:hprof=cpu=samples,depth=100,interval=20,lineno=y,thread=y mainClass
This will sample CPU usage up to 100 layers deep in stack traces every 20 ms. This will be dumped to the log file when the application terminates, or when it receives a SIGQUIT.
Example usage:
jps # to get Java Process pid
kill -3
Whilst appears useful, it seems somewhat clunky to get a meaningful output. Or perhaps needs more practice.
jstack
JStack is considerably simpler in that it just prints the stack trace for each thread, and then leaves it up to the caller to parse/analyse. This makes its operation simple to script up and allows precise control over when it should be used and the results it will collect.
A possibly useful parser might be ParseJStack which has a good writeup at weblogs.java.net.
Flight Recorder
With Java 8 comes flight recorder, which looks perhaps the most promising option of the lot.
Details: docs.oracle.com
