The Neo4j Java FlightRecorder Integration

This package allows you to use Java Mission Control and FlightRecorder to monitor select parts of the Neo4j internals. This is pretty low-level and undocumented, so if you don’t already have a good idea about what is going on here, then be prepared for a pretty steep learning curve.

Building the package

Just run mvn clean package and find neo4j-jfr.jar in the cli/target directory. Copy it to the <neo4j-install-dir>/lib directory.

Turning on Java FlightRecorder

Add the following to neo4j-wrapper.conf:

wrapper.java.additional=-XX:+UnlockCommercialFeatures
wrapper.java.additional=-XX:+FlightRecorder
wrapper.java.additional=-XX:FlightRecorderOptions=stackdepth=500
wrapper.java.additional=-XX:+UnlockDiagnosticVMOptions
wrapper.java.additional=-XX:+DebugNonSafepoints

Note that this requires an Oracle JDK, and that there are special licensing terms for FlightRecorder.

Configuration

Add the following to your neo4j.properties file:

dbms.tracer=jfr

Neo4j will now report page cache and transaction events to FlightRecorder, next time it starts. There is a slight performance overhead to enabling this, but it’s pretty small. There is a more significant performance overhead when a recording is being produced.

You can grab recordings through Java Mission Control, or from the command line. In Java Mission Control, you have to remember to enable the Neo4j events in the Flight Recording wizard. You can then export the Flight Recorder configuration from Mission Control, and use it as a profile when you grab recordings from the command line. You can alternatively generate a configuration file either with just the Neo4j events, or with Neo4j’s events merged into the .jfc configuration files that ship with your JVM. Use e.g. java -jar cli/target/neo4j-jfr.jar <$JAVA_HOME/jre/lib/jfr/default.jfc >neo4j.jfc to obtain neo4j.jfc which you can now copy into $JRE_HOME/lib/jfr. You can then start the flight recording with jcmd like this:

jcmd <PID> JFR.start duration=60s name=MyRecording filename=recording-output.jfr settings=neo4j

This will put the recording-output.jfr file in the working directory of the profiled program, that you can then open with Java Mission Control.

Common Problems

1. The JVM crashes in VM_Version_Ext::resolve_cpu_information_details() when I enable the FlightRecorder flags

   The virtual machines in Amazon EC2 occasionally report that they have 0 CPU cores: https://community.oracle.com/thread/3527326. Try upgrading Java, or use different AMIs or machine types.