Java Platform, Standard Edition Troubleshooting Guide Release 9 E61074-05 October 2017 Java Platform, Standard Edition Troubleshooting Guide, Release 9 E61074-05 Copyright © 1995, 2017, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency- specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle. Contents Preface Audience xii Documentation Accessibility xii Related Documents xii Conventions xii Part I General Java Troubleshooting 1 Prepare Java for Troubleshooting Set Up Java for Troubleshooting 1-1 Enable Options and Flags for JVM Troubleshooting 1-1 Gather Relevant Data 1-3 Make a Java Application Easier to Debug 1-4 2 Diagnostic Tools Diagnostic Tools Overview 2-1 Java Mission Control 2-2 Troubleshoot with Java Mission Control 2-3 What Are Java Flight Recordings 2-3 Types of Recordings 2-4 How to Produce a Flight Recording 2-5 Use Java Mission Control to Produce a Flight Recording 2-5 Use Startup Flags at the Command Line to Produce a Flight Recording 2-9 Use Triggers for Automatic Recordings 2-10 Inspect a Flight Recording 2-11 How to Get a Sample JFR to Inspect 2-11 Range Navigator 2-12 General Tab 2-12 Memory Tab 2-13 Code Tab 2-16 iii Threads Tab 2-17 I/O Tab 2-19 System Tab 2-19 Events Tab 2-19 The jcmd Utility 2-19 Useful Commands for the jcmd Utility 2-21 Troubleshoot with the jcmd Utility 2-22 Native Memory Tracking 2-22 Use NMT to Detect a Memory Leak 2-23 How to Monitor VM Internal Memory 2-23 JConsole 2-27 Troubleshoot with the JConsole Tool 2-28 Monitor Local and Remote Applications with JConsole 2-29 The jdb Utility 2-30 Troubleshoot with the jdb Utility 2-31 Attach a Process 2-31 Attach to a Core File on the Same Machine 2-32 Attach to a Core File or a Hung Process from a Different Machine 2-32 The jinfo Utility 2-33 Troubleshooting with the jinfo Utility 2-35 The jmap Utility 2-35 Heap Configuration and Usage 2-36 Heap Histogram 2-37 Permanent Generation Statistics 2-38 The jps Utility 2-40 The jstack Utility 2-41 Troubleshoot with the jstack Utility 2-41 Stack Trace from a Core Dump 2-41 Mixed Stack 2-42 The jstat Utility 2-43 The visualgc Tool 2-45 Control+Break Handler 2-45 Thread Dump 2-46 Detect Deadlocks 2-47 Heap Summary 2-48 Native Operating System Tools 2-48 DTrace Tool 2-49 Probe Providers in Java HotSpot VM 2-49 Improvements to the pmap Utility 2-50 Improvements to the pstack Utility 2-51 Custom Diagnostic Tools 2-51 iv Java Platform Debugger Architecture 2-51 NMT Memory Categories 2-52 Postmortem Diagnostic Tools 2-52 Hung Processes Tools 2-54 Monitoring Tools 2-54 Other Tools, Options, Variables, and Properties 2-55 The java.lang.management Package 2-57 The java.lang.instrument Package 2-57 The java.lang.Thread Class 2-58 JVM Tool Interface 2-58 The jrunscript Utility 2-58 The jsadebugd Daemon 2-59 The jstatd Daemon 2-59 Thread States for a Thread Dump 2-59 Troubleshooting Tools Based on the Operating System 2-59 3 Troubleshoot Memory Leaks Debug a Memory Leak Using Java Flight Recorder 3-1 Detect a Memory Leak 3-1 Find the Leaking Class 3-2 Find the Leak 3-3 Understand the OutOfMemoryError Exception 3-4 Troubleshoot a Crash Instead of OutOfMemoryError 3-7 Diagnose Leaks in Java Language Code 3-7 Get a Heap Histogram 3-8 Monitor the Objects Pending Finalization 3-9 Diagnose Leaks in Native Code 3-10 Track All Memory Allocation and Free Calls 3-10 Track All Memory Allocations in the JNI Library 3-10 Track Memory Allocation with Operating System Support 3-11 Find Leaks with the dbx Debugger 3-12 Find Leaks with the libumem Tool 3-14 4 Troubleshoot Performance Issues Using JFR JFR Overhead 4-1 Find Bottlenecks 4-2 Garbage Collection Performance 4-4 Synchronization Performance 4-6 I/O Performance 4-7 v Code Execution Performance 4-8 Part II Debug JVM Issues 5 Troubleshoot System Crashes Determine Where the Crash Occurred 5-1 Crash the Native Code 5-2 Crash in the Compiled Code 5-3 Crash in the HotSpot Compiler Thread 5-4 Crash in the VM Thread 5-4 Crash Due to Stack Overflow 5-4 Find a Workaround 5-5 Working Around Crashes in the HotSpot Compiler Thread or Compiled Code 5-6 Working Around Crashes During Garbage Collection 5-8 Working Around Crashes Caused by Class Data Sharing 5-9 Microsoft Visual C++ Version Considerations 5-10 6 Troubleshoot Process Hangs and Loops Diagnose a Loop Process 6-1 Diagnose a Hung Process 6-2 Deadlock Detected 6-2 Deadlock Not Detected 6-4 No Thread Dump 6-4 Oracle Solaris 8 Thread Library 6-5 7 Handle Signals and Exceptions Handle Signals on Oracle Solaris, Linux, and macOS 7-1 Handle Exceptions on Windows 7-1 Signal Chaining 7-3 Handle Exceptions Using the Java HotSpot VM 7-4 Console Handlers 7-5 Signals Used in Oracle Solaris, Linux, and macOS 7-5 Part III Debug Core Library Issues vi 8 Time Zone Settings in the JRE Native Time Zone Information and the JRE 8-1 Determine the Time Zone Data Version in Use 8-2 Troubleshoot Problems with TZupdater 8-2 Determine the Default Time Zone on Windows 8-3 Check the Default Time Zone JRE Reports 8-3 Determine the Setting in the Control Panel 8-4 Check for Automatic Daylight Saving Time Adjustment 8-4 Set the Default Time Zone in the Control Panel 8-5 Check -Duser.timezone System Property 8-5 Special Tools in Windows 7 8-6 JRE Internal Representation of Time Zone Mappings 8-6 Part IV Debug Client Issues 9 Introduction to Client Issues Java SE Desktop Technologies 9-1 General Steps to Troubleshoot an Issue 9-3 Identify the Type of Issue 9-3 Java Client Crashes 9-4 Performance Problems 9-4 Behavior Problems 9-5 Basic Tools 9-6 Java Debug Wire Protocol 9-6 10 AWT Debug Tips for AWT 10-1 Layout Manager Issues 10-2 Key Events 10-2 Modality Issues 10-3 AWT Crashes 10-4 Focus Events 10-5 How to Trace Focus Events 10-5 Native Focus System 10-6 Focus System in Java Plug-in 10-7 Focus Models Supported by X Window Managers 10-7 Miscellaneous Problems with Focus 10-8 Data Transfer 10-9 vii Debug Drag-and-Drop Applications 10-10 Frequent Issues with Data Transfer 10-10 Other Issues 10-12 Splash Screen Issues 10-12 Tray Icon Issues 10-13 Pop-up Menu Issues 10-13 Background or Foreground Color Inheritance 10-13 AWT Panel Size Restriction 10-13 Hangs During Debugging of Pop-up Menus and Similar Components on X11 10-14 Window.toFront()/toBack() Behavior on X11 10-14 Heavyweight or Lightweight Components Mix 10-15 11 Java 2D Pipeline Rendering and