Using Dtrace with Sun Studio Tools to Understand, Analyze, Debug, and Enhance C
Total Page:16
File Type:pdf, Size:1020Kb
Using DTrace with Sun Studio Tools to Understand, Analyze, Debug, and Enhance C ... Page 1 of 17 United States Worldwide Java Solaris Communities About Sun How to Buy My Account Cart Sun Microsystems Documentation docs.sun.com Browse Documentation Titles Browse Product Documentation Help Table of Contents Using DTrace with Sun Studio Tools to D Program Structure Understand, Analyze, Debug, and Enhance DTrace Architecture Providers and Probes Complex Applications Actions and Predicates By Nasser Nouri, April 2010 Principal Buffers Security Privileges In general, a complex application is implemented by several engineers. And in most cases, the engineer who is responsible for debugging or increasing the performance of the application is not the original DTrace and Sun Studio Tools software designer or developer. Understanding the Behavior of Applications Hence the software engineer might encounter some difficulties in understanding, analyzing, or pid.d debugging the application for reasons such as lack of documentation, lack of good coding practices, or The Script ufunc.d the complexity of the application itself. The D Script Measuring the Performance of Enhancing an existing application can be divided into two different stages: Applications functime.d • Exploration or discovery stage The D Script Measuring the Performance of • Enhancement or creation stage Multi-threaded Applications The lwptime.d D Script In the first stage, the software developer wants to understand the behavior of the application, how the functions are being called, and how the data travel through the software stacks. The software developer Memory Allocations, Leaks, has to have a good understanding of the application before advancing to next stage. and Access Errors The hello.c Program In the second stage, the software developer makes enhancements or creates a new solution based on The mfs.d Script the knowledge that is acquired in the first stage. The Output of the mfs.d D Script The Solaris Dynamic Tracing (DTrace) facility can be used by developers for the exploration or Debugging Applications discovery stage. It helps them to funnel their attention to the problem area of a complex software The stop.d D Script system. Tracing Arbitrary Instructions The errorpath.d D Script Sun Studio tools can be used by developers for the enhancement or creation stage. Once they have The Output of the errorpath.d identified the problem area of a complex software system using DTrace, they can use Sun Studio tools Script to zoom into the problem area to pinpoint the culprits and implement the optimal solutions. The signal-send Probe in the proc Provider Developers will benefit most when they use Sun Studio tools during the development phase. However, sig.d once the software is deployed, DTrace becomes very instrumental at customer sites to diagnose the The D Script software and produce valuable information for developers. Statically Defined Tracing for User Applications This article describes how DTrace can be used in conjunction with Sun Studio tools to understand, The libCrun.d File analyze, and debug software applications. First the article describes the DTrace utility and its Chime Visualization Tool for architecture. Next it shows how different tools can be used to fix the software deficiencies and the DTrace performance problems. In Conclusion This article is not intended to describe all of the functionality of DTrace and the Sun Studio tools. It is intended to give enough information to developers so they can make wise decisions about selecting the right tool for the right task. The NetBeans DTrace GUI Plugin is a graphical user interface (GUI) DTrace that can be installed into the Sun Studio IDE using the NetBeans Plugin Portal (choose Tools > Plugins in the IDE). You can run D scripts from the GUI, even those that are embedded in shell scripts. In fact, the DTrace GUI Plugin runs all of the D scripts that are packaged in the DTraceToolkit. The DTraceToolkit is a collection of useful documented scripts developed by the OpenSolaris DTrace community. You can visualize the output of DTrace using Chime. Chime is a graphical tool for visualizing DTrace aggregations. In particular, its ability to display data over time adds a missing dimension to system observability. Chime is fully integrated with the NetBeans DTrace GUI Plugin. Documentation for the NetBeans DTrace GUI Plugin is available at http://wiki.netbeans.org/DTrace . D Program Structure The DTrace utility includes a new scripting language called D. The D language is derived from a large subset of C programming language combined with a special set of functions and variables to help make tracing easy. Each D program consists of a series of clauses, each clause describing one or more probes to enable, and an optional set of actions to perform when the probe fires. https://docs.oracle.com/cd/E19205-01/820-4221/ 9.2.2018 Using DTrace with Sun Studio Tools to Understand, Analyze, Debug, and Enhance C ... Page 2 of 17 probe descriptions /predicate / { action statements; } Probe descriptions are specified using the form provider :module :function :name . Omitted fields match any value. • Provider - The name of the DTrace provider that is publishing this probe. The provider name typically corresponds to the name of the DTrace kernel module that performs the instrumentation to enable the probe. • Module - The name of the module in which the probe is located. This name is either the name of a kernel module or the name of a user library. • Function - The name of the program function in which the probe is located. • Name - The name that represents the location in the function, such as entry or return. Predicates are expressions enclosed in slashes (/ /) that are evaluated at probe firing time to determine whether the associated actions should be executed. Probe actions are described by a list of statements separated by semicolon (;) and enclosed in braces ({}). DTrace Architecture The core of DTrace - including all instrumentation, probe processing and buffering - resides in the kernel. Processes become DTrace consumers by initiating communication with the in-kernel DTrace component using the DTrace library. The dtrace command itself is a DTrace consumer, since it is built on top of the DTrace library. Figure 1 Overview of the DTrace Architecture Providers and Probes DTrace providers are loadable kernel modules that communicate with the DTrace kernel module. When you use DTrace, each provider is given an opportunity to publish the probes it can provide to the DTrace framework. You can then enable and bind your tracing actions to any of the probes that have been published. To create a probe, the provider specifies the module name and function name of the instrumentation point; plus a semantic name for the probe. Each probe is thus uniquely identified by a 4-tuple: < provider, module, function, name > The DTrace framework provides sixteen different instrumentation providers that among them offer observability into many aspects of the system. The providers are named as follows: dtrace provider probes related to DTrace itself lockstat lock contention probes profile probe for firing at fixed intervals fbt function boundary tracing provider syscall probes at entry/return of every syscall sdt “statically defined probes” user definable probe sysinfo probe kernel stats for mpstat and sysinfo tools vminfo probe for vm kernel stats https://docs.oracle.com/cd/E19205-01/820-4221/ 9.2.2018 Using DTrace with Sun Studio Tools to Understand, Analyze, Debug, and Enhance C ... Page 3 of 17 proc process/LWP creation and termination probes sched probes for CPU scheduling io provider probes related to disk IO mib probes in network layer in kernel fpuinfo probe into kernel software FP processing pid probe into any function or instruction in user code plockstat probes user level sync and lock code fasttrap non-exposed probe, fires when DTrace probes are called Please refer to the Solaris Dynamic Tracing Guide at http://docs.sun.com/app/docs/doc/819–3620 for complete descriptions of these providers. DTrace also provides a facility for developers of user applications to define customized probes in application code to augment the capabilities of the pid provider. These probes are called static probes. Actions and Predicates Actions are executed only when the predicate evaluates to true. Actions enable your DTrace programs to interact with system outside of DTrace. The most common actions record data to a DTrace buffer. Other actions are available, such as stopping the current process, raising a specific signal on the current process, or ceasing tracing altogether. Some of these actions are destructive in that they change the system, albeit in a well-defined way. These actions can only be used if destructive actions have been explicitly enabled. By default, data recording actions record data to the principal buffer. The data recording actions are: trace takes a D expression as its argument and traces the result to the directed buffer tracemem copies the memory from the address specified by addr into the directed buffer for the length specified by nbyte printf traces D expressions printa enables users to display and format aggregations stack records a kernel stack trace to the directed buffer ustack records a user stack trace to the directed buffer jstack an alias for ustack()() , will result in a stack with in situ Java frame translation The destructive actions are: stop forces the process that fires the enabled probe to stop raise sends the