Write your own monitoring software with RRDtool
Tobias Oetiker
Motivation Write your own monitoring About RRDtool Programming
software with RRDtool Summary
Tobias Oetiker
ISG.EE - ETH Zürich
OSCON 2005 Portland, Oregon, August 3, 2005 Write your own monitoring Netflow Data presented with software with RRDtool Fluxscope Tobias Oetiker
Motivation
About RRDtool
Programming
Summary Write your own monitoring Content software with RRDtool
Tobias Oetiker
Motivation The world today Motivation About data collection A graph says more than The world today 1’000 words About data collection About RRDtool A graph says more than Programming 1’000 words Summary
About RRDtool
Programming
Summary Write your own monitoring Numbers everywhere software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words
About RRDtool 15 users online I Programming I 1.34 PB free tape space Summary
I 31’223 ifInOctets
I 100 degrees in the server room
I 87 Mb/s transfer rate Write your own monitoring All these questions software with RRDtool
Tobias Oetiker
Motivation The world today About data collection I Are we running out of A graph says more than 1’000 words
diskspace? About RRDtool
I Is there a pattern in the high Programming server load? Summary
I Is there really global warming?
I Can we do something about it?
I Collect that data! Write your own monitoring How to collect data software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words
About RRDtool
I collect everything, analyze Programming
later (aka never) Summary
I data with known properties makes life simpler
I from research to production Write your own monitoring Requirements for a solution software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words
About RRDtool
Programming I handle counters and gauges Summary I be nice to the server
I setup for specific task
I data-pre processing
I and forget about it . . . Write your own monitoring Condor use at UW software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words
About RRDtool
Programming
Summary Write your own monitoring Making data accessible software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words I graphical About RRDtool
representation Programming
I eye candy for the Summary people
I analysis at a glance
I make my boss look good to his boss Write your own monitoring Data post-processing software with RRDtool
Tobias Oetiker
Motivation The world today About data collection A graph says more than 1’000 words
About RRDtool
Programming I highlight interesting Summary properties
I combine data from several sources
I add extra Information Write your own monitoring Content software with RRDtool
Tobias Oetiker
Motivation
About RRDtool Motivation History Features Graphing features About RRDtool Programming Summary History Features Graphing features
Programming
Summary Write your own monitoring 1995: MRTG was only a start software with RRDtool
Tobias Oetiker
Motivation
About RRDtool History Features Graphing features
Programming I logfiles in text format Summary I gnuplot for graphs
I 1996 - MRTG is used for amazing tasks
I more performance
I more flexibility Write your own monitoring The rrd TOOL software with RRDtool
Tobias Oetiker
Motivation
About RRDtool History Features Graphing features I a building block Programming I basis for a better Summary MRTG (2nd system)
I basis for hundreds of other tools
I Google says: “industry standard” Write your own monitoring The Round Robin Database software with RRDtool
Tobias Oetiker
Motivation
I lossy storage, fixed file size About RRDtool History I current data is the most Features Graphing features
interesting Programming
I consolidation functions for Summary feature extraction
I artificial data-sources
I holt-winters aberrant behavior detection
I constant step size
I fixed disk space Write your own monitoring Feeding Data software with RRDtool
Tobias Oetiker
Motivation combat jitter by taking About RRDtool I History Features acquisition time into Graphing features
account. Programming
Summary I preserve data-volume
I on-the-fly data validation
I deal with unknown data
I time is a one way street Write your own monitoring Basic Graphing software with RRDtool
Tobias Oetiker
Motivation
About RRDtool History Features I auto scaling Graphing features
I auto labeling Programming Summary I sensible defaults
I quick results
I anti-aliased output
I multiple formats (png/eps/pdf/svg) Write your own monitoring Advanced Graphing software with RRDtool
Tobias Oetiker
I change colors, fonts, Motivation About RRDtool sizes History Features I data from several Graphing features databases Programming Summary I data processing with RPN math
I alpha transparency (rrggbbaa)
I support of locale based character encoding
I truetype fonts Write your own monitoring Content software with RRDtool
Tobias Oetiker
Motivation Motivation About RRDtool Programming Language bindings Basic operations About RRDtool Advanced Graphing Summary
Programming Language bindings Basic operations Advanced Graphing
Summary Write your own monitoring On the command line software with RRDtool
Tobias Oetiker
Motivation
About RRDtool
Programming Just another Unix command. Language bindings Basic operations Advanced Graphing
code Summary rrdtool cmd arg example rrdtool info demo.rrd Write your own monitoring Via STDIN software with RRDtool
Tobias Oetiker
Save startup time by feeding several commands. Motivation About RRDtool code Programming Language bindings echo cmd arg | rrdtool - Basic operations Advanced Graphing example Summary echo info demo.rrd | rrdtool -
I several commands
I fast operation
I simple interface Write your own monitoring perl module software with RRDtool
Tobias Oetiker
Motivation I love coding in perl . . . About RRDtool Programming code Language bindings Basic operations use RRDs; Advanced Graphing my $return = RRDs::cmd arg; Summary example use RRDs; use Data::Dumper my $ret = RRDs::info "demo.rrd"; print Dumper $ret; Write your own monitoring Other Bindings software with RRDtool
Tobias Oetiker
Motivation
About RRDtool
Programming Language bindings RRDcgi Basic operations I Advanced Graphing I Perl Pipes - RRDp Summary
I Python
I Tcl
I and more externally Write your own monitoring Creating Round Robin Databases software with RRDtool
Tobias Oetiker
I what data do I want to look at? Motivation
I how long do I want to keep it? About RRDtool Programming I what are the interesting properties of my data? Language bindings Basic operations Advanced Graphing I what to keep in one rrd file? Summary Command rrdtool create filename [–start start time][–step step] [DS:ds-name:DST:heartbeat:min:max] [RRA:CF:xff:steps:rows]
DST: COUNTER, GAUGE, DERIVE, ABSOLUTE, New in 1.2: COMPUTE CF: AVERAGE, MIN, MAX, LAST, New in 1.2: Aberrant Behavior Detection. Write your own monitoring Create Example software with RRDtool
Tobias Oetiker example Motivation rrdtool create example.rrd --step=60\ About RRDtool DS:in:COUNTER:600:0:1000 \ Programming Language bindings Basic operations RRA:AVERAGE:0.5:1:100 \ Advanced Graphing
RRA:AVERAGE:0.5:10:100 \ Summary RRA:MAX:0.5:10:100
I resolution 60 seconds
I COUNTER data source
I updates at least every 600 seconds
I accept rates between 0 and 1000
I three archive with 100 entries
I store unknown unless half are known. Write your own monitoring Adding Data into a Round Robin software with RRDtool Database Tobias Oetiker
Motivation
About RRDtool
Programming Language bindings Command Basic operations rrdtool update filename [–template ds:ds:. . .] Advanced Graphing time:value:value. . . [. . . ] Summary
I data value and acquisition time
I timestamps have to increase with every update
I instead of time use N for now
I templates are for redundancy only Write your own monitoring Update Example software with RRDtool
Tobias Oetiker
Motivation
About RRDtool
Programming Language bindings Basic operations Advanced Graphing
example Summary rrdtool update example.rrd \ --template in N:39344 Write your own monitoring Data re-bin-ing software with RRDtool
Tobias Oetiker
Motivation 700 octets/sec About RRDtool 1 2 Programming 280k Language bindings octets Basic operations Advanced Graphing
Summary
400s 3
300s 300s
Data Aquisition Data Storage
300s 300s Write your own monitoring Data Consolidation software with RRDtool
Tobias Oetiker RRA with 1 PDP Motivation
About RRDtool
Programming Language bindings Basic operations Advanced Graphing
Summary AVERAGE RRA with 2 PDP
AVERAGE RRA with 3 PDP Write your own monitoring The first graph software with RRDtool
Tobias Oetiker
Motivation
About RRDtool
Programming Language bindings Command Basic operations rrdtool graph graph.png DEF:var=rrdfile:DS-name:CF Advanced Graphing Summary LINE1:var#rrggbb:label
1. prepare data (DEF) 2. draw graph (LINE1) Write your own monitoring Data processing with RPN software with RRDtool (remember the HP days) Tobias Oetiker
Motivation
What if you have collected octets About RRDtool
but want to present bits? Programming Language bindings Basic operations Calculated time series and variables Advanced Graphing CDEF:var=RPN expression Summary
Examples
a,8,* a ∗ 8 (bit to byte conversion) a,b,+ a + b a,b,c,IF if (a!=0) then b else c a,1800,TREND half-hour sliding window average a,b,c,LIMIT if (a>b AND a Tobias Oetiker Motivation Calculated single value variable About RRDtool Programming Language bindings VDEF:var=RPN expression Basic operations Advanced Graphing var is associated with a time and a data value. Summary Examples a,TOTAL rate multiplied with interval a,AVERAGE average value of a a,95,PERCENT find 95-percentile Write your own monitoring MRTG like Graph software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings Example Basic operations Advanced Graphing rrdtool graph graph.png \ Summary DEF:in=data.rrd:input:AVERAGE \ DEF:out=data.rrd:output:AVERAGE \ AREA:in#ff0000:incoming \ LINE2:out#00ff00:outgoing Write your own monitoring Graph with marked maximum software with RRDtool Tobias Oetiker Motivation About RRDtool Example Programming Language bindings Basic operations rrdtool graph graph.png \ Advanced Graphing DEF:a=data.rrd:a:AVERAGE \ Summary VDEF:max=a,MAXIMUM \ LINE2:a#00ff00:outgoing \ LINE1:max#ff0000:maximum\\g \ VRULE:max#ff0000 \ GPRINT:max:"at %.2lf" Write your own monitoring Graph with marked maximum software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings Basic operations Advanced Graphing Summary Write your own monitoring Graph with 95 percentile software with RRDtool Tobias Oetiker Motivation Example About RRDtool Programming rrdtool graph.png \ Language bindings Basic operations DEF:in=data.rrd:input \ Advanced Graphing DEF:out=data.rrd:output \ Summary CDEF:base=in,out,MAX \ VDEF:n95=base,95,PERCENT \ AREA:in#0000b0:incoming \ AREA:out#00ff0080:outgoing LINE1:base#000000:maximum \ LINE1:n95#ff0000:"95%-tile "\ GRPINT:n95:"at %.2lf %s" Write your own monitoring Graph with 95 percentile software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings Basic operations Advanced Graphing Summary Write your own monitoring Mailserver Statistics software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Language bindings Basic operations Advanced Graphing Summary Write your own monitoring Content software with RRDtool Tobias Oetiker Motivation About RRDtool Programming Motivation Summary About RRDtool Programming Summary Write your own monitoring Summary software with RRDtool Tobias Oetiker Motivation I carefully design your About RRDtool RRD structure Programming Summary I use RPN math to beat your data into form I write your own monitors, RRDtool does the boring stuff I look good More information on . . . http://www.rrdtool.org