IN ACTION Understanding data with graphs Philipp K. Janert FOREWORDS BY COLIN D. KELLEY AND THOMAS WILLIAMS MANNING Gnuplot in Action Gnuplot in Action Understanding Data with Graphs PHILIPP K. JANERT MANNING Greenwich (74° w. long.) For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. Sound View Court 3B fax: (609) 877-8256 Greenwick, CT 06830 email: [email protected] ©2010 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Development editor: Nermina Miller, Tom Cirtin Copyeditor: Benjamin Berg Manning Publications Co. Proofreader: Katie Tennant Sound View Court 3B Typesetter: Dottie Marsico Greenwich, CT 06830 Cover designer: Marija Tudor ISBN 978-1-933988-39-9 Printed in the United States of America 1 2 3 4 5 6 7 8 9 10 – MAL – 14 13 12 11 10 09 The purpose of computing is insight, not numbers. —R. W. Hamming The purpose of computing is insight, not pictures. —L. N. Trefethen brief contents PART 1 BASICS ..........................................................................1 1 ■ Prelude: Understanding data with gnuplot 3 2 ■ Essential gnuplot 16 3 ■ Working with data 29 4 ■ Practical matters 49 PART 2 POLISHING .................................................................65 5 ■ Doing it with style 67 6 ■ Decorations 90 7 ■ All about axes 110 PART 3 ADVANCED GNUPLOT ................................................131 8 ■ Three-dimensional plots 133 9 ■ Color 152 10 ■ Advanced plotting concepts 175 11 ■ Terminals in depth 200 12 ■ Macros, scripting, and batch operations 222 vii PART 4 GRAPHICAL ANALYSIS WITH GNUPLOT .......................243 13 ■ Fundamental graphical methods 245 14 ■ Techniques of graphical analysis 273 15 ■ Coda: Understanding data with graphs 301 contents foreword xvii foreword xix preface xxi acknowledgments xxiii about this book xxv PART 1 BASICS .................................................................1 Prelude: Understanding data with gnuplot 3 1 1.1 A busy weekend 4 Planning a marathon 4 ■ Determining the future 6 1.2 What is graphical analysis? 9 Data analysis and visualization concepts 10 ■ Why graphical analysis? 12 ■ Limitations of graphical analysis 12 1.3 What is gnuplot? 13 Gnuplot isn’t GNU 13 ■ Why gnuplot? 14 ■ Limitations 15 1.4 Summary 15 ix x CONTENTS Essential gnuplot 16 2 2.1 Simple plots 16 Invoking gnuplot and first plots 17 ■ Plotting data from a file 20 ■ Abbreviations and defaults 23 2.2 Saving and exporting 24 Saving and loading commands 25 ■ Exporting graphs 25 One-step export script 27 2.3 Summary 28 Working with data 29 3 3.1 Managing large data sets 30 Multiple data sets per file: index 30 ■ Records spanning multiple lines: every 31 3.2 Smoothing and summarizing data 32 Plotting unsorted data files 32 ■ Smoothing noisy data 35 3.3 Math with gnuplot 38 Mathematical expressions 38 ■ Built-in functions 38 User-defined variables and functions 39 ■ Complex numbers 40 3.4 Data transformations 41 Simple data transformations 41 ■ Pseudocolumns and the column function 42 3.5 Plotting functions and data 43 Tricks and warnings 44 3.6 Logarithmic plots 44 How do logarithmic plots work? 44 3.7 Summary 47 Practical matters 49 4 4.1 Managing options 50 4.2 Data files 51 Permissible formats and options 51 4.3 Strings 55 Quotes 55 ■ String operations 55 ■ String applications 57 Crazy example: plotting the Unix password file 58 4.4 Generating textual output 59 The print command 59 ■ The set table option 60 CONTENTS xi 4.5 Interacting with gnuplot 61 Getting help 61 ■ Command history 61 ■ Hot keys and mousing 62 ■ Reading data interactively 63 4.6 Summary 64 PART 2 POLISHING ........................................................65 Doing it with style 67 5 5.1 Choosing plot styles 68 Inline style directives 68 ■ Terminal capabilities 69 Global style directives 70 5.2 Plot styles 70 Core styles 71 ■ Box styles 72 ■ Styles with errorbars or ranges 78 ■ Filled styles 81 ■ Other styles 84 5.3 Customizing styles 85 Custom line styles 86 ■ Specifying color 87 Worked example: half-tone shading 87 5.4 Summary 89 Decorations 90 6 6.1 Quick start: minimal context for data 91 6.2 Digression: locations on a graph 92 6.3 Additional graph elements: decorations 94 Common conventions 94 ■ Arrows 94 ■ Text labels 97 Objects 99 6.4 The graph’s legend or key 100 Turning the key on and off 101 ■ Placement 101 Layout 101 ■ Explanations 102 ■ Appearance 104 Default settings 104 6.5 Worked example: features of a spectrum 104 6.6 Overall appearance 106 Size and aspect ratio 106 ■ Borders and margins 108 6.7 Summary 109 All about axes 110 7 7.1 Multiple axes 111 Terminology 111 ■ Plotting with two coordinate systems 112 Should you do it? 113 xii CONTENTS 7.2 Selecting plot ranges 115 7.3 Tic marks 116 Major tic marks 116 ■ Minor tic marks 117 ■ Formatting the tic labels 118 ■ Reading tic labels from file 122 Grid and zero axes 123 7.4 A worked example 123 7.5 Special case: time series 124 Turning numbers into names: months and weekdays 124 General time series: the gory details 127 7.6 Summary 130 PART 3 ADVANCED GNUPLOT .......................................131 Three-dimensional plots 133 8 8.1 Basics 135 8.2 Options for surface and contour plots 136 Surface plots 136 ■ Contour lines 139 8.3 Coordinate axes and view point 141 Borders 142 ■ View point 143 8.4 Plotting data from a file using splot 145 Grid format 146 ■ Matrix format 148 ■ Smooth surfaces 149 8.5 Summary 151 Color 152 9 9.1 Defining palettes 153 Color spaces: a refresher 153 ■ The palette option 154 9.2 Creating colored graphs with palettes 157 The pm3d mode 157 ■ The colorbox 158 ■ Other ways to use color 160 9.3 Using color for data representation 161 Thoughts on palette design 162 ■ Some sample palettes 165 Words of caution 168 9.4 Case studies 169 A smoothly varying function 169 ■ A complex figure 171 9.5 Summary 173 CONTENTS xiii Advanced plotting concepts 175 10 10.1 Multiplot 176 Regular arrays of graphs with layout 177 ■ Graphs within a graph 179 ■ Graphs aligned on a common axis 181 10.2 Higher math and special occasions 183 Parametric plots 183 ■ Non-Cartesian coordinates 184 Vector fields 188 10.3 Curve fitting 190 Background 190 ■ Using the fit command 191 ■ Worked example 195 ■ Should you do it? 197 10.4 Summary 199 Terminals in depth 200 11 11.1 Exporting graphs to file 201 11.2 Common terminal options 202 Size 202 ■ Fonts 202 ■ Enhanced text mode 202 Miscellaneous appearance options 205 ■ Flushing output channels 205 11.3 Standard graphics file formats 206 Bitmaps 206 ■ SVG 208 11.4 Print-quality output 209 PostScript 209 ■ Using PostScript plots with LaTeX 211 PDF 217 11.5 Interactive terminals 218 wxt 218 ■ x11 219 ■ aqua 219 ■ windows 219 11.6 Other terminals 220 11.7 Summary 221 Macros, scripting, and batch operations 222 12 12.1 Strings and string macros 223 12.2 Calling other programs from gnuplot 224 Executing a command in a subshell 225 ■ Capturing the output of a subprocess 225 ■ Input/output redirection (Unix only) 226 Example: watermarking plots 227 12.3 Calling gnuplot from other programs 228 Batch operations 228 ■ Invoking gnuplot from other programs 229 ■ Example: creating a font table 232 xiv CONTENTS 12.4 Slideshows with pause and reread 232 12.5 Configuring your workspace 234 Creating custom hot key bindings 236 12.6 Gnuplot for the web 239 Using Gnuplot as a CGI script 239 ■ Using gnuplot as a subprocess to a CGI script 241 12.7 Summary 241 PART 4 GRAPHICAL ANALYSIS WITH GNUPLOT ............. 243 Fundamental graphical methods 245 13 13.1 Relationships 246 Scatter plots 246 ■ Logarithmic scales 252 13.2 Counting statistics 256 Jitter plots and histograms 256 ■ Kernel density estimates 258 Cumulative distribution functions 259 ■ Consider using median and percentiles 261 13.3 Ranked data 262 13.4 Multivariate data 264 Parallel coordinate plots 264 ■ Multivariate analysis 269 Star plots 270 ■ Historical perspective: computer-aided data analysis 271 13.5 Summary 272 Techniques of graphical analysis 273 14 14.1 The core principle of graphical analysis 274 14.2 Iteration and transformation 275 A case study in iteration: car data 275 ■ Making data comparable: monitoring quantities in a control chart 278 ■ Honor the data: truncation and responsiveness 280 14.3 Changing the appearance to improve perception 284 Banking 284 ■ Judging lengths and distances 287 Enhancing quantitative perception 289 ■ Plot ranges and the matter of zero 291 ■ A tough problem: the display of changing compositions 292 CONTENTS xv 14.4 Housekeeping 296 The lifecycle of a graph 296 ■ Input data files 296 ■ Output files 298 14.5 Reminders for presentation graphics 298 14.6 Summary 300 15 Coda: Understanding data with graphs 301 appendix A Obtaining, building, and installing gnuplot 303 appendix B Gnuplot reference 309 appendix C Resources 345 index 351 foreword Thomas Williams was a CS undergrad and I was an EE/CS undergrad at Villanova Uni- versity.