Week 7 " Reuse Your Own Code Top 20 Tools of All Time CS 212 – Spring 2008 (

Software Tools ! Use the language that best fits your task

! Think small " Write little programs that test various concepts " Test them! " Comment them! " Build collections of these little programs Week 7 " Reuse your own code Top 20 Tools of All Time CS 212 – Spring 2008 (http://uk.gizmodo.com/)

Languages for Different Domains Scripting Languages

! General purpose ! Concurrent/distributed ! A script is a sequence of ! Example scripting languages: " Examples: Lisp, Algol, PL/1, processes common commands made into a Unix shell, Python, Perl, Scheme, Java, Python " Control of multiple threads single program Tcl (Tool command language) ! Systems programming " Examples: Ada, Oz, Smalltalk, " Unix uses shell scripts " Emphasis on efficiency and Java " The shell is the interactive ! Some Python code: tight control of data ! Educational interface to Unix structures " " Examples: Basic, Haskell, You can combine commands class Stack (object): " Examples: C, C++, Forth, Pascal, Python, Scheme, from the Unix shell to create def __init__ (self): Modula-2 Smalltalk programs self.stack = [ ] ! Scripting ! Various other domains def put (self, item): self.stack.append(item) " ! Examples: Unix shell, Perl, " Discrete event simulation: A scripting language is usually def get (self): Python, Ruby, Tcl Simula " Easy to learn return self.stack.pop() " Web scripting: Javascript " Interpreted instead of def isEmpty (self): " Realtime applications: Ada compiled return len(self.stack) == 0 " Text processing: Snobol, Perl " Printing: Postscript " …

A Programming Language Controversy Programming Language Weirdness

! “Go To Statement Considered ! Weird languages Harmful” " Whitespace " Edsger Dijkstra, Communications # Only spaces, tabs, and newlines are significant of the ACM (March 1968) # A great language for security since a program can be printed onto plain paper and stored without worrying about an adversary reading the code ☺ ! Sparked long-running discussion on " var'aq whether “go to” is necessary or # Based on the grammatical structure of the Klingon language desirable ! Led to concept of structured " Proponents of “go to” presented programming examples where code was more " Idea: Code is clearer if we ! Weird concepts readable using “go to” restrict ourselves to just a " Polyglot code " At the time few control structures # Code that is valid for multiple languages # No break " Loops have single entry, single # Usually takes advantage of the different ways that comments are indicated # No continue in the different languages # No exceptions exit " Quine # A program whose only output is its own source code # Not considered valid to use the empty program

1 Integrated Development Environments Unix

! An IDE usually includes ! You should know how to use ! Original version by Ken Thompson ! Philosophy (Bell Labs) in 1969 " " Source code editor (usually a debugger! Almost everything is a text file " Little programs (utilities) to do with color highlighting) " Place breakpoints ! An interactive, multi-user little tasks " Compiler or interpreter " Step through code operating system (not the first " Connect programs with pipes & redirection " Tools for “build # Step over such system, but an early one) # % who | sort | lpr automation” (i.e., keeps # Step into # Print an alphabetical list of who is track of what needs to be # Step out of… ! Unix is closely tied to the active on the system ! Linux is an open software version recompiled) " Examine current call-stack development of C of Unix " Debugger " Unix was originally written in PDP- " Examine values of active 7 Assembly Language " Since 1991 " Class browser (for variables " Then in B # Linus Torvalds (the kernel) languages with classes) # Some debuggers allow you " Then in C # Richard Stallman (GNU) " to change a variable value " B and C were basically created to Widely used for high-performance write Unix computing ! Examples: DrJava, Eclipse " In Eclipse: As you type, ! Debuggers are usually much ! Mac OS X is built on Unix gives you list of options + more effective than placing documentation print-statements

Regular Expressions Makefiles

! Common goal: search/match/do ! Some of the rules for regular ! Used when ! Once you have a makefile stuff with strings expressions compiling/recompiling a large " You recompile whatever is " A regular character matches system (many interdependent necessary by typing make ! Idea: use special strings to itself files) match other strings " A . matches any character " Checks which files have ! To create a makefile " * implies 0 or more changed and only recompiles " Some characters are meta- " Common strategy is to find occurrences (of preceding those that are necessary characters some examples and modify item) " Because of dependencies, them " more than just the changed + implies 1 or more " There are automated tools for ! Regular expressions are closely files can need to be occurrences building makefiles related to finite state " \ implies following character is recompiled automata (CS 381/481) treated as a regular character " Also keeps track of compiler ! Modern IDEs often provide " [ … ] matches any one options character from within the ! Why not recompile everything? tools for managing the build process brackets; - can be used to " Expensive indicate a range " Order of compilation can be ! A regular expression in Java important "((\\.[0-9]+)|([0-9]+\\.[0-9]*))"

Memory Management Garbage Collection

! Modern programs are ! Manual memory management ! Want to keep any object ! Once “not-in-use” objects " Long running bugs that can be reached from are found " Make dynamic use of " Dangling pointers program’s variables " Can reclaim the memory memory # Memory has been freed, " Either directly or through for re-use but part of the code is other objects that can be " Can also compact memory still trying to use it reached ! Garbage collector # I.e., move all the “in-use” " Memory leaks " Program’s variables = objects to another " Some languages (e.g., Java, # Memory that is no longer anything in the call stack memory block (without C#) use a garbage used, but is not freed gaps between objects) collector to reclaim unused # Long running program ⇒ memory run out of memory " Other languages (e.g., C, C++) require programmers to manage their own ! There are tools to help memory catch such bugs " E.g., purify for C, C++

2 Garbage Collector Schemes Use of Standard Data Structures

! Mark and Sweep ! For either scheme ! Packages for widely-useful ! For example, Java provides " Mark every object as “not-in- " Can “stop the world” data structures " Interfaces use” " Can interleave (i.e., take turns) " Java Collections # List, Map, Set " Starting from the call stack, " Can run concurrently Framework " visit every reachable object, Classes marking it as “in-use” " C++ STL (Standard # ArrayList, LinkedList, ! Java’s current garbage " Everything still marked “not- Template Library) HashMap, TreeMap, in-use” can be reclaimed collector HashSet, TreeSet " A 2-tier scheme (old " " Provide tools for Algorithms generation; new generation) # Arrays.sort, ! Reference Counting # Sorting & searching " A mark-and-sweep method Arrays.search,… " Every object keeps a count of # Iteration " With compaction how many pointers reference # List it # Set " When count is zero, memory ! Java’s garbage collection # Map (or dictionary) can be reclaimed scheme has changed as new # Stack " Problem: cycles! Java versions were released # Queue # Priority Queue

Version Control Profiling Tools

! Allows you to keep track of ! CVS (Concurrent Version ! People are notoriously bad at predicting the most changes for a large project System) computationally expensive parts of a program " Can back up to old version " Open source " Rule of thumb (Pareto Principle): 80% of the time is spent in if changes create problems " Widely used tool for 20% of the code " Multiple contributors can version control " No use improving the code that isn’t executed often work on the system " Maintains a history of all " How do you determine where your program is spending its time? changes made ! Part of the data produced by a profiler (Python) " Supports branching, ncalls tottime percall cumtime percall filename:lineno(function) allowing several lines of 2521 0.227 0.000 1.734 0.001 Drawing.py:102(update) 7333 0.355 0.000 0.983 0.000 Drawing.py:244(transform) development 4347 0.324 0.000 4.176 0.001 Drawing.py:64(draw) " Provides mechanisms for 3649 0.212 0.000 1.570 0.000 Geometry.py:106(angles) merging branches back 56 0.001 0.000 0.001 0.000 Geometry.py:16(__init__) 343160 9.818 0.000 12.759 0.000 Geometry.py:162(_determinant) together when desired 8579 0.816 0.000 13.928 0.002 Geometry.py:171(cross) ! SVN (Subversion) 4279 0.132 0.000 0.447 0.000 Geometry.py:184(transpose) " An alternative to CVS ! Java has a built-in profiler (hprof); there are many others

More Advanced Profiling Documentation Generators

! Need additional profiling ! Example: ! Comments (esp. specifications) are as important as tools for applications that VTune Performance the code itself " Are multithreaded Analyzer (from Intel) " Determine successful use of code " Use multiple cores " Can monitor # Memory usage " Determine whether code can be maintained # Performance during file " Creation/maintenance = 1/10 I/O ! Documentation belongs in code (or as close to it as # Thread overhead and synchronization possible) # Load balancing " “Code evolves, documentation drifts away” # Idle time " Put specs in comments next to code when possible # Communication bottlenecks " Need to document a complicated method? # Write a paragraph at the top # Or break method into smaller, clearer pieces

3 Example Documentation Generator: Javadoc How Javadoc is Produced

indicates Javadoc comment Java source code javadoc Linked HTML web /** (many files) pages * Constructs an empty HashMap with the specified initial * capacity and the default load factor (0.75). * Javadoc keywords ! An important Java * @param initialCapacity the initial capacity. * @throws IllegalArgumentException if the initial capacity is negative. documentation tool */ public HashMap(int initialCapacity) { this(initialCapacity, DEFAULT_LOAD_FACTOR); ! Extracts documentation from } classes, interfaces can include HTML " Requires properly formatted /** * Constructs an empty HashMap with the default initial capacity comments * (16) and the default load factor (0.75). */ ! Produces browse-able, public HashMap() { this.loadFactor = DEFAULT_LOAD_FACTOR; hyperlinked HTML web pages threshold = (int)(DEFAULT_INITIAL_CAPACITY * DEFAULT_LOAD_FACTOR); table = new Entry[DEFAULT_INITIAL_CAPACITY]; init(); }

A List of Software Tools Some Useful Javadoc Tags (from Wikipedia)

@return description ! Revision control: Bazaar, Bitkeeper, ! Parser generators: Lex, Yacc, Parsec Bonsai, ClearCase, CVS, Git, GNU arch, ! Bug Databases: gnats, Bugzilla, Trac, " Use to describe the return value of the method, if any Mercurial, Monotone, PVCS, RCS, SCM, Atlassian Jira, LibreSource SCCS, SourceSafe, SVN, LibreSource ! Debuggers: gdb, GNU Binutils, valgrind " E.g., @return the sum of the two intervals Synchronizer ! Memory Leaks/Corruptions Detection: ! @param parameter-name description Interface generators: Swig dmalloc, Electric Fence, duma, Insure++, ! Build Tools: Make, automake, Apache Purify, Aard " Describes the parameters of the method Ant, SCons, Rake, Flowtracer ! Code coverage: GCT, CCover ! " E.g., @param i the other interval Compilation and linking tools: GNU ! Source-Code Clones/Duplications toolchain, gcc, Microsoft Visual Studio, Finding: CCFinder CodeWarrior, Xcode, ICC @author name ! Refactoring Browser (e.g., Eclipse) ! Static code analysis: lint, Splint ! Code Sharing Sites: Freshmeat, Krugle, ! @deprecated reason Search: grep, find Sourceforge, ByteMyCode, UCodit ! Text editors: emacs, vi ! Source code generation tools @see package.class#member ! Scripting languages: Awk, Perl, Python, ! Documentation generators: Doxygen, REXX, Ruby, Shell, Tcl {@code expression} help2man, POD, Javadoc, Pydoc/Epydoc " Puts expression in code font

! No hammer? No screw or screwdriver? ! Why the rifle and not the cannon? Why the watch and not the clock? ! No electricity?