Teach Yourself Perl 5 in 21 Days

Total Page:16

File Type:pdf, Size:1020Kb

Teach Yourself Perl 5 in 21 Days Teach Yourself Perl 5 in 21 days David Till Table of Contents: Introduction ● Who Should Read This Book? ● Special Features of This Book ● Programming Examples ● End-of-Day Q& A and Workshop ● Conventions Used in This Book ● What You'll Learn in 21 Days Week 1 Week at a Glance ● Where You're Going Day 1 Getting Started ● What Is Perl? ● How Do I Find Perl? ❍ Where Do I Get Perl? ❍ Other Places to Get Perl ● A Sample Perl Program ● Running a Perl Program ❍ If Something Goes Wrong ● The First Line of Your Perl Program: How Comments Work ❍ Comments ● Line 2: Statements, Tokens, and <STDIN> ❍ Statements and Tokens ❍ Tokens and White Space ❍ What the Tokens Do: Reading from Standard Input ● Line 3: Writing to Standard Output ❍ Function Invocations and Arguments ● Error Messages ● Interpretive Languages Versus Compiled Languages ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 2 Basic Operators and Control Flow ● Storing in Scalar Variables Assignment ❍ The Definition of a Scalar Variable ❍ Scalar Variable Syntax ❍ Assigning a Value to a Scalar Variable ● Performing Arithmetic ❍ Example of Miles-to-Kilometers Conversion ❍ The chop Library Function ● Expressions ❍ Assignments and Expressions ● Other Perl Operators ● Introduction to Conditional Statements ● The if Statement ❍ The Conditional Expression ❍ The Statement Block ❍ Testing for Equality Using == ❍ Other Comparison Operators ● Two-Way Branching Using if and else ● Multi-Way Branching Using elsif ● Writing Loops Using the while Statement ● Nesting Conditional Statements ● Looping Using the until Statement ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 3 Understanding Scalar Values ● What Is a Scalar Value? ● Integer Scalar Values ❍ Integer Scalar Value Limitations ● Floating-Point Scalar Values ❍ Floating-Point Arithmetic and Round-Off Error ● Using Octal and Hexadecimal Notation ❍ Decimal Notation ❍ Octal Notation ❍ Hexadecimal Notation ❍ Why Bother? ● Character Strings ❍ Using Double-Quoted Strings ❍ Escape Sequences ❍ Single-Quoted Strings ● Interchangeability of Strings and Numeric Values ❍ Initial Values of Scalar Variables ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 4 More Operators ● Using the Arithmetic Operators ❍ Exponentiation ❍ The Remainder Operator ❍ Unary Negation ● Using Comparison Operators ❍ Integer-Comparison Operators ❍ String-Comparison Operators ❍ String Comparison Versus Integer Comparison ❍ Comparison and Floating-Point Numbers ● Using Logical Operators ❍ Evaluation Within Logical Operators ❍ Logical Operators as Subexpressions ● Using Bit-Manipulation Operators ❍ What Bits Are and How They Are Used ❍ The Bit-Manipulation Operators ● Using the Assignment Operators ❍ Assignment Operators as Subexpressions ● Using Autoincrement and Autodecrement ❍ The Autoincrement Operator Pre-Increment ❍ The Autoincrement Operator Post-Increment ❍ The Autodecrement Operator ❍ Using Autoincrement With Strings ● The String Concatenation and Repetition Operators ❍ The String-Concatenation Operator ❍ The String-Repetition Operator ❍ Concatenation and Assignment ● Other Perl Operators ❍ The Comma Operator ❍ The Conditional Operator ● The Order of Operations ❍ Precedence ❍ Associativity ❍ Forcing Precedence Using Parentheses ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 5 Lists and Array Variables ● Introducing Lists ● Scalar Variables and Lists ❍ Lists and String Substitution ● Storing Lists in Array Variables ● Accessing an Element of an Array Variable ❍ More Details on Array Element Names ● Using Lists and Arrays in Perl Programs ❍ Using Brackets and Substituting for Variables ● Using List Ranges ❍ Expressions and List Ranges ● More on Assignment and Array Variables ❍ Copying from One Array Variable to Another ❍ Using Array Variables in Lists ❍ Substituting for Array Variables in Strings ❍ Assigning to Scalar Variables from Array Variables ● Retrieving the Length of a List ● Using Array Slices ❍ Using List Ranges in Array-Slice Subscripts ❍ Using Variables in Array-Slice Subscripts ❍ Assigning to Array Slices ❍ Overlapping Array Slices ❍ Using the Array-Slice Notation as a Shorthand ● Reading an Array from the Standard Input File ● Array Library Functions ❍ Sorting a List or Array Variable ❍ Reversing a List or Array Variable ❍ Using chop on Array Variables ❍ Creating a Single String from a List ❍ Splitting a String into a List ❍ Other List-Manipulation Functions ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 6 Reading from and Writing to Files ● Opening a File ❍ The File Variable ❍ The Filename ❍ The File Mode ❍ Checking Whether the Open Succeeded ● Reading from a File ❍ File Variables and the Standard Input File ❍ Terminating a Program Using die ❍ Reading into Array Variables ● Writing to a File ❍ The Standard Output File Variable ❍ Merging Two Files into One ● Redirecting Standard Input and Standard Output ● The Standard Error File ● Closing a File ● Determining the Status of a File ❍ File-Test Operator Syntax ❍ Available File-Test Operators ❍ More on the -e Operator ❍ Testing for Read Permission-the -r Operator ❍ Checking for Other Permissions ❍ Checking for Empty Files ❍ Using File-Test Operators with File Variables ● Reading from a Sequence of Files ❍ Reading into an Array Variable ● Using Command-Line Arguments as Values ❍ ARGV and the <> Operator ● Opening Pipes ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 7 Pattern Matching ● Introduction ● The Match Operators ❍ Match-Operator Precedence ● Special Characters in Patterns ❍ The + Character ❍ The [] Special Characters ❍ The * and ? Special Characters ❍ Escape Sequences for Special Characters ❍ Matching Any Letter or Number ❍ Anchoring Patterns ❍ Variable Substitution in Patterns ❍ Excluding Alternatives ❍ Character-Range Escape Sequences ❍ Matching Any Character ❍ Matching a Specified Number of Occurrences ❍ Specifying Choices ❍ Reusing Portions of Patterns ❍ Pattern-Sequence Scalar Variables ❍ Special-Character Precedence ❍ Specifying a Different Pattern Delimiter ● Pattern-Matching Options ❍ Matching All Possible Patterns ❍ Ignoring Case ❍ Treating the String as Multiple Lines ❍ Evaluating a Pattern Only Once ❍ Treating the String as a Single Line ❍ Using White Space in Patterns ● The Substitution Operator ❍ Using Pattern-Sequence Variables in Substitutions ❍ Options for the Substitution Operator ❍ Evaluating a Pattern Only Once ❍ Treating the String as Single or Multiple Lines ❍ Using White Space in Patterns ❍ Specifying a Different Delimiter ● The Translation Operator ❍ Options for the Translation Operator ● Extended Pattern-Matching ❍ Parenthesizing Without Saving in Memory ❍ Embedding Pattern Options ❍ Positive and Negative Look-Ahead ❍ Pattern Comments ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Week 1 Week 1 in Review Week 2 Week 2 at a Glance ● Where You're Going Day 8 More Control Structures ● Using Single-Line Conditional Statements ❍ Problems with Single-Line Conditional Statements ● Looping Using the for Statement ❍ Using the Comma Operator in a for Statement ● Looping Through a List: The foreach Statement ❍ The foreach Local Variable ❍ Changing the Value of the Local Variable ❍ Using Returned Lists in the foreach Statement ● The do Statement ● Exiting a Loop Using the last Statement ● Using next to Start the Next Iteration of a Loop ● The redo Statement ● Using Labeled Blocks for Multilevel Jumps ❍ Using next and redo with Labels ● The continue Block ● The goto Statement ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 9 Using Subroutines ● What Is a Subroutine? ● Defining and Invoking a Subroutine ❍ Forward References to Subroutines ● Returning a Value from a Subroutine ❍ Return Values and Conditional Expressions ● The return Statement ● Using Local Variables in Subroutines ❍ Initializing Local Variables ● Passing Values to a Subroutine ❍ Passing a List to a Subroutine ● Calling Subroutines from Other Subroutines ● Recursive Subroutines ● Passing Arrays by Name Using Aliases ● Using the do Statement with Subroutines ● Specifying the Sort Order ● Predefined Subroutines ❍ Creating Startup Code Using BEGIN ❍ Creating Termination Code Using END ❍ Handling Non-Existent Subroutines Using AUTOLOAD ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 10 Associative Arrays ● Limitations of Array Variables ● Definition ● Referring to Associative Array Elements ● Adding Elements to an Associative Array ● Creating Associative Arrays ● Copying Associative Arrays from Array Variables ● Adding and Deleting Array Elements ● Listing Array Indexes and Values ● Looping Using an Associative Array ● Creating Data Structures Using Associative Arrays ❍ Linked Lists ❍ Structures ❍ Trees ❍ Databases ❍ Example: A Calculator Program ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 11 Formatting Your Output ● Defining a Print Format ● Displaying a Print Format ● Displaying Values in a Print Format ❍ Creating a General-Purpose Print Format ❍ Choosing a Value-Field Format ❍ Printing Value-Field Characters ❍ Using the Multiline Field Format ● Writing to Other Output Files ❍ Saving the Default File Variable ● Specifying a Page Header ❍ Changing the Header Print Format ● Setting the Page Length ❍ Using print with Pagination ● Formatting Long Character Strings ❍ Eliminating Blank Lines When Formatting ❍ Supplying an Indefinite Number of Lines ● Formatting Output Using printf ● Summary ● Q&A ● Workshop ❍ Quiz ❍ Exercises Day 12 Working with the File System ● File Input and Output Functions ❍ Basic Input and Output Functions ❍ Skipping and Rereading Data ❍ System Read and Write Functions ❍ Reading Characters

  1090
Recommended publications
  • PHP Programming Cookbook I
    PHP Programming Cookbook i PHP Programming Cookbook PHP Programming Cookbook ii Contents 1 PHP Tutorial for Beginners 1 1.1 Introduction......................................................1 1.1.1 Where is PHP used?.............................................1 1.1.2 Why PHP?..................................................2 1.2 XAMPP Setup....................................................3 1.3 PHP Language Basics.................................................5 1.3.1 Escaping to PHP...............................................5 1.3.2 Commenting PHP..............................................5 1.3.3 Hello World..................................................6 1.3.4 Variables in PHP...............................................6 1.3.5 Conditional Statements in PHP........................................7 1.3.6 Loops in PHP.................................................8 1.4 PHP Arrays...................................................... 10 1.5 PHP Functions.................................................... 12 1.6 Connecting to a Database............................................... 14 1.6.1 Connecting to MySQL Databases...................................... 14 1.6.2 Connecting to MySQLi Databases (Procedurial).............................. 14 1.6.3 Connecting to MySQLi databases (Object-Oriented)............................ 15 1.6.4 Connecting to PDO Databases........................................ 15 1.7 PHP Form Handling................................................. 15 1.8 PHP Include & Require Statements.........................................
    [Show full text]
  • Using the GNU Compiler Collection (GCC)
    Using the GNU Compiler Collection (GCC) Using the GNU Compiler Collection by Richard M. Stallman and the GCC Developer Community Last updated 23 May 2004 for GCC 3.4.6 For GCC Version 3.4.6 Published by: GNU Press Website: www.gnupress.org a division of the General: [email protected] Free Software Foundation Orders: [email protected] 59 Temple Place Suite 330 Tel 617-542-5942 Boston, MA 02111-1307 USA Fax 617-542-2652 Last printed October 2003 for GCC 3.3.1. Printed copies are available for $45 each. Copyright c 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being \GNU General Public License" and \Funding Free Software", the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below). A copy of the license is included in the section entitled \GNU Free Documentation License". (a) The FSF's Front-Cover Text is: A GNU Manual (b) The FSF's Back-Cover Text is: You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development. i Short Contents Introduction ...................................... 1 1 Programming Languages Supported by GCC ............ 3 2 Language Standards Supported by GCC ............... 5 3 GCC Command Options .........................
    [Show full text]
  • Object-Oriented Programming Basics with Java
    Object-Oriented Programming Object-Oriented Programming Basics With Java In his keynote address to the 11th World Computer Congress in 1989, renowned computer scientist Donald Knuth said that one of the most important lessons he had learned from his years of experience is that software is hard to write! Computer scientists have struggled for decades to design new languages and techniques for writing software. Unfortunately, experience has shown that writing large systems is virtually impossible. Small programs seem to be no problem, but scaling to large systems with large programming teams can result in $100M projects that never work and are thrown out. The only solution seems to lie in writing small software units that communicate via well-defined interfaces and protocols like computer chips. The units must be small enough that one developer can understand them entirely and, perhaps most importantly, the units must be protected from interference by other units so that programmers can code the units in isolation. The object-oriented paradigm fits these guidelines as designers represent complete concepts or real world entities as objects with approved interfaces for use by other objects. Like the outer membrane of a biological cell, the interface hides the internal implementation of the object, thus, isolating the code from interference by other objects. For many tasks, object-oriented programming has proven to be a very successful paradigm. Interestingly, the first object-oriented language (called Simula, which had even more features than C++) was designed in the 1960's, but object-oriented programming has only come into fashion in the 1990's.
    [Show full text]
  • The Machine That Builds Itself: How the Strengths of Lisp Family
    Khomtchouk et al. OPINION NOTE The Machine that Builds Itself: How the Strengths of Lisp Family Languages Facilitate Building Complex and Flexible Bioinformatic Models Bohdan B. Khomtchouk1*, Edmund Weitz2 and Claes Wahlestedt1 *Correspondence: [email protected] Abstract 1Center for Therapeutic Innovation and Department of We address the need for expanding the presence of the Lisp family of Psychiatry and Behavioral programming languages in bioinformatics and computational biology research. Sciences, University of Miami Languages of this family, like Common Lisp, Scheme, or Clojure, facilitate the Miller School of Medicine, 1120 NW 14th ST, Miami, FL, USA creation of powerful and flexible software models that are required for complex 33136 and rapidly evolving domains like biology. We will point out several important key Full list of author information is features that distinguish languages of the Lisp family from other programming available at the end of the article languages and we will explain how these features can aid researchers in becoming more productive and creating better code. We will also show how these features make these languages ideal tools for artificial intelligence and machine learning applications. We will specifically stress the advantages of domain-specific languages (DSL): languages which are specialized to a particular area and thus not only facilitate easier research problem formulation, but also aid in the establishment of standards and best programming practices as applied to the specific research field at hand. DSLs are particularly easy to build in Common Lisp, the most comprehensive Lisp dialect, which is commonly referred to as the “programmable programming language.” We are convinced that Lisp grants programmers unprecedented power to build increasingly sophisticated artificial intelligence systems that may ultimately transform machine learning and AI research in bioinformatics and computational biology.
    [Show full text]
  • Cygwin User's Guide
    Cygwin User’s Guide Cygwin User’s Guide ii Copyright © Cygwin authors Permission is granted to make and distribute verbatim copies of this documentation provided the copyright notice and this per- mission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this documentation under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this documentation into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Cygwin User’s Guide iii Contents 1 Cygwin Overview 1 1.1 What is it? . .1 1.2 Quick Start Guide for those more experienced with Windows . .1 1.3 Quick Start Guide for those more experienced with UNIX . .1 1.4 Are the Cygwin tools free software? . .2 1.5 A brief history of the Cygwin project . .2 1.6 Highlights of Cygwin Functionality . .3 1.6.1 Introduction . .3 1.6.2 Permissions and Security . .3 1.6.3 File Access . .3 1.6.4 Text Mode vs. Binary Mode . .4 1.6.5 ANSI C Library . .4 1.6.6 Process Creation . .5 1.6.6.1 Problems with process creation . .5 1.6.7 Signals . .6 1.6.8 Sockets . .6 1.6.9 Select . .7 1.7 What’s new and what changed in Cygwin . .7 1.7.1 What’s new and what changed in 3.2 .
    [Show full text]
  • A First Course to Openfoam
    Basic Shell Scripting Slides from Wei Feinstein HPC User Services LSU HPC & LON [email protected] September 2018 Outline • Introduction to Linux Shell • Shell Scripting Basics • Variables/Special Characters • Arithmetic Operations • Arrays • Beyond Basic Shell Scripting – Flow Control – Functions • Advanced Text Processing Commands (grep, sed, awk) Basic Shell Scripting 2 Linux System Architecture Basic Shell Scripting 3 Linux Shell What is a Shell ▪ An application running on top of the kernel and provides a command line interface to the system ▪ Process user’s commands, gather input from user and execute programs ▪ Types of shell with varied features o sh o csh o ksh o bash o tcsh Basic Shell Scripting 4 Shell Comparison Software sh csh ksh bash tcsh Programming language y y y y y Shell variables y y y y y Command alias n y y y y Command history n y y y y Filename autocompletion n y* y* y y Command line editing n n y* y y Job control n y y y y *: not by default http://www.cis.rit.edu/class/simg211/unixintro/Shell.html Basic Shell Scripting 5 What can you do with a shell? ▪ Check the current shell ▪ echo $SHELL ▪ List available shells on the system ▪ cat /etc/shells ▪ Change to another shell ▪ csh ▪ Date ▪ date ▪ wget: get online files ▪ wget https://ftp.gnu.org/gnu/gcc/gcc-7.1.0/gcc-7.1.0.tar.gz ▪ Compile and run applications ▪ gcc hello.c –o hello ▪ ./hello ▪ What we need to learn today? o Automation of an entire script of commands! o Use the shell script to run jobs – Write job scripts Basic Shell Scripting 6 Shell Scripting ▪ Script: a program written for a software environment to automate execution of tasks ▪ A series of shell commands put together in a file ▪ When the script is executed, those commands will be executed one line at a time automatically ▪ Shell script is interpreted, not compiled.
    [Show full text]
  • Automating Your Sync Testing
    APPLICATION NOTE By automating system verification and conformance testing to ITU-T synchronization standards, you’ll save on time and resources, and avoid potential test execution errors. This application note describes how you can use the Paragon-X’s Script Recorder to easily record Tcl, PERL and Python commands that can be integrated into your own test scripts for fast and efficient automated testing. AUTOMATING YOUR SYNC TESTING calnexsol.com Easily automate synchronization testing using the Paragon-X Fast and easy automation by Supports the key test languages Pre-prepared G.8262 Conformance recording GUI key presses Tcl, PERL and Python Scripts reduces test execution errors <Tcl> <PERL> <python> If you perform System Verification language you want to record i.e. Tcl, PERL SyncE CONFORMANCE TEST and Conformance Testing to ITU-T or Python, then select Start. synchronization standards on a regular Calnex provides Conformance Test Scripts basis, you’ll know that manual operation to ITU-T G.8262 for SyncE conformance of these tests can be time consuming, testing using the Paragon-X. These tedious and prone to operator error — as test scripts can also be easily tailored well as tying up much needed resources. and edited to meet your exact test Automation is the answer but very often requirements. This provides an easy means a lack of time and resource means it of getting your test automation up and remains on the ‘To do’ list. Now, with running and providing a repeatable means Calnex’s new Script Recorder feature, you of proving performance, primarily for ITU-T can get your automation up and running standards conformance.
    [Show full text]
  • Assignment 4 Computer Science 235 Due Start of Class, October 4, 2005
    Assignment 4 Computer Science 235 Due Start of Class, October 4, 2005 This assignment is intended as an introduction to OCAML programming. As always start early and stop by if you have any questions. Exercise 4.1. Twenty higher-order OCaml functions are listed below. For each function, write down the type that would be automatically reconstructed for it. For example, consider the following OCaml length function: let length xs = match xs with [] -> 0 | (_::xs') = 1 + (length xs') The type of this OCaml function is: 'a list -> int Note: You can check your answers by typing them into the OCaml interpreter. But please write down the answers first before you check them --- otherwise you will not learn anything! let id x = x let compose f g x = (f (g x)) let rec repeated n f = if (n = 0) then id else compose f (repeated (n - 1) f) let uncurry f (a,b) = (f a b) let curry f a b = f(a,b) let churchPair x y f = f x y let rec generate seed next isDone = if (isDone seed) then [] else seed :: (generate (next seed) next isDone) let rec map f xs = match xs with [] -> [] Assignment 4 Page 2 Computer Science 235 | (x::xs') -> (f x) :: (map f xs') let rec filter pred xs = match xs with [] -> [] | (x::xs') -> if (pred x) then x::(filter pred xs') else filter pred xs' let product fs xs = map (fun f -> map (fun x -> (f x)) xs) fs let rec zip pair = match pair with ([], _) -> [] | (_, []) -> [] | (x::xs', y::ys') -> (x,y)::(zip(xs',ys')) let rec unzip xys = match xys with [] -> ([], []) | ((x,y)::xys') -> let (xs,ys) = unzip xys' in (x::xs, y::ys) let rec
    [Show full text]
  • Learning Perl Through Examples Part 2 L1110@BUMC 2/22/2017
    www.perl.org Learning Perl Through Examples Part 2 L1110@BUMC 2/22/2017 Yun Shen, Programmer Analyst [email protected] IS&T Research Computing Services Spring 2017 Tutorial Resource Before we start, please take a note - all the codes and www.perl.org supporting documents are accessible through: • http://rcs.bu.edu/examples/perl/tutorials/ Yun Shen, Programmer Analyst [email protected] IS&T Research Computing Services Spring 2017 Sign In Sheet We prepared sign-in sheet for each one to sign www.perl.org We do this for internal management and quality control So please SIGN IN if you haven’t done so Yun Shen, Programmer Analyst [email protected] IS&T Research Computing Services Spring 2017 Evaluation One last piece of information before we start: www.perl.org • DON’T FORGET TO GO TO: • http://rcs.bu.edu/survey/tutorial_evaluation.html Leave your feedback for this tutorial (both good and bad as long as it is honest are welcome. Thank you) Yun Shen, Programmer Analyst [email protected] IS&T Research Computing Services Spring 2017 Today’s Topic • Basics on creating your code www.perl.org • About Today’s Example • Learn Through Example 1 – fanconi_example_io.pl • Learn Through Example 2 – fanconi_example_str_process.pl • Learn Through Example 3 – fanconi_example_gene_anno.pl • Extra Examples (if time permit) Yun Shen, Programmer Analyst [email protected] IS&T Research Computing Services Spring 2017 www.perl.org Basics on creating your code How to combine specs, tools, modules and knowledge. Yun Shen, Programmer Analyst [email protected] IS&T Research Computing
    [Show full text]
  • Building Performance Measurement Tools for the MINIX 3 Operating System
    Building Performance Measurement Tools for the MINIX 3 Operating System Rogier Meurs August 2006 Contents 1 INTRODUCTION 1 1.1 Measuring Performance 1 1.2 MINIX 3 2 2 STATISTICAL PROFILING 3 2.1 Introduction 3 2.2 In Search of a Timer 3 2.2.1 i8259 Timers 3 2.2.2 CMOS Real-Time Clock 3 2.3 High-level Description 4 2.4 Work Done in User-Space 5 2.4.1 The SPROFILE System Call 5 2.5 Work Done in Kernel-Space 5 2.5.1 The SPROF Kernel Call 5 2.5.2 Profiling using the CMOS Timer Interrupt 6 2.6 Work Done at the Application Level 7 2.6.1 Control Tool: profile 7 2.6.2 Analyzing Tool: sprofalyze.pl 7 2.7 What Can and What Cannot be Profiled 8 2.8 Profiling Results 8 2.8.1 High Scoring IPC Functions 8 2.8.2 Interrupt Delay 9 2.8.3 Profiling Runs on Simulator and Other CPU Models 12 2.9 Side-effect of Using the CMOS Clock 12 3 CALL PROFILING 13 3.1 Introduction 13 3.1.1 Compiler-supported Call Profiling 13 3.1.2 Call Paths, Call and Cycle Attribution 13 3.2 High-level Description 14 3.3 Work Done in User-Space 15 3.3.1 The CPROFILE System Call 15 3.4 Work Done in Kernel-Space 16 3.4.1 The PROFBUF and CPROF Kernel Calls 16 3.5 Work Done in Libraries 17 3.5.1 Profiling Using Library Functions 17 3.5.2 The Procentry Library Function 17 3.5.3 The Procexit Library Function 20 3.5.4 The Call Path String 22 3.5.5 Testing Overhead Elimination 23 3.6 Profiling Kernel-Space/User-Space Processes 24 3.6.1 Differences in Announcing and Table Sizes 24 3.6.2 Kernel-Space Issue: Reentrancy 26 3.6.3 Kernel-Space Issue: The Call Path 26 3.7 Work Done at the Application
    [Show full text]
  • Memory Tagging and How It Improves C/C++ Memory Safety Kostya Serebryany, Evgenii Stepanov, Aleksey Shlyapnikov, Vlad Tsyrklevich, Dmitry Vyukov Google February 2018
    Memory Tagging and how it improves C/C++ memory safety Kostya Serebryany, Evgenii Stepanov, Aleksey Shlyapnikov, Vlad Tsyrklevich, Dmitry Vyukov Google February 2018 Introduction 2 Memory Safety in C/C++ 2 AddressSanitizer 2 Memory Tagging 3 SPARC ADI 4 AArch64 HWASAN 4 Compiler And Run-time Support 5 Overhead 5 RAM 5 CPU 6 Code Size 8 Usage Modes 8 Testing 9 Always-on Bug Detection In Production 9 Sampling In Production 10 Security Hardening 11 Strengths 11 Weaknesses 12 Legacy Code 12 Kernel 12 Uninitialized Memory 13 Possible Improvements 13 Precision Of Buffer Overflow Detection 13 Probability Of Bug Detection 14 Conclusion 14 Introduction Memory safety in C and C++ remains largely unresolved. A technique usually called “memory tagging” may dramatically improve the situation if implemented in hardware with reasonable overhead. This paper describes two existing implementations of memory tagging: one is the full hardware implementation in SPARC; the other is a partially hardware-assisted compiler-based tool for AArch64. We describe the basic idea, evaluate the two implementations, and explain how they improve memory safety. This paper is intended to initiate a wider discussion of memory tagging and to motivate the CPU and OS vendors to add support for it in the near future. Memory Safety in C/C++ C and C++ are well known for their performance and flexibility, but perhaps even more for their extreme memory unsafety. This year we are celebrating the 30th anniversary of the Morris Worm, one of the first known exploitations of a memory safety bug, and the problem is still not solved.
    [Show full text]
  • Statically Detecting Likely Buffer Overflow Vulnerabilities
    Statically Detecting Likely Buffer Overflow Vulnerabilities David Larochelle [email protected] University of Virginia, Department of Computer Science David Evans [email protected] University of Virginia, Department of Computer Science Abstract Buffer overflow attacks may be today’s single most important security threat. This paper presents a new approach to mitigating buffer overflow vulnerabilities by detecting likely vulnerabilities through an analysis of the program source code. Our approach exploits information provided in semantic comments and uses lightweight and efficient static analyses. This paper describes an implementation of our approach that extends the LCLint annotation-assisted static checking tool. Our tool is as fast as a compiler and nearly as easy to use. We present experience using our approach to detect buffer overflow vulnerabilities in two security-sensitive programs. 1. Introduction ed a prototype tool that does this by extending LCLint [Evans96]. Our work differs from other work on static detection of buffer overflows in three key ways: (1) we Buffer overflow attacks are an important and persistent exploit semantic comments added to source code to security problem. Buffer overflows account for enable local checking of interprocedural properties; (2) approximately half of all security vulnerabilities we focus on lightweight static checking techniques that [CWPBW00, WFBA00]. Richard Pethia of CERT have good performance and scalability characteristics, identified buffer overflow attacks as the single most im- but sacrifice soundness and completeness; and (3) we portant security problem at a recent software introduce loop heuristics, a simple approach for engineering conference [Pethia00]; Brian Snow of the efficiently analyzing many loops found in typical NSA predicted that buffer overflow attacks would still programs.
    [Show full text]