CSCI-GA.3033.003 Scripting Languages Anonymous Functions

• CSCI-GA.3033-003 NYU • 6/14/2012 Context, Modules, Glue (Perl)

Anonymous functions

• Function definition can omit name sub [id] [proto] [attrs] [{…}] CSCI-GA.3033.003 Higher-order function: • Example script: has function param or #!/usr/bin/env perl returns function result Scripting Languages use warnings; use strict; sub callFn { my ($fn, $x) = @_; &$fn($x); } 6/14/2012 sub printIt { my ($it) = @_; print "printIt $it\n”; } #pass reference to named subroutine; prints "printIt Hello" callFn(\&printIt, "Hello"); Context and Modules (Perl) #pass reference to anonymous subroutine; prints "lambda Hi" Scripting as Glue callFn(sub { my ($it) = @_; print "lambda $it\n" }, "Hi"); • Useful for call-backs, e.g., sort/compare

Perl Outline Idiomatic Perl

• Regular expressions (continued) • Interpreter specification: #! • Type conversions • Regular expressions extract data • Programming in the large – Captured groups: $1, $2, … • Hashes and arrays store data • Scripting as glue – Nested references build up data structures • Default operand: $_ • Parameter array: @_ • String interpolations report results

Perl Soap-box Example from 5/24/2012, Revisited Readable Perl #!/usr/bin/perl -w %cup2g = ( flour => 110, sugar => 225, butter => 225 ); %volume = ( cup => 1, tbsp => 16, tsp => 48, ml => 236 ); • Use pragmata: warnings, strict, %weight = ( lb => 1, oz => 16, g => 453 ); while (<>) { maybe also diagnostics, sigtrap my ($qty, $unit, $ing) = /([0-9.]+) (\w+) (\w+)/; if ($cup2g{$ing} && $volume{$unit}) { • Avoid default $_ $qty = 1.0 * $qty * $cup2g{$ing} / $volume{$unit}; $unit = 'g'; • Name your subroutine parameters } elsif ($volume{$unit}) { $qty = 1.0 * $qty * $volume{ml} / $volume{$unit}; • Use my, avoid local $unit = 'ml'; } elsif ($weight{$unit}) { • Use /x modifier on regular expressions $qty = 1.0 * $qty * $weight{g} / $weight{$unit}; $unit = 'g'; – Add whitespace, line breaks, comments } printf("%d $unit $ing\n", $qty + .5); • Use modules for large projects } © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 5 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 6

Concepts Perl Inside a Regular Expression Chomsky Hierarchy Kind Construct Syntax Example Pattern Matches Type Languages Automata Rules Essentials Character Itself b b 0 Recursively Turing machine α→β Concatenation e e bc bc enumerable 1 2 Alternative (or) e1|e2 a|bc a, bc 1 Context Linear bounded B α γ→αδγ Repetition (≥0) e* a* , a, aa, aaa sensitive Turing machine Grouping (e) (a|b)c ac, bc 2 Context free Pushdown A→β Quantifier Optional e? (a|b)? , a, b automaton (BNF) Repetition (≥1) e+ a+ a, aa, aaa 3 Regular Finite state A→b, C→dE Repetition e{n} a{3} aaa machine (regexp) Char class Custom class […] [a-c] a, b, c Wildcard character , , • Formal regexp only has “essentials” . . a 3 : Shortcut class \… \d 0,1,2,…,9 • Perl adds shortcuts and non-regular features Assertion Zero-width anchor $,^,\b,… \b (word boundary) © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 7 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 8

Perl Perl Non-Regular Perl Regex Features Non-Regular Perl Regex Examples • Non-regular = not essential feature, and can not be emulated by essential features Description Regex Matched – Backtracking engine, may take exponential time Square (\d+)x\1 123x123 • Backreferences in same pattern: \1, \2, … (repeated) • Zero-width assertions: Palindrome (\w+)\w(??{reverse $1}) redivider – Look-ahead positive (?=…), negative (?!…) (mirrored) – Look-behind positive (?<=…), negative (?' x length $1}) <<>> • Match-time code execution: (?{…}) parentheses • Match-time interpolation: (??{…}) • Backtracking suppression: (?>…)

Concepts Outline Natural and Artificial Languages English Perl • Regular expressions (continued) Concept Examples Feature Examples • Type conversions Noun fish Variable $line Pronoun it, they Default variable $_, @_ • Programming in the large Inflection …s Sigil @… • Scripting as glue Singular apple Scalar $i Plural oranges Array @row Verb cook Function sort Irregular verb read Operator <…> Infinitive to be Function reference &cmp Context go fish Context if(@a)… © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 11 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 12

Perl Perl Context Primitive Conversions

• When you use a value in a different context Value Context than the value’s type, Perl converts it Boolean Number String Reference – E.g., using array as scalar -> int length 0 false "" undefined Number identity • Context provided by: !=0 true "value" – Operator: e.g., string . string "" false 0 symbolic – Function: e.g., list identity print String "0" false 0 reference – Assignment: e.g., @a = list Other true prefix – Statement: e.g., if (boolean) {…} Undefined false 0 "" undefined • Functions provide context for parameters, and can react to context for return value Reference true addr "typ(adr)" identity

Perl Concepts List Conversions Weak/Strong, Static/Dynamic Typing

Value Context Strong typing Scalar List (explicit conversions) ML Scalar identity 1-element list Scheme Java List literal last element flattened sublists VBA Array length identity PHP empty "0" empty list Hash JavaScript non-empty "size/buckets" k , v , k , v ,… C 1 1 2 2 Perl true iff match all groups m// Weak typing end of file "" empty array (implicit conversions) assembler <…> not EOF line all lines Static typing Dynamic typing (compile-time checks) (runtime checks)

Perl Perl Functions and Context Prototype Example

• Prototype provides context for parameters • Example from page 227 of “Programming Perl”, 3rd ed. Wall, Christiansen, Orwant. O’Reilly, 2000. – sub [id] [proto] [attrs] [{…}] • See also the Error module on CPAN. – No prototype: list operator Function definitions Example usage – proto = (protoChar*) sub try(&$) { try { scalar, list, hash, file handle – $ @ % * my ($try, $catch) = @_; die "phooey"; – & subroutine (in first slot: bare block, no comma) eval { &$try }; } #not end of the call! – ; separates mandatory from optional arguments if ($@) { catch { – \ adds backslashes on actuals local $_ = $@; /phooey/ and &$catch; print "unphooey\n"; checks context for return value • wantarray } }; – true = list, false = array, undefined = void context } sub catch (&) { $_[0] } © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 17 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 18

Perl Outline Modules

• Module = file p/q.pm that starts with • Regular expressions (continued) declaration package p::q • Type conversions – Group library functions for reuse • Programming in the large – E.g., Math::BigInt, arbitrary precision arithmetic • Module import: access with unqualified names • Scripting as glue – Compile-time use p; runtime require p; – Disable locally: no p; • Pragma = pragmatic module – Changes Perl behavior in fundamental way – E.g., strict, forces variable declarations

Perl Concepts Module Example Pluggable Type Systems use strict; use warnings; # put this code in a package apple; # file apple.pm sub create { • User chooses between multiple alternative my ($weight, $color) = @_; return { WEIGHT => $weight, COLOR => $color }; optional type systems, e.g., in Perl: } no implicit declarations sub pluck { – use strict "vars"; my $ref_to_hash = $_[0]; – use strict "refs"; no string→ref conversion return $ref_to_hash->{COLOR} . " apple"; } – use strict "subs"; no bareword strings sub prepare { my ($ref_to_hash, $how) = @_; – use warnings; same as perl -w return $how . "d " . pluck($ref_to_hash); – perl -T taint-flow checking } 1 # return true = success • Motivation: trade-off between ease of writing, #!/usr/bin/perl use strict; use warnings; use apple; # note the "use apple" maintainability, robustness, and security our $fruit = apple::create(150, "red"); # from a different file print apple::prepare($fruit, "slice"), "\n"; # "sliced red apple" – Similar to “Option Explicit” in VBA

Perl Perl Structure of a Perl Application Using Objects

Compilation unit Usually, file; also, -e, eval arg #!/usr/bin/perl Import class module Package Named global namespace use strict; use warnings; use Apple; may vary by block; usually: one per file our $a1 = new Apple(150, "green"); Constructor call forms our $a2 = Apple->new(150, "green"); (indirect obj vs. arrow) Module p/q.pm file with package p::q $a2->{COLOR} = "red"; Set hash entry Pragma Module that changes language print $a1->prepare("slice"), "\n"; Method call Class Package used to bless reference print $a2->prepare("squeeze"), "\n"; (arrow form) Subroutine Named subroutines don’t nest Statements In subroutine or directly in file

Concepts Perl Classes and Objects Defining Classes use strict; use warnings; package Fruit; sub new { Fruit class = package used my ($cls, $weight) = @_; return bless({ WEIGHT => $weight }, $cls); WEIGHT to bless references instance name : blessing package } sub pluck { new() my $self = $_[0]; name $a1 : Apple return "fruit(" . $self->{WEIGHT} . "g)"; pluck() Apple } prepare() WEIGHT = 150 sub prepare { WEIGHT my ($self, $how) = @_; COLOR = “green” return $how . "d " . $self->pluck(); objects } fields (hash keys) COLOR 1 $a2 : Apple (blessed new() references) class method pluck() WEIGHT = 150 prepare() COLOR = “red” • Invocant (class name or object) is first argument instance methods fields (hash keys+values) •bless( ref,pkg) associates object with class

Perl Concepts Inheritance in Perl Virtual Dispatch use strict; use warnings; package Fruit; use strict; use warnings; package Fruit; sub new { Fruit sub new { Fruit my ($cls, $weight) = @_; my ($cls, $weight) = @_; return bless({ WEIGHT => $weight }, $cls); WEIGHT return bless({ WEIGHT => $weight }, $cls); WEIGHT } } sub pluck { new() sub pluck { new() my $self = $_[0]; my $self = $_[0]; return "fruit(" . $self->{WEIGHT} . "g)"; pluck() return "fruit(" . $self->{WEIGHT} . "g)"; pluck() } prepare() } prepare() sub prepare { sub prepare { Use class of my ($self, $how) = @_; my ($self, $how) = @_; return $how . "d " . $self->pluck(); return $how . "d " . $self->pluck(); reference to } } 1 Apple 1 decide which Apple use strict; use warnings; use Fruit; package Apple; use strict; use warnings; use Fruit; package Apple; our @ISA = ("Fruit"); WEIGHT our @ISA = ("Fruit"); method to callWEIGHT sub new { sub new { my ($cls, $weight, $color) = @_; COLOR my ($cls, $weight, $color) = @_; COLOR return bless({ WEIGHT => $weight, COLOR => $color }, $cls); return bless({ WEIGHT => $weight, COLOR => $color }, $cls); } new() } new() sub pluck { sub pluck { my $self = $_[0]; pluck() my $self = $_[0]; pluck() return $self->{COLOR} . " apple"; return $self->{COLOR} . " apple"; } prepare() } prepare() 1 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 27 1 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 28

Perl Outline Gluing Perl using Bash: Pipes

• Regular expressions (continued) • Bash (Bourne again shell) is an interactive command prompt for Unix or Linux • Type conversions • At the bash shell prompt, prog1 | prog2 • Programming in the large pipes STDOUT of prog1 to STDIN for prog2 • Scripting as glue • Perl scripts work well in pipelines: read from STDIN, write to STDOUT • Idiomatic Perl facilitates one-liners, e.g.: ls -l | perl -e'while(<>){/(\d+)\s(\S+\s+){3}(\S+)$/;print "$1 $3\n"}' | sort -rn | head -5 • Good for one-time use, bad for readability

Perl Perl Gluing Bash using Perl: `…` Gluing Perl using Perl: eval

• Perl provides many common Unix • Perl scripts can interpret embedded Perl commands as library functions code using eval – chdir, chmod, kill, mkdir, rmdir, … #!/usr/bin/perl print "please enter an operator, e.g., '+':\n"; • Like bash, Perl has file test operators $op = <>; chomp $op; – -d, -e, -f, -r, -w, -x, … $command = 'print (42 ' . $op . ' 7)'; • Perl programs can embed shell print "running the command '$command':\n"; eval $command; commands with `…` print "\n"; – Returns output as string • Module Safe provides restricted eval – Error code goes in $? (reval) as sandbox for untrusted code

Perl Perl Gluing Text using Perl: Heredocs Common Perl Mistakes

• Heredoc = multi-line text embedded in Perl Detected Message Typical example #!/usr/bin/perl Compiler Missing curleys if($x) print "x" print "-Error-\n", <

– print(<

Soap-box Perl Evaluating Perl Perl Culture

Strengths Weaknesses • Perl haikus and other poetry • String processing • Write-only language print STDOUT q, Just another Perl hacker, – Regular expressions • Need references to unless $spring – Interpolation nest arrays/hashes • Obfuscated Perl contest • One-liners • Grafted-on features – Object-orientation $_ = "wftedskaebjgdpjgidbsmnjgc"; – Many operators – Exception handling tr/a-z/oh, turtleneck Phrase Jar!/; – Implicit operands • Language specified print; • Vibrant community by implementation • perl.org, cpan.org, perlmonks.org, pm.org • Portability

Perl Administrative Suggestions for Perl Practice Last Slide • Pick up your graded homework and quiz • hw04 gets you points, but you may want to do more at your own leasure • Today’s lecture • Next lecture • Sort CSV file by user-specified column – Context – HTML, HTTP • Pretty-print C code as HTML – Modules – Server-side scripting • Turn PID and PPID from ps into tree – Object-oriented Perl – PHP • Use Perl to translate gnuplot into VBA code that draws a Powerpoint slide – Scripting as glue • Write a Perl poem

Perl Perl

#!/usr/bin/perl Scopes and Visibility package p; Package Example our $x = 'px'; { caller of eval (if any) package q; our ($x, $y) = ('qx', 'qy'); print "x ('qx' from block) $x \n"; … {… {…} …} … print "p::x ('px' qualified access) $p::x \n"; {…} print "q::x ('qx' qualified access) $q::x \n"; (block) Name lookup: (block)(block) } (block) (outer blocks) { (compilation unit) package r::s; # name can include multiple qualifier levels (package) our $x = 'rsx'; { package q; # can reopen in different block, even different file our $z = 'qz'; • Compilation unit = file, -e, or eval argument print "x ('rsx' block hides package) $x \n"; print "y ('qy' disallowed by 'use strict') $y \n"; • Package = global named namespace print "z ('qz' from block) $z \n"; print "q::x ('qx' qualified access) $q::x \n"; – Can declare same package in multiple blocks, print "r::s::x ('rsx' qualified access) $r::s::x\n"; } or even in multiple compilation units } – Only one package in effect at any point print "x ('px' from compilation unit) $x \n"; print "q::z ('qz' qualified access) $q::z \n"; print "r::s::x ('rsx' qualified access) $r::s::x\n"; © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 39 © Martin Hirzel CSCI-GA.3033.003 NYU 6/12/12 40

Perl Typeglobs and Symbol Tables

• Package p::q uses symbol table %p::q:: – Symbol table entry $p::{'x'} is typeglob *p::x • Typeglob = symbol table entry – E.g., *x holds $x, @x, %x, … – *x{SCALAR} is the same as \$x, and similarly for ARRAY, HASH, CODE, IO – *x{GLOB} is the same as \*x – *x{PACKAGE} returns package containing x – *x{NAME} returns name of typeglob • Perl internally manipulates typeglobs to implement features, e.g., function “use”