System Integration
Scripting Languages • Why use these languages for systems work • Remember Operating Systems? Fun and Futures
Course: 67557 Hebrew University אלי וט י פה – Lecturer: Elliot Jaffe
System popen
• Use system call in libc • System with piped output #include
Perl Futures
• Execute operation in /bin/sh return output • Perl • Redirect STDIN, STDOUT as needed • Python $output = `cmd ` $output = `cmd 2>&1 ` • Business Use $output = `cmd 2>/dev/null ` $output = `cmd 2>&1 1>/dev/null ` $output = `cmd 3>&1 1>&2 2>&3 3>&-`
1 Perl 6 Perl History
• A story of Hubris • Dec 18, 1987: Perl 1.0 released – $calers, @rrays, %ssociative_arrays Or • June 5, 1988: Perl 2.0 released – Regular expressions • Oct 18, 1989: Perl 3.0 released • Dreaming is easy, building is very hard – GPL • May 21, 1991: Perl 4.0 released – Array in scalar context gives length
Perl History (2) Perl History (3)
• Oct 18, 1994: Perl 5.0 released • Mar 28, 2000: Perl 5.6.0 released – OO, Lexical scoping, Perldoc, POD • July 18, 2000: • Mar 13, 1995: Perl 5.001 released – Larry Wall believes that Perl 5 has run out of • Feb 29, 1996: Perl 5.002 released steam • Oct 10, 1996: Perl 5.003 released – Perl 6 is born – Estimated time of arrival: 2001 • May 5, 1997: Perl 5.004 released • 2002: Perl 5.80 released • Jul 22, 1998: Perl 5.005 released
Why Perl 6 It all began so simply
• Internals are a mess • Separate the parser from the interpreter
• Extension is very painful Source Code Parser • Porting is very hard Syntax Tree Compiler • Fear of losing the development community Byte Code Optimizer • Need a new focus
Better Byte Code Runtime
2 Syntax Trees Parse Tree vs. AST
• Parse Tree Parse Tree AST – Encode the sequence of rules used to understand a given input expr + – Includes punctuation, and intermediate nodes plus • Abstract Syntax Tree 3 4 – Condensed form of a Parse tree excluding punctuation and intermediate nodes 3 + 4
Remember the Alamo Parrot
• Microsoft .Net introduced in 2000 • Based the Monty Python “Parrot” sketch • Virtual Machine based answer to Java • Build a common runtime for Python, Perl6 • All MS languages would run on .Net VM • Expected release date: End of 2002 • Support for I/O, Threads, Multiple Scoping • If MS could do it, how hard can it be? schemes, Performance, Platform • Let’s build a generic VM for Perl Independence, Extensions • We’ll call it Parrot • Current 1.0 release date: End of 2005
Where is Perl6? Big Eyes
• Myth: Its easy to design a new language • Goal is to have Perl6 be self-hosting • 361 RFC’s in 2000 • I.E. The Perl6 compiler should be written in Perl6 • RFC 28: Perl should stay Perl • Proposed Solution 1: – (In your dreams) – Port Perl5 to Parrot (Ponie) – Write Perl6 Parser in Perl5 • By end of 2004, Language specification is – Use Perl5-Perl6 translator “stable but not final” – Status: Perl5 is hard, translator does not exist
3 Perl6 PUGs
• Proposed Solution 2: • Proposed Solution 3: – Implement Perl6 parser in C – Implement Perl6 in another language (Haskell) – Output Abstract Syntax Tree (AST) – Use this to test AST generator and Parrot AST – Build Parrot compiler for Perl6 AST compiler – Status: Proposed in early 2005 – Who cares about performance – Took 3 weeks to implement basic language – Still missing many pieces
Where does it stand? Reality
• Perl6 Language: • Perl6 is years away – Still not final, cannot be closed until there is a working implementation • Perl5 is very stable • Parrot 1.0: end of 2005? – Latest major release is 5 years old • PUGs – 7800+ modules in CPAN – Currently 6.0.10: 100 times slower than Perl5 – Modules will not run as-is in Perl6! – Complete: 6.283185 – no projected date • Perl6 compiler – Partial implementation, lacks developers
Python Multi - Python
• April 1999: Python 1.5.2 • Python – Unix, Windows • Dec 2001: Python 2.2 (very stable) • Jython – Python in Java • Nov 2004: Python 2.4 (stable) • IronPython – Python on .NET • Psycho – Python on JVM • Improvements include minor language changes, bug fixes
4 Futures Using OpenSource in real-life
• Less focus on interpreters • Language Issues • Less focus on language extensions • Business Issues • More focus on standard libraries • Operational Issues – Learn from Java – Learn from Perl • Examples – PictureVision – Google
Language Specific Issues Perl Specific Issues
• How do you convince a “professional” • How do you share code with another organization that Perl/Python is stable organization? enough for production use? • They may not be using your language! – Used to be a major problem – Now acceptable for in-house use Solution: – Rare in published software • Share Data Files – Competes with Micro$oft .NET • Define Interfaces as Soap, IPC, RPC
Perl Specific Issues Large System Issues
• How do you protect your product rights? • Configuration Management – Trust – Versioning – Obfuscation • Perl/Python – interpreter can change – Contract Language (lawyers) • Modules – if you rely on modules, then can change • Secondary software – Databases – The biggest drawback to using Scripting – pbm, jpeg, gimp Languages!
5 Large System Issues Perl in Business
• Configuration Management (cont) • PictureVision used .5 Million lines of Perl – Install Process – Update Process • Implemented AOL’s You’ve Got Pictures – Rollback Process – Disaster Recovery • These are harder because you don’t have a • Supported hundreds of thousands of single executable to install concurrent users • Instead you have a complex system of files
PictureVision used Perl for: Python at MS
• Web: • Microsoft believe it or not! – CGI, Modperl • Administration: • 1996: "It's called what?" – Installation, Maintenance, Monitoring • 1997: "You actually shipped Python code?" • Infrastructure: (MerchantServer 1.0) – Remote Procedure calls • 1998: "Nice prototype. We'll rewrite it in • Distributed Services: the next version." And they did, in C++. – Global Naming Service
Python at Google Google’s Reasons
• "Python has been an important part of • Highly adaptable to changing requirements – You need a language that is very flexible, so you can adapt your Google since the beginning, and remains so tools during development as the system grows and evolved. Today – Changes in computing environment • Rapid development dozens of Google engineers use Python, and – For new and experienced developers we're looking for more people with skills in – The market moves very very quick; you want to be able to keep up with it. If it takes two years for you to respond to something that is this language" needed today, you're behind the curve. -- Peter Norvig, Director of Search Quality • Easy to maintain – You can come back a year later, look at that code, and understand at Google what is going on
http://www.sauria.com/~twl/conferences/pycon2005/20050325/Python%20at%20Google.html
6 Google’s Uses Course Take Aways
• Development Support • Perl and Python are useful tools – Build, Packaging and Deployment system • Network Infrastructure – You can use them without fear – Used for moving data around • Dev Data Center • Monitoring • There is more to life than C++ and Java – All live servers user Python to monitor health • Log generation and Analysis • Online Services • Language Design is hard – Google Groups, code.google.com
Ruby Ruby
• Began in 1993 • Everything is an object • V1.0 in 1995 • Variables are object attributes • Every function is a method • Ruby inherited much from Perl – (both are precious stones) • Similar to Perl, Python and SmallTalk – Lists, Hashes, Regular expressions
Ruby examples More Ruby examples
-199.abs # 199 a = [1, 'hi', 3.14, 1, 2, [4, 5]]
"ruby is cool".length # 12 a[2] # 3.14
"Rick".index("c") # 2 a.reverse # [[4, 5], 2, 1, 3.14, "hi", 1]
“Happy Days?".split(//).uniq.sort.join a.flatten.uniq # [1, "hi", 3.14, 2, 4, 5] # " ?DHapsy"
7 Still more ruby Ruby In Use
a = [1, 'hi', 3.14] • Lost of uses, but not in the same league as a.each {|item| puts item} Perl and Python # Prints each element • Interesting language with potential (3..6).each {|num| puts num} • probably an also-ran # Prints the numbers 3 to 6
8