Linux Journal | August 2011 | Issue

™ Writing Facebook Applications

Understanding and Using Git

Since 1994: The Original Magazine of the Linux Community Use AUGUST 2011 | ISSUE 208 | www.linuxjournal.com Podcasts to Share BUILD A Your Linux BEOWULF Knowledge HPC SYSTEM with the Add Nagios FedoraLiveCD Notifications Project to Your Screen Window LINUX STANDARD Make BASE JavaScript and Its Role Programming in the Easy with Community HOW TO PLAN CoffeScript YOUR OWN LINUX EVENT WITH GARETH GREENAWAY OF SCALE

$5.99US $5.99CAN HOW-TO: 08 UNISON THE FILE

SYNCHRONIZATION PROGRAM 0 09281 03102 4 10 Gig On Board

Blazing Fast, Embedded 10Gb Ethernet

10G Rackmount Servers in the iX-Neutron server line feature the Intel® Xeon® Processor 5600/5500 Series, and come with 10GbE networking integrated onto the motherboard. This eliminates the need to purchase an additional expansion card, and leaves the existing PCI-E slots available for other expansion devices, such as RAID controllers, video cards, and SAS controllers.

For more information on the iX-1204-10G, or to request a quote, visit: http://www.iXsystems.com/neutron

30% cost savings/port over equivalent Dual- Port 10 GB PCI Express add-on card solution KEY FEATURES:

IPMI NIC . Supports Dual 64-Bit Six-Core, Quad- Core or Dual-Core, Intel® Xeon® Processor 5600/5500 Series GigE NICS . 1U Form Factor with 4 Hot-Swap SAS/ 10GbE NICS SATA 3.5” Drive Bays . Intel® 5520 chipset with QuickPath Interconnect (QPI) . Up to 192GB DDR3 1333/1066/800 SDRAM ECC Registered Memory (18 DIMM Slots) . 2 (x8) PCI-E 2.0 slots + 1 (x4) PCI-E 2.0 (in x8 slot -Low-Profile - 5.5” depth) . Dual Port Intel® 82599EB 10 Gigabit SFP+ - Dual Port Intel® 82576 Gigabit Ethernet Controller . Matrox G200eW Graphics . Remote Management - IPMI 2.0 + IP-KVM with Dedicated LAN . Slim DVD 10Gb Ethernet . 700W/750W Redundant AC-DC 93%+ Adapters High-Efficiency Power Supply

Call iXsystems toll free or visit our website today! 1-855-GREP-4-IX | www.iXsystems.com

Intel, the Intel logo, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and/or other countries. 1&1 DUAL HOSTING

No other web host offers more expertise, know- how and quality service than 1&1. Double Security: Your website is simultaneously hosted in 2 locations in our high tech data center! High-speed Global Network: 210 GBit/s Connectivity Environmentally Responsible: 100% Renewable Energy Solid Technical Foundation: 1,000 In-house Developers

SUMMER SPECIAL: 1&1 DUAL ADVANCED PACKAGE & * 1 YEAR FREE! Q2 FREE Domains QUNLIMITED Trafﬁ c QASP, .NET, AJAX, LINQ, PHP5, Perl, SSI QFREE Private Domain Registration QDNS Management Q5 Microsoft® SQL Databases Q500 E-mail Accounts Q1&1 SiteAnalytics Q24/7 Toll-free Customer Support Q50 FTP Accounts Q1&1 WebsiteBuilder

Need more domains? ® .com with FREE Private Registration just $4.99/ﬁ rst year.*

1-877-GO-1AND1 www.1and1.com 1-855-CA-1AND1 www.1and1.ca

* 24 month minimum contract term required for Dual Advanced offer. Set-up fee and other terms and conditions may apply. .com offer valid fi rst year only. After fi rst year, standard pricing applies. Visit www.1and1.com for full promotional offer details. Program and pricing specifi cations and availability subject to change without notice. 1&1 and the 1&1 logo are trademarks of 1&1 Internet AG, all other trademarks are the property of their respective owners. © 2011 1&1 Internet, Inc. All rights reserved. AUGUST 2011 CONTENTS Issue 208

FEATURES 34 A CONFERENCE PRIMER WITH GARETH GREENAWAY Geek Wrangling 101, with an expert. Shawn Powers 42 FACEBOOK APPLICATION DEVELOPMENT Port your Web app to Facebook. Mike Diehl 48 LINUX STANDARD BASE: STATE OF AFFAIRS LSB and what it means for the Linux community. Jeff Licquia, Stew Benedict and Vladimir Rubanov 54 GIT A guide to what many consider the best version control tool ever created. Henry Van Styn

ON THE COVER • How to Plan Your Own Linux Event with Gareth Greenaway of SCALE, p. 34 • Writing Facebook Applications, p. 42 • Understanding and Using Git, p. 54 • Build a Beowulf HPC System with the FedoraLiveCD Project, p. 64 • Linux Standard Base and Its Role in the Community, p. 48 • Use Podcasts to Share Your Linux Knowledge, p. 60 • Add Nagios Notifications to Your Screen Window, p. 24 • Make JavaScript Programming Easy with CoffeeScript, p. 16 • How-To: Unison the File Synchronization Program, p. 73

2 | august 2011 www.linuxjournal.com They say work smarter, not harder. They must be using our processor.

The next generation of intelligent server processors The Intel® Xeon® processor 5600 series automatically regulates power consumption to combine industry-leading HQHUJ\HIÀFLHQF\ZLWKLQWHOOLJHQWSHUIRUPDQFHWKDWDGDSWV to your workload. Check out the new intelligent features of the Xeon® 5600 at intel.com/itcenter.

Servers from iXsystems feature the Intel® Xeon® processor 5600 series.

iX2216-10G

5IFiX2216-10GGFBUVSFTEVBMPOCPBSE t%VBM*OUFM® Xeon® 5600 Series Processors *OUFM®&#(JHBCJU4'1 1PSUT t6'PSN'BDUPSXJUITJYUFFO4"44"5")PU4XBQ ® EVBMPOCPBSE*OUFM (JHBCJU1PSUT %SJWF#BZT Enterprise Servers BOE%*..TMPUTTVQQPSUJOHVQUP(# t0O#PBSE%VBM1PSU*OUFM®&#(JHBCJU4'1 for Open Source PG%%3&$$3FHJTUFSFENFNPSZ*EFBMGPS www.iXsystems.com )1$ %BUB$FOUFS 7JSUVBMJ[BUJPO $MVTUFSJOH 1-855-GREP-4-IX BOE$MPVE$PNQVUJOHBQQMJDBUJPOT iX1204-10G 5IFiX1204-10GGFBUVSFTUIFMBUFTU*OUFM® ® Xeon® 5600 Series Processors Intel is not responsible for and $16TCBTFEPOUIFONBOEON t%VBM*OUFM has not verifi ed any statements OFYUHFOFSBUJPONJDSPBSDIJUFDUVSF5IFOFYU t6'PSN'BDUPSXJUI)PU4XBQ4"44"5"%SJWF#BZT or computer system product- TUFQJOJOUFMMJHFOUQFSGPSNBODF BVUPNBUFE t0O#PBSE%VBM1PSU*OUFM®&#(JHBCJU4'1 specifi c claims contained herein. FOFSHZFGýDJFODZ BOEþFYJCMFWJSUVBMJ[BUJPO

© 2010, Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Sponsors of Tomorrow, Intel Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others. AUGUST 2011 CONTENTS Issue 208 COLUMNS 16 Reuven M. Lerner’s At the Forge Introducing CoffeeScript

22 Dave Taylor’s Work the Shell Calculating Day of the Week

24 Kyle Rankin’s Hack and / Nagging Notifications

77 Kyle Rankin and Bill Childers’ Tales from the Server Room It’s Always DNS’s Fault! 30 BLU-RAY

80 Doc Searls’ EOF First Brazil, Then the World INDEPTH 60 Podcasting Do you know something about Linux? A podcast is an excellent way to share your knowledge with others. Charles Olsen 64 How to Build a Beowulf HPC System Using the FedoraLiveCD Project Build a Red Hat-based Beowulf Cluster using a 73 UNISON kickstart file and tools from the FedoraLiveCD project. Howard Powell 69 Radio Dramas in Linux Tips and tricks for creating radio dramas with open-source software. Dan Sawyer 73 Unison, Having It Both Ways The file synchronization tool that supports bidirectional updates of files and directories. Adrian Klaver

IN EVERY ISSUE 8 Current_Issue.tar.gz 10 Letters 12 UPFRONT 28 New Products 30 New Projects 69 RADIO DRAMAS 65 Advertisers Index 79 Marketplace

USPS LINUX JOURNAL (ISSN 1075-3583) (USPS 12854) is published monthly by Belltown Media, Inc., 2121 Sage Road, Ste. 310, Houston, TX 77056 USA. Periodicals postage paid at Houston, Texas and at additional mailing offices. Cover price is $5.99 US. Sub scription rate is $29.50/year in the United States, $39.50 in Canada and Mexico, $69.50 elsewhere. POSTMASTER: Please send address changes to Linux Journal, PO Box 16476, North Hollywood, CA 91615. Subscriptions start with the next issue. Canada Post: Publications Mail Agreement #41549519. Canada Returns to be sent to Pitney Bowes, P.O. Box 25542, London, ON N6C 6B2

4 | august 2011 www.linuxjournal.com More TFLOPS, Fewer WATTS Microway delivers the fastest and greenest ﬂoating point throughput in history 2.5 TFLOPS

Enhanced GPU Computing with Tesla Fermi 480 Core NVIDIA® Tesla™ Fermi GPUs deliver 1.2 TFLOP single precision & 600 GFLOP double precision performance! New Tesla C2050 adds 3GB ECC protected memory New Tesla C2070 adds 6GB ECC protected memory Tesla Pre-Conﬁgured Clusters with S2070 4 GPU servers WhisperStation - PSC with up to 4 Fermi GPUs OctoPuter™ with up to 8 Fermi GPUs and 144GB memory

New Processors 12 Core AMD Opterons with quad channel DDR3 memory 8 Core Intel Xeons with quad channel DDR3 memory Superior bandwidth with faster, wider CPU memory busses Increased efﬁciency for memory-bound ﬂoating point algorithms

Conﬁgure your next Cluster today! 10 TFLOPS 5 TFLOPS www.microway.com/quickquote 508-746-7341

FasTree™ QDR InﬁniBand Switches and HCAs

45 TFLOPS 36 Port, 40 Gb/s, Low Cost Fabrics Compact, Scalable, Modular Architecture Ideal for Building Expandable Clusters and Fabrics MPI Link-Checker™ and InﬁniScope™ Network Diagnostics

FasTree 864 GB/sec Achieve the Optimal Fabric Design for your Speciﬁc Bi-sectional Bandwidth MPI Application with ProSim™ Fabric Simulator Now you can observe the real time communication coherency of your algorithms. Use this information to evaluate whether your codes have the potential to suffer from congestion. Feeding observed data into our IB fabric queuing-theory simulator lets you examine latency and bi-sectional bandwidth tradeoffs in fabric topologies.

GSA Schedule Contract Number: GS-35F-0431N

pC2_Microway.indd 1 7/15/10 9:20:43 AM Executive Editor Jill Franklin [email protected] Senior Editor Doc Searls [email protected] Associate Editor Shawn Powers [email protected] Art Director Garrick Antikajian [email protected] Products Editor James Gray [email protected] Editor Emeritus Don Marti [email protected] Technical Editor Michael Baxter [email protected] Senior Columnist Reuven Lerner [email protected] Security Editor Mick Bauer [email protected] Hack Editor Kyle Rankin [email protected] Virtual Editor Bill Childers [email protected]

Contributing Editors Ibrahim Haddad • Robert Love • Zack Brown • Dave Phillips • Marco Fioretti • Ludovic Marcotte Paul Barry • Paul McKenney • Dave Taylor • Dirk Elmendorf • Justin Ryan

Proofreader Geri Gale

Publisher Carlie Fairchild [email protected]

General Manager Rebecca Cassity [email protected]

Senior Sales Manager Joseph Krack [email protected]

Associate Publisher Mark Irgang [email protected]

Webmistress Katherine Druckman [email protected]

Accountant Candy Beauchamp [email protected]

Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA

Editorial Advisory Panel Brad Abram Baillio • Nick Baronian • Hari Boukis • Steve Case Kalyana Krishna Chadalavada • Brian Conner • Caleb S. Cullen • Keir Davis Michael Eager • Nick Faltys • Dennis Franklin Frey • Alicia Gibb Victor Gregorio • Philip Jacob • Jay Kruizenga • David A. Lane Steve Marquez • Dave McAllister • Carson McDonald • Craig Oda Jeffrey D. Parent • Charnell Pugsley • Thomas Quinlan • Mike Roberts Kristin Shoemaker • Chris D. Stark • Patrick Swartz • James Walker

Advertising E-MAIL: [email protected] URL: www.linuxjournal.com/advertising PHONE: +1 713-344-1956 ext. 2

Subscriptions E-MAIL: [email protected] URL: www.linuxjournal.com/subscribe PHONE: +1 818-487-2089 FAX: +1 818-487-4550 TOLL-FREE: 1-888-66-LINUX MAIL: PO Box 16476, North Hollywood, CA 91615-9911 USA Please allow 4–6 weeks for processing address changes and orders PRINTED IN USA

LINUX is a registered trademark of Linus Torvalds. Learn Drupal & jQuery FROM THE COMFORT OF YOUR LIVING ROOM

The Lullabot Learning Series includes everything you need to become a Drupal & jQuery expert from the comfort of your living room! The videos are available in both DVD format and high-defi nition video download. Purchase the videos at http://store.lullabot.com Current_Issue.tar.gz

I’ll Be in My Hole

ech folks tend to get a bad rap as basement- are standards and collaboration. The Linux Standards dwelling creatures unable to function in Base attempts to keep the various flavors of Linux T polite society. And really, that’s true for only compatible with each other. Sometimes that works 80% of us. Of course, I’m teasing, but it’s hard to better than others, but we have the current “State deny Linux users and open-source enthusiasts tend of Affairs” regarding the LSB for you this month. to belong to unique communities. We thought it Jeff Licquia, Stew Benedict and Vladimir Rubanov SHAWN POWERS might be nice to focus on our Linux community this provide an overview of the LSB and talk about how month, and so the articles you’re about to read to be compliant. Henry Van Styn follows them up should feel as cozy as a Hobbit hole. with an article on Git. Once a group of people start Reuven M. Lerner knows his audience and to collaborate on software, revision control is vital. introduces a new programming language called Henry discusses Git and how its distributed model CoffeeScript. The coffee-/java-/bean-naming makes it so powerful. convention is nothing new, but CoffeeScript makes As magazine publishers, we surely understand JavaScripting fun and easy. If you’ve ever been the importance of communication when it comes frustrated with JavaScript, you’ll want to read Reuven’s to community. With the Linux world, however, we article. Following that, Dave Taylor continues his communicate in multiple ways. One of those methods, series on determining the day of the week. If you which is near and dear to my heart, is podcasting. truly do live in a basement or Hobbit hole, you Kyle Rankin and I do the Linux Journal Insider podcast might need your computer to determine what day every month, and if we can do it, you can do it. it is, as sunlight might be a rare sight. Charles Olsen walks through the process of creating Kyle Rankin doesn’t help the geek stereotype a podcast from the planning stages all the way to when everything he does is within a single, dark the publishing stages. Content, of course, is up to console window. Kyle uses a screen session to connect you. Dan Sawyer takes a slightly different angle, to his favorite programs (all command-line) from and instead of podcasting, he shows how to edit anywhere. This month, he shows how to get Nagios radio dramas. If you tend to make stuff up in your to send notifications to a screen session instead of podcast, you already may be walking the line e-mail or SMS messages. Bill Childers is back with us between fact and fiction, so we wanted to make this month as well, and he and Kyle continue their sure you were prepared either way. new column, Tales from the Server Room. If you’re What group of geeks would be worth their a sysadmin, you’ll probably relate to their misery and salt without allowing their computers to form a maybe learn something along the way. community as well? Whether you want multiple No community is a community without gatherings. computers to share a filesystem with Unison (Adrian Granted, in our world, those gatherings often are Klaver shows how), or if you want your systems in IRC rooms or in on-line forums. We do have our to form a cohesive community of their own in a meatspace meetups, however, and if you’ve never Beowulf cluster (Howard Powell describes how to do been to a LinuxFest, I highly recommend attending it with Fedora Live CDs), this issue has you covered. one. This month, I interview Gareth Greenaway We’ve also got the full lineup you expect every month. from the Southern California Linux Expo (SCALE) UpFront articles, new product announcements—lots and ask him what it takes to host a Linux confer- of goodies to soothe your geek itch. And, of course, ence. Gareth is easy to talk to, and he has some if the thought of being a part of the Linux community practical advice for anyone interested in attending a offends you, well, feel free to read this magazine by LinuxFest or even hosting their own. To contrast the yourself. We won’t tell.I idea of getting together in person, Mike Diehl talks about Facebook application development. Whether Shawn Powers is the Associate Editor for Linux Journal. He’s also the Gadget you’re interested in integrating services or creating Guy for LinuxJournal.com, and he has an interesting collection of vintage a new zombie-pony-themed Farmville, you’ll want Garfield coffee mugs. Don’t let his silly hairdo fool you, he’s a pretty ordinary to read Mike’s article. guy and can be reached via e-mail at [email protected]. Or, swing Some of the core values of our Linux community by the #linuxjournal IRC channel on Freenode.net.

8 | august 2011 www.linuxjournal.com

letters

production logs on hosts behind a load low-power television stations. His under- balancer. You don’t have to set up standing of the situation is that the FCC handlers like swatch, and you also can dictates to broadcasters the power just use chip to see of all of your logs at which they can transmit based on like a pimped version of tail. several technical factors. Those factors include things like terrain, desired broadcast I’ve used it here at Yahoo pretty frequently coverage, distance from other broadcasts during the past month for deployments running on close frequencies and so on. and bug investigations. In fact, the FCC sometimes will dictate that the signal can’t be broadcasted Here’s a blog post, if you’re omnidirectional. I’m close to the Canadian interested: codefury.net/2011/04/ border, and some stations are not allowed video-chip-a-log-file-monitor-multiplexer. to broadcast toward Canada. -- Kenny Katzgrau Before researching, my guess was that broadcast power was based on fees paid Thanks Kenny. Sharing tools is one of to the FCC. As it turns out, that’s not the the things that makes the Open Source case. The fees do vary widely based on community so great. I appreciate you location, however, and usually frequencies sharing with us!—Ed. are auctioned off to the highest bidder.

May 2011 Cover Receiving Digital TV Signals, Finally, you might want to check out My manga reading/drawing daughter saw No Longer Analog Signals www.antennaweb.org. It offers some the cover of the May 2011 issue of LJ and Regarding the new digital TV world: I am great tools to help you get the best asked me why there was manga on it. I told receiving three stations via digital signal on possible reception.—Ed. her, “Linux is just cool like that!” Then, she my digital-to-analog-converter TV, with my asked if there was an article about manga outdoor antenna. Two other stations, in 777 Permissions in the magazine. I told her about past issues the same area, hardly come in at all. The While reading the article titled “Numeric covering Blender, GIMP and so on, which first group transmits at 1MW power; the Relativity” by Joey Bernard on page 14 prompted a discussion about the great other group transmits at only 0.25MW of the May 2011 issue, I was a little open-source software she uses every day. power. My signal strength meter seems disappointed to notice that the instruc- consistent with these numbers (above). tions for installing the Einstein Toolkit Thanks for the great cover! My question is, why is hardly anything contain a chmod 777. Being a systems -- written on the transmit power of the administrator, this made my eye twitch Eric Miller sending stations? Why do some “stations” a bit. The world write bit should be get to transmit at 1MW, while others can used only under rare circumstances. In That’s awesome! It’s great when things like transmit only at one-fourth of this? All fact, I would argue that it should be cover art can make Linux relevant for people the complaints about digital TV signals used only for common storage locations, who otherwise might not care.—Ed. abound, but nobody ever seems to such as /tmp, and even then, it should mention the power used by the sending be accompanied by the sticky bit (that Utility I Wrote (Open Source, stations to send their signals. I believe this is, 1777). In most cases, 700 or 755 of Course) has to be the main reason for fair-to-bad permissions are most appropriate. I’m a longtime reader and fan of LJ, TV receptions of digital signals, not all -- and I thought you might be interested the malarkey you read—for example, Elliott Forney in a tool I wrote for system admins and airplanes, leaves, slight temperature developers named chip. It’s basically changes and so on. I get better reception Sky Survey Using Linux a logfile multiplexer and monitor: in cold, cloudy, rainy weather than on There’s an incredible survey of the entire https://github.com/katzgrau/chip. “nice days”! And, I live more than 60 night sky, using a laptop powered by miles from the transmitters of these five Fedora Linux, and I thought Linux Journal I remember you wrote about swatch stations. Who decides who gets what MWs? readers may be interested: skysurvey.org. (www.linuxjournal.com/article/4776), -- -- and chip basically does what swatch does, R. E. Mackey Jim Gagnon but additionally, it can do it on multiple remote files. That becomes super handy I didn’t know the answer to this, so I How cool! Thanks for the link.—Shawn, when you want to monitor multiple called a friend of mine who manages local the space nut.

10 | august 2011 www.linuxjournal.com Linux to the Rescue! started with the home-schooling community Situation: 1998-vintage Compaq small-form- in Ontario, Canada. What “Baby Penguins” factor desktop (Pentium-II 400MHz) running are you going to target? Fedora Core 5 as a home firewall/DHCP/DNS -- server. After a power failure, it refuses even Dean Anderson to POST. The lights come on, but nobody’s home. I’d buy a used Dell SFF system to You’re totally preaching to the choir here, At Your Service replace it, but I’m looking at several hours of as I’ve been pushing Linux in education for work to install and configure with Fedora more than a decade. (I am the Technology 14. There goes my evening plans, sigh. Director for a school district in Michigan.) MAGAZINE I didn’t know about Japan and coffee, PRINT SUBSCRIPTIONS: Renewing your Brilliant idea: move the old hard disk to the but it’s a great story and encouraging for subscription, changing your address, paying your new system and see what happens, but I’m those of us in the trenches. I do hope the invoice, viewing your account details or other quite skeptical based on my experiences next generation “gets it” when it comes subscription inquiries can instantly be done on-line, with Windows and changing out hardware. to the advantages Linux and open source www.linuxjournal.com/subs. Alternatively, can provide. Great job!—Ed. within the U.S. and Canada, you may call us toll-free 1-888-66-LINUX (54689), or Result: it boots! It works! Nary a hiccup. internationally +1-818-487-2089. E-mail us at I’m back in business in 15 minutes. I got Dropbox [email protected] or reach us via postal mail, my evening back, thanks to Linux. I received the June 2011 issue of LJ today, Linux Journal, PO Box 16476, North Hollywood, CA and as usual, I proceeded to read it cover 91615-9911 USA. Please remember to include your -- complete name and address when contacting us. Jim Garrison to cover on the day it arrived. I immediately noticed your article on Dropbox, as it is an DIGITAL SUBSCRIPTIONS: Digital subscriptions I feel obliged to recommend a system integral part of my daily toolkit. of Linux Journal are now available and delivered as PDFs anywhere in the world for one low cost. upgrade, as FC5 reached end of life back in Visit www.linuxjournal.com/digital for more 2007. I also feel obliged to note how awe- For my part, I find one of the key uses of information or use the contact information above some it is that a system running FC5 still is Dropbox is to turn regular desktop applica- for any digital magazine customer service inquiries. running well! And, you gotta love Linux. It tions into Web-synchronized applications. As LETTERS TO THE EDITOR: We welcome makes most hardware, “just work”.—Ed. many LJ readers do, I work across different your letters and encourage you to submit laptops and desktops, but I would like to them at www.linuxjournal.com/contact or It Starts with the Kids keep some key apps immediately available, mail them to Linux Journal, PO Box 980985, A few years ago, I read a book called The with the same data, no matter where I am Houston, TX 77098 USA. Letters may be edited Culture Code by Clotaire Rapaille. One working, such as time-tracking and my to-do for space and clarity. concept that he explained intrigued me. list. There are many Web-based solutions for WRITING FOR US: We always are looking He described how in the 1970s, Japan’s this, but I found a couple client applications for contributed articles, tutorials and real- consumption of coffee was 0%. If you I prefer to use for their feature sets. world stories for the magazine. An author’s guide, a list of topics and due dates can be were to go to a restaurant and ask for found on-line, www.linuxjournal.com/author. coffee, it was not on the menu. How do For time-tracking, I use Klok 2 you impose a new beverage into a nation (www.getklok.com), and for task lists, ADVERTISING: Linux Journal is a great that does not have coffee in its vocabulary? I use Task Coach (www.taskcoach.org). resource for readers and advertisers alike. Request a media kit, view our current You start by making candies infused with By moving the data files for these applica- editorial calendar and advertising due coffee flavor. Who eats these candies? tions into Dropbox, I immediately have my dates, or learn more about other advertising Children. This is exactly what was done. time-tracking and to-do lists synchronized and marketing opportunities by visiting us They whet a generation’s appetite with caf- across all my working environments. on-line, www.linuxjournal.com/advertising. Contact us directly for further information, feine, and in around the year 2000, coffee Thanks Dropbox for the synchronizing [email protected] or +1 713-344-1956 ext. 2. accounted for billions and billions of Japan’s feature, and thanks to the Linux Journal GDP. What is my point? Next paragraph. team for a great publication. -- Last month, I was honored with the oppor- Randy Earl ON-LINE tunity to expose home-schooling parents WEB SITE: Read exclusive on-line-only content on to a presentation on “Free Software to I use Dropbox in a similar way as well! Linux Journal’s Web site, www.linuxjournal.com. Augment your Child’s Learning”. In this I find it works great to sync Pidgin prefer- Also, select articles from the print magazine presentation, I attempted to cover various ences, so I don’t need to enter all my are available on-line. Magazine subscribers, digital or print, receive full access to issue open-source projects as well as operating account information on each computer I archives; please contact Customer Service for systems that are freely available to everyone. use. I also keep my desktop background further information, [email protected]. If we start to infuse the idea of Linux and photos (all space photos from NASA) FREE e-NEWSLETTERS: Each week, Linux open source in the conscience of the in sync, so they are available on every Journal editors will tell you what's hot in the world generation coming after us, maybe, just computer. Syncing is awesome technology, of Linux. Receive late-breaking news, technical tips maybe, Linux would be synonymously in and I really look forward to open-source and tricks, and links to in-depth stories featured the minds of people when they say “PC”, tools maturing to the point that they on www.linuxjournal.com. Subscribe for free instead of the other OSes available. I’ve rival Dropbox’s abilities.—Ed. today, www.linuxjournal.com/enewsletters.

www.linuxjournal.com august 2011 | 11 UPFRONT NEWS + FUN diff -u WHAT’S NEW IN KERNEL DEVELOPMENT

This edition of diff -u is dedicated to David Brownell, a kernel hacker who XBMC, Now gave a lot his time and creativity to Linux, and a lot of help and encouragement with Less XB! to many open-source developers. Rest in peace, David. May your name linger long in the source tree. Xbox Media Center (XBMC) is Pekka Enberg has been working on a helpful virtualization tool to make one of those projects whose it relatively straightforward to boot a virtual system on a running Linux box. There name makes less and less sense are various virtualization tools available, but Pekka says this one is intended to as time goes on. Sure, people still be a clean, lightweight alternative that anyone could use. Tony Ibbs has released KBUS, available at kbus-messaging.org. It’s a mes- are using XBMC on an actual sage-passing framework, intended to support software running on embedded Microsoft Xbox, but for the most systems. The main idea is that it will be very reliable: a process that dies before part, XBMC now is run on com- it can send a needed message still will have a message sent by KBUS on its puters. In fact, recent versions of behalf; messages flying through the system will all arrive in a predictable order, XBMC installed on an ION-based making their significance easier to interpret; and messages that are sent by one process are as nearly as possible guaranteed to arrive at their destination. nettop makes just about the Dan Rosenberg kicked off a new drive to increase Linux security, when he perfect media center. Version posted some patches to hide the /proc/slabinfo file from regular users. It 10 (Dharma) introduced a fancy turned out that his approach was too much of a blunt instrument, with too plugin system that allows XBMC little gain, for certain folks’ taste. But in the process of debating its various to be extended beyond its built-in merits, a bunch of folks, including Linus Torvalds, dove into the memory allocation code, trying to figure out ways of preventing various security exploits media-playing abilities. The next that had been seen in the recent past. version, currently in development, David Johnston alerted the kernel folks to the idea that some of his code will focus partially on recording as had been included in the kernel, without attributing him as the author and well as playback. without having been licensed by him to be included in the source tree. It When it comes to performance, appeared to be a clear copyright violation. David actually said he’d be pleased and honored to have his code in the kernel; he just wanted to make it’s hard to beat XBMC. It’s faster sure the attribution was correct, and the license issues sorted out. So a few and more responsive than a Boxee folks worked on finding out how the code had gotten into the kernel in the Box, has local media playback first place and how to fix the attribution. These kinds of debates always are unlike the Roku, and is open fascinating, because they put everyone in the awkward position of already source, unlike Microsoft’s media having done something wrong and trying to figure out the best legal and ethical way to back out of it again. center options. It does currently A bit of political wrangling: David Brown tried to remove Daniel Walker lack in premium on-line stream- and Bryan Huntsman from the MAINTAINERS file, as the official maintainers ing, but that’s largely because of ARM/Qualcomm MSM machine support. Bryan was fine with this, but the live version is based on Linux. Daniel had not given his approval. It turned out that Bryan and David were It’s a trade-off I’m willing to both industry folks, and Daniel wanted to remain a maintainer, partly in order to make sure they “did right by the community”. As Linus Torvalds pointed out make. (I actually keep a Roku during the discussion, there was no real reason for any good maintainers to be for that purpose and use XBMC supplanted if they didn’t want to step down. Their contribution could be only for everything else.) positive, and so the argument by the industry folks amounted to an exclusivity Check out the latest features that Linus said he found distasteful. And as Pavel Machek pointed out, the and download a copy for your fact that David had thought it would be okay to do this in the first place was a good enough argument that he wasn’t yet ready to be a sole maintainer. operating system at xbmc.org. —ZACK BROWN —SHAWN POWERS

12 | august 2011 www.linuxjournal.com [ UPFRONT ]

NON-LINUX FOSS

Most people are familiar with Thunderbird. In fact, I used its predecessor back when it was just a part of the Netscape Communicator suite. Thunderbird has come a long way in recent months, however, and it’s still being developed very actively. If you haven’t checked out some of its new features, now might be the time to do so. Thunderbird supports Gmail-style archiving, tabs and better searching, and it’s completely extensible with add-ons. Oh, and if you’re stuck with Windows or OS X, no problem! Thunderbird always has been a cross-platform e-mail solution, and it still is. It makes the transition from one platform to another easier for users, and if you don’t like your e-mail to be kept on some Web company’s servers, it might be the ideal e-mail solution for you. Check it out at getthunderbird.com. —SHAWN POWERS

BOOKS LENS

This screenshot, from the project page, shows the search results for “Dora”.

If you are an Ubuntu user and a fan of the new Unity interface, you might be interested in a new lens in development by David Callé. The Books Lens provides a real-time search interface for e-books. It currently interfaces with Google Books, Project Gutenberg and Forgotten Books. By the time you read this, that list probably will have grown. The Books Lens instantly finds metadata on your search, including book covers, and gives you direct links to the books themselves. For us book nerds, it’s quite a handy little tool. Check it out yourself at https://launchpad.net/unity-books-lens, or install the PPA with: sudo apt-add-repository ppa:davicd3/books-lens

Then, install the package unity-books-lens. —SHAWN POWERS [ UPFRONT ]

velocity. So, you would have two files in the To CFD, or Not to CFD? 0 subdirectory describing these two fields. You also need to set any important physical One area that chews up a lot of cycles on OpenFOAM is library-based, you can properties in files stored in the Dictionaries machines around the world is CFD. What build your own utilities or solvers by using subdirectory. These files end with Properties. is CFD? CFD is short for Computational OpenFOAM as a base. This is very useful This is where you would define items like Fluid Dynamics. The general idea is to in a research environment where you may viscosity. The last step in pre-processing is model the flow of gases and liquids (or be trying something no one else ever has. to set up the control file. This file is named fluids) as they interact with solid surfaces. A model in OpenFOAM is called a controlDict and is located in the system This type of modeling is used in designing case. Cases are stored as a set of files subdirectory. As an example, let’s say you aircraft, automobiles, submarines and fan within a case directory. Many other CFD wanted to start at t=0, run for 10 seconds blades—basically, anything that travels packages use a single file instead. A direc- with a timestep of 0.1 seconds. This section through water or air. As you increase the tory allows you to separate the data from of the control file would look like this: complexity of the surfaces, or increase the the control parameters from the properties. complexity of the flow (such as going Case files can be edited using any text startFrom startTime; from subsonic to supersonic), the amount editor, such as emacs or vi. Pre-processing startTime 0; of computer time needed to model it involves creating all of the files required stopAt stopTime; goes up. One of the big packages avail- for the case you are investigating. stopTime 10; able to do CFD is the suite of programs The first step is mesh generation. deltaT 0.1; made by Ansys. Several groups here at Your fluid (be it a liquid or a gas) is broken my university use it. But, there is an down into a collection of discrete cells, You also set output parameters in open-source option available, OpenFOAM called a mesh. OpenFOAM includes a this control file. You can set how often (www.openfoam.com). This month, I number of utilities that will generate a OpenFOAM writes output with the describe what you can accomplish with mesh based on a description of the writeControl keyword. So, let’s say you OpenFOAM. The OpenFOAM Project geometry of your fluid. For example, the want to write out the results every 10 includes binary packages as deb files blockMesh utility generates simple meshes timesteps. That section of the control file or RPM files. You also can download of blocks, and the snappyHexMesh utility would look like this: a source code package of files or even generates complex meshes of hexahedral download directly from the Git repository. or split-hexahedral cells. If you want to writeControl timeStep; OpenFOAM (Open Source Field generate a basic block mesh, you would writeInterval 10 Operation and Manipulation) basically is lay out the details in a dictionary file, a set of C++ libraries that are used in the called blockMeshDict, in the subdirectory This tells OpenFOAM to write out various processing steps. OpenFOAM, constant/polyMesh within your case results every 10 timesteps into separate just like most other CFD packages, breaks subdirectory. The file starts with: subdirectories for each write. These subdirec- down the work to be done into three tories would be labeled with the timestep. separate steps. The first step is called pre- FoamFile You also can set the file format, the processing. In pre-processing, you define { precision of results and file compression, the problem you are trying to model. This version 2.0; among many other options. involves defining the boundary conditions format ascii; Before you actually start the solver given by the solid objects in your model. class dictionary; step, it probably is a good idea to check You also describe the characteristics of the object blockMeshDict; the mesh to be sure that it looks right. fluid you are trying to model, including } There is a post-processing tool called viscosity, density and any other properties paraFoam that you can use. If you are in that are important for your model. The Case files, in general, start with a your case directory, calling paraFoam will next step is called the solver step. This is header of this format, describing what load it up. Or, you can specify another where you actually solve the equations each case file consists of. In this particular directory location with the command-line that describe your model. The third step file, you can define sections containing argument -case xxx, where xxx is the is called post-processing. This is where vertices, blocks, edges or patches. Once case directory you are interested in. you take a look at the results and visualize you have created this file, you can run the The next step is to run a solver on your them so that you can see what happens in blockMesh utility to process this dictionary model and see what you get. Solvers tend your model. An obvious consequence of file and generate the actual mesh data to be specialized, in order to solve one this breakdown is that most of the com- that will be used in the computation. particular class of problem efficiently. So putational work takes place during the The next step is to set your initial condi- OpenFOAM comes with a rather large set solver phase. The pre- and post-processing tions. This is done by creating a subdirectory of standard solvers out of the box. If you steps usually can be done on a desktop, called 0 and placing the relevant case files are interested in solving a laminar flow while the solver step easily can use up 50 here. These case files would contain the ini- problem for an incompressible fluid, you or 100 processors. OpenFOAM includes tial values and boundary conditions for the likely would use icoFoam. Or, if you are several pre- and post-processing utilities, fields of interest. For example, if you were interested in looking at cavitation, you could along with several solvers. But the real looking at a fluid flowing through a cavity, use cavitatingFoam. Or, you may want power comes from the fact that, because you may be interested in the pressure and a solver for internal combustion engines

14 | august 2011 www.linuxjournal.com [ UPFRONT ]

(engineFoam). Take a look at the documentation to see what is available. They Said It Once you know which solver you are going to use, you can run it by simply If at first you don’t succeed, call executing the relevant binary from it version 1.0. within your case directory, or you can —Unknown use the -case command-line argument to point to another case directory. WE APOLOGIZE FOR THE The last step is the post-processing INCONVENIENCE—God step, where you actually look at the —Douglas Adams, So Long results and see what happened inside and Thanks for All the Fish your model. Here, you can use the supplied utility paraFoam. It can accept All Your Base Are Belong To Us a -case command-line argument, just —Zero Wing (Nintendo Game) like all of the other utilities I’ve discussed. DesktopNova has many options to set, and You then can manipulate the data and Now I am become Death, the a convenient method to set them. look at various time slices. You can destroyer of worlds. generate isosurface and contour plots, —J. Robert Oppenheimer vector plots or streamline plots. You DesktopNova even can create animations, although Klaatu barada nikto. One of the things that frustrates not directly. You can make paraFoam From 1951’s The Day the Earth me about GNOME is the lack of output image files representing movie Stood Still wallpaper rotation integration. frames. So they would be saved with Thankfully, several tools are avail- filenames of the form: able to remedy that shortcoming. My current favorite solution is fileroot_imagenum.fileext READERS’ CHOICE AWARDS 2011 DesktopNova by Stefan Haller. DesktopNova not only lets you Then, you can take this sequence Vote for your Linux and open- of images and run them through source favorites in this year’s add multiple folders to your something like the convert utility Readers’ Choice Awards at wallpaper rotation group, it also from ImageMagick to bundle them www.linuxjournal.com/rc11. optionally allows you to show a Polls will be open August 2–22, together into a single movie file. tray icon to interact with it. 2011, and we’ll announce the As a final comment, in both the DesktopNova comes precom- winners in the December 2011 pre- and post-processing steps, there issue of Linux Journal. piled for several distributions, and are utilities that allow you to convert it already may be in your repository. to and from file formats used by other When combined with the hundreds CFD packages, including fluent, CFX of space photos I’ve downloaded and gambit, just to name a few. This LinuxJournal.com from NASA, DesktopNova makes comes in handy if you are collaborating my computing experience out with other people who happen to This month’s issue is all about com- of this world! (Yes, I think that be using one of these other packages. munity. The Linux and Open Source counts as the cheesiest thing I’ve If you are working with someone community is what keeps all of us written all year.) who is using fluent, you would motivated and having fun with use fluentMeshToFoam in your pre- what we do. These communities Check it out at sites.google.com/ processing step and foamMeshToFluent range in size and scope from a site/haliner/desktopnova. in your post-processing step. small LUG in your hometown, to —SHAWN POWERS This short article provides only a the distributed development teams small taste of OpenFOAM. If you are on the largest Linux distributions, interested in fluid dynamics, you defi- to other Open Source communities nitely should take a look at OpenFOAM. that exist to support projects like Drupal. They are all part of the larger ecosystem The home page includes links to great that is open-source software. We also like to think of LinuxJournal.com as its documentation and several tutorials. own community, and we hope you’ll join us. The installation package also includes Visit www.linuxjournal.com/participate to connect with Linux Journal several example cases, so you can see staff and others, check out our forums and learn about community events. how standard problem sets are handled. We encourage you to chime in with your comments, and let your opinions be You usually can use one of these known. It’s a great way to give and get feedback from our writers and readers. examples as a jumping-off point for We also invite you to visit us on Freenode on our IRC channel, #linuxjournal. your own problem. Check it out and There, you’ll find Linux Journal staff and readers who always are eager to share see if it can help you in your work. stories and sometimes lend a hand. See you there!—KATHERINE DRUCKMAN —JOEY BERNARD

www.linuxjournal.com august 2011 | 15 COLUMNS AT THE FORGE

Introducing CoffeeScript

REUVEN M. LERNER An attractive, new language that makes JavaScript programming fun and easy.

For many years, JavaScript had a bad reputation which then is executed. among developers. Sure, we would use it when we Why would you want to compile a program into needed to do something dynamic in the browser, but JavaScript? First, because modern JavaScript engines we also would avoid it as much as possible. As I (and are quite speedy. Compiling to JavaScript ensures that others) have written numerous times in the past year, the execution will be fast, and that it will work on a however, JavaScript is going through a multidimensional wide variety of platforms and in many contexts—similar renaissance. It executes quickly, and it increasingly is in many ways to the growing number of languages standardized and stable across browsers and imple- (for example, Clojure and Scala) that compile to JVM mentations. You can use it in the traditional role of bytecodes. Second, if you can compile into JavaScript, browser-side applications, either by itself or using a you can take advantage of the libraries and frameworks framework, such as jQuery. You even can build entire that already work with JavaScript. MVC applications in the browser using systems like The other reason to use a language other than Backbone.js, as I described in this space the past two JavaScript, but that compiles into JavaScript, is that you months. And on the server, you can use node.js to can use a syntax that’s more accessible and less prone create high-performance applications. to errors. I’m a longtime user of both Ruby and Python, It’s great that JavaScript has improved in many ways. and I found that CoffeeScript incorporated many of At the same time, the language contains many legacy my favorite features from both languages to create a issues—not in terms of capabilities, but in terms of language with easy-to-understand syntax and powerful the syntax and grammar. You can do great things with features. If you know Ruby, Python and JavaScript, you’ll JavaScript, but it’s easy to write code that has unexpected likely be able to learn this language in a relatively side effects, whose variables don’t have the scope you short time period. expect, or whose functions operate just differently Now, it’s true that CoffeeScript has been around enough from your intention to cause problems. for about 18 months at the time of this writing, and (Another thing that hasn’t improved is its name that it already has attracted a number of fans. But, and the fact that although everyone calls it “JavaScript”, several events have made it even more interesting during it’s officially known as “ECMAScript”, named for the the past few months. First, Brendan Eich, Mozilla’s standards body that approved it.) CTO and the inventor of JavaScript, said earlier this One of the more interesting solutions to this problem year that CoffeeScript may well provide some useful has been the introduction of languages that are supersets syntax ideas for the next version of JavaScript, known of JavaScript. One of the best-known examples is as “Harmony”. Douglas Crockford, well known for his Objective-J, which bolts an Objective-C-like syntax, object ideas, writing and lectures about JavaScript, apparently structure and class library onto JavaScript. The Cappuccino has lent his support to CoffeeScript as well. framework for in-browser MVC applications is written Even more significant, from my perspective, is in Objective-J and provides a programming experience that the Ruby on Rails core team has announced similar to that of the Cocoa platform for the Macintosh. that CoffeeScript will be a standard part of Rails, Another solution has been to run a preprocessor on starting with version 3.1. (Version 3.1 also will feature JavaScript code, as in the case of Google Closure. Closure jQuery as the standard framework, replacing compiles JavaScript into JavaScript, but in so doing, Prototype, and the incorporation of SASS, a CSS optimizes and minimizes your JavaScript code. macros language.) I’m not going to explore the Another approach, taken by Jeremy Ashkenas details of CoffeeScript and Rails here, but the fact (who, incidentally, also created the Backbone.js that a large, growing, active and influential Web framework) was to create an entirely new language framework is adopting CoffeeScript will give that compiles into JavaScript. This language, which he JavaScript language architects like Eich a chance to called CoffeeScript, is not designed to be run directly see how it works “in the wild” and gather feedback (although you can do so, with the right tools). Rather, from developers before deciding what will (and won’t) CoffeeScript code is compiled into JavaScript code, go into the next version of JavaScript.

16 | august 2011 www.linuxjournal.com So, this month, I want to introduce some of the from standard JavaScript. JavaScript, after all, is all basics of CoffeeScript. I must admit I was skeptical of about functions—those that you can pass as parameters, learning yet another new language, especially one that and those that you can execute with parentheses, compiles into JavaScript. But after a bit of playing and much like in Python. You can start to see the differences experimentation, I see why people are so excited about as soon as you define a function though: it. Just where CoffeeScript will be in another few years remains to be seen, but it’s certainly a useful tool to coffee> hello_world = (name) -> console.log "Hello there, #{name}!" have right now, particularly if you’re working on Web applications whose code has become increasingly twisted First and foremost, you can see that defining a and complex with JavaScript functions. function in CoffeeScript is really a variable assignment. The function-definition syntax is still a bit weird by my Installing and Running standards, using -> to separate the parameters and body You can download and install CoffeeScript in a number of an anonymous function. Parameters are named in of ways. One method is to download and compile the parentheses, with commas separating them if necessary. source from the GitHub account on which it is stored In this particular function body, I’m also using and developed. Just say: Ruby-style string interpolation to print the user’s name. If you’re tired of using + or a third-party library git clone http://jashkenas.github.com/coffee-script/ just to interpolate string values, this is likely to be a useful feature for you. and you will have the source on your machine. The CoffeeScript compiler depends on node.js, so it’s not Basic Syntax surprising that you also can download and install it To execute the function, just give the CoffeeScript REPL using the node.js package manager, npm: (read-eval-print loop) the name of the function, followed by parentheses (that is, execution): npm install coffee-script coffee> hello_world() Finally, some Linux packages (of various flavors) exist for CoffeeScript, for those who would prefer to CoffeeScript functions return their final value, install it alongside other packages, with dependencies much as Ruby methods do. In the case of this hello_world taken care of. function, that means the function is returning Once you’ve installed it, you can involve the “undefined”. You can make that a bit better by having interactive command-line prompt, and start coding the function return the string, rather than print it: at the coffee> prompt. You can print something immediately, using the coffee> hello_world = (name) -> "Hello there, #{name}!" built-in console.log function: Then, you can say: coffee> console.log "Hello, world" coffee> console.log hello_world('Reuven') What’s console.log? It’s a built-in function on the Hello there, Reuven! “console” object—for example: or: coffee> console.log.toString() console.log.toString() coffee> hello_world('Reuven') function () { 'Hello there, Reuven!' process.stdout.write(format.apply(this, arguments) + '\n'); } depending on whether you want to get the string back or just print it to the console. As you can see, this lets you take a look at the CoffeeScript provides the basic JavaScript data JavaScript behind the scenes. If you do the same thing types, but it adds a great deal of syntactic sugar, as with the “console” object, you see something that looks well as additional methods, that make those data even more like the JavaScript you know and love (or not): types easier to deal with. For example, CoffeeScript uses Python-style triple-quoted strings. coffee> console.toString() The “existential” operator, a question mark (?), console.toString() allows you to determine whether a variable contains [object Object] something other than null or undefined values. This is better than just checking for the truth of a value, which In some ways, I haven’t yet done anything different will lead to (literal!) false negatives if a value contains

www.linuxjournal.com august 2011 | 17 COLUMNS AT THE FORGE

0 or the empty string. For example: languages. Unlike Python, however, colons aren’t necessary after the “if” and “else” statements. v = 0 CoffeeScript arrays are like JavaScript arrays, but with all sorts of nice syntax added. Want a ten-element if v array? Just use this: console.log "yes" else a = [0..9] console.log "no" and you’ll get it, using the built-in “range” operator. if v? (This doesn’t work for letters though.) You can retrieve console.log "yes" ranges as well: else console.log "no" a[5..7]

The above code example shows how CoffeeScript You can assign to a range, without worrying about implements if/else control blocks. There also is an the precise length of what you’re splicing in: “unless” operator, which (as you might expect) inverts the output from if. CoffeeScript also provides postfix a[5..7] = ['a', 'b'] “if” and “unless” statements—something you might recognize from Perl and Ruby. You also can retrieve substrings using this syntax: You also can see another element of CoffeeScript above, this one taken from Python. Whitespace and coffee> alphabet[5..10] indentation are significant, and allow you to remove alphabet[5..10] much of the curly braces and begin/end of many other 'fghijk'

Silicon Mechanics is proud to announce their sponsorship of Wave Vidmar and the Ocean Row Solo North Atlantic Challenge 2011, a solo row from west-to-east across the North Atlantic.

Expedition Goals Departure: July 2011, Cape Cod Education and research: Wave’s Science Education Adventure Destination: England Program reaches hundreds of thousands of students around Duration: 140 days (estimated) the world. Philosophy Boat “It’s not just one man rowing across an ocean, it’s hundreds Liberty is 24 feet long, with a 6.5 foot beam, and weighs or thousands of people rowing across an ocean. It’s a 1400 pounds fully laden. It is the most advanced, technologi- cooperative effort, that helps push the boundaries and limits, cally equipped ocean row boat in the world. Shaped and giving real-life examples to others in what they too can designed using 3D modeling programs, the boat was tested possibly achieve.” under simulated extreme conditions long before it was ever constructed. Equipment Mission Control “We only work with the best of the best. When you trust Silicon Mechanics and Intel have partnered to donate the your life to equipment and/or services, you can’t afford to Rackform iServ workstation that will run mission control have second-best. We only accept sponsors that have superior functions for the expedition. The Intel® Xeon® Processor product, value, and policies.” E3-1200 Series is at the heart of his customized high-performance system.

For more information about our For more information about Wave Vidmar and Ocean Row Solo, Intel Xeon Processor-based servers visit www.oceanrowsolo.com. visit www.siliconmechanics.com/xeon To track the expedition, visit www.siliconmechanics.com/ors. CoffeeScript strings, like JavaScript strings, are first_name: 'Reuven' immutable. Thus, you cannot assign to a substring: last_name: 'Lerner' coffee> alphabet[5..10] = [5] Once again, using Python-style indentation instead of alphabet[5..10] = [5] curly braces makes it more compact but no less readable. [ 5 ] Loops and Comprehensions coffee> alphabet You can loop through arrays with for..in: alphabet 'abcdefghijklmnopqrstuvwxyz' for number in a console.log number JavaScript objects, which can be used like hashes (aka “dictionaries” or “associative arrays”) work just I should point out that this will print each number as they do in JavaScript, but with some easier syntax. on a line by itself and will return an array of undefined For example, in JavaScript, you would write: values (because console.log doesn’t return a value, and “for” returns an array). person = { Objects can be accessed using the similar for..of loop: first_name: 'Reuven', last_name: 'Lerner' for key, value of person } console.log "key = '#{value}'"

In CoffeeScript, you can say: Note that for..of loops work on any JavaScript object. Because every object can contain properties, person = and because arrays are objects, you even can assign

When you partner with Silicon Mechanics, you get more than affordable, high-quality HPC — you get a team of Experts pulling right along with you. COLUMNS AT THE FORGE

properties to arrays: doing this only when you want, not by mistakenly forgetting to add a “var” somewhere. a.foo = 'bar' Second, properties can be accessed and assigned to as if they were variables, using Ruby-style @varname Using a for..in loop, you’ll continue to see the values syntax. So, when you say x=5 in a CoffeeScript program, in the “a” array. But using a for..of loop, you’ll get not you’re assigning the value 5 to the lexical value x. When only the array elements (whose keys are the indexes), you say @x=5 though, you’re assigning the value 5 to but also “foo”. Additionally, note that if you’re interested the property x on the current object. only in the properties defined on an object, rather than Finally, “this” is scoped dynamically in JavaScript, on its prototype, you can add the “own” keyword: changing value to reflect the object to which the current function is attached. This means that if you aren’t for own key, value of person careful, you can write callback functions that reference console.log "key = '#{value}'" “this”, but accidentally end up referring to the wrong object. CoffeeScript lets you define functions not only In this particular case, there will be no difference. with -> (the “thin arrow”), but also with => (the “fat But if “person” were to have a prototype, and if the arrow”). The difference is that when you define functions prototype were to have keys of its own, you would see with =>, they are bound to the value of “this” when it the prototype’s keys and values, and not just those for was defined, allowing access to the defining context’s the “person” object. properties using @propname syntax. But the need for these for..in and for..of loops All of these things come together in CoffeeScript’s is reduced dramatically in CoffeeScript because of object model, which extends JavaScript’s such that comprehensions, an idea that comes from Python that you can work with something resembling a traditional takes some getting used to, but that offers a great class-instance model, rather than the built-in, deal of power and simplicity once you do so, allowing JavaScript-style prototype-based model. Prototypes still you to combine map, reduce and filter in a single state- exist and work—and indeed, CoffeeScript’s classes ment. For example, you can take a list of numbers: compile into JavaScript prototypes. But, you can get beyond prototype-style inheritance by declaring classes coffee> a = [100..110] with a constructor. Here is a simple example of the [ 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110 ] sort of thing you can do:

And you then can multiply each number by 5, much class Person as you would do with “map” in Ruby or Python, but constructor: (firstName='NoFirst', lastName='NoLast') -> using “comprehension” syntax: @firstName = firstName @lastName = lastName coffee> i*5 for i in a Person.count++ [ 500, 505, 510, 515, 520, 525, 530, 535, 540, 545, 550 ] @count: 0 You also can select out some values. Suppose, for example, you want to operate only on those elements p1 = new Person() that are odd. You can say: console.log p1 console.log Person.count coffee> i*5 for i in a when i%2 [ 505, 515, 525, 535, 545 ] p2 = new Person('Reuven') console.log p2 Objects console.log Person.count As I wrote above, everything in JavaScript (and, thus, in CoffeeScript) is an object, and every object has properties. p3 = new Person('Reuven', 'Lerner') (Another way to think about this is to say that everything console.log p3 in JavaScript is a hash, with key-value pairs.) Understanding console.log Person.count and working with this can be a bit confusing at first, particularly because of the way the “this” keyword When you run the above file, you get the operates. CoffeeScript offers several solutions to this following output: problem, as well as the scoping and variable issues that drive JavaScript programmers crazy. { firstName: 'NoFirst', lastName: 'NoLast' } First, scoping of all variables is lexical and is obvious 1 from the indentation. You can attach variables to the { firstName: 'Reuven', lastName: 'NoLast' } global object, or other objects, but you will end up 2

20 | august 2011 www.linuxjournal.com { firstName: 'Reuven', lastName: 'Lerner' } 3

This not only shows how you can use CoffeeScript classes in a way similar to traditional classes, but also that you can have default values for function parameters by declaring their values before the -> sign. You also can see how naturally the use of the @propname syntax fits into this object model. The above constructor looks almost like a Ruby method, rather than a JavaScript one, with a clear distinction between local variables and instance variables.

Conclusion CoffeeScript is an attractive, new language that makes JavaScript programming fun and easy. You can think of it as JavaScript with a new syntax or as an easy-to-learn language that integrates into JavaScript applications, with many ideas taken from Python and Ruby. It removes many of the syntactic bloat and issues associated with traditional JavaScript, simultaneously providing a language that’s easier to write and maintain, but also faster to execute, than raw, unoptimized JavaScript. CoffeeScript is a language we all should watch in the coming months and years. It probably won’t replace JavaScript altogether, but it definitely will make it easier to accomplish certain tasks. Next month, I’ll look at how CoffeeScript can be integrated into your browser-side Web applications, especially those using jQuery. That’s the direction the Ruby on Rails community seems to be moving toward with its 3.1 release, and even if your favorite framework doesn’t adopt CoffeeScript, understanding how they work together probably will be useful.I

Reuven M. Lerner is a longtime Web developer, architect and trainer. He is a PhD candidate in learning sciences at Northwestern University, researching the design and analysis of collaborative on-line communities. Reuven lives with his wife and three children in Modi’in, Israel.

Resources

The home page for CoffeeScript, including documentation, quick references, FAQs and annotated source code, is at jashkenas.github.com/coffee-script. There is an active and growing community of CoffeeScript users, with an IRC channel (#coffeescript) and Wiki at GitHub.

For a good introduction to CoffeeScript, see the presentation written by Jacques Crocker, available at coffeescript-seattlejs.heroku.com.

Finally, the Pragmatic Programmers have released (at the time of this writing) an excellent pre-release “beta book”, written by active CoffeeScript user Trevor Burnham. If you’re interested in learning more about this interesting little language, I highly recommend this book. It’s mostly aimed at beginners, but given the limited number of advanced CoffeeScript programmers out there, this should not bother you. COLUMNS WORK THE SHELL

Calculating Day of the Week

DAVE TAYLOR Working on the nuances of this day-of-the-week shell script.

For those of you playing along at home, you’ll 8 9 10 11 12 13 14 recall that our intrepid hero is working on a shell 15 16 17 18 19 20 21 script that can tell you the most recent year that a 22 23 24 25 26 27 28 specific date occurred on a specified day of the 29 30 31 week—for example, the most recent year when Christmas occurred on a Thursday. Look! A perfectly formed month, so it’s easy to There are, as usual, nuances and edge cases that figure out the DOW for once. But, that’s not really make this calculation a bit tricky, including the need fair for our testing, so let’s move forward a month to recognize when the specified date has already to June and look at June 3 instead. That’s DOM=3, passed during the current year, because if it’s July DOW=6 (Friday): and we’re searching for the most recent May 1st that was on a Sunday, we’d miss 2011 if we just June 2011 started in the previous year. Su Mo Tu We Th Fr Sa In fact, as any software developer knows, the core 1 2 3 4 logic of your program is often quite easy to assemble. 5 6 7 8 9 10 11 It’s all those darn corner cases, those odd, improbable 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 The solution I’m going to work with is The solution I’m going to work with is likely likely more complicated than necessary, more complicated than necessary, but it’s mine and I’m sticking with it. but it’s mine and I’m sticking with it. Here’s the idea. As awk goes through the lines, it easily can ascertain NF (number of fields). If NF < 7, we have a month where the first day starts on situations where the program needs to recognize and a DOW other than Sunday. Any matching date for respond properly that makes programming a detail- the first week of June 2011, for example, would oriented challenge. It can be fun, but then again, it can have NF = 4. be exhausting and take weeks of debugging to ensure Look back at June though, because it’s important excellent coverage. to recognize that the last week of the month has a That’s where we are with this script too. On problem too. It has NF=5. Because any match in that months where the first day of the month is a line must have DOM > 7, however, we can address Sunday, we’re already set. Give me a numeric date, this nuance later. Stay tuned, as they say. and I can tell you very quickly what day of the week The formula we can use to calculate day of it is. Unfortunately, that’s only 1/7th of the possible week for the first week of a month, however, month configurations. given all this information and with i the day of the month is DOW=i+(7-NF). A few test cases verify What DOW Is That DOM? that it works: For purposes of this discussion, let’s introduce two acronyms: DOM is Day Of Month, and DOW is Day June 3 = i=3, NF=4 DOW=(7-4)+3 = 6 Of Week. May 3, 2011, has DOM=3 and DOW=3, July 1 = i=1, NF=2 DOW=(7-2)+1 = 6 as it’s a Tuesday. May 2 = i=2, NF=7 DOW=(7-7+2 = 2 The cal utility shows this month like this: For any date that doesn’t occur on that first May 2011 week, however, we can ignore all these complicated Su Mo Tu We Th Fr Sa calculations and simply get the day of the week. 1 2 3 4 5 6 7 How do you tell if it’s in the first week? Another

22 | august 2011 www.linuxjournal.com test. Search for the matching DOM and then look at the matching How do you tell if it’s in the first line number. If it’s not line 1, we have to calculate the day of week from the matching cal output line: week? Another test. Search for awk "/$expr/ { for (i=1;i<=NF;i++) the matching DOM and then look { if (\$i~/${day}/) { print i }}}" at the matching line number.

In my previous columns, I was creating this overly complicated regular expression to match all the edge cases (literally, the cases DOW of 2 may 2011 = 2 when the match was the first or last day of a week). Instead, DOW of 16 may 2011 = 2 here’s a new plan that’s faster and less complicated. We’ll use sed to pad each calendar with leading and trailing spaces: Looks good! Next month, we’ll tie this all together. We have a function cal june 2011 | sed 's/^/ /;s/$/ /' that calculates day of week for a given date, we already have figured out how to parse user input to get a desired day of Now our regular expression to match just the specified date week for a specified month/day pair, and we know how to and no others is easy: figure out if the starting point for our backward date search is the current year (for example, whether we’re past that point [^0-9]DATEINQUESTION[^0-9] in the year already).I

Further, awk easily can give us that NF value too, so here’s Dave Taylor has been hacking shell scripts for a really long time, 30 years. He’s the author a rough skeleton of the DOW function for a given day of the of the popular Wicked Cool Shell Scripts and can be found on Twitter as @DaveTaylor and month, month and year: more generally at www.DaveTaylorOnline.com. figureDOM() { day=$1; caldate="$2 $3" expr="[^0-9]${day}[^0-9]" Open Frame Panel PC NFval=$(cal $caldate | sed 's/^/ /;s/$/ /' | \ awk "/$expr/ { print NF }") DOW="$(( $day + ( 7 - $NFval ) ))" Ÿ Fanless ARM9 400MHz CPU } Ÿ 4 Serial Ports & SPI Ÿ Open Frame Design That works if we search only for matches that are in the first Ÿ 10/100 BaseT Ethernet week of the month, but that, of course, is unrealistic, so here’s a Ÿ SSC-I2S Audio Interface better, more robust script: Ÿ 2 High Speed USB 2.0 Ports Ÿ SD/MMC Flash Card Interface Ÿ figureDOW() Battery Backed Real Time Clock Ÿ Up to 1 GB Flash & 256 MB RAM { Ÿ Linux with Eclipse IDE or WinCE 6.0 day=$1; caldate="$2 $3" Ÿ JTAG for Debuging with Real-Time Trace expr="[^0-9]${day}[^0-9]" Ÿ WVGA (800 x 480) Resolution with 2D Accelerated Video 2.6 KERNEL cal $caldate | sed 's/^/ /;s/$/ /' > $temp Ÿ Wide Input Voltage from +12 Vdc to +26 Vdc NRval=$(cat $temp | awk "/$expr/ { print NR }") Setting up a Panel PC can be a puzzling experience. However, the NFval=$(cat $temp | awk "/$expr/ { print NF }") PPC-E7+ Compact Panel PC comes ready to run with the Operating if [ $NRval -eq 3 ] ; then System installed on Flash Disk. Apply power and watch either the DOW="$(( $day + ( 7 - $NFval ) ))" Linux X Windows or the Windows CE User Interface appear on the vivid color LCD. Interact with the PPC-E7+ using the responsive else integrated touch-screen. Everything works out of the box, allowing DOW=$(cat $temp | awk "/$expr/ you to concentrate on your application, rather than building and { for (i=1;i<=NF;i++) { if (\$i~/${day}/) { print i }}}") configuring device drivers. Just Write-It and Run-It. fi For more info visit: www.emacinc.com/panel_pc/ppc_e7.htm /bin/rm -f $temp

} Since 1985 OVER A few quick tests: 25 YEARS OF SINGLE BOARD SOLUTIONS EQUIPMENT MONITOR AND CONTROL DOW of 3 june 2011 = 6 DOW of 1 july 2011 = 6 Phone: (618) 529-4525 · Fax: (618) 457-0110 · www.emacinc.com

www.linuxjournal.com august 2011 | 23 COLUMNS HACK AND /

Nagging Notifications Why let your e-mail do all the nagging? Let screen alert you to Nagios issues.

KYLE RANKIN In the February 2011 issue, I wrote about screen, for the information I needed. Honestly, this same the console window manager, and how I configure its method should work pretty well for just about any hardstatus line to show notifications along the bottom monitoring program you might use, as long as it has a of my terminal window. Although some people like Web interface and you have enough regex-fu to parse their desktop environment to fire up notifications when the HTML for the data you need. they have a new e-mail or IM, because I spend a good I originally wrote the script so that if the Nagios deal of my time within screen, it has my focus, and it status was okay, it would print that, and if there were makes sense to put important notifications there. In any critical or warning alerts, it would output those that February 2011 article (see Resources), I introduced statistics instead. I realized that I wanted screen to print how to set up the hardstatus line and demonstrated a okay in green, warnings in yellow and critical alerts in custom script I use to show when I have new e-mail. red. That way, I might notice problems even if I wasn’t For this article, I expand on the topic of screen looking directly at my terminal at the time. To accomplish notifications with a new notification script I’ve found this, I actually needed to run the script three different incredibly useful. Ever since I’ve had more than a handful times within screen. of servers, I’ve relied on monitoring programs like Nagios The script below takes just two arguments: the to keep track of server health. Although monitoring Nagios host to poll (with an optional user name and software has its own method of notifications via e-mail password if you use one) and the type of status to or SMS, I’ve found it valuable to have my current report. I chose the color codes green, yellow and red to represent okay, warning and critical statuses, respectively. I found the http://nagioshostname/cgi-bin/ Although monitoring software has its own nagios3/tac.cgi page was the simplest to scrape and method of notifications via e-mail or SMS, I’ve had all of the information I needed for the script:

found it valuable to have my current Nagios #!/usr/bin/perl

health right there in my screen session. # usage: nagios_scraper.pl [user:password@]nagios_host STATUS # where STATUS is green, red, yellow, or all Nagios health right there in my screen session. It not only provides a backup to my mail notifications, it also $nagios_host=shift; saves me from having a Nagios window open in my $show=shift; browser all the time. If you are new to screen and haven’t set up a open TAC, "wget --timeout=2 -q -O - custom hardstatus line, check out my February 2011 ¯http://$nagios_host/cgi-bin/nagios3/tac.cgi |"; @tac = ; article first to get up to speed. Instead of revisiting how close TAC; to configure a .screenrc file from scratch, I’m assuming you already have a basic .screenrc set up, and instead, foreach $line (@tac){ I’m skipping ahead to how to add this Nagios script to if ($line =~ /(\d+) Down/){ $hosts_down = $1; } your existing screen session. elsif($line =~ /(\d+) Unreachable/){ $hosts_unreachable = $1; } elsif($line =~ /(\d+) Up/){ $hosts_up = $1; } Screen Scraping for Screen elsif($line =~ /(\d+) Pending/){ $hosts_pending = $1; } When I set about writing this script, I realized there elsif($line =~ /(\d+) Critical/){ $services_critical = $1; } are a number of different ways to capture the current elsif($line =~ /(\d+) Warning/){ $services_warning = $1; } health of Nagios. Although I didn’t spend a lot of time elsif($line =~ /(\d+) Unknown/){ $services_unknown = $1; } looking into it, I imagine there are lower-level APIs I elsif($line =~ /(\d+) Ok/){ $services_ok = $1; } could query, but honestly, all I really wanted was to elsif($line =~ /(\d+) Pending/){ $services_pending = $1; } know if Nagios was all green (okay) or had any } warnings or critical alerts (yellow or red), and if so, how many. To accomplish that, I decided the simplest # remove the username and password from the output method was to scrape one of the Nagios status pages $nagios_host =~ s/.*\@//;

24 | august 2011 www.linuxjournal.com only if it truly needs my attention. if($show eq "green" && ($hosts_down == 0 && $services_critical == 0 Name this script nagios_scraper.pl, put it either in ¯&& $services_warning == 0)){ /usr/local/bin for everyone to use or your user’s ~/bin/ print "$nagios_host: OK"; directory, make sure it is executable, and then test it } against your Nagios server to make sure you have the elsif($show eq "red" && ($hosts_down > 0 || $services_critical > 0)){ syntax right. For instance, if you had no user name or print "$nagios_host: ${hosts_down}D ${services_critical}C "; password set up for Nagios, and your Nagios server } was named naggyhost, you would type the following elsif($show eq "yellow" && $services_warning > 0){ command to test if everything was okay: print "$nagios_host: ${services_warning}W "; } $ /usr/local/bin/nagios_scraper.pl naggyhost green elsif($show eq "all"){ print "${hosts_down}D ${hosts_up}U ${services_critical}C Type the following to test for critical alerts: ¯${services_warning}W ${services_ok}OK"; } $ /usr/local/bin/nagios_scraper.pl naggyhost red

As you can see, I actually collect a lot more statistics Or, type the following to test see all statuses: than I ultimately use, just in case I want to refer to them later. The important thing to note in this script is $ /usr/local/bin/nagios_scraper.pl naggyhost all that in each of the green, red and yellow statuses, I print something only if there’s something of that status I do recommend that you set up a user name and to print. This is crucial, because I don’t want to clutter password for your Nagios Web access if you haven’t my hardstatus line, and I want to see yellow or red text already. Because the user name and password you use COLUMNS HACK AND /

for this script ultimately will end up in plain text, I recommend setting up an account for the Nagios Web interface that can log in but can see only the Nagios status and can’t submit any changes (like maintenance modes and acknowledgements). Let’s assume I set up an account called readonly with a password of n0wr1t3 on naggyhost. I would call the script like this:

$ /usr/local/bin/nagios_scraper.pl readonly:n0wr1t3@naggyhost red

Again, if the script doesn’t provide any output in one of the modes, it could just mean that the status doesn’t currently apply. If you want to test that for sure, run the script with the all argument instead of Figure 1. Everything is okay. green, yellow or red to see the full status.

We Will, We Will, Nag You Once you have tested the script and have it working, the next step is to add it to your ~/.screenrc. The first step is to add three new backtick configuration lines to ~/.screenrc that will call nagios_scraper.pl each with green, red and yellow statuses. In my case, I assume you might have a few backtick commands defined, so I start with command 110:

backtick 110 27 27 /usr/local/bin/nagios_scraper.pl ¯readonly:n0wr1te@naggyhost red backtick 111 61 61 /usr/local/bin/nagios_scraper.pl ¯readonly:n0wr1te@naggyhost yellow Figure 2. One host is down alert with five critical services. backtick 112 73 73 /usr/local/bin/nagios_scraper.pl ¯readonly:n0wr1te@naggyhost green into your existing screen session. Figures 1 and 2 show I’ve set each of these commands to run at different what the hardstatus line will look like either when the intervals. I want to check for critical alerts more frequently status is okay or when there are critical alerts. than warnings or when everything is okay, so I run What amazes me the most the more I dig in to the command with the red argument every 27 seconds. screen notifications is just how simple it is to add new I then run it with yellow and green every 61 and 73 scripts to the list once you get the hang of it. Even if you seconds. Note that I set these intervals to be at odd times. don’t use screen, it wouldn’t be too difficult to modify I’ve realized the value in staggering my screen notification the script so that it outputs to a desktop notification scripts so they don’t risk all running at the same time, so instead (see my December 2009 column for details).I to help with that I try to choose odd intervals. Once you have defined the backtick lines, the Kyle Rankin is a Sr. Systems Administrator in the San Francisco Bay Area next step is to add them to your hardstatus string and the author of a number of books, including The Official Ubuntu Server so they show up in green, yellow and red. In my Book, Knoppix Hacks and Ubuntu Hacks. He is currently the president of the case I pasted in: North Bay Linux Users’ Group.

%{+b r}%110`%{+b y}%111`%{= g}%112` Resources so that my hardstatus string modified from my previous article would be: “Status Messages in Screen” by Kyle Rankin, LJ, February 2011: www.linuxjournal.com/ hardstatus string '%{= w}%Y-%m-%d %c | %l | %101`| article/10950 ¯%{+b r}%110`%{+b y}%111`%{= g}%112`' “Message for You Sir” by Kyle Rankin, LJ, Now save your changes to your ~/.screenrc, and December 2009: www.linuxjournal.com/ either start up a new screen session or type Ctrl-A : article/10612 and type source ~/.screenrc to load these changes

26 | august 2011 www.linuxjournal.com

NEW PRODUCTS

Compuware’s Gomez Platform

Compuware’s integrated application performance management (APM) solution not only has been upgraded but also rechristened the Compuware Gomez Platform. Gomez, says Compuware, “optimizes mobile and Web application performance across the entire application delivery chain, from data centers, through the cloud, to the edge of the Internet”. This update includes new functionality in both Compuware’s on-premise APM products (formerly called Vantage) and its software as a service (SaaS) APM products, now all operating under the Gomez product brand name. New features and capabilities in the new Gomez release include real-user monitoring, mobile-readiness assessment, real-user transaction analytics, integration with Google Page Speed and Internet health map with last-mile data. www.compuware.com

Noble Harmony

Noble Systems has released the next generation (that is, version 2) of Noble Harmony, a browser access tool for remote and mobile contact center management. Version 2 builds on the earlier edition’s flexible, browser-agnostic design, adding improved user-interface layout and customization. The net effect is that managers and supervisors can operate effectively without being tethered to an individual station by allowing them to monitor real-time activities with mobile devices and tablets. More specifically, they now can arrange how KPI information is displayed on the dashboard, speed access to frequently used screens, query a vastly expanded array of variables, set alerts to be triggered on specific activities and parameters, find and analyze individual campaigns and agents, and manage list assignments. www.noblesys.com

Protecode’s Developer Assistant

The folks at Protecode have expanded the reach of Developer Assistant, a tool that the company calls “the industry’s first solution for real-time management of open-source licensing and copyrights”. Previously available only for the Eclipse IDE, the tool now supports all operating systems and all software development tools worldwide. Developer Assistant, says Protecode, operates unobtrusively in the background and requires no training for developers, allowing them to concentrate on development rather than open-source license management. Operating directly at the developer workstation, it uses the code analysis services of Protecode’s Enterprise Server and compares the code structure of a file to signatures of millions of files stored in Protecode Global IP Signatures. There, it identifies the licensing and copyright obligations of the file and provides instant feedback to the developer as the code is put together or brought in from the Internet, an external storage device or a corporate repository. www.protecode.com

Lucid Imagination’s Lucid Certified Developer Exam If your daily grind involves development with Apache Lucene/Solr technology, surf on over to Lucid Imagination to find out more about its new Lucid Certified Developer Exam. The company calls the exam “the industry’s only standardized benchmark of open-source search knowledge, skills and experience”. Created by some of the most esteemed experts and contributors for the Lucene/Solr Project, the Lucid Certified Developer Exam provides developers with a way to validate their expertise and differentiate themselves in the competitive job market, as well as helps employers select new hires with more confidence and more accurately assess knowledge gaps in the current employee base. www.lucidimagination.com/certification

28 | august 2011 www.linuxjournal.com NEW PRODUCTS

Sencha Animator

Sencha is encouraging interactive designers to switch to its new Sencha Animator v.1, “the industry’s first purpose-built tool for creating CSS3-based animations that deliver plug-in-free rich-media experiences”. Designed for interactive designers and advertising creatives who want to create cross-platform animations for playback on WebKit browsers and touchscreen mobile devices, Sencha Animator brings static Web pages to life and ensures high-fidelity playback on all leading mobile devices and desktop browsers. Key product features include full CSS3 support, interactive timeline and intuitive object property controls, an intuitive timeline interface and exporting of animations to readable HTML. www.sencha.com InMage Scout

From the backup and recovery news desk comes breaking news from InMage on version 6 of InMage Scout—a single, easy-to-use solution for local backup and replication of physical and virtual environments that is based on continuous data protection (CDP) technology. InMage claims that the upgraded InMage Scout becomes the only unified product on the market featuring CDP- based backup and remote disaster recovery support for most major enterprise platforms, including Linux, AIX, Windows, VMware, Solaris, XenServer and Hyper-V. The new Scout also features both asynchronous IP and synchronous FC-based data protection without imposing software-based storage virtualization on customers. With this launch, InMage Scout 6.0 also introduces the PureSplit architecture, which allows customers to use their existing storage without having to virtualize it though server appliances. www.inmage.com

Queplix QueCloud

Integrate your disparate cloud-based, SaaS and on-premise applications with Queplix’s new QueCloud, easily and without coding. This can be done using QueCloud, says Queplix, at low cost and with a great deal of automation while avoiding common drawbacks, such as long service engagements, manual processes and the constant wholesale movement of entire datasets. QueCloud cites its Application Software Blades technology that provides automatic connection to data sources, complemented by data virtualization that brings all of the structure of the data sources into the system instantly. From there, the entire data integration process merely involves checking off boxes to align the data and the business rules desired. Once applications are integrated, QueCloud keeps critical customer, product and financial data consistent between applications. Absent from the process are typical elements of ETL technology, such as coding, SQL, complex widgets and wire diagrams. www.queplix.com

Thomas Lah, Steve O’Connor and Mitchel Peterson’s Building Professional Services (Prentice Hall)

Ready to take your world-class skills out of the cubicle and into the big, brutal world? Before you take that jump, sit down long enough to digest the new book Building Professional Services: The Sirens’ Song from Thomas Lah, Steve O’Connor and Mitchel Peterson. There certainly is money to be made, but the pitfalls are also deep and treacherous, say the authors. These leading experts present a comprehensive guide to creating professional services organizations, managing them to maturity and delivering quality services while earning superior margins. Readers will learn how to adopt a complete, practical framework for delivering a full spectrum of professional services—from support and education services to managed, consulting and productized services. Besides emphasizing the key factors that drive success—revenue, references and repeatability—the book covers frameworks for organization, project delivery, solutions development and operational infrastructure. www.informit.com/store

Please send information about releases of Linux-related products to [email protected] or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content.

www.linuxjournal.com august 2011 | 29 NEW PROJECTS Fresh from the Labs Blu-ray Special This month’s article is in a different format from usual; normal service will resume next month. I recently scraped together some cash to buy a Blu-ray drive, and like every other user, I found there was no immediate way to play the disc other than reboot into an OS I won’t mention. Therefore, this month, I cover Blu-ray-specific projects, highlighting individual parts of what later will become a whole in basic playback of Blu-ray discs—I hope. libbluray—Blu-ray Playback Library www.videolan.org/developers/ libbluray.html According to the libbluray Web site: “libbluray is an open-source library designed for Blu-ray discs playback for media players, like VLC or MPlayer. This research project is developed by an international team of developers from Doom9.” The Web site lists the project’s features as follows:

I Portability: currently supported platforms are GNU/Linux, Windows and Mac OS X. Dependencies are very limited.

I Freedom: libbluray is released under a Free Software license, ensuring it will I’m watching a Blu-ray movie under Linux. How? Read through to the end, but I guarantee stay free. you won’t like it.

I Features: the library integrates navigation, playlist parsing, menus and BD-J. If your distro uses sudo: research project provides, through an open-source library, a way to I Legal: libbluray is DRM-circumvention- $ sudo make install understand how the AACS works. free, and thus, safe to integrate in This research project is mainly your software. If your distro uses root: developed by an international team of developers from Doom9. I Fun: libbluray is a fun-to-hack-on $ su NB: this project doesn’t offer project in its starting phase. # make install any key or certificate that could be used to decode encrypted Installation To install libbluray, you The main note here is that “Most copyrighted material. need to grab a binary from somewhere commercial Blu-ray are protected by (like the Ubuntu Personal Package Archive, AACS or BD+ technologies, and this According to the project page, libaacs for instance), or you need to grab the library is not enough to play back those features the following: source. To get the source, you need to discs”. This is simply a library, and you’ll have git installed. Once you have git, need a player like MPlayer or VLC com- I Portability: currently supported plat- enter the following command: piled against it. This is the first of several forms are GNU/Linux, Windows, Mac libraries I’m going to cover, so let’s move OS X. The main dependency is libgcrypt $ git clone on to the next. for all cryptographic functions. git://git.videolan.org/libbluray.git libaacs—AACS Reading I Freedom: libaacs is released under a Once that downloads, change into the Library Free Software license, ensuring it will new directory and compile like so: www.videolan.org/developers/libaacs.html stay free. To quote the libaacs Web site: $ cd libbluray I Legal: libaacs does not include any key $ ./bootstrap libaacs is a research project to or certificate and respects copyright. $ ./configure implement the Advanced Access $ make Content System specification. This I Fun: libaacs is a fun-to-hack-on project

30 | august 2011 www.linuxjournal.com in its starting phase. traces of AACS.” Also according to the docu- Installation The installation follows mentation, here are its key fea- the same procedure as that for libbluray. If tures so far: you’re grabbing the source, enter: I Dual-Core supported $ git clone decryption of EVO/M2TS files git://git.videolan.org/libaacs.git (for hard-disk-to-hard-disk speed records). Change into the new directory with the following command: I Support for every pack type of an EVO (including in-place $ cd libaacs decryption of ADV_PCKs, excluding Sequence Key Then, repeat the next few lines from Sections). This snazzy-looking program is DumpHD. It decrypts HD my libbluray installation instructions. DVD and Blu-ray discs onto your hard drive. I Decryption of every ARF libbdplus—Purpose protection type. Unconfirmed path (/media/INCEPTION, in my case), (URL Unknown) I Multiple files (currently CLI only) as well as to browse for a destination. Perhaps the missing link in basic Blu-ray or complete disc mode. From here, you need to check only playback, I translated this from a French whether you want audio, video or both, Web site: “libbdplus is a library for I Usage of a key database to get the and then click Dump. decrypting protected BD + Blu-ray discs. decryption keys or direct retrieval of Part of my problem appeared to be Currently in development, the project will the keys off the source disc. something to do with external libraries certainly be hosted by the VideoLAN team and helper applications, and their installation if legal clarification is achieved.” I Supports HD-DVDs for and pathing. So far, the documentation However, finding this library seems to Standard/Advanced Content (but available is strewn haphazardly across be a mystery in itself, which I’m guessing not both on the same disc), Blu-ray README files, forums, man pages and so may be kept under wraps for legal ROM BDMV. on, all as separate elements that are hard reasons. On the first page of Googling, I to track down. The same applies for the came across this e-mail from the MPlayer I Experimental Blu-ray recordable support actual program files themselves, which is archives: “Can you send me a link to (with multiple CPS Units, BDMV, BDAV not fun. If anyone wants to help these guys the hidden libbdplus git? Or the source with Aux Directories and Thumbnails). with packaging, hosting, documentation snapshot? Thanks.” (anything to make this a cohesive package), Indeed, I’ve seen references on other I Automatic BD+ removal using the I swear I will cover this program again mailing lists referring to this trio of BDVM Debugger or manually by but in proper detail. libraries being the key to basic playback. supplying a correct Conversion Table Although I couldn’t confirm it with my (currently CLI only). MakeMKV one-and-only Blu-ray disc (and meager www.makemkv.com download limit), it seems that unencrypt- I Streaming output of EVO/M2TS files If you’ve reached this point of the article ed discs can be played back with the first to stdout. filled with despair and just want to watch library, and these last two are needed a film with your new drive, there is one for encrypted discs. I Very much console output for free. last resort: MakeMKV. Before I continue, this is shareware— DumpHD—HD Decrypter I GUI. that horrid software model from the forum.doom9.org/ 1980s and 1990s. Nevertheless, installa- showthread.php?t=123111 Although I don’t have space to cover tion is quite easy, and its GUI integration While I must stress from the outset that the installation properly, I did find this is simply excellent (it managed to include I couldn’t get this to rip my only disc, it program in the Ubuntu repository, and clever program choices when I right-clicked looks to be invaluable nonetheless. Also as this is a trickier program, I hope it’ll be on the titlebar). developed by the Doom9 folks, like libbluray in your distro archive too. Using DumpHD Still, you get only a trial period for Blu-ray and libaacs, DumpHD is a sleek GUI program at least is easy. The only real input required functionality, at which point you have to for decrypting HD-DVD and Blu-ray discs from the user is to browse for the disc’s pay for the program. Okay, you’re thinking, and dumping the files onto your hard disk. According to the documentation: “DumpHD is a HD-DVD/Blu-ray decrypter While I must stress from the outset that based on the idea of BackupHDDVD I couldn’t get this to rip my only disc, it by muslix64. The goal of this program is to make a perfect backup without any looks to be invaluable nonetheless.

www.linuxjournal.com august 2011 | 31 NEW PROJECTS

Hopefully, we as a community can focus our efforts in whatever form we can— hosting, coding, documentation and so on—and reach the same point of maturity as projects like libdvdcss.

massive files. Ripping some- is available at themediaviking.com/ thing that’s 30Gb makes software/bluray-linux. something that’s almost 30Gb And, that’s about as much coverage as as well. I guess this is to make I’m willing to give a shareware program. the quality as 1:1 as possible, MakeMKV: this expensive bit of shareware always gets but I couldn’t find anything to Conclusion the job done in ripping Blu-ray discs to hard drive, but lower the bitrate for storage Linux and Blu-ray may be in a bit of a sorry at quite a price. (half the point of ripping). state for now, but (encrypted) DVD also had Maybe I missed something in many of the same problems in its first few that doesn’t sound too bad. How much is the GUI, but I don’t think so. years. When I first tried DVDs under Linux it? It costs 50 euros. Last time I looked, When you get to the ripping stage, (longer ago than I care to remember), that’s about eleventy-billion in US dollars. you need to choose which title(s) to rip, projects like xine were in their infancies. Only I might pay 50 euros for a commercial and be sure you choose the right ones unencrypted DVDs (of which there weren’t software player, maybe, but for a ripper, on the first go. After a big wait, your many around) could be played, and play- no, sorry. Anyway, I have 30 days of trial MKV should be ready to watch. I hope back methods were primitive. No menus or usage, and I’m desperate just to watch a you chose the right title! navigation, no chapters and individual .vob film with my new drive, so let’s push on. Still, this isn’t watching Blu-ray. It’s files were played one at a time. The Next Head to the Web site’s Download page, ripping it. And, it takes some time too. Chapter button in xine would skip to the where at the top, it has a link to a forum “Can we watch the film at your place, next .vob file (if memory serves correctly). for Linux users. There you’ll find all the honey?” “Certainly. I’ll just rip it first and Now we take Linux DVD playback for information with excellent instructions for guess at which titles are the film and the granted—at least for those of us in countries installation, with links to two tarballs you’ll special features, and hope I get it right where libdvdcss isn’t a problem. And, unless need. The installation went perfectly for on the first go.” Less than cool. I’m using something OSS like VideoLAN, me, and all I had to do was download and Note: just after writing that section, playing DVDs under Windows nowadays is extract the tarballs, open a terminal in each I found a guide on how to stream from actually more of a hassle than when I’m in folder, and copy and paste some commands. MakeMKV to VideoLAN. This saves on my native Linux. So what about Blu-ray? The GUI is intuitive and largely self- ripping time, but it also has jittery perfor- Currently, the projects for decrypting explanatory, but I urge you to make sure mance, and when I tried it with MPlayer, it and playing Blu-ray discs are spread around the output folder is on a partition with was very glitchy. For those wanting to try the Net, disparate and unconnected. The enough free space to accommodate these their luck with this alternative, the guide streaming method mentioned at the end of the last section of this article shows it is at least possible to play encrypted discs, albeit in a bastardized form. Hopefully, we as a community can focus our efforts in whatever form we can—hosting, coding, documentation and so on—and reach the same point of maturity as projects like libdvdcss. If I’ve made any mistakes along the way, I implore you to write in and correct me! I’d rather provide some kind of help with this issue than contribute misinformation.I

John Knight is a 27-year-old, drumming- and bass-obsessed maniac, studying Psychology at Edith Cowan University in Western Australia. He usually can be found playing a kick-drum far too much.

Brewing something fresh, innovative or mind-bending? Send e-mail to [email protected]. The final product: a 27-gig mkv rip that I can finally watch. Worth it? Not sure.

32 | august 2011 www.linuxjournal.com The newly updated LINUX JOURNAL ARCHIVE is here!

ALL 200 ISSUES!

The archive includes all 200 issues of Linux Journal, from the premiere issue in March 1994 through December 2010. In easy-to-use HTML format, the fully searchable, space-saving archive offers immediate access to an essential resource for the Linux enthusiast: Linux Journal.

www.LinuxJournalStore.com A CONFERENCE PRIMER WITH GARETH GREENAWAY Can’t make it to a LinuxFest this year? Host your own! SHAWN POWERS

34 | august 2011 www.linuxjournal.com www.linuxjournal.com august 2011 | 35 FEATURE A Conference Primer with Gareth Greenaway

Every time I attend a Linux conference, I’m reminded of one thing: that we need to host the show comes from I’m so glad I’m not in charge. I have a difficult time getting my family our generous sponsors. Several of them return each year and have a regular presence out the door to go to school on time every morning. The thought of on our show floor, and we greatly appreciate arranging dozens of sessions, hundreds of volunteers and thousands their support. We end up providing various discounts for our attendees to attend the of attendees is just overwhelming. I’ve always been quite certain every show—these range from student discounts conference has a “nervous-breakdown room” where the people in to discounts for various user groups and open-source projects. charge go to rock back and forth in a fetal position a few times a day. Although that might be true for many people, as it would certainly SP: What is the most expensive part of putting on a Linux conference? be for me, my opinion changed when I attended my first Southern GG: There are a few aspects that could California Linux Expo (SCALE) a couple years ago. There certainly was easily qualify for the most expensive part. The first is the show network, which is plenty of chaos to be had, but in the hallway outside the vendor area, utilized by our exhibit floor, our show I met Gareth Greenaway. Gareth was casually talking to the group registration system, our show wireless of us from Linux Journal, sipping on a cup of coffee and wearing networks and audio/video equipment used for monitoring talks. Each of these areas, flip-flops. When I learned that he was one of the people in charge including each booth on the show floor, of SCALE, I figured he already had cracked and had just come from ends up being a separate managed network. The downside is that providing this kind a fetal-ball-rocking session in the nervous-breakdown room. As it of network infrastructure can be very turns out, he’s just really good at what he does. When we decided expensive, while the upside is that SCALE ends up with a rock-solid network. to do an issue on Community, I thought he would be the perfect The other aspect is our audio and person to interview about running a Linux conference. Thankfully, visual equipment. We made the decision a few years back to handle the audio and in his typically calm and casual way, he agreed. visual equipment ourselves, realizing how expensive renting it directly from the SP: What is your job, specifically, when it attendee registration for the show should venue can be. Luckily, we had some very comes to SCALE every year? open, and when the “early-bird” registration talented individuals step forward and fill GG: Currently, I fill two roles within should end and the ticket price should the need. Although we’re handling this the SCALE organization. The first role is increase. We also have several milestones element ourselves, it still can end up being Conference Operations. This role is the for behind-the-scenes tasks, such as the quite an expense, especially with the primary contact between the venue and the various committees submitting their number of speaker sessions that we need SCALE “team”. In this capacity, I also deal budgets for the show, determining the to accommodate now. At the SCALE 9x with several of the vendors that make the placement of booths on the show floor, Friday sessions, we had nine concurrent show happen, such as the booth decorators. signing various contracts and so on. sessions running, each room with its own The second role is that of Community Planning for the show usually begins full A/V setup. Relations. This role is responsible for work- around May or June. Around September ing with the community to get the word is when we typically open up the Call for SP: What is the most difficult aspect out about the show, working directly Presentations. In December, we open up of planning SCALE? And, what’s the with the speakers’ chairs to draw speaker the registration system, with the early-bird most rewarding? submissions from community groups and ticket prices going until around early GG: For me, the most difficult aspect is projects, and directing the Community January. The Call for Presentations closes that the show lasts only a few days. We Relations committee’s efforts to recruit near the end of December or early plan things out for almost a full year, and dotORG exhibitors from the Free and January. Around this time, things really it often feels like it’s over too quickly. Open Source community. get busy for most of the committees. A few years ago, I jokingly said that it [Note: the conference is usually sched- reminded me of a scene from one of my SP: Given a one-year calendar, could you uled to take place toward the end of favorite movies, Brewster’s Millions. The describe the milestones and activities that February each year.] main character inherits millions of dollars take place to prepare for SCALE? If some and has to spend it all to get his full things are planned more than a year in SP: Most conferences charge an entry inheritance. Shortly after he gets his first advance, mention those as well. fee, accept sponsorships and get money inheritance, he has an interior designer GG: The big milestones for the show from vendors for having booths on-site. design him an office, but it’s never quite include things like the dates for the Call Without divulging specific dollar amounts, right. He explains to her that he wants to for Presentations, when the Call for what percentage of funding comes from walk in and say “I want to die in this Presentations should open and when it what sources? office!” Finally in one of the final scenes, should close. Other milestones are when GG: Generally, a good portion of the funds after he’s spent all his money and has

36 | august 2011 www.linuxjournal.com “WE’VE PUT TOGETHER A LIST LINUX PRESENTER TIPS OF THOSE SPEAKERS I People like to look at pictures and listen to speakers. I try to keep my PowerPoint presentations simple and with few words. Not everyone presents that way, but WHO ARE TYPICALLY it works for me.

LATE, AND AS THE I Be excited about your topic, even if you’re not. I just gave a very lively and exciting DEADLINE GETS CLOSER talk at Penguicon about backups, pretty much the most boring topic on the planet. WE SEND OUR GOON I Give 110% to your audience, whether it’s 2,000 people or two people. They came SQUADS TO THEIR to see you, likely at great expense; give them the respect they deserve. HOMES, FORCING THEM I Leave some time for questions if possible. TO GO THROUGH THE I Leave the room when your time slot is over. If there is a crowd, lead them into the SUBMISSION PROCESS!” hallway to be respectful of the next speaker. I Make bullet points short, and elaborate on them when you talk. If you just read nothing, Brewster walks into the office your bullet points to the crowd, the crowd can (and will) do it without you. This and exclaims “I want to die in this office.” needs to be bold, highlighted and repeated. Your talk elaborates your bullet Immediately after that, all the rented points. If you simply read the presentation, it makes for a horribly boring session. furniture is taken back and the office is torn down. This is always how I feel I Don’t be arrogant. Your audience is full of your peers, not your students. Always when SCALE is being packed up. assume there are people in the audience who know more about the topic than There are a few aspects about organizing you do (there usually are). SCALE that I consider especially rewarding. The first being the opportunity to help I It’s important to take questions from the audience, but don’t let the questions spread the word about many of the free (or one audience member) dominate the talk. If questions start to get out of and open-source projects that aren’t able hand, tell the audience you’ll take further questions at the end or after the talk. to be as vocal as some of the others. A few years ago, we invited the creators of the I Fact-check your presentation. Geeks love to correct a presenter, so be careful QIMO Project to attend SCALE as a dotORG with generalizations and look for places in your talk where you might be exhibitor, and they ended up being one of inaccurate. Make sure any sources you use are authoritative and up to date. the more-popular exhibitor booths during the show and gained quite a bit of exposure I Avoid demos unless absolutely necessary. Screenshots, screen-scrape because of being at SCALE. videos and example commands usually can make your point and are less We’ve also given quite a few people likely to break in the middle of your talk. Nothing’s worse for a presenter’s who have never spoken at a conference reputation or an audience’s patience than a broken demo. If you must use a the opportunity to speak. One of my demo, test it thoroughly and have a backup plan in case it fails. Preparing for favorite sessions that we’ve had at SCALE things to go completely wrong is something that you have to do. I gave a talk was a session that took place during our on virtualization at Penguicon a couple years ago. I prepared for the talk and Friday sessions a few years back. The ran through the demo at least four times. But on the day of the presentation, daughter of one of our volunteers, Larry the virtual networking I needed for the talk failed to run. I played it off, spent Cafiero, and two daughters of Fedora/Red a couple minutes getting comments and tips from the peanut gallery in the Hat contributor Karsten Wade, gave a talk crowd on how to troubleshoot it, but then abandoned it, and just went off to on what they liked and disliked about free talk about what was left in the slide. The talk wasn’t as good as I’d hoped, and open-source software. It was a great and it wasn’t what people came to see, but I think it wasn’t a total wreck moment as these three young girls were either, simply because I had enough material and was prepared in the event explaining to a room full of adults, many that it went sideways. of whom were developers, what the software should and shouldn’t do. I Practice your talk at a LUG. Linux Users Groups usually are quite grateful to have speakers and are a great place to practice your talk in front of a similar audience SP: As someone who speaks at confer- before the “main event”. ences, I know I’m horrible at getting talks submitted in time. I can’t imagine I’m the I Try not to take yourself too seriously. There will be mistakes, flubs and people only speaker with that tendency. Do you doing silly stuff during your talk. Be ready to blow off mistakes or joke about find it difficult to arrange quality sessions them. You can turn a bad moment into something that will make the crowd laugh. in a timely manner? They won’t remember the flub, and you come off as very entertaining.

www.linuxjournal.com august 2011 | 37 FEATURE A Conference Primer with Gareth Greenaway

GG: We’ve put together a list of those speakers TIPS FOR ATTENDING who are typically late, and as the deadline gets closer we send our goon squads to their homes, A LINUX EVENT forcing them to go through the submission process! In all seriousness, it can be difficult, I Always plan a backup talk to attend. Sometimes a talk’s summary and we’ve had to resort to extending the call for is written months before the presentation, and often presenters papers a few times. Usually this works to our will completely rework a talk. Other times, the talk just isn’t what benefit though, as we see a flood of submissions you were expecting. Usually you can tell in the first five minutes for really great talks. Unfortunately, the flood whether a talk is worth the time, and if you already have a backup of talks also leaves us with more talks than talk planned, you’ll be able to get there before you miss too we can accommodate in the main conference. much of it. Fortunately, many of those talks end up being great candidates for the SCALE Friday sessions I Attend the “hallway track”. Some of the most rewarding time you’ll or the UpSCALE, our version of Ignite. spend at a conference is the hallway track or the conversations you have with geeks in the hallway outside formal talks. It’s definitely SP: Who is your dream speaker, and why? worth it to get over any social anxiety and strike up a conversation GG: I really have two answers to this one. An with fellow geeks to find out what cool things other people are obvious choice would be Linus Torvalds. I believe up to. there is an unwritten rule somewhere that your event really can’t be considered a true open- I All things being equal, choose an expert. Sometimes you’ll have a source event until you’ve had Linus speak. I’d few different talks you want to attend at the same time. If it’s difficult imagine he’s got a whole blessing he does to decide, add extra points if one of the talks is being given by either too, a big ceremony and probably a closing a known good presenter or the absolute expert on the topic. dance number as well. My other dream speaker isn’t anyone in I Don’t be “that guy”. There’s always the person in the audience particular but all the people who don’t think they who wants to prove he or she is smarter than the presenter and have what it takes to speak at a conference. tries to dominate the talk. You’ll get better answers and a better One of the things I love seeing at events is an presentation from presenters if you actually let them present. If you overflowing room for an unknown speaker. have detailed questions, most presenters are more than happy to field questions after the talk is over. SP: What is the most stressful part of the planning process? I Always throw in at least one “wild card”. It’s easy if you are a GG: The most stressful part is the last few programmer to stay in the developer track, or if you are a weeks leading up to the show—wondering sysadmin to stay in the sysadmin track—after all, that’s why if everything is going to come together and tracks were invented. Be open to mixing it up though, and wondering if anyone will show up. This is choose at least one wild-card talk in a subject you may have also when I start dreaming about all the only indirect interest in. things that could possibly go wrong. Usually the scenarios end up being pretty ridiculous I Silence your phone. This goes for your computer too. It’s really though. One of those scenarios had the show distracting for everyone to hear that Ubuntu startup sound, a taking place on a farm, with speakers giving ringtone or the Angry Birds noises in the middle of a talk. talks in a chicken coop.

I Get your schedule figured out, and plan which talks you want to go to. SP: How many people are involved in putting If you have a smartphone or PDA, put them in your calendar, along on SCALE (not including guest speakers)? with what room they’re in. That way, you’re not scrambling trying GG: There are roughly ten core members that to figure out where you need to be and when. work on the show each year, but that number is growing as more and more people get involved, I If you’re on social media, find the event’s hashtag and which is really great to see, because new Tweet/Facebook-post with the hashtag. Also, search for that members bring new ideas and make sure that hashtag. You’ll wind up finding lots of sidebar events and the show stays fresh and fun. It works very meeting a whole bunch more folks that way. It’s the “hallway much like an open-source project—someone track”, but virtualized. will see a missing or existing element that needs to be added or needs attention, then I If it’s a multi-day event, bring half the clothes and twice the money. he or she jumps in. We’ve had several people Hotel food is expensive, and drink is doubly so. start off simply as volunteers during the show, who are not core members of the team. I Pack a power strip. SP: How did SCALE start? I Be ready to have a lot of fun. GG: Several years ago, I proposed to some

38 | august 2011 www.linuxjournal.com members of the local Linux Users Group Since then, we’ve hosted the show crouching under my desk, reaching up to that we host a one-day event, with the at various locations, expanded the the keyboard to hit the Enter key, because idea of it being a gathering of the local speaker track to five concurrent sessions, I had read horror stories of people blowing LUGs in the area. We had speakers from added specialty Friday sessions and up their monitors. I wouldn’t say I neces- the local LUGs giving presentations on a expanded the exhibitor floor to more sarily miss those days, but I do think it’s variety of topics, and exhibitors included than 100 exhibitor booths. important to realize and appreciate how LUG members showcasing their favorite far things have come. open-source projects and a handful of SP: You must be passionate about Free and open-source software definitely local commercial companies with Linux- Linux to take on something like this. has come a long way. It’s given so many based products. We called the event What’s your story? people so many opportunities to use and LUGFest and ended up hosting the event GG: I actually got involved with free and work with a variety of software that they four times, every six months. open-source software while looking for a otherwise wouldn’t have had access to. While planning the fifth event, a job. I was hunting for a job as a computer member of one of the other local LUGs programmer and came across one that SP: How has the economy dip in recent told me that some students at University sounded interesting. I ended up asking years affected the turnout of both guests of Southern California and University of about their environment and was told that and vendors? California Los Angeles and members of they used UNIX. I had never heard of GG: When the economy dipped a few the USCLUG and UCLALUG, respectively, UNIX, so I started doing some research years ago, we definitely saw that reflected were looking into the possibility of hosting and came across two free UNIXes, Linux in many aspects of the show. For obvious a conference on the USC campus. I made and FreeBSD. I don’t remember exactly reasons, many sponsors and commercial contact with that team, and we ended up why, but I ended up going the Linux exhibitors were more reluctant to fund the meeting, then planning out the first route, and I ended up doing a Slackware show and have their employees host a SCALE event. The first show was held at install. After writing 40+ floppy disks, I booth. We saw fewer companies out at the the Davidson conference center on the had a Linux machine. That definitely was a show recruiting versus other years as well. USC campus, and there were roughly 20 different time. I can vividly remember the One aspect that remains a constant, how- exhibitors and two speaker tracks. first time I set up an X graphical session— ever, is our attendance. Many of us feared FEATURE A Conference Primer with Gareth Greenaway

get everything in writing. And, if you’re LINUX COMMUNITY EVENTS holding an event at a hotel, be sure to account for the 20% service charge that CHECK TO SEE IF THERE’S A COMMUNITY-ORGANIZED EVENT IN YOUR REGION: gets charged to everything! One of the most interesting experiences I Flourish—Chicago, Illinois: www.flourishconf.com that I’ve had planning SCALE was the year the show ended up being the same week- I LinuxFest Northwest—Bellingham, Washington: linuxfestnorthwest.org end that a certain awards show takes place in Southern California. Before the show I Northeast GNU/Linux Fest—Worcester, Massachusetts: began, while hunting around the venue www.northeastlinuxfest.org for a missing package for one of sponsors, I ended up in the room where the awards I Ohio Linux Fest—Columbus, Ohio: ohiolinux.org show tickets were being stored. Following the show, buildings surrounding the venue I Penguicon—Troy, Michigan: penguicon.org were topped with snipers as a form of security for the awards show. I SCALE—Los Angeles, California: www.socallinuxexpo.org SP: What advice can you give to someone I SouthEast LinuxFest—Spartanburg, South Carolina: southeastlinuxfest.org who is (or readers who are?) considering starting a regional event themselves? I Florida Linux Show: www.floridalinuxshow.com GG: Start small, don’t try to do too much the first year. For a more complete list of Linux-related events, including popular corporate-organized events as Make sure roles and responsibilities well as international events, visit www.linuxjournal.com/events. are well defined and well communicated. Nothing is worse than having too many cooks in the kitchen all trying to do the same thing. that with the economy in the position it software or events, but it shouldn’t just Take chances, and allow others to make was that we would see a downturn in the be about the paycheck. mistakes. The event doesn’t need to be number of people who attended. While it absolutely perfect, and it won’t be. didn’t shoot up at all, it certainly didn’t SP: How do you arrange the session Don’t take things personally. People plummet, which was a pleasant surprise. schedule? Is it your fault when two sessions are going to complain; it’s just a fact of In recent years, as a hopeful sign that I want to go to occur at the same time? life. Be polite, and listen to what they the economy is heading back to a more That said, is there a certain methodology have to say. There likely will be some healthier place, we’ve seen an uptick in or software that helps arrange tracks to useful information in the complaints. both sponsorship and attendance. This minimize topic overlap? Open communication is key. It definitely past SCALE (SCALE 9x), we saw quite a GG: We do our best to arrange the talks should be a team effort. jump in the number of attendees that in a way that two talks we think will be came out for the show as well as the popular don’t overlap. Unfortunately, SP: As always, Gareth, speaking with you number of exhibitors. we’re not always right. We’ve definitely was great. Thanks for taking the time to had some talks that were wildly popular, talk to us. And, to LJ readers, if you’ve SP: Many are wondering if there’s a place with attendees spilling out into the hallways ever considered starting a community for a major national show again—a and sitting in the aisles. That being said, event, I’d highly recommend attending LinuxWorld circa 1998—or if the trend is it’s absolutely my fault when there are two one of the regional events in your area. toward regional community-driven events sessions that you want to go to occur at Talk to the people who make conferences like SCALE. What are your thoughts? the same time. happen; they even might be willing to give GG: I think the time for commercially you some advice. And, if you’re ever in driven events has passed, and now it’s SP: Share some experiences or details Southern California in late February, be sure the time for community events. That that would be surprising for those not to come to SCALE! Tell Gareth we sent you. being said, I think having a governing familiar with planning and running a If you’re planning to speak at or attend a organization for national or international Linux conference. Linux event in the future, see the sidebars events could be an interesting and impor- GG: One thing that always surprises to this article for some tips from myself and tant addition. The community events are everyone about SCALE is that everyone editors Kyle Rankin and Bill Childers.I so successful because of the passion for involved is a volunteer; no one is paid the subject that the organizers bring to to work on the show. Shawn Powers is the Associate Editor for Linux Journal. the table. There definitely is a different Advice I would give to others wanting He’s also the Gadget Guy for LinuxJournal.com, and he has an dynamic when people work on some- to plan a similar event would be to start interesting collection of vintage Garfield coffee mugs. Don’t let thing for a paycheck vs. for the love of it. small and definitely not try to do every- his silly hairdo fool you, he’s a pretty ordinary guy and can be I’m not against people being paid for thing yourself. When dealing with reached via e-mail at [email protected]. Or, swing by their work on free and open-source vendors and any venues, make sure to the #linuxjournal IRC channel on Freenode.net.

40 | august 2011 www.linuxjournal.com

Facebook Application Development

Facebook development, for friends and profit. / Mike Diehl

o you have a Facebook page? If you do, you are among previous jobs and some of my son’s activities. I’ve managed to the 600 million users who actively use the social-networking reconnect with my best friend from high school after 20 years. service. I’ll admit I initially resisted the temptation to join I’ve even managed to reconnect with my best friend from the DFacebook. After all, I’ve got e-mail, instant messaging third grade! Since I’m 43 years old, I think that’s quite an and VoIP. What do I need with yet another form of communication? accomplishment. Like I said, it’s very powerful technology. Eventually, temptation got the better of me, and I gave it a try. For But for me, there’s more. I also get links to blogs and news the record, I also experimented with MySpace and Twitter, but those articles from like-minded friends as well as pointers to breaking sites just didn’t seem to do anything for me. These days, every time I news that’s of interest to me. Many times, I’ve heard about events go to my computer, I check my servers, my e-mail and my Facebook. on Facebook before I saw them on TV or read about them in the I never thought it would get to this point, but it turns out that newspaper. Lately, I’ve been investigating ways to use Facebook Facebook is a surprisingly powerful piece of technology. So far, to promote my new business, and I’m sharing some of what I’ve I’ve connected with friends from church, college, high school, found in this article.

42 | august 2011 www.linuxjournal.com First, let’s take care of some of the easy stuff. Everyone’s probably been to a Web site and seen a Facebook “Like” button. When users click on this button, they are asked to log in to Facebook, if they’re not Figure 1. Like Button already. Then, the button click causes a brief message to be added to users’ Facebook pages, indicating that they like the particular Web site. All of the users’ Facebook friends then can see this message. Assuming the user’s full name, you would use something like this: that many of one’s friends share the same interests, this is a very simple way to get a message out to potentially interested people. select name from user where uid=1690577964 Adding a Like button to a Web site is as easy as embedding a little HTML code into your page. Facebook even has a wizard to The resulting URL would like like this: help customize the button: https://developers.facebook.com/ docs/reference/plugins/like. https://api.facebook.com/method/fql.query?query=select When I played with this wizard, I was given code that looked ¯name from user where uid=1690577964 like the code shown in Listing 1. This code results in a Web browser displaying something that resembles Figure 1. Amazingly, your results look like this: Facebook also provides methods that allow a Web site to post to users’ Facebook pages as well as add a person or page to their friends list. These methods are documented at https://developers.facebook.com/docs/reference/dialogs. Mike Diehl Before I go any further, let me pass on a little tidbit of information I came across while researching this article. Facebook supports a primitive chat function that allows users to exchange interactive messages with their friends. This chat function is FQL requires that you specifically list each of the fields in based on the XMMP protocol, which is the same protocol that which you are interested. That means you can’t simply use Google Talk and Jabber use. The details are documented at select * from... as you would be tempted to do in a https://developers.facebook.com/docs/chat. Essentially, you language like SQL. So, it’s important to know what tables Adding a Like button to a Web site is as easy as embedding a little HTML code into your page. need to know your Facebook user name and password. Then, you and columns are available to you. This is well documented prepend your user name to @chat.facebook.com to form a Jabber at: https://developers.facebook.com/docs/reference/fql. ID. When I plugged this information in to Kopete, I was able to At the risk of duplicating what’s already on the Web site, chat with my Facebook friends using Kopete. I also was able to get here’s a list of a few of the more interesting tables that you can real-time updates when my friends logged in and out of Facebook. query (from the Web site): Facebook also provides a powerful method of querying its database for various types of information. Using a language I comment: query this table to obtain comments associated with called Facebook Query Language (FQL), you can ask for all one or more feed comment XIDs. kinds of information about a Facebook user, subject to access restrictions. The easiest way to do this is to use the HTTP GET I connectioncomment: query this table to return a user’s friends access method. This method allows you to execute a query and the Facebook pages to which the user is connected. using a standard HTTP GET. The results are returned in XML format. All you have to do is append your query to the end of I friend: query this table to determine whether two users are this URL: https://api.facebook.com/method/fql.query?query=. linked together as friends. For example, if you had a user ID number and wanted to find I like: query this table to return the user IDs of users who like a given Facebook object (video, note, link, photo or photo album). Listing 1. Like Button Code I notification: query this table to get the notifications for the I status: query this table to return one or more of a user’s statuses.

www.linuxjournal.com august 2011 | 43 FEATURE Facebook Application Development

I stream: query this table to return posts from a user’s stream You can query against quite a few other tables, and I’ve not or the user’s profile. fully explored what information is available. I can tell you that e-mail addresses and phone numbers aren’t readily available using I user: query this table to return detailed information from a this query method. Also, users are able to set permissions that user’s profile. determine what can be retrieved.

Listing 2. Facebook JavaScript Demonstration

1 #!/usr/bin/perl 50 function(rows) { 2 51 document.getElementById("who").innerHTML = 3 ¯rows[0].name + "\'s Minions!"; 4 print < ¯friend where uid1={0} order by uid2', r.session.uid); 9 56 query2.wait( 10 test page 57 function(rows) { 11 58 var i = 0; 12 59 var friends = ""; 13 60 14 61 while (rows[i]) { 15 62 var friend = " ¯src=http://graph.facebook.com/" + rows[i].uid2 17

¯+ "/picture> "; 18 63 friends = friends + friend; 19

64 20 65 if ((i%20) == 19) { friends = friends + "
"; } 21

66 i++; 22

67 } 23

68 24 69 document.getElementById("result").innerHTML = friends; 25 70 } 26 42 function t (response) { 87 43 88

44 if (response.session) { 89 46 90 47 var query = FB.Data.query('select name, uid from user 91 ¯where uid={0}', r.session.uid); 92 48 93 EOF 49 query.wait( 94 ;

44 | august 2011 www.linuxjournal.com That said, you could use these methods in a server-side program to get all kinds of information about a user. But, the real power comes from being able to perform these queries on the client’s computer via JavaScript. (I’ll show you how that’s done, shortly.) Before you can do anything even remotely fun with Facebook’s JavaScript API, you need to become a developer. Becoming a developer is a simple matter of adding the “developers” application to your Facebook profile and filling out a form, which is available at https://www.facebook.com/developers. Once you’ve filled Figure 2. Create New Application out the form and agreed to the Facebook terms of service, you can start adding and configuring your new applications. You’ll be issued an application key for each application you define. (I’ll explain documented at https://www.facebook.com/terms.php. how to use that key shortly.) Now that all the formalities are out of the way, let’s Facebook has received a lot of bad publicity lately because of define our first application. Start the process by going to its often-changing and sometimes poorly implemented privacy https://www.facebook.com/developers/createapp.php, policies. But, from reading its terms of service, which all developers and create a new application (Figure 2). Once you’ve created have to agree to abide by, it appears that Facebook is really your new application, you have to configure it. trying to do the right thing. Some items from the terms of There are a half-dozen pages of configurable options, so service, for example, stipulate that you can’t spam your users. I won’t go into too much detail with them. Most of them are That is, you can’t create pre-canned messages and blast them fairly intuitive and well documented. As long as you understand out to multiple users. You always must provide an “opt-out” the process that Facebook performs when a user accesses for any action your application performs. You’re not allowed to your application, you won’t find this very difficult. Because store personal data for use outside your application and so on. understanding the process is critical, I’ll discuss it a bit before Facebook’s terms of service all seem pretty rational and are looking at some more code.

Develop.Devvelop. Deploy.Deploy. Scale.Scale. FuFullll root access on yyourour own virtual servserverer for as little as $$19.95/mo19.95/mo

0XOWLSOH0XOWLSOH / /LQX[LQX[ G GLVWULEXWLRQVLVWULEXWLRQV W WRR F FKRRVHKRRVH I IURPURP :HEEDVHG:HEEDVHG G GHSOR\PHSOR\PHQWHQW ) )LYHLYH J JHRJUDSKLFDOO\HRJUDSKLFDOO\ GLYHUVHGDWDFHQWHUV'HGLFDWHG,3DGGUHVV3UHPLXPEDQGZLGWKGLYHUVHGDWD FHQWHUV 'HGLFDWHG ,3DGGUHVV 3UHPLXP EDQGZLGWKSURYLGHUVFRUH603;HQSURYLGHUV FRUH603;HQ LQVWDQFHV2XWRIEDQGFRQVROHDFFHVV3ULYDWHEDFNHQGQHWZRLQVWDQFHV 2XW RI EDQG FRQVROH DFFHVV 3ULYDWH EDFNHQG QHWZRUUNIRUFOXVWHULQJ,3IDLORYHUN IRU FOXVWWHULQJ ,3 IDLORYHU VXSSRUWIRUKLJKDYDLODELOLW\(DVLO\XSJUDGHRUDGGDGGLWLRQDVXSSRUWIIRRUKLJK DYDLODELOLW\(DVLO\XSJUDGHRUUDGGDGGLWLRQDO/O/LQRGHV)UHHPDQDJHG'16LQRGHV )UHH PDQDJHG'16

)RUPRUHLQIRUPDWLRQYLVLW)RUPRUHLQIRUPDWLRQYLVLW wwww.linode.comww.linode.com oorr FFDOOXVDWDOOXVDW 6609-593-710309-5933-7103 FEATURE Facebook Application Development

The key concept is that all Facebook applications occupy a space on the Facebook interface known as a canvas. The canvas is where your application’s content will be rendered and is embedded inside the rest of the Facebook look and feel. Essentially, a Facebook application developer simply uses the Facebook JavaScript API and server-side code to perform the application’s function. Thus, you can write your server-side code in any language you like. When users access your application, they use a URL that looks like http://apps.facebook.com/APP_NAME, where APP_NAME is the name given to your application. So, if I created an application called mikestestpage, it would be accessible as http://apps.facebook.com/mikestestpage. You’ll notice that this URL points to a Facebook page. When the user’s browser loads this page, Facebook sends a page back to the user, as expected. However, Facebook also embeds the content of your application’s canvas code. The end result is a seamless integration between the Facebook user experience and your program code. To keep this from getting too abstract, let’s look at the code shown in Listing 2. This is a simple Perl script, but it’s meant to be as language- agnostic as possible. All it does is output some static HTML. Lines 1–13 are simple boilerplate that produce a valid HTTP and HTML header. Line 13 loads the Facebook JavaScript API. I discovered that my application didn’t work unless I included the code in lines 16 and 17; Facebook must use that DOM container internally. The Figure 3. My Minions rest of the code is the fun part. Lines 19–23 create the simple application’s layout by creating a who and a result DOM container. The rest of the code is devoted in lines 57–70 loops over that result set. Each time through the loop, to populating those containers. It’s all done in the JavaScript starting you build up an IMG tag that points to the thumbnail picture of each on line 27. of the user’s friends. On line 65, you ensure that you start a new row The key concept is that all Facebook applications occupy a space on the Facebook interface known as a canvas.

Lines 31–38 are where you initialize the Facebook API by passing of pictures after every 20 pictures are rendered. Finally, you send all it an object that contains, among other things, your application’s of the image tags, en masse, to the result DOM container. The unique identifier. Line 40 calls the getLoginStatus method and passes browser then begins to load the images. it two parameters. The first parameter is a callback function that gets And, that’s the bulk of the application, except for that little piece called when this method returns. The second parameter has to do of magic on line 89. The tag is a tag that Facebook interprets with authentication caching; I found that if I set it to True, my callback before it sends the results to your application’s canvas. In this case, was called twice, which didn’t work very well. you’re asking Facebook to provide you with a Like button that your The main code is in the callback function in lines 42–83. Line 44 users can press to indicate to their friends that your program is worth checks to see if the user is logged in. If not, the else statement block trying. There is an entire class of such tags, known as the Extensible on lines 73–82 is executed. In this simple application, users will be FaceBook Markup Language (XFBML), and I’ve barely scratched the given the opportunity to log in and then must reload the page. More surface of what’s available. This article was meant as a teaser to let sophisticated applications would reload the page for them, of course. you know that this functionality is available. That leaves lines 45–71. Line 47 creates an FQL query, as discussed Figure 3 shows the results of this simple application. earlier. All this query does is get the user’s name. Lines 49–53 use the As you can see, Facebook has a huge user base and provides wait method to get the results and process them using a callback many powerful ways to interact with those users. I’ve demonstrated function that you pass implicitly as a parameter. Here, all you’re doing a couple neat tricks you can use with your own Web pages, as well is creating a string value with the query results and placing it inside as provided an introduction to the Facebook Javascript API. I’ve also the who DOM container. The fact that this method is called wait is a briefly shown how to integrate your application into the Facebook bit of a misnomer. This is not a blocking call. This function will be framework. I hope this has been fun, and I hope it piques your called asynchronously as the results are returned to the client. interest in Facebook application development.I In lines 55 and 56, you set up another query and process the results in another callback function. Only this time, the query selects Mike Diehl operates Diehlnet Communications, LLC, a small IP phone company. Mike lives in all of the current user’s friends by UID number. The callback function, Albuquerque, New Mexico, with his wife and three sons. He can be reached at [email protected].

46 | august 2011 www.linuxjournal.com th 20 USENIX SECURITY SYMPOSIUM San Francisco, CA t August 8–12, 2011 Join us for a 5-day tutorial and refereed technical program for researchers, practitioners, system administrators, system programmers, and others interested in the latest advances in the security of computer systems and networks.

The Program Begins with 4 Days of Training Taught by Co-Located Workshops Include: Industry Leaders, Including: EVT/WOTE ’11: 2011 Electronic Voting • Richard Bejtlich on TCP/IP Weapons School 3.0 Technology Workshop/Workshop on (2 Day Class) Trustworthy Elections, August 8–9, 2011 • Rik Farrow on SELinux—Security Enhanced Linux • Jim DelGrosso on an Overview of Threat Modeling CSET ’11: 4th Workshop on Cyber Security • SANS on Hacker Detection for Systems Administrators Experimentation and Test, August 8, 2011 (2 Day Class) FOCI ’11: USENIX Workshop on Free and And Continues with a 3-Day Technical Program Open Communication on the Internet, Including: August 8, 2011 Keynote Address WOOT ’11: 5th USENIX Workshop on Offensive Charles Stross, Hugo award-winning author, on Technologies, August 8, 2011 “Network Security in the Medium Term: 2061–2561 AD” Paper Presentations HealthSec ’11: 2nd USENIX Workshop on 35 refereed papers that present new research in a Health Security and Privacy, August 9, 2011 variety of subject areas, including securing smart phones, HotSec ’11: 6th USENIX Workshop on Hot understanding the underground economy, and privacy- and Topics in Security, August 9, 2011 freedom-enhancing technologies Plus: MetriCon 6.0: Sixth Workshop on Security • Invited Talks • Poster Session Metrics, August 9, 2011 • Panel Discussions • Birds-of-a-Feather Stay Connected... • Rump Session Sessions (BoFs) http://www.usenix.org/facebook

http://twitter.com/usenix #SEC11

An overview of the LSB, the state of LSB specifications, tools, compliance process and lessons learned.

JEFF LICQUIA, STEW BENEDICT and VLADIMIR RUBANOV

inux is a dynamic and exciting platform, and its presence This is the problem that motivated the creation of the Linux L in the marketplace is now an undeniable fact. But, to an Standard Base (LSB). By reducing the differences between individual outsider, it can be tempting to ask “which Linux?” Given Linux distributions from the application developers’ point of view, the several leading implementations and several variations the LSB greatly reduces the costs involved with porting applications throughout the world, it’s easy to wonder how the platform to different distributions, as well as lowers the cost and effort maintains any consistency. involved in after-market support of those applications.

48 | august 2011 www.linuxjournal.com For many independent software vendors (ISVs), supporting Linux I IBM PPC 64. also becomes a question of “which one?” Either they choose a particular Linux vendor/release and build, test and support that one (or I IBM 31-bit S/390. several). Or, they build against an older distribution, hoping to pick up only “stable” interfaces, shipping their own builds of known problem I IBM 64-bit zSeries. libraries and wish for the best. By defining a known, consistent set of interfaces, the LSB gives ISVs a better chance of building By providing a number of guarantees about a typical Linux dis- an application that will work across multiple distributions. tribution, LSB eases the burden for ISVs wanting to write applica- The LSB consists of the following key components: tions for Linux and support users on their preferred distributions.

I Specification: a set of requirements that compliant applications LSB Tools and distributions must meet. The LSB workgroup provides a number of tools to support the specification. The most important ones are discussed below, and I Tools: a set of tools, tests and informational systems that help Figure 2 presents a general overview of the tools. develop the LSB in a collaborative manner as well as help automate the certification process for applications and distributions.

I Certification: a process to ensure the two-way promise of platform/ application compatibility using the tools and the specification.

The LSB is a core standard for the Linux operating system that encourages interoperability between applications and the platform. Figure 1 shows the components and processes of the LSB.

Figure 2. LSB Tools

The LSB SDK allows application developers to build their applications in a special environment to ensure that all external Figure 1. LSB Components and Processes dependencies of the resulting binaries meet the LSB requirements. The LSB SDK includes special header files that contain only LSB- The LSB specification defines a common set of elements across defined functions and data types, stub libraries that export only multiple Linux distributions that covers the following: LSB-defined symbols, and compiler wrappers lsbcc and lsbc++ that launch underlying GCC components with proper settings to use I Packaging and installing guidelines. the special headers and stub libraries. Using the LSB SDK is easy— one simply should replace calls to GCC with calls to the compiler I Shared libraries and their interfaces. wrappers (for example, by changing CC/CCX variables). Special .pc files are provided for those developers using pkg-config. Also, I Configuration files. integration of the SDK with Eclipse is supported via the LSB Eclipse Plugin, which adds additional project types (LSB executable, LSB I File placement (Filesystem Hierarchy Standard = FHS). shared and static libraries) to the Eclipse environment and allows one to manage all the settings in a visual way. I System commands. The Linux App Checker is used by ISVs and application developers to check their applications for portability across multiple Because it is a binary specification, the LSB is divided into both Linux distributions. The tool is not limited to checking just LSB general and processor-specific components. The following computing requirements; it also enables general portability analysis based on architectures now are supported: an internal knowledge base of distribution contents. Currently, the knowledge base contains information about 70 different distribution I x86 (IA32). versions. App Checker accepts as input a set of components comprising an application’s package: binaries and .sos, possibly in I x86-64 (AMD64/EM64T). various directories or/and packed in .rpm/tar.gz (arbitrary mix) files. Results of the analysis are visualized in interlinked HTML reports. I Intel IA64 (Itanium). For example, external dependencies (libraries and interfaces) of the application as a whole (internal dependencies between compo- I IBM PPC 32. nents are excluded) are visualized with the info of the “portability

www.linuxjournal.com august 2011 | 49 FEATURE Linux Standard Base: State of Affairs

degree” for each dependency and recommendations on how to I Analyze and compare distributions (statistics on Linux elements improve portability. An interesting aspect is that the tool can in each distribution, which distributions provide/miss a particular differentiate between required libraries (registered as DT_NEEDED version of a component, library, command, interface and so on). records in ELF) and libraries actually used by the application. The LSB Distribution Checker is a framework for distribution 3) LSB workgroup services for structuring standardization developers and OSVs (operating system vendors) to check their process and decision support, for example: systems’ conformance to the LSB. It is actually a whole stack of software components. The bottom layer consists of actual tests that I Statistics on LSB evolution (which element appeared/disappeared check specific requirements for particular components in the target in which LSB version). system under test. The majority of these tests are runtime unit tests, but there also are special checkers, including static verification mech- I Analytical data on which popular elements (used by numerous anisms. The middle layer is the framework to control test execution applications and provided by majority of distributions) are not and results collection. And finally, the top layer is the user interface yet in LSB—this serves as a basis for expanding the standard. over all the components. The browser-based interface supports “one button” execution of all certifications tests. Alternatively, the user can I Service data like the test coverage and documentation coverage select specific tests to run, which can be useful for upstream devel- of particular libraries. opers interested in testing only particular components. Note that the LSB Distribution Checker can be used not only for LSB compliance All the tools are integrated—there are links in the Application testing, but also as a general QA testing framework. During devel- and Distribution checkers to pages in the Navigator. Also, the opment of the tests and their regular execution, the LSB workgroup checkers are integrated with the Certification system and allow has identified hundreds of bugs in various distributions and one to submit test data to the Linux Foundation for certification, upstream components, which were reported and acknowledged right from within the checkers. by the upstream and distribution developers. The LSB Navigator is an integrated on-line informational system LSB Certification over a corresponding database that holds structured information LSB certification is a two-way promise of platform/application about LSB elements, about the Linux ecosystem, as well as service compatibility. For distributions, being certified is a guarantee that data used in the standardization analysis and decision making. all required LSB commands and interfaces are available on the Correspondingly, one can distinguish three parts of the Navigator: platform. For ISVs, it is a guarantee that their applications do not 1) Interactive on-line version of the LSB specification, which rely on anything not provided by either the LSB or the ISV itself in might be much more useful in a practical perspective compared its application package or dependent packages. Both distributions to the plain specification text. It provides searchable and browse- and applications can be certified. able information about standardized and non-standardized The vendor submits the test results to the Certification Linux modules, libraries, interfaces, types and so on. For example, Authority (the Linux Foundation). The Linux Foundation reviews given an interface name, one can find out the following in the results and either issues the certification or requests the two clicks: vendor to correct issues and/or resubmit tests with questionable results. The vendor will need to sign a Trademark License I Whether the interface is in LSB. Agreement to use the LSB trademark and pay any necessary fees. The best way to get started with certification is to use the I Recommendations on using the interface or any of its alternatives. checkers—the LSB Distribution Checker for OSVs and the Linux

I Direct links to the interface’s documentation.

I What libraries in which distributions provide the interface.

I How many registered applications use the interface.

I Which open-source tests check the interface (can be used as interface usage examples).

2) Analytical data about popular Linux distributions and applications, for example:

I Interface “popularity” lists—how many applications use each interface (separately for LSB and non-LSB interfaces).

I See what external libraries and interfaces are required and actually used by popular applications.

I See which elements modern Linux distributions consist of (particular versions on various hardware platforms). Figure 3. Auto-Catalytic Effect in the Linux Ecosystem

50 | august 2011 www.linuxjournal.com Figure 4. LSB Release Timeline

Application Checker for ISVs. Once the vendor has addressed any I Best-effort dynamic linking: perhaps the largest difference issues found by the tools, the checkers can upload the test data between an LSB-compliant application and a native Linux appli- to the Certification system to begin the certification process. cation involves the use of the LSB dynamic linker. This has been Certification puts the application or distribution on the Linux used in the past to resolve difficult compatibility problems, Foundation’s official certification list, and it provides marketing allowing the native environment to differ from the LSB while materials for developers to advertise their certification. You still supporting LSB applications. LSB 4.0 debuted a new build also will be allowed to use the LSB trademark/branding in your strategy that allows an LSB application to run using the native packaging/advertising. environment when the LSB dynamic linker is not present. LSB certification secures a number of benefits for the whole Linux ecosystem: Figure 5 shows the list of currently certified LSB 4.0 distributions. LSB version 4.1, released in March 2011, has the following main I App vendors need to do only one port. changes from LSB 4.0:

I Distro vendors get more apps ported to the Linux platform. I Additional symbols: a number of additional symbols have been added based on ISV demand. These are primarily symbols from I End users can choose their distro and their apps without glibc and related libraries, with a few new symbols brought in vendor lock-in . by uplifts of GTK and Cairo.

There is an auto-catalytic effect driven by the LSB (Figure 3). I ALSA: ALSA has been a trial-use module in previous releases. For 4.1, the workgroup has promoted ALSA to a required LSB Releases component and included both shallow and normal-level tests Figure 4 shows the history of LSB releases. LSB 4.0, introduced a for the ALSA interfaces. number of major improvements: I Enhancements to printing: printing enhancements include the I Expanded coverage: seven completely new libraries and addition of ipp/HTTP interfaces from CUPS, as well as a GTK around 1,000 new interfaces were added as compared to uplift that includes the common printing dialog. its predecessor, LSB 3.2. Four completely new test suites and many new tests in the existing test suites expanded the Lessons Learned test coverage significantly. The LSB workgroup has been working on the standard since the early 2000s, moving together with the evolving Linux ecosystem. I Version independence in tools: with the advent of backward- We’ve learned a number of things along the way that are worth compatible standards starting with LSB 3.0, the proliferation of sharing with the community: different LSB versions and the collection of corresponding tools and tests to support them, things were becoming confusing. I Need for upstream documentation: good standards should rely LSB 4.0 introduced version-independent tools and tests that on a good documentation for the elements they define. In the support all LSB versions starting from 3.0, which means users case of LSB, the most burning point is poor upstream docu- can select which LSB version they target when using the new mentation of library interfaces. A number of good interfaces tools, making it easy to move back and forth on the version could not be included in the standard due to this reason. We stack as needed without the need for re-installation. would like to encourage upstream developers to pay attention

www.linuxjournal.com august 2011 | 51 FEATURE Linux Standard Base: State of Affairs

to the documentation quality for their components as it helps develop consistent applications.

I Test what you document: upstream components should have good functional tests that can check cor- rectness of the component’s implementa- tion against the requirements stated in the documentation. Ideal conformance tests should have an explicit linkage between particular checks in the tests and particular requirements in the documentation, thus enabling traceability during testing.

I New methods for developing interface standards: LSB is the biggest interface standard in the world. It embraces dozens Figure 5. LSB Certified Distributions of thousands of elements. Previous interface standardization efforts counted far has been defined largely by the LSB’s role as a trailing standard; fewer elements (for example, POSIX includes just about 1,800 we document and test for the behavior designed and developed interfaces). The huge number of elements makes it mandatory to by the community, rather than guiding the community to the use special methods and tools for developing the standard; other- behavior we design. On occasion, we act as a clearinghouse when wise, it is impossible to manage the complexity. The LSB work- different parties introduce incompatible behavior, but we intentionally group, jointly with ISPRAS (Institute for System Russian Academy avoid “kingmaker” roles even then, preferring to encourage the of Sciences), has developed a number of leading-edge methods parties involved to come to a solution on their own. and supporting tools to make development of the standard sys- Another of LSB’s community benefits is our recently accelerated tematic and manageable. These include automatic generation of testing strategy, which has resulted in the discovery of numerous some parts of the LSB specification and tools, a systematic process compatibility bugs at an early stage, allowing them to be fixed and supporting tools for identifying candidates for inclusion into before they can make it into the wider world. To our knowledge, LSB and moving them through the standardization process stages. we are the only group performing wide multi-distro compatibility testing in the Linux community. I Continuous testing—find problems early: our approach to The challenge, going forward, is to communicate the value of testing has changed with time. In the past, our focus on testing the LSB to the Open Source Development community more clearly. has been limited to ad hoc QA on our own software and the To this end, we have been looking at making better use of social auditing of results from customer tests. At some point, the LSB media, such as Facebook and Twitter, and in generating more workgroup started running its own regular tests of both the statistics about our contributions. enterprise and community distributions, as well as development A number of items on the rolling project plan page could be can- versions of those distributions, and it has been filing bugs with didates for future versions of LSB. The workgroup generally finalizes distributions and upstream projects for found problems. These the plan for the next release at a face-to-face session, weighing the have resulted in quick deployment of fixes before they reach demand/return for the development work to add new modules with the public, both of our tests and of the products being tested. the available resources we have to work on the LSB. Contributions to LSB are always welcome, and if a contributor was to provide a LSB and the Community complete module with specifications and tests, it certainly would The relationship between the LSB and the greater Linux community increase the chances of being added to the next version of LSB.

52 | august 2011 www.linuxjournal.com Conclusion Interoperability often is more about having a forum to resolve The LSB provides a standardized and tested common set of differences rather than any one approach being correct. interfaces for ISVs to target, thus addressing the platform frag- LSB has gone through a long evolution together with the Linux mentation problem. LSB’s goal is to make the ISV’s incremental ecosystem. Looking forward to advancing the standard further, the LSB cost of targeting each additional distro approach zero. The LSB workgroup appeals to the community to participate and contribute to Certification mark is available to distros and apps alike. By the effort to help secure the success of the Linux platform as a whole. enabling ISVs to target the LSB rather than any one distro, the LSB enables portability across distributions. Also, the LSB serves Acknowledgements as a place where distros, upstream maintainers and ISVs can The authors would like to acknowledge all companies, developers, come together to solve problems and prevent regressions. community members, Linux Foundation members and employees, the ISPRAS team and everyone else who contributed to the LSB efforts. Such an industry-wide initiative as LSB would be impossible Resources without the wide support and talented people involved.I

LSB Workgroup Home Page www.linuxfoundation.org/ Jeff Licquia is a software developer at the Linux Foundation. His Linux experience dates to the collaborate/workgroups/lsb early days of Linux, and he has extensive experience bringing Linux into the workplace effectively. He has been working on the LSB since 2005. LSB Downloads: www.linuxfoundation.org/collaborate/ workgroups/lsb/download Stew Benedict is a member of the technical staff at the Linux Foundation. He previously worked as a distribution developer for Mandrake/Mandriva, working on PPC/IA64 ports and various LSB Navigator: linuxfoundation.org/navigator security initiatives. He has been involved with LSB, first as a distribution representative and later as a developer, for several years. LSB Certification System: www.linuxfoundation.org/lsb-cert Vladimir Rubanov is Head of Department at the Institute for System Programming of the Russian LSB Project Plan: www.linuxfoundation.org/en/ Academy of Sciences (ISPRAS) and the Director of the LinuxTesting.org Verification Center. He and ProjectPlan42 his team have been actively involved in the LSB workgroup since 2005. Vladimir holds MSc and PhD degrees in Computer Science from MIPT University (“Russian MIT”).

RACKMOUNT SERVERS Git

Revision control perfected. HENRY VAN STYN

In 2005, after just two weeks, Linus differ significantly from more modern tools like Mercurial. This Torvalds completed the first version of is one of the reasons Git is intimidating to many prospective Git, an open-source version control system. users. But, if you throw away your assumptions of how version control should work, you’ll find that Git is actually simpler than Unlike typical centralized systems, Git is based on a distributed most systems, but capable of more. model. It is extremely flexible and guarantees data integrity In this article, I cover some of the fundamentals of how while being powerful, fast and efficient. With widespread and Git works and stores data before moving on to discuss basic growing rates of adoption, and the increasing popularity of usage and work flow. I found that knowing what is going on services like GitHub, many consider Git to be the best version behind the scenes makes it much easier to understand Git’s many control tool ever created. features and capabilities. Certain parts of Git that I previously Surprisingly, Linus had little interest in writing a version control had found complicated suddenly were easy and straightforward tool before this endeavor. He created Git out of necessity and after spending a little time learning how it worked. frustration. The Linux Kernel Project needed an open-source I find Git’s design to be fascinating in and of itself. I peered tool to manage its massively distributed development effectively, behind the curtain, expecting to find a massively complex and no existing tools were up to the task. machine, and instead saw only a little hamster running in a Many aspects of Git’s design are radical departures from wheel. Then I realized a complicated design not only wasn’t the approach of tools like CVS and Subversion, and they even needed, but also wouldn’t add any value.

54 | august 2011 www.linuxjournal.com Git Object Repository Git, at its core, is a simple indexed name/value database. It stores pieces of data (values) in “objects” with unique names. But, it does this somewhat differently from most systems. Git operates on the principle of “content-addressed storage”, which means the names are derived from the values. An object’s name is chosen automatically by its content’s SHA1 checksum—a 40-character string like this:

1da177e4c3f41524e886b7f1b8a0c1fc7321cac2

SHA1 is cryptographically strong, which guarantees a different checksum for different data (the actual risk of two different pieces of data sharing the same SHA1 checksum is infinitesimally small). The same chunk of data always will have the same SHA1 checksum, which always will identify only Figure 1. An example directory structure and how it might be stored in Git as tree and that chunk of data. Because object names are blob objects (I truncated the SHA1 names to six characters for readability). SHA1 checksums, they identify the object’s content while being truly globally unique—not just to one repository, but to all repositories everywhere, forever. only one level of names and references. Because SHA1s of the To put this into perspective, the example SHA1 listed above referenced objects are within its content, a tree’s SHA1 exactly happens to be the ID of the first commit of the Linux kernel into identifies and verifies the data throughout the structure; a a Git repository by Linus Torvalds in 2005 (2.6.12-rc2). This is a lot checksum resulting from a series of checksums verifies all the more useful than some arbitrary revision number with no real underlying data regardless of the number of levels. meaning. Nothing except that commit ever will have the same ID, Consider storing a change to and you can use those 40 characters to verify the data in every file the file README illustrated in Note: throughout that version of Linux. Pretty cool, huh? Figure 1. When committed, this Git stores all the data for a repository in four types of objects: would create a new blob (with a One potential problem blobs, trees, commits and tags. They are all just objects with an new SHA1), which would require a of the content-addressed SHA1 name and some content. The only difference between them new tree to represent “foo” (with design is that two large is the type of information they contain. a new SHA1), which would require files with minor differences a new tree for the top directory must be stored as different Blobs and Trees (with a new SHA1). objects. However, Git A blob stores the raw data content of a file. This is the simplest of While creating three new optimizes these cases by the four object types. objects to store one change might using deltas to eliminate A tree stores the contents of a directory. This is a flat list of seem inefficient, keep in mind that duplicate data between file/directory names, each with a corresponding SHA1 representing aside from the critical path of tree objects wherever possible. its content. These SHA1s are the names of other objects in the objects from changed file to root, The size-reduced data is repository. This referencing technique is used throughout Git every other object in the hierarchy stored in a highly efficient to link all kinds of information together. For file entries, the remains identical. If you have a manner in “pack files”, referenced object is a blob. For directory entries, the refer- gigantic hierarchy of 10,000 files which also are further enced object is a tree that can contain more directory entries, and you change the text of one file compressed. This operates in turn referencing more trees to define a complete and ten directories deep, 11 new objects transparently underneath potentially unlimited hierarchy. allow you to describe both the old the object repository layer. It’s important to recognize that blobs and trees are not and the new state of the tree. themselves files and directories; they are just the contents of files and directories. They don’t know about anything outside their Commits own content, including the existence of any references in other A commit is meant to record a set of changes introduced to a pro- objects that point to them. References are one-way only. ject. What it really does is associate a tree object—representing a In the example shown in Figure 1, I’m assuming that the files complete snapshot of a directory structure at a moment in time— MyApp.pm and MyApp1.pm have the same contents, and so by with contextual information about it, such as who made the definition, they must reference the same blob object. This behavior change and when, a description, and its parent commit(s). is implicit in Git because of its content-addressable design and A commit doesn’t actually store a list of changes (a “diff”) works equally well for directories with the same content. directly, but it doesn’t need to. What changed can be calculated As you can see, directory structures are defined by chains of on-demand by comparing the current commit’s tree to that of its references stored in trees. A tree is able to represent all of the parent. Comparing two trees is a lightweight operation, so there data in the files and directories under it even though it contains is no need to store this information. Because there actually is

www.linuxjournal.com august 2011 | 55 FEATURE Git

nothing special about the parent commit other than chronology, invoking the git command from within the working copy. There is one commit can be compared to any other just as easily regardless no special server process or extra overhead, and you can have as of how many commits are in between. many repositories on your system as you like. All commits should have a parent except the first one. Commits You can turn any directory into a working copy/repository just usually have a single parent, but they will have more if they are by running this command from within it: the result of a merge (I explain branching and merging later in this article). A commit from a merge still is just a snapshot in time git init like any other, but its history has more than one lineage. By following the chain of parent references backward from the Next, add all the files within the working copy to be tracked current commit, the entire history of a project can be reconstructed and commit them: and browsed all the way back to the first commit. A commit is expanded recursively into a project history in git add . exactly the same manner as a tree is expanded into a directory git commit -m "My first commit" structure. More important, just as the SHA1 of a tree is a fingerprint of all the data in all the trees and blobs below it, the SHA1 You can commit additional changes as frequently or infrequently of a commit is a fingerprint of all the data in its tree, as well as all as you like by calling git add followed by git commit after each of the data in all the commits that preceded it. modification you want to record. This happens automatically because references are part of an If you’re new to Git, you may be wondering why you need to call object’s overall content. The SHA1 of each object is computed, in git add each time. It has to do with the process of “staging” a set part, from the SHA1s of any objects it references, which in turn of changes before committing them, and it’s one of the most common were computed from the SHA1s they referenced and so on. sources of confusion. When you call git add on one or more files, they are added to the Index. The files in the Index—not the working Tags copy—are what get committed when you call git commit. A tag is just a named reference to an object—usually a commit. Think of the Index as what will become the next commit. It simply Tags typically are used to associate a particular version number provides an extra layer of granularity and control in the commit with a commit. The 40-character SHA1 names are many things, process. It allows you to commit some of the differences in your but human-friendly isn’t one of them. Tags solve this problem by working copy, but not others, which is useful in many situations. letting you give an object an additional name. You don’t have to take advantage of the Index if you don’t There are two types of tags: object tags and lightweight tags. want to, and you’re not doing anything “wrong” if you don’t. Lightweight tags are not objects in the repository, but instead are If you want to pretend it doesn’t exist, just remember to call git simple refs like branches, except that they don’t change. (I explain add . from the root of the working copy (which will update the branches in more detail in the Branching and Merging section below.) Index to match) each time and immediately before git commit. You also can use the -a option with git commit to add changes Setting Up Git automatically; however, it will not add new files, only changes to If you don’t already have Git on your system, install it with your existing files. Running git add . always will add everything. package manager. Because Git is primarily a simple command-line The exact work flow and specific style of commands largely tool, installing it is quick and easy under any modern distro. are left up to you as long as you follow the basic rules. You’ll want to set the name and e-mail address that will be The git status command shows you all the differences recorded in new commits: between your working copy and the Index, and the Index and the most recent commit (the current HEAD): git config --global user.name "John Doe" git config --global user.email [email protected] git status

This just sets these parameters in the config file ~/.gitconfig. This lets you see pending changes easily at any given time, The config has a simple syntax and could be edited by hand just and it even reminds you of relevant commands like git add to as easily. stage pending changes into the Index, or git reset HEAD to remove (unstage) changes that were added previously. User Interface Git’s interface consists of the “working copy” (the files you directly Branching and Merging interact with when working on the project), a local repository The work you do in Git is specific to the current branch. A branch stored in a hidden .git subdirectory at the root of the working is simply a moving reference to a commit (SHA1 object name). copy, and commands to move data back and forth between them, Every time you create a new commit, the reference is updated to or between remote repositories. point to it—this is how Git knows where to find the most recent The advantages of this design are many, but right away commit, which is also known as the tip, or head, of the branch. you’ll notice that there aren’t pesky version control files scattered By default, there is only one branch (“master”), but you can throughout the working copy, and that you can work off-line have as many as you want. You create branches with git branch without any loss of features. In fact, Git doesn’t have any concept and switch between them with git checkout. This may seem of a central authority, so you always are “working off-line” unless odd at first, but the reason it’s called “checkout” is that you are you specifically ask Git to exchange commits with your peers. “checking out” the head of that branch into your working copy. The repository is made up of files that are manipulated by This alters the files in your working copy to match the commit at

56 | august 2011 www.linuxjournal.com the head of the branch. Branches are super-fast and easy, and they’re a great way to try out new ideas, even for trivial things. If you are used to other systems like CVS/SVN, you might have negative thoughts associated with branches—forget all that. Branching and merging are free in Git and can be used without a second thought. Run the following commands to create and switch to a new local branch named “myidea”: git branch myidea git checkout myidea

All commits now will be tracked in the new branch until you switch to another. You can work on more than one branch at a time by switching back and forth between them with git checkout. Branches are really useful only because they can be merged back together later. If you decide that you like the changes in myidea, you can merge them back into master: git checkout master Figure 2. gitk git merge myidea However, if there are conflicts—places where the same part Unless there are conflicts, this operation will merge all the of a file was changed differently in each branch—Git will warn changes from myidea into your working copy and automatically you and update the affected files with “conflict markers” and not commit the result to master in one fell swoop. The new commit commit the merge automatically. When this happens, it’s up to will have the previous commits from both myidea and master you to edit the files by hand, make decisions between the versions listed as parents. from each branch, and then remove the conflict markers. To FEATURE Git

complete the merge, use git add on each formerly conflicted file, The fetch line above defines the remote tracking branches. and then git commit. This “refspec” specifies that all branches in the remote repository After you merge from a branch, you don’t need it anymore under “refs/heads” (the default path for branches) should be and can delete it: transferred to the local repository under “refs/remotes/origin”. For example, the remote branch named “master” will become a git branch -d myidea tracking branch named “origin/master” in the local repository. The lines under the branch section provide defaults—specific If you decide you want to throw myidea away without to the master branch in this example—so that git pull can be merging it, use an uppercase -D instead of a lowercase -d as called with no arguments to fetch and merge from the remote listed above. As a safety feature, the lowercase switch won’t let master branch into the local master branch. you delete a branch that hasn’t been merged. The git pull command is actually a combination of the git To list all local branches, simply run: fetch and git merge commands. If you do a git fetch instead, the tracking branches will be updated and you can compare them git branch to see what changed. Then you can merge as a separate step:

Viewing Changes git merge origin/master Git provides a number of tools to examine the history and differences between commits and branches. Use git log to Git also provides the git push command for uploading to a view commit histories and git diff to view the differences remote repository. The push operation is essentially the inverse between specific commits. of the pull operation, but since it won’t do a remote “checkout” These are text-based tools, but graphical tools also are available, operation, it is usually used with “bare” repositories. A bare such as the gitk repository browser, which essentially is a GUI repository is just the git database without a working copy. It is version of git log --graph to visualize branch history. See most useful for servers where there is no reason to have editable Figure 2 for a screenshot. files checked out. For safety, git push will allow only a “fast-forward” merge Remote Repositories where the local commits derive from the remote head. If the local Git can merge from a branch in a remote repository simply by head and remote head have both changed, you must perform a transferring needed objects and then running a local merge. full merge (which will create a new commit deriving from both Thanks to the content-addressed storage design, Git knows which heads). Full merges must be done locally, so all this really means objects to transfer based on which object names in the new is you must call git pull before git push if someone else commit are missing from the local repository. committed something first. The git pull command performs both the transfer step (the “fetch”) and the merge step together. It accepts the URL of the Conclusion remote repository (the “Git URL”) and a branch name (or a full This article is meant only to provide an introduction to some of “refspec”) as arguments. The Git URL can be a local filesystem Git’s most basic features and usage. Git is incredibly powerful and path, or an SSH, HTTP, rsync or Git-specific URL. For instance, has a lot more capabilities beyond what I had space to cover here. this would perform a pull using SSH: But, once you realize all the features are based on the same core concepts, it becomes straightforward to learn the rest. git pull user@host:/some/repo/path master Check out the Resources section for some sites where you can learn more. Also, don’t forget to read the git man page.I Git provides some useful mechanisms for setting up relation- ships with remote repositories and their branches so you don’t have Henry Van Styn is the founder of IntelliTree Solutions, an IT consulting and software development to type them out each time. A saved URL of a remote repository is firm located in Cincinnati, Ohio. Henry has been developing software and solutions for more than called a “remote”, which can be configured along with “tracking ten years, ranging from sophisticated Web applications to low-level network and system utilities. branches” to map the remote branches into the local repository. He is the author of Strong Branch Linux, an in-house server distribution based on Gentoo. Henry A remote named “origin” is configured automatically when a can be contacted at www.intellitree.com. repository is created using git clone. Consider a clone of Linus Torvald’s Kernel Tree mirrored on GitHub: git clone https://github.com/mirrors/linux-2.6.git Resources

If you look inside the new repository’s config file (.git/config), Git Home Page: git-scm.com you’ll see these lines set up: Git Community Book: book.git-scm.com [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/* Why Git Is Better Than X: whygitisbetterthanx.com url = https://github.com/mirrors/linux-2.6.git [branch "master"] Google Tech Talk: Linus Torvalds on Git: remote = origin www.youtube.com/watch?v=4XpnKHJAok8 merge = refs/heads/master

58 | august 2011 www.linuxjournal.com

INDEPTH

Podcasting If you want to contribute to the Linux community but would rather talk than develop software or write documentation, podcasting may be the ideal approach for you. CHARLES OLSEN

If you like to talk and have something interesting to say, you may additional hosts means that you don’t have to know everything, enjoy podcasting. A podcast is simply an audio file in MP3 or Ogg and you don’t have to do all the talking. format that people can download and listen to on their own schedule. In the first few episodes of mintCast, each host recorded sepa- A few years ago, I created a podcast called mintCast, which rately, then we edited the recordings together. In later episodes, was designed to help new users get started with Linux. I had been we recorded at the same time, allowing us to converse and play listening to several podcasts, but I knew nothing about producing off each other. Doing it that way made the shows more interesting, one myself. This article summarizes what I learned during the and it was easier for the hosts to keep going. It did, however, add following two years. It should save you some time and pain, if complexity to the process of recording the show. you decide to create a podcast. Entire books have been written about podcasting, and you Recording should read some of them if you want to study the subject in There are many ways to record a podcast. Sometimes I used a small depth. To keep this article to a manageable length, I focus on handheld recorder that saved the recordings in MP3 format and had what I learned while producing mintCast. a USB port that allowed me to copy the files over to my PC. Most of the time, I used Audacity to record the show. Audacity Plan is an open-source program that can record, edit and play audio The first step is to choose your topic and target audience. If your files. You can install it from your distro’s repositories. topic is too broad or too narrow, it will be hard to build an audience. A good quality microphone makes a big difference when you’re Going Linux has a great example. The home page describes recording a podcast. It doesn’t need to be expensive, but do some the target audience (“New to Linux, upgrading from Windows research and read reviews before choosing a microphone. to Linux, or just thinking about moving to Linux?”) and what it A headset may seem to be a good choice, and that’s what provides (“practical, day-to-day advice on how to use Linux and I worked with. It causes problems, though—unless you’re very its applications”). Potential listeners can see at a glance if this is disciplined about always keeping your hands away from your head, a podcast they want to listen to. it’s very easy to bump the headset, making an ugly noise on the A podcast episode usually has multiple segments. Typical seg- recording. A microphone boom can help avoid this. ments might include an introduction, personal news, industry news, Although Audacity has a lot of power and features, it’s very opinions and commentary, interviews, tutorials, tips, recommendations easy to use. The default settings are fine for recording spoken of software or Web sites, and listener feedback. Whichever segments voice. You can tweak the input settings by watching the meter you decide to use, it’s a good idea to stay consistent. while speaking a few words, then click the big red Record button That doesn’t mean you need to be absolutely rigid. A typical when you’re ready to start. episode of mintCast would have these segments: introduction, per- It becomes more complicated if you have two hosts and you’re sonal news, Linux news, main topic, Web site recommendation and not sitting in the same room. On mintCast, we solved this problem listener feedback. The main topic would be a tutorial, interview or by talking on the telephone, and each of us recorded our own audio discussion about a new distro release. This approach gave us a con- in Audacity. We could then assemble the recordings as side-by-side sistent format, with enough variety to keep it fresh and interesting. tracks in Audacity. This gave us a very high recording quality. You should write an outline before you start recording. Include When more hosts joined the team, or when we did user the topics you want to talk about and the points you want to interviews, we needed to get three or four people on the call at make. If your show has multiple hosts, Google Documents is an the same time. We did not have the ability to do conference effective way to share your notes, because all the hosts can read calls from our personal phones, and we didn’t want to ask our and even edit the document. interview subjects to try to use Audacity. Do not write a script of exactly what you want to say, and do We tried Skype and Skype Call Recorder, but never managed not read directly from your notes. It’s very obvious to listeners to get that working. I know other podcasts use this software, and when you do this, and it sounds bad. It’s acceptable to read a I certainly would try it again if I were doing a podcast now. brief quotation, for example, when you’re reading from a news Some podcasts use the Talkshoe Web site. Once you’ve created item. Otherwise, just write out the points you want to make, a free Talkshoe account, you can create a call series and invite and use the list only as a reminder as you’re speaking. others to the calls. You can allow them to participate fully or just Will you work alone, or with other hosts? I always found it listen. Calls can be recorded and then are available to listen to or easier to record a show with one or two other hosts. Having download at any time.

60 | august 2011 www.linuxjournal.com INDEPTH

You can listen to Talkshoe calls on your computer or telephone. song at the beginning of the show, a longer version of the theme You can participate by calling in from a telephone or by using for the end of the show, and a five-second clip from the theme Skype or Ekiga. Talkshoe also sets up a chat room, allowing listeners that we used as a bumper between segments. I put all of those on a live show to interact with the hosts. clips into the audio track. Talkshoe can be your start-to-finish solution to podcasting. Audacity allows you to select part of the audio, then cut or It allows you to promote, broadcast, record and publish your copy and paste it. A scrolling display shows a waveform of your recordings. Why, then, should you bother with any of the other audio for each track, making it easy to select exactly the part of tools and processes in this article? the clip you want to work with. You can zoom in or out on the For mintCast, I could not bring myself to publish our shows waveform display. After doing a few shows, I could recognize my without a considerable amount of editing. Each of the hosts had “uhs” from the waveform before I even heard them. a verbal tic—for example, frequently saying “uh”, “you know” or If your audio is in multiple tracks, be careful about deleting “basically”—that was too embarrassing to leave in the recording. material. If you delete a clip from one track, you’ve now changed In Talkshoe, the audio is published exactly as it is recorded. the timing for the rest of that track, and it will be out of sync with We also wanted to edit in theme music at the beginning and the other tracks. If you delete a clip, be sure to select and delete that end of the show. We couldn’t do that with Talkshoe. time frame from all tracks so that they will stay in sync. Audacity Still, we often used Talkshoe to record the show. I then would allows you to click and drag across all of the tracks if you need to. download the recording and do the necessary editing before But, if you need to remove unneeded noise from one track publishing it to our own Web site. while a host is talking on another track, you don’t want to delete The mintCast team now is using Mumble to record shows. it. Just select the noise in the waveform, then click the Generate Mumble is a free and open-source voice chat program. It uses a menu and choose Silence. This will silence that part of that track, nonstandard protocol, so you need to set up a Mumble server to without affecting that track’s timing compared to the other tracks. which the Mumble clients can connect. With Mumble server and Audacity has several useful tools on the Generate and Effect client version 1.2.3, you have the ability to record the Mumble menus. If you want to remove a sound without silencing it, you call. You then can load the recording into Audacity for editing.

Recording Tips Before making your first recording, it’s a good idea to do a test recording for at least five to ten minutes. Make sure your recording levels are good, and that it sounds like you expect. Listen for background noise—things that you normally don’t even notice, like ceiling fans and air conditioners, might interfere. Even low-quality microphones seem to pick up everything you don’t want on the recording. If you’re eating, drinking or typing, listeners will know it. Make sure you know where your mute button is. If your microphone doesn’t have a mute button, leave the sound control panel open on your PC and find the mute control. A lot of things can happen that you don’t want on the recording, and the mute button can save you a lot of tedious editing. Don’t worry about making a perfect recording. Unless you’re using Talkshoe to publish the podcast exactly as recorded, you can edit out most mistakes. We often found ourselves rambling or stumbling over our words, and then we would take a moment to collect our thoughts and then go back and start over at the beginning of the sentence or paragraph. These “do-overs” helped us produce a better show.

Editing Once you have the show recorded, the best tool for editing is Audacity. It’s in the repo of every major distro, and it’s very easy to use, even if you have no previous experience editing audio. If you recorded in Audacity, you’re ready to start editing. If you recorded with some other tool, Audacity can open files in any audio format. If you have each host’s audio in a separate recording, you can add new audio tracks and paste each recording into a separate track. I also like to add a separate track for music. We had a theme

www.linuxjournal.com august 2011 | 61 INDEPTH

also can select the clip and Generate a tone, a chirp or a noise that sounds like static. Linux Podcasts It was not unusual for our initial recording levels to come out way too low. The Effect menu includes an Amplify command, Going Linux (goinglinux.com): for computer users who just allowing us to select an entire track (or any portion of it) and want to use Linux to get things done. amplify the volume. If the volume varies too much in different parts of the recording, Linux Action Show (www.jupiterbroadcasting.com/ the Compressor command on the Effect menu can bring the levels show/linuxactionshow): video broadcast with Linux news closer together. The volume will be increased for the quiet parts and reviews. and reduced for the loud parts. If you do have a consistent noise in the background, such as Linux Outlaws (www.linuxoutlaws.com): two pragmatic Linux an air conditioner, the Noise Removal command on the Effect users talk about the latest developments in free and open menu often can remove it. You need a few seconds of audio software and culture (warning: explicit language). where you’re not talking, where the only sound is the background noise you want to remove. When you click the Noise Removal Linux Reality (linuxreality.com): an excellent podcast, no command, a dialog box pops up explaining how to use it. longer actively produced but still available for download. Save often as you work. Although crashes were not frequent, Audacity did crash more than any other program I’ve used in Linux. mintCast (www.mintcast.org): generally focuses on helping Finally, Audacity can export the finished file to the appropriate users get started with Linux. audio format. Podcasts are typically MP3, though many also are offered in Ogg format. The Linux Link Tech Show (www.tllts.org): a live weekly Webcasted audio show dealing with the Linux computer Music operating system and any other geeky subjects that the Remember that all those CDs and MP3s you’ve purchased over hosts find interesting at the time. the years are copyrighted—you can’t use those songs in your show. However, there are sources where you can get music TuxRadar (tuxradar.com/podcast): news, reviews, rants, raves, without spending thousands of dollars. chit-chat, geekspeak and more. The Podcast Themes Web site has music specifically created for podcasts. A few songs are available for free, and many others are Ubuntu UK (podcast.ubuntu-uk.org): all the latest news and available at a low cost. Various licenses allow you to buy exclusive issues facing Ubuntu Linux users and free software fans in general. or non-exclusive rights to the music. Unique Tracks is another Web site that offers royalty-free Many more podcasts about Linux, or that frequently cover music. They offer individual songs and albums in many different Linux topics, exist. You can find dozens more podcasts at styles: classical, uplifting, rock, heavy metal, hip-hop, New Age the Linux Link: www.thelinuxlink.net. and many others. Many of the songs include the full-length song as well as shorter versions, and some even include a loop of five to ten seconds, which is ideal for a bumper. listeners can discuss the show. On simpler sites, they may be able to leave comments only on the blog post. If nothing else, at least Publishing get a Gmail account and publish the address. Once you have a completed recording, you need to publish it Read the comments and consider them seriously, especially the so that others can download it. If you recorded in Talkshoe, it’s negative comments. I know it’s painful to read criticism after you’ve already published. Even if you recorded in Talkshoe, you still can put so much work into the podcast, but those criticisms often can publish it to your own site. help you make a better podcast. Encourage listeners to write in with Any Web-hosting company will offer a template for publishing a comments and suggestions, and thank them for their feedback. podcast. If your site uses WordPress, you’ll need the PodPress plugin. In any case, you’ll have a Web location to upload the finished files. Conclusion Then, create a blog entry on your site and link the audio files. When I first started using Linux, the podcasts I listened to really Although your site probably already offers a feed, you can use helped me a lot. I was reading books and magazines, of course, Google FeedBurner to get information allowing you to analyze but listening to a friendly voice really helps make a new topic your traffic. Once you’re set up in FeedBurner, it’s a good idea to accessible. It’s also nice to be able to learn new things while submit your podcast to the iTunes store, the Zune Marketplace driving to and from work. and Podcast Alley. All of those will make it easier for people to If you like to talk and know a thing or two about Linux, find your podcast. why not help others increase their mastery of Linux?I

Feedback Charles Olsen has been working in IT help desk and technical training for more years than he will Once you have listeners, some will give you feedback about the admit. For two years, he was lead host of mintCast, a podcast by the Linux Mint community show. It’s best if your Web site is set up with a forum where for all users of Linux. You can find mintCast at www.mintcast.org or, if you must, in iTunes.

62 | august 2011 www.linuxjournal.com Learn Drupal & jQuery FROM THE COMFORT OF YOUR LIVING ROOM

How to Build a Beowulf HPC System Using the FedoraLiveCD Project Build a Beowulf cluster without disks to optimize cost and reliability, and simplify software maintenance. HOWARD POWELL

The FedoraLiveCD Project allows anyone to create a custom nodes will run entirely from an initial ramdisk, so they will bootable CD or PXE boot image with little effort. For large HPC exhibit very good I/O for normal OS disk operations. systems, this greatly simplifies the creation of diskless compute The cluster I’ve designed is set up for MPI-based computation. nodes, leading to higher reliability and lower costs when The master node runs a queue system where jobs can be submitted designing your cluster environment. The network and CPU and farmed out to the compute nodes to run within the allotted overhead for a diskless setup are minimal, and the compute resources. Because my compute nodes are diskless, the goal is to

Listing 1. Example dhcpd.conf File

# match if substring(option vendor-class-identifier, 0, 9) = # DHCP Server Configuration file. ¯"PXEClient"; # see /usr/share/doc/dhcp*/dhcpd.conf.sample next-server 10.1.0.1; # filename "pxelinux.0"; ddns-update-style interim; allow booting; } allow bootp; option dns-domain-search-list code 119 = string; host c0 { subnet 10.0.0.0 netmask 255.255.0.0 { hardware ethernet A4:BA:DB:1E:71:2D; default-lease-time 604800; fixed-address 10.1.0.254; max-lease-time 1209600; option host-name "c0"; option routers 10.0.0.1; } option ip-forwarding off; option subnet-mask 255.255.0.0; host c1 { range dynamic-bootp 10.0.0.100 10.0.0.254; hardware ethernet A4:BA:DB:1E:71:3A; } fixed-address 10.1.0.253; option host-name "c1"; subnet 10.1.0.0 netmask 255.255.0.0 { } default-lease-time 604800; max-lease-time 1209600; host c2 { option routers 10.1.0.1; hardware ethernet A4:BA:DB:1E:71:47; option ip-forwarding off; fixed-address 10.1.0.252; option ntp-servers 10.1.0.1; option host-name "c2"; option subnet-mask 255.255.0.0; } option domain-name-servers 10.1.0.1; option time-offset -5; host c3 { option domain-name "cluster"; hardware ethernet A4:BA:DB:1E:71:54; option interface-mtu 9000; fixed-address 10.1.0.251; } option host-name "c3"; } class "pxeclients" {

64 | august 2011 www.linuxjournal.com Advertiser Index CHECK OUT OUR BUYER'S GUIDE ON-LINE. produce a simple and streamlined operating system with as few Go to www.linuxjournal.com/buyersguide where you can learn libraries and utilities as necessary to get the nodes to interact with more about our advertisers or link directly to their Web sites. the master job scheduler. Software that is needed by jobs (such as Thank you as always for supporting our advertisers by buying the MPI libraries) can be shared via NFS from the master node. The their products! compute nodes simply have a kernel and the basic libraries needed to start a job. User account information can be shared via a local LDAP service running on the master node or by any method you Advertiser URL Page # already may have available in your environment. 1&1 INTERNET INC. www.oneandone.com 1 To prepare a diskless cluster, your master node will need some amount of reasonably fast local disk storage and at least 10/100 ABERDEEN, LLC www.aberdeeninc.com C3 Ethernet, preferably gigabit Ethernet. Your diskless nodes will need Ethernet hardware that can PXE boot from a network interface; most ARCHIE MCPHEE www.mcphee.com 79 modern hardware supports this. These nodes will need to be on the DIGI-KEY CORPORATION www.digi-key.com 79 same physical subnet, or you will have to configure your dhcpd service to respond or relay between subnets. Your diskless nodes also should DRUPALCON london2011.drupal.org 27 have sufficient physical memory (RAM) to hold the OS image plus have enough room to run your programs—a few gigabytes of RAM EMAC, INC. www.emacinc.com 23 should be sufficient if you keep your OS image simple. GENSTOR SYSTEMS, INC. www.genstor.com 21 For the rest of this article, I assume your cluster is based on a Red Hat-derived distribution, as this is based on a Fedora-specific tool. I’m HOSTINGCON/INET INTERACTIVE www.hostingcon.com 9 going to demonstrate an environment where all of the cluster nodes IXSYSTEMS, INC. www.ixsystems.com C2, 3 can communicate with the master on a private Ethernet subnet.

Your boot server needs to run just two services for diskless LINODE, LLC www.linode.com 45 booting: DHCP and TFTP. DNSMasq can be substituted for DHCP and TFTP, but I demonstrate using separate DHCP and TFTP services LINUX JOURNAL STORE www.linuxjournalstore.com 33 because that’s how I set up my own cluster. For convenience, you LOGIC SUPPLY, INC. www.logicsupply.com 39, 61 may choose to install bind or some other DNS to make communication between nodes more friendly. To deploy custom rpm files LULLABOT www.lullabot.com 7, 63 quickly, you may want to have access to a local repository shared MICROWAY, INC. www.microway.com C4, 5 via Apache or another Web service. Local rpm repositories also are a viable method to deploy custom rpm files. OEM PRODUCTION www.polywell.com 57 First, install DHCP via yum: OHIO LINUX FEST www.ohiolinux.org 59 yum -y install dhcp tftp-server syslinux POLYWELL COMPUTERS, INC. www.polywell.com 79

The file /etc/dhcpd.conf should be created, and in this config RACKMOUNTPRO www.rackmountpro.com 25 file, you need to define your subnet and a pxeclients class that SILICON MECHANICS www.siliconmechanics.com 18, 19, 53 simply locates the bootable pxelinux image on disk. You also need TECHNOLOGIC SYSTEMS www.embeddedx86.com 13

USENIX SECURITY SYMPOSIUM www.usenix.org/sec11/lj 47 Listing 2. Example tftp File

UTILIKILTS www.utilikilts.com 79 service tftp { socket_type = dgram protocol = udp ATTENTION ADVERTISERS wait = yes user = root November 2011 Issue #211 Deadlines server = /usr/sbin/in.tftpd Space Close: August 22; Material Close: August 30 server_args = -s /tftpboot disable = no Theme: Hack This bind = 10.1.0.1 BONUS DISTRIBUTIONS: per_source = 11 Utah Open Source, USENIX OSDI, SHAREPOINT cps = 100 2 Contact Joseph Krack, +1-713-344-1956 ext. 118, flags = IPv4 [email protected] }

www.linuxjournal.com august 2011 | 65 INDEPTH

Listing 3. Example nodes-ks.cfg

### System language xkeyboard-config lang en_US.UTF-8 nscd nss_ldap autofs ### System keyboard gpm keyboard us ntp compat-gcc-34-g77 compat-libf2c-34 ### System timezone compat-libstdc++-296 timezone America/New_York compat-libstdc++-33 dapl dapl-utils ### Root password dhcp rootpw abcd1234 dmidecode hwloc iscsi-initiator-utils ### System authorization information libXinerama auth --useshadow --enablecache libXmu libXpm libXp ### Firewall configuration libXt # Firewalls are not necessary in a cluster, usually man firewall --disabled mesa-libGL nfs-utils openssh ### Disables Selinux openssh-clients selinux --disable openssh-server pciutils sysklogd ### Repositories tvflash repo --name=Your-Custom-Repo --baseurl= vim-minimal http://your.custom.repo/ vim-enhanced repo --name=base --baseurl= http://mirror.centos.org/centos/5/os\$basearch/ repo --name=newrepo --baseurl=file:///tmp/localrepo ### Pre-install scripts

### Post-install scripts ### Enable and disable some services %post services --enabled=gpm,ipmi,ntpd --disabled=nfs ### Here you can run any shell commands you wish to ### further customize your nodes. ### Package install information %packages ### Sets up DHCP networking on the compute nodes bash cat << EOF > ifcfg-eth0 kernel DEVICE=eth0 syslinux BOOTPROTO=dhcp passwd ONBOOT=yes policycoreutils MTU=1500 chkconfig EOF authconfig rootfiles mv ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0 comps-extras

66 | august 2011 www.linuxjournal.com INDEPTH

If you have multiple network interfaces Listing 4. Example cluster-ssh-keys.spec on your master node, you can choose %define name cluster-ssh-keys to bind TFTP to one interface by using %define version 1.0 the bind command. %define release 1

Summary: ssh keys for cluster compute nodes to define the diskless hosts definition for each node by associating Name: %{name} the bootable MAC address of each node with a static IP that Version: %{version} you define for that node. I also chose to include the host-name Release: %{release} option, so that my diskless hosts will know a name other than Group: System Environment/Base localhost.localdomain once they are booted. License: GPL Next, you need to enable the TFTP dæmon. Red Hat BuildArch: noarch systems launch TFTP via xinetd—I simply needed to enable the BuildRoot: %{_builddir} /etc/xinetd.d/tftp config file and start xinetd. If you have multiple URL: http://your.custom.url network interfaces on your master node, you can choose to bind Distribution: whatever TFTP to one interface by using the bind command. Vendor: You Once configured, both services should be added to the default Packager: your email runlevel and started: %description chkconfig dhcpd on This provides the ssh keys necessary for compute chkconfig xinetd on nodes on a diskless cluster. service dhcpd start service xinetd start %prep exit 0 Now for the fun part—creating the OS image. RPMForge hosts a version of the livecd-tools package, which can be %build installed via yum: exit 0 yum install livecd-tools %install exit 0 The live CD tools require a Red Hat kickstart file—templates can be found via Google and as part of the livecd-tools package. A %clean template kickstart is generated by anaconda on any freshly installed exit 0 system in the root home directory as /root/anaconda-ks.cfg. Of particular interest here are the %packages and the %post %files sections. In %packages, you can choose exactly which programs %defattr(-,root,root) you need or want installed on the initial ramdisk image and avail- /etc/ssh able to the OS at boot. I recommend choosing as little as you can in order to keep the initrd small and streamlined. In %post, you can add any shell commands you need in order to customize your solve this, I have generated a template host key that I then deploy compute nodes further—for example, by editing config files for copies of to each of my diskless compute nodes via an rpm file. To needed services. The example kickstart provided here works with build your own version of this rpm, you need to create a spec file a RHEL- or CentOS 5.5-based distribution. (see the example) and copy the host keys from /etc/ssh to the location If you review my example kickstart file, you’ll notice that I’ve specified by BuildRoot in the spec file. The rpmbuild command gener- specified DHCP as the boot protocol for the network on each of ates the rpm, and this rpm can be included in a local yum repository the compute nodes. Because the dhcpd service already knows by specifying its name to the %packages section of your kickstart: about the Ethernet MAC address of my diskless compute nodes, the nodes simply will re-request an IP address during boot and be rpmbuild -bb sshkeys.spec reassigned the same one. Remember that no unique information is stored on the node’s OS image, so using DHCP is the easiest By setting up SSH with the same host key on each node, way to assign IPs to each diskless node. I’ve defeated some of the security of SSH by allowing the One special situation to note: because the compute nodes are possibility of man-in-the-middle attacks between my master diskless, each time SSH starts on a node, it generates a new set of node and compute nodes. However, in my cluster environment host keys. When the node reboots, it generates a new set of different where compute nodes communicate on a private and dedicat- keys, leading to an impossible-to-maintain situation for SSH users. To ed channel and do not have a direct connection to the outside

www.linuxjournal.com august 2011 | 67 INDEPTH

Listing 5. Example mknodes.sh Listing 6. Example exports File

#!/bin/bash /local-disk 10.0.0.0/255.0.0.0(rw,async)

/etc/init.d/nscd stop

cd /local-disk/nodes/ Listing 7. Example fstab File

livecd-creator --config=/local/nodes/nodes-ks.cfg \ master:/local-disk /local-disk nfs _netdev 0 0 --fslabel=cluster -t /local-disk/nodes/

livecd-iso-to-pxeboot /local-disk/nodes/cluster.iso creates the image and cleans up any temporary files for me. Once the files have been copied to tftpboot, it’s time to boot rsync -av /local-disk/nodes/tftpboot/ /tftpboot/ a compute node. If all goes well, the diskless client will request a DHCP address, and your DHCP server will respond with an IP and rm /local-disk/nodes/cluster.iso the location of the TFTP server and image to download. The client rm -rf /local-disk/nodes/tftpboot then should connect to the TFTP server, download the image and launch the OS you just created. /etc/init.d/nscd start Problems with the PXE boot process can be diagnosed by using any network protocol analyzer, such as Wireshark. Once the image is loaded and the kernel is alive, you should see the normal world, this shouldn’t be a problem. boot process on the screen of the diskless compute node. Another idea that might simplify your SSH environment is As noted before, specialized user-level software (such as the MPI to consider enabling host-based SSH authentication (so users libraries in my case) can be distributed to your nodes via standard don’t have to generate private and public keys while on your NFS shares. On your NFS server (it can be the same as your master User home directories can be shared via NFS or via a high-performance, cluster-based filesystem, such as PVFS2 or Lustre. cluster). The root SSH environment is hardened against SSH node), simply define a new share in /etc/exports and enable NFS: host-based authentication, so you’ll either have to work around this security measure or set up SSH public/private key- chkconfig nfs on chains for the root account on your new cluster. Normal users service nfs start should have no problems with host-based SSH authentication, so long as the UIDs are common among the entire cluster. Your nodes need to add an entry for the NFS server either to Once your kickstart has been customized to your liking, their local fstab files or via some other method like autofs. the rest of the setup is simple. Just run the livecd-creator User home directories can be shared via NFS or via a high- script to generate an ISO image, then use the livecd-isto-to-pxe performance, cluster-based filesystem, such as PVFS2 or Lustre. NFS script to convert that into something TFTP can use. is reliable when disk I/O is not very intensive or mostly read-only, When compiling the OS image, some active dæmons may but it breaks down if your code relies heavily on large numbers of interfere with the build process. Of particular note, SELinux files or heavy, simultaneous I/O operations to disk. must be permissive or disabled, and if you use the nameserver Please keep in mind that any customizations of the environment cache dæmon (nscd), you may need to disable it temporarily on the diskless nodes are not maintained between reboots. In fact, while the build process runs or else risk a corrupted image: it’s perfectly okay to cold-reset a diskless node; the OS image cannot be corrupted like it could be if it were on a local disk. This simplifies setenforce 0 troubleshooting strange node problems. If a reboot doesn’t clear service nscd stop the problem (and assuming no other diskless nodes show the same livecd-creator --config=nodes-ks.cfg --fsla- problem), it’s almost certainly a hardware bug—this alone can save bel=Compute_nodes hours of time when working with a large cluster.I livecd-iso-to-pxe Compute_nodes.iso rsync -av tftpboot/ /tftpboot/ Howard Powell is the sole sysadmin at the University of Virginia Astronomy Department. He’s built service nscd start three generations of Linux-based high-performance computing clusters to support the Virginia Institute of Theoretical Astronomical, which are used to study cool things like what’s happening I’ve chosen to write all of this into a handy shell script that around black holes in our universe. He lives near Charlottesville, Virginia.

68 | august 2011 www.linuxjournal.com INDEPTH

Radio Dramas in Linux A little care and some excellent FLOSS software can yield some ear-popping audio goodness. DAN SAWYER

Podcasting and satellite radio have brought back the age of talking, and dipping down to –40db (at the lowest) for audio fiction. The Golden Age of radio may have run through whispers or low voices. Check your work with the spectrum the 1930s and 1940s, but with the advent of Audible.com, analyzer, and double-check it by listening with a good pair Podiobooks.com and the hundreds of streaming stations, the of headphones. true Golden Age of audio fiction is now. Getting your power cleaned might require a power condi- Everywhere you look, people are producing their audio tioner (a $60 Fuhrman will do fine) or a battery-operated fiction with...Garageband? If Macs are the price of entry, recorder like the Zoom H4, and minimizing cable runs to count me out. Excellent though Apple is on a number of reduce the chance of radio interference. Getting your room points, I’ve never been able to reconcile myself to paying a quiet enough may require installing foam pads in the windows premium for the privilege of being a member of a totalitarian- and removing all computers from the room. A good handbook minded gadget-nut club. on setting up a home recording studio wouldn’t go amiss. But audio fiction has been a dear love of mine since childhood, And, of course, when recording voice, always record at and at the urging of Scott Sigler, I joined the rising wave four as high a sample rate as possible. For best results, if you’re years ago. I’ve done fairly well for myself, producing my way, on recording on Linux, record in either Audacity or Ardour. Linux. What follows is a quick-and-dirty guide to making a radio (Watch those xruns; they will cause skips, so make sure you drama that sounds amazing, rather than just adequate, using have JACK well set up for your equipment if you’re using only FOSS tools. Ardour.) Also, make sure you’re working with Planet CCRMA, 64Studio, UbuntuStudio or another “Studio” distribution— Pre-Production and Production you’ll need that real-time kernel with all its special patches if I’m gonna rocket through these really quick, because they’re you want this to work like a dream, rather than a nightmare. not software-intensive, but they are important—desperately important. Without good quality audio to start with, you Cleanup and Editing might as well go home and forget the whole thing. Most of the cleanup you’ll need to do actually will come in The goal is to get good performances, with good technical the next step, and you can do that easily with in-line effects in quality, in a semi-dead room. Good performances come from Ardour—but I’ll get to that in a moment. First, let’s assume a good director and good actors working together, either that, despite your best engineering-fu, you haven’t quite been through a well-annotated script in a remote studio, or through able to get a clean enough signal. You’ve got white noise at recording with the director and actor in the same studio— –50db—no buzz, just some annoying spread-spectrum hiss— work styles vary, as may your mileage. If you’re starting this so how do you deal with it? from scratch, experiment to find out what works best for you This is the kind of thing the noise removal tool in Audacity and your actors. was born for—and the one thing it’s really good at. Here’s A semi-dead room is important as well. A number of the how to make the most of it. qualities of the human voice only manifest in space—a close mic Select a short clip of room tone (where your actor isn’t in a completely dead room can rob some voices of their richness, talking, and there’s no noise other than that which you wish while micing a few inches away in a room with a few diffuse to reduce), bring up the noise removal tool, and click “sample”. reflective surfaces will help bring out the texture and timbre in Then, select the entire area you want to quash (typically the your actor’s voice as the sound waves play phase games in the whole raw dialogue track) and open the noise remover again. space. Put some time into tuning your room and doing experiments, This time, with your frequency blend set at 130 and your and you’ll be well rewarded. Also, a good pop screen (about reduction set at –8db (over the years, I’ve found these settings $30) and a copy of The Yamaha Sound Reinforcement Handbook give the best trade-off of noise reduction/quality preservation will save you hours of grief on the far end. for spoken word material), then click OK. Good technical quality comes through good equipment, Once you’ve run this—and, if you’ve got really bad audio, clean power and quiet. Ultimately, you want a signal that’s run it a second time with a new noise sample—export your free of ground loop buzz with an ambient noise floor of audio as a .wav file. Do not overwrite your original .wav file— below –60db. Don’t try cheating by reducing your recording if anything’s gone wrong that you haven’t caught yet, you gain—this won’t help you. When you amp it back up in post, want to be able to go back to the unaltered file. you’ll just hear all the noise again and some extra besides. When you’re done with your cleanup, bring up Ardour and Instead, aim for a mean signal ratio of more than –20db for a build your project. For the purposes of this article, I’m going to normal speaking voice, peaking up to 0 for shouting or loud assume you have one room in which the characters are conversing,

www.linuxjournal.com august 2011 | 69 INDEPTH

that there will be sound FX and some music as well. Your track layout, then, will look like this: two mono tracks for dialogue, two stereo tracks for sound FX and one stereo track for music. You’re also going to need one stereo bus for the reverb rooms—all of which I’ll get to in the next section. For now, lay out your tracks, pull in your audio and lay it down. Now comes the edit—arguably the most arduous part of the process. Select your performance keepers, trim them and lay them in. Adjust their timing, and time them with your audio effects. Ardour’s non-destructive slide-edit and automat- able faders make this process much quicker and easier than it is in other programs—trust me on this one. In my misspent youth, I edited three full-length films and several short films and corporate videos in destructive editors like Audacity, and that’s a kind of hell I wouldn’t wish on my worst enemy. In dramatic terms, the edit is where your emotion is created, sustained and communicated. Get the edit close to correct, then give your ears a break before moving on to the next step—you’re going to need yourself in top form to make the Figure 1. Controlling the intensity with which the audio hits the reverb is magic happen. key to managing the characteristics of your room.

Reverb, Panning and Mixing: Where the Magic Happens My 18-hour full-cast audiobook Down From Ten took place entirely in a single house. Since the house and the house’s geography are key to the drama, it was essential to make the environment a character, with every room as distinct and soni- cally interesting as the characters who were performing in it. That sense of place is one of the things mentioned frequently in reviews by listeners—“immersive”, they call it. But I’m not alone in doing this. Dirk Maggs of Above The Title Productions made his name pioneering this sort of deep audio texturing for BBC radio, and it’s the kinds of touches and flourishes I’m about to describe that make the difference between a pleasant and listenable production and a full-blown audio movie. To create that sense of place, start by building a “room”. The characteristics of the materials, angles and distances in a space are what makes the difference between a kitchen and a living room, or a cathedral and a park outdoors. For this sample Figure 2. Piping the Audio through the Reverb Room by Creating a Send project, let’s assume you need a bedroom and a cathedral, because most of us know what each of those sound like. This is where that stereo bus comes in. Rename it “Cathedral”. From your two dialogue tracks, put “sends” in before the faders (in Ardour, everything is inserted by right- clicking above the fader in the mixer window). Route those sends to “Cathedral”—left-to-left, right-to-right. Double-click on each send, and adjust your send level and pan—this is how hard your sound will hit the reverb room and on what side. You can’t automate these, but there’s rarely a need. Set the send level at –12db or a little lower for now, and leave the pan pot dead center. Now, duplicate these settings on the Figure 3. Setting Up the Reverb Room Using the TAP Reverberator other dialogue track, and then set up another send on the sound FX track. your poison. For my purposes, I’m using the somewhat limited Popping over to the Cathedral bus, insert a reverb. Linux but most excellent TAP Reverberator. Because we’re doing this has a number of excellent reverbs available in LADSPA, LVR2 room as a cathedral, we’ll use the “Cathedral HD” preset. As and VST flavors (just less than a dozen at my last count). Pick you bring it up, you’ll notice things like wet/dry balance, decay

70 | august 2011 www.linuxjournal.com INDEPTH

arriving at a stop just off-center at the same time the volume fader reaches its maximum. Set the pot back to “play” and give it a listen. You’ll hear the character walking into the room (without footsteps), approaching through sonic space. (If you want footsteps, put them on the sound FX track and repeat the same automation moves for them.) Now that you’ve got a handle on how these things interact when you start moving pots and faders around, it’s time to mix your project. Do it now, adjusting the relative volume levels of everything (and their positions in the room) in “write” mode, then switch back to “play”. Watch the redline while you’re doing it—you don’t want to create a mix Figure 4. Setting the Pan Pot to “Write” that clips on exports. Keep everything below 0db, and you’ll be fine. time and other controls—adjusting these will give you an Go ahead. I’ll be here when you get back. innate feeling of the space you’re in. For these purposes, let’s just use the default setting. You now have your room. If you click play, you’ll hear everything playing in that room. But, what if it’s too live? The reverb might be too hot in certain frequency ranges—cathedrals tend to sound a bit sharp, because the processor simulates all that stone and vaulting you find in a real cathedral. You can finesse your room further by adding an EQ to the cathedral bus—for example, the LADSPA “Multiband EQ” plugin—and selectively damping some frequencies. For voice in a cathedral, dumping the extreme low end and extreme high end, and subtly damping 800Hz, 1KHz and 2,400Hz can sweeten up the room wonderfully. You can control (and automate) the volume of your reverb with the fader on the bus. Now, the real magic happens. Let’s say you want to have one character enter the room from the left and approach the other character, who the audience is standing near (in audio space). If the POV character’s voice is on dialogue 1 and the visitor is on dialogue 2, here’s what you do. Set dialogue 2’s fader to “write”, and start it off at a low volume. Push play, and gradually raise the volume as the character crosses the room to the other character. When he reaches his destination, his fader’s endpoint volume should be the same as the other character’s fader volume level. You’ll notice that you’re already hearing that sense of “approach”, as if the character is walking straight toward you—that’s because as you change the fader volume on the track, you’re changing the wet/dry balance for the reverb. The drier the signal, the closer it seems; the wetter the signal, the farther away it seems. But, you wanted the character to come in from the left, not walk toward you from deep in the distance. So, set that fader back to “play” and return the transport to the beginning of the walk-in. Now, set the track’s pan pot to “write”, and drag it all the way off left. Press play, and, in tandem with the increasing volume, slide that pot from extreme left to just left of center,

www.linuxjournal.com august 2011 | 71 INDEPTH

and sopranos. These frequencies create vocal warmth when they’re higher than the others, and damping them down Best Plugins for makes for a cold, emotionally distant sound.

Spoken Word 4. Authority comes in the low registers: 150–400 for most men and 250–600 for most women. Boosting these lends more gravity I Tube Amp (overdriven): for subtle distortion, as of a and urgency to the voice, cutting them does the opposite. voice coming over a speaker. But, when it comes to constructing a good drama mix, I Ring Modulate: for radio interference, mechanization, you’ll also want to take a look at your music and sound FX. and alien vocal effects. Because ambient room tone, most music and many sound effects are very strong in the mid-range frequencies, they easily I EQ: for damping, booming, flattening and sweetening. can swamp your voices, even if the music is at a very low volume Good EQ plus a good actor can yield dozens of distinct compared to your voices. Your actors can get lost in the mix, vocal textures from a single mouth. even when you’ve been sure not to mix the music too hot. To solve this, slap an EQ on your music and effects channels, I Reverb (with a light touch): for a tiny bit of extra and cut those middle frequencies just a bit—5db usually does resonance to add a sense of authority or gravity. the trick, and adding another subtle reduction up around Experiment with all the available FOSS reverbs— 2,500Hz will make room for your consonants. Doing this lets they all have subtly different characteristics and are the music and sound FX fill in the sound around your actors, useful in different situations. rather than running over them, and the resulting sound will be much cleaner and more pleasing.

Mastering EQ, for That Extra-Special Touch Once you’ve finished your editing, mixing, reverb, EQ and all Without a doubt, EQ is the single-most powerful tool in audio- the other various sweetening and tweaking you want to do, drama engineering. It’s through EQ tweaks that you can turn a it’s time to export. Assuming you’ve got a good mix that doesn’t regular spoken voice into a tinny phone voice, a voice shouting clip—or clips only on one or two occasions—you won’t need through a wall from a neighboring room or a voice of great a compressor. Throw the LADSPA Declipper on the master bus authority. For the human voice, a good 15-band parametric EQ just for safety and export your project. (such as the Multiband EQ plugin that comes with almost all Linux If you’re destined for CD distribution, you may want to pull JAMin distros) will do just fine. As each voice is different, there aren’t into this—it’s a bit esoteric if you’re new to this game, but well worth learning. The easiest way to use it is by setting up an insert on the master bus (see Dave Phillips’ Without a doubt, EQ is the single-most powerful many excellent articles on LinuxJournal.com for instructions on using JAMin). tool in audio-drama engineering. If you’re aiming for MP3 distribution, import your exported .wav into Audacity and use it to master your MP3. This gives you the chance to do a many universal recommendations—even so, here’s a few that will final waveform inspection, looking for pops, skips or dead spots apply almost anywhere: (as can sometimes happen on export with some systems) and to do your tagging as the file spools out. 1. Dump everything below 125Hz. Almost all human voices bottom out at around 125. Anything lower is noise, and Wrapping It Up that noise can screw up your reverb, so just mute the low And that’s really about it. Whether you’re soundtracking a film, bands. Ditto for anything above 16KHz. doing a radio drama or an audiobook, the procedure and tools are very similar. A few plugins (far fewer than most people think 2. Vocal crispness comes in above 2KHz. Your consonants are you need), a little care and some excellent FLOSS software can up here—the difference between good articulation and bad yield some ear-popping audio goodness. Once you learn the is the quality of the consonants. Damping these frequencies basics here, all that’s really left is practice. Which reminds me, makes the voice muddy, boosting them makes it crisp. You I have another audiobook due next week. Best get back to it.I can tune the performance subtly by tweaking the consonants. Similarly, if your actor has harsh consonants, damping these Dan Sawyer is the founder of ArtisticWhispers Productions, a small audio/video studio in the subtly will take out those nasty sibilants and dentals. San Francisco Bay Area where he produces full-cast audiobooks and radio dramas. He is the author of The Clarke Lantham Mysteries, the Parsec-nominated The Antithesis Progression, 3. Vocal richness and texture comes in the middle of the Down From Ten and several short stories. You can find out more about him and his various range: 300–600 for tenors and around 500–800 for altos flavors of insanity at www.jdsawyer.net.

72 | august 2011 www.linuxjournal.com INDEPTH

Unison, Having It Both Ways Unison is a program for bidirectional synchronization of files using the rsync algorithm. ADRIAN KLAVER

Unison is a file synchronization tool that supports bidirectional on UNIX-like systems. On Windows systems, it’s either updates of files and directories. It uses the rsync algorithm to limit $USERPROFILE\.unison or $HOME\.unison or c:\.unison, the size of updates to only what has changed. The program runs depending on the settings for USERPROFILE and HOME. on UNIX-based systems as well as Windows machines and can Now, let’s move on to the actual setup: desktop <--> EC2 sync between them. It uses SSH as the default transport method. server <--> laptop. The important part of the above is that each Unison originated as a research project at the University <--> represents a different Unison root pair. The desktop Unison of Pennsylvania, and although that project has moved on program has no knowledge of what is on the laptop, only what it (Harmony/Boomerang), the program continues on. This has led can see on the EC2 server and vice versa. The role of the EC2 to some misconceptions as to the status of the program that are Unison instance in this scenario is to keep a particular set of files answered at the URL provided in the Resources section of this in sync with either the desktop or laptop, depending on who is article. The short version is that the program is very much alive asking. In simplest terms, the EC2 Unison is a server to the and in active use by many. In this article, I demonstrate the desktop/laptop Unison clients, although this is not strictly true setup and basic usage of Unison. due to the bidirectional nature of Unison. Either Unison instance The first step is installing Unison on your machine. It is avail- in a pairing can serve or receive files. Connection between the able in package repositories, although probably as an older version. desktop/laptop and the EC2 server is done over SSH using For instance, on my version of Kubuntu, 10.04, the package public/private key authentication. For the purposes of this article, version is 2.27.57. The latest stable version is 2.40.61, at the time I am using a subset of the paths I normally keep in sync. of this writing. A lot of usability, performance and cross-platform Although it is possible to run Unison by supplying arguments improvements have been made between those versions, so for this to the program on the command line, there is a better way—that article, I use 2.40.61. For Windows and Mac machines, current is, to create a profile file for each root pair that is synced. These binaries are available from the Web site’s download page. For are *.prf files stored in the ~/.unison directory. When you start other platforms, it is necessary to build from source. Unison in text mode, you can supply it a some_name argument Complete instructions are available in the manual’s Install that maps to a some_name.prf file. In GUI mode, Unison searches section (see Resources). Unison can be built as either a text or the ~/.unison directory for *.prf files and presents them as choices GUI (GTK) version, assuming the appropriate libraries are available. (Figure 1). For this article, I am using lj_article.prf (Listing 1). The GUI version also can be run in text mode via a command-line I use this same profile on both the desktop and laptop switch, so it is the most flexible. The text-only version is handy for machines. Let’s go through the lines and detail what they mean. servers where graphical libraries are not installed. Note: when I built from source, the Quit button did not show up in the GUI toolbar. As you will see later, that is more annoying than fatal. Listing 1. ~/.unison/lj_article.prf Unison can be used and customized in a variety of ways, but trying to cover all the bases would deplete the magazine’s ink #Preferences available 2.27.57+ except as noted allowance for this issue. Instead, I demonstrate the way I use root = /home/aklaver Unison on a daily basis as an illustration of what is possible. My root = ssh://alabama/lj_article basic setup consists of Unison on three machines: my home path = software_projects/linux_journal_articles desktop, my laptop and an Amazon EC2 instance. path = software_projects/aws Before going into my actual setup, bear with me as I explain path = software_projects/track_stocks the basic operating principles behind Unison. The starting point path = .unison for Unison are two roots. These are just directory paths that will ignore = Name *~ be synced. Both paths can be local, or one can be local and the # BelowPath in version 2.40.61+ other remote. The preferred method of connecting local to remote ignore = BelowPath .unison/* is SSH, and sections in the manual and wiki (Resources) describe ignorenot = Name *.prf how to set up Windows to use SSH. backup = Name * Unison also has a socket method where an instance of the backuploc = central program is set up as a server listening on a socket. The catch is backupdir = unison_backup that the data is transferred unencrypted, so use at your own risk. maxbackups = 3 To keep track of state and configuration information, Unison # Copy preferences version 2.30.4+ creates a private directory. In the absence of an ENVIRONMENT copythreshold = 100 variable saying otherwise, that directory is $HOME/.unison

www.linuxjournal.com august 2011 | 73 INDEPTH

contrast to the regular Path 'path' form that matches only the path specified. I am doing this to avoid syncing the archive files and the backup directory (more on that later). Now I have a quandary. I have said that I want to sync the .unison path with path=.unison and ignore it with ignore = BelowPath .unison/*. At that point, nothing would be synced, and the preferences would be useless. That is where the ignorenot preference comes in. By specifying Name *.prf, I am saying “disregard what I said about ignoring what is in .unison for the specific case of *.prf files”. The end result is that I sync only the profile files in ~/.unison. The next four preferences configure the backup option. The presence of a backup= preference signals that backups are to be done. A backup is done when a file is changed or deleted by Figure 1. Profile Choices Unison. The backed-up file is kept on the machine where the change is applied by Unison. So if you make a change to a file An aside, what I talk about here are called preferences in the on your local root, when it is synced to the remote root and the Unison manual. The first two labeled root are the root pair I’ll change is applied to the remote file, the previous version of the be syncing. The paths can be either absolute or relative to the remote file will be saved on the remote machine. In this case, directory in which Unison is started. The first root is an abso- using Name * means back up everything. It is possible to use the lute one defining my home directory. The second takes a little path specifications mentioned above to restrict that. There also is more explaining. The ssh:// indicates that SSH is used to a backupnot that operates like ignorenot. make the connection to the remote machine. In this case, I The backuploc preference specifies where the backups have used an SSH alias “alabama” to refer to the remote EC2 are to be stored. The options are local and central, where host. This expands as aklaver@host_name and assumes the local has the backup files being stored alongside the original default SSH port of 22. Should your machine be running at a files, and central moves the backup files to a location as different port, the form is ssh://user@host_name:port_number. defined in backupdir. With backuploc=central and no The /lj_article portion is the directory path relative to backupdir preference, the backups will be found in the my home directory on alabama. If I wanted to specify an directory backup/ in ~/.unison. This is why I have the absolute directory path, it would look something like ignore=BelowPath .unison/* preference in the profile. ssh://alabama//home/aklaver/lj_article (note the Although in this case, I am using ~/unison_backup to store two forward slashes). backups, I have other profiles using ~/.unison/backup. With just the two roots specified, running Unison would sync The maxbackups preference is self-explanatory; it restricts everything in my local home directory with the lj_article directory the number of backed-up files to the three most-recent on the remote machine. This is not what I want, so I have versions. In its absence, the default is two versions. I use the included some path preferences. A path preference is exactly central method of backup, because I don’t want to sync the that—a path to some part of the root that you want to sync. backups. Per my article, “Using rdiff-backup and rdiffWeb to Presence of a path preference restricts the sync to the paths Back Up and Restore” (LJ, December 2010), I use rdiff-backup specified. A path is relative to the root of the replica, and the to keep versioned backups already. I do like to keep Unison separator character is the forward slash (it will be converted as backups close at hand though, as insurance if I make an needed). The important part is that paths are relative to the inappropriate change to a profile and cause a file or files to root. This means that the data I sync on either my desktop or disappear, or in case I make the wrong decision on which laptop is relative to /home/aklaver and becomes relative to direction to propagate a change. Besides, I am a firm believer /home/aklaver/lj_article on the EC2 server. A path preference that you cannot have too many backups. can point to a single file, a directory or a symbolic link (on The copythreshold preference is one of the performance Unixen). The path preference is inclusive, in that if you specify enhancements in recent versions. Since 2.30.4, it has been possible a directory, it syncs everything in that directory and below. to tune Unison when doing whole file transfers or when redoing To filter what is synced in a particular path, or in general, an interrupted transfer. The rsync code, as used by Unison, is ignore preferences are used. The first form used here, Name designed more for doing changes to files than for moving over 'name' is one of four types; the others being Path 'path', entire files. Setting a copythreshold to a positive integer causes Regex 'regex' and BelowPath 'path' (this is new to 2.40.16). Unison to farm out the job of copying complete files. If set to 0, There is a lot of power here, explained fully in the Path all whole file transfers will be farmed out. Otherwise, a number Specification and Ignoring Paths sections of the manual. The greater than 0 refers to a file size in kilobytes above which Unison ignore = Name *~ preference uses globbing patterns to will use another program. By default, that program is rsync, ignore any tilde files in the paths. The second ignore preference although that can be changed using copyprog. Unison also uses ignore=BelowPath .unison/* is more specific. It causes rsync to resume interrupted large file transfers. anything in .unison and paths below it to be ignored. This is in In the same vein, Unison 2.30.4+ will, without any setting

74 | august 2011 www.linuxjournal.com needed, keep track of an interrupted transfer of a new directory. Whatever has been transferred will be stored in a temporary directory away from the intended destination, on the receiving side, and on the next transfer, it resumes with the untransferred files. At this point, running this profile is anti-climatic. The first run pops up a warning about there not being archive files present for the root pair (Figure 2). The archive files are where Unison stores its information about the roots and files synced. Pressing Enter starts the process of the initial scan and the population of the archive files. After the scan is done, the GUI presents the choices available for file transfers (Figure 3). In this case, because the remote path is empty, all arrows point from local to alabama. There are two options here: arrow down through each choice and press Enter, or type the letter g as a shortcut for the Go button Figure 4. A Sync with Changes to Both Roots and start the transfer of all the files at once. On subsequent transfers, the choices for each directory/file may well be different, depending on the changes made on each root (Figure 4). The choice presented is not set in stone, and you can change it with the arrow and Skip buttons on the toolbar, dealing with each file as necessary. For batch actions on the files, use the Actions menu (Figure 5). Also worth mentioning is the Ignore menu item. It is very handy in that it will write an ignore preference to your profile matching the item you select using the ignore type you select. With version 2.40.1+, a profile editor is built in to the GUI interface. So, if you

Figure 5. Action Menu Showing Bulk Action Choices

decide you want to undo the ignore preference, simply go to the Synchronization menu and then Change Profile. As mentioned previously, when I built the GUI, it did not have the Quit button. Typing q as a shortcut still works, or you can go through the Synchronization menu to get to Quit. So, what happens if a file has been changed on both roots since the last sync? The default action is to skip syncing that particular file, and this is represented by a ? in the file listing (Figure 6). It is left up to the user to decide what to do in that situation. That said, preferences can be set to force Unison’s hand Figure 2. First Sync of Root Pair, Archive Warning (search for “force” in the manual). Should you decide to sync, some tools are available to help you make a decision on what to do with a file. For nonbinary files, you can use the Diff button to look at the diff between a pair of files to help figure things out. There also is the Merge button. This requires some extensive preparation, and I have not used it since I first tried Unison many years ago (at the time there were known issues with the merge code). Since then, considerable work has gone into making merge functional, and it is on my to-do list to give the new code a spin. The manual provides detailed instructions for setting up merge, if you are so inclined. With all these files flying around, how safe is your data? For a more complete discussion, see the Invariants section of the manual. The essence of that section is that Unison guarantees certain behavior for the paths synced as well as its own information about Figure 3. Syncing First Time the process, at any moment in time. For paths, that is that they

www.linuxjournal.com august 2011 | 75 INDEPTH

Adrian Klaver lost a term paper once, due to no backups. Ever since, he has had a bit of an obsession with backup utilities. When not worrying about the state of his data, he can be found trying to outsmart his digital camera.

Resources

Unison Web Site: www.cis.upenn.edu/~bcpierce/unison

Figure 6. Sync with Changes to Both Roots, Showing Diff Also Harmony/Boomerang Project: www.seas.upenn.edu/~harmony are either at their original state or their completed changed state. Project Status: www.cis.upenn.edu/~bcpierce/unison/ The same principle applies to Unison’s own private data—it is status.html either in its original form or reflects the current successfully completed syncs. My own experience is that Unison is very for- Manual: www.cis.upenn.edu/~bcpierce/unison/download/ giving of operator error. A sync interrupted either intentionally releases/stable/unison-manual.html or by mistake is recoverable. I use Unison on a daily basis to help me keep track of file Wiki: https://alliance.seas.upenn.edu/~bcpierce/wiki/index.php changes on my laptop and desktop. Using an EC2 server as an intermediary host, I am able to bounce the changes from either “Using rdiff-backup and rdiffWeb to Back Up and Restore” by of my personal machines off the cloud and back to the other Adrian Klaver, LJ, December 2010: www.linuxjournal.com/ machine. For many years now, Unison has been a must-have article/10701 program for me. I hope you find it useful too.I TALES FROM THE SERVER ROOM

It’s Always DNS’s Fault! How do you fix a DNS server that isn’t broken? Find out in this episode of Tales from the Server Room. KYLE RANKIN It’s always better to learn from someone else’s company’s DNS infrastructure was set up. It had two mistakes than from your own. In this column, Kyle main data centers: A and B. Each data center had a Rankin or Bill Childers tells a story from his years as a load-balanced pair of DNS servers set up as active- systems administrator, and the other chimes in from passive, and the public virtual IP addresses for each time to time. It’s a win-win: you get to learn from their were published as the NS records for each domain we experiences, and they get to make snide comments to serviced. That would cause each data center to service each other. Today’s episode is narrated by Bill. half the DNS load for any set of requests, and due to each data center having a load-balanced pair of DNS Some Days, You’re the Pigeon... servers, we could tolerate a failure of a DNS server BILL CHILDERS I was suffering, badly. We had just finished an all-night without any degradation in customer-facing service. switch migration on our production Storage Area [Kyle: The beauty of a system like this is that even Network while I was hacking up a lung fighting though DNS has automatic failover if you have more walking pneumonia. Even though I did my part of the than one NS record, if a DNS server is down, you all-nighter from home, I was exhausted. So when my generally have to wait the 30 seconds for it to time pager went off at 9am that morning, allowing me a out. That 30-second delay was too long for our needs, mere four hours of sleep, I was treading dangerously so with this design, we could take down any individual close to zombie territory. DNS server and the load balancer would just forward I looked at the pager and saw that someone had requests to the remaining server in the data center.] pushed the dreaded “Panic Button”, a Web-based tool Anyway, I continued troubleshooting. On a hunch, we’d made that would alert the larger IT team to an I started running nslookups against a few domains unknown high-priority issue. I sat up, reeling and asked from my home—maybe the problem was visible only my wife to begin the caffeine IV drip that would wake from the outside. Oddly enough, nslookups succeeded me up while I slowly started banging synapses together, for the most part, except for those pointing to one of hoping for a spark. According to the report, our DNS our most active sites, which had Akamai as a content infrastructure was timing out on a lot of requests, delivery network (CDN). Akamai requires that you causing overall site slowdown. I had to re-read that configure your DNS using CNAME, or alias, records, so e-mail several times for it to sink into my oxygen-and- that its CDN can spider and cache your content. The sleep-deprived brain. How could DNS be timing out, CNAME records look something like the following: and why hasn’t our internal monitoring caught that? We monitored the DNS servers and service levels inter- I A CNAME pointing www.ourdomain.com to nally, and if performance was bad, I should have been ourdomain.com.edgesuite.net. the first to know. Something smelled really funny, and it wasn’t me, despite the pneumonia-induced fever. I A CNAME pointing origin-www.ourdomain.com [Kyle: I’ll pretend I didn’t see the “something to ourdomain.com. smelled funny” comment, as it’s too easy. The funny thing here was that we had a long-standing tradition Surely enough, external requests that hit data-center of DNS being blamed whenever there was any sort of A would time out and wind up failing over to data-center networking problem. I’ve said before that people tend B. Typical DNS timeouts put this on the order of 30 sec- to blame the technology they understand least. This onds, which is unacceptable for any kind of commercial case was one of the first times that it actually seemed Web site. Since Akamai was involved, and the main site (at least on the surface) to be a DNS issue.] I found that was affected was utilizing Akamai, I made I started checking on things as I dialed in to the the call to Akamai support for assistance. conference call for this issue. Our monitoring system said [Kyle: I can’t count how many times I’ve used nothing was awry, and response times for DNS were personal servers that are colocated outside a corporate normal. I ran a few nslookups past the DNS server, and network to troubleshoot problems. It can be invaluable it replied in its usual speedy fashion with the expected to have a perspective on the health of a network result. I flipped through the logs as well, and they showed service that’s completely detached from your corporate nothing out of the ordinary. What was going on? network. Think of it as another reason to keep your At this point, I probably should describe how the home server on 24/7.]

www.linuxjournal.com august 2011 | 77 TALES FROM THE SERVER ROOM

...and Some Days You’re the Statue. However, without any signs of anything At this point, I had the Akamai folks looking at the issue from their end, and after a couple hours of back-and-forth troubleshooting, wrong on the DNS server other than they announced that the problem was not with their setup, and it had to be something in our DNS servers. However, all the tests I the failed queries, I was reluctant did within the data center returned correctly and instantly. It was just to bounce the server, because only tests from the outside that timed out and failed to data-center B, and even those were sporadic. By this time it was after noon, if the error condition cleared, we’d and even the caffeine IV drip was wearing off. I was tired, sick, have no further way to collect and my brain was not firing on all cylinders. It was at about this time that I started getting e-mail messages information about the problem. from my pointy-headed boss about how I was “not doing anything” to fix the problem, and he wanted status updates every half-hour on how things were progressing. I replied that I either could update him every half-hour or work on the problem like I had been. and I remembered that the DNS servers were fronted by a load That e-mail message made my cell phone ring, instantly. My boss balancer. On a hunch, I asked the network engineer if he had was on the line, demanding I reboot the DNS server in an attempt to noticed any issues with the load balancer. He investigated and saw “fix” the problem. However, without any signs of anything wrong on weirdness in the log files of the unit. After a little more conversation the DNS server other than the failed queries, I was reluctant just to with him, he agreed there was a problem on the primary load bounce the server, because if the error condition cleared, we’d have balancer, and the decision was made to fail over to the backup no further way to collect information about the problem. load balancer. Once the failover happened, the DNS issue we Kyle wound up finding an odd bug in an older version of BIND were seeing cleared up again. All along, we were fighting a where a counter rollover caused weird things to happen. That bug flaky load balancer, not an issue on the DNS server. supposedly was fixed in the version we were running, but it was a lead to go on, so I made the call, reluctantly, to restart the DNS Lessons Learned service on the primary DNS server. Much to my surprise, once we Several lessons came out of this issue. The biggest one is that it’s did that, the timeouts stopped occurring. All of a sudden, our DNS easy to lose sight of all the technologies that come into play infrastructure was back up to 100%, and site performance returned in a modern data center. My team was responsible for the to its normal levels. UNIX systems, so we naturally tested and troubleshot the [Kyle: It’s worth noting that the DNS process on this system servers, but initially didn’t think that the network possibly had been up and stable for more than a year. Although it was could be a problem. Always be sure to look outside your realm technically possible that an internal uptime counter rollover (like of responsibility, as the problem may lie there. the 498-day uptime rollover on older Linux kernels) could cause [Kyle: It’s funny, because I’ve known people who default to strange behavior, it really seemed like grasping at straws, and looking outside their realm of responsibility when there is a I was surprised when it seemed to fix the problem. That, of problem. We should have been clued off when we noticed that course, brought up other questions though—were we going internal DNS requests always worked while external ones (ones to have to bounce our DNS service every year?] through the load balancer) were flaky. But like Bill said, once we My boss called me after the site performance returned. To this rebooted the service and the problem disappeared, there wasn’t day, I don’t know if the call was to gloat about his “solution” being any troubleshooting left to do.] correct, or if he called to chastise me for waiting so long to restart Another lesson was one I already knew, but it was highlighted the DNS server. I explained that although the issue was no longer that day. Rebooting a server that’s misbehaving is an absolute last occurring, we had zero information as to the root cause of the issue, resort, as you’ll wind up losing the problem in the first place, and and that it was not “fixed”. Rebooting things randomly is what you’ll never figure out what the root cause is. Windows admins do when things act up. UNIX system administrators In all, although this issue did cost the company money that tend to try to reach the heart of the issue. At the end of the call, it day due to the poor site performance, it was a good learning was apparent he didn’t care about a fix. He simply wanted the site experience. I think of this incident a lot when designing new back to its normal performance. I finally passed out, exhausted, yet infrastructure or when faced with a new and unknown problem. feeling worried that we had not seen the last of this issue. It reminds me to be thorough when troubleshooting, to look at every possibility and not assume anything.I Listen to Your Hunches Fast-forward a couple weeks later. I’m feeling better, the pneumonia’s Kyle Rankin is a Sr. Systems Administrator in the San Francisco Bay Area and the author of a number of defeated, and I’m back at work. True to my gut feeling, the issue books, including The Official Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks. He is currently the spontaneously re-occurred again. People around the office started president of the North Bay Linux Users’ Group. panicking, blaming the DNS infrastructure and flailing about in general. Kyle and I immediately set to troubleshooting again, but Bill Childers is an IT Manager in Silicon Valley, where he lives with his wife and two children. He just like the time before, we couldn’t find a single thing wrong enjoys Linux far too much, and he probably should get more sun from time to time. In his spare with the DNS server. This time though, I was more on the ball, time, he does work with the Gilroy Garlic Festival, but he does not smell like garlic.

78 | august 2011 www.linuxjournal.com LINUX JOURNAL MARKETPLACE 79 | august 2011 OLY OLY LD-001 POLYWELL $199 Intel® / AMD® CPU Intel® efficient Energy Quiet and Low starts at Voltage Platform Voltage $7,750 $4,999 - - - Silent Eco Green PC Green Eco Silent 9015H 15Bay 45TB 30TB - Dual Gigabit LAN 10 1, 0, 6, - RAID-5, Hot Spare - Hot Swap, Mac Windows, - Linux, - E-mail Notification Case - Tower www.linuxjournal.com [email protected] 888.765.9686 888.765.9686 www.polywell.com/us purposes only and may be trademarks of their respective owners. $9,999 More Choices, Excellent Service,More Choices, Excellent Great Prices! Netdisk 8000V Quiet Performance - Dual Gigabit LAN 10 1, 0, 6, - RAID-5, Hot Spare - Hot Swap, - E-mail Notification Case - Tower 9020H 20Bay 60TB - 4x Gigabit LAN 10 1, 0, 6, - RAID-5, Hot Spare - Hot Swap, Mac Windows, - Linux, - E-mail Notification Case - Tower $26,999 $12,950 72TB 144TB 20 Years of Customer Satisfaction of Customer 20 Years Industry's Longest Warranty, 5-Year Service Class Customer First com 4U-24Bay 4U-24Bay NAS/iSCSI/SAN Storage RAID-6, 4x Giga/10Gbit LAN Mix SAS/SATA, 5U-48Bay 5U-48Bay RAID-6, NAS/iSCSI/SAN Storage RAID-6, 4x GigaLAN SATA, 4U24A 1461 San Mateo Ave. South San Francisco, CA 94080 650.583.7222 Fax: 650.583.1974 650.583.7222 Fax: CA 94080 South San Francisco, Ave. 1461 San Mateo 5048A $6,999 36TB Quiet Storage NAS/SAN/iSCSI Quiet Storage - 4x Gigabit LAN 10 1, 0, 6, - RAID-5, Hot Spare - Hot Swap, Mac Windows, - Linux,

$26,999 $36,999 2U-12Bay 2U-12Bay 2x GigaLAN RAID-6, II, SATA NAS/iSCSI/SAN Storage

Polywell OEM Services, Your Virtual Manufacturer OEM Services,Polywell Your American made Utility Kilts for Everyday Wear Everyday Kilts for Utility made American Prototype Development with Linux/FreeBSD Support Small Scale to Mass Production Manufacturing Fulfillment, Shipping and RMA Repairs 144TB 222TB Polywell Computers, Inc Inc Computers, Polywell and combinations thereof are trademarks of NVIDIA Corporation. Other names are for informational GeForce nForce, NVIDIA, ION,

2U12B Polywell Solutions Polywell 72 Bay 74 Bay NetDisk 8074A EOF

First Brazil, Then the World A high-leverage business project for free software and low-cost hardware. DOC SEARLS

Too often we hear a project or an idea dismissed of São Paulo has agreed to design the thin-client with others in the same business. because it tries to “boil the ocean” instead of hardware and license the design to manufacturers, something more modest, like poaching an egg. who will compete to provide components and The most recent developments are timely. But the problem with modesty is that we’ve seen finished gear at the best prices. This will provide Says maddog: the ocean boiled, many times. Linux is the prime some needed income to the university as well. example. Linus himself half jokingly talked about “But the main part of this isn’t technical”, The other main driver for Project “world domination” in the mid-’90s, long before maddog says. “It’s economic. Most of our work Cauã, Douglas Conrad of OpenS it actually happened. Other examples go back as will go into running the pilot project and Tecnologia, has put together a far as J.C.R. Licklider, whose writings on “man- preparing educational and documentation home-theater thin-client solution computer symbiosis” and the “Intergalactic materials. We want to make starting and that we will start selling through the Computer Network” predated the personal com- maintaining a business as easy as possible, “Entrepreneurs”. This will show puter and the Internet by several decades. Both for as many people as possible.” people that the SA/E (system free software and open source also were inten- Thin clients and fat servers make equally administrator/entrepreneur) model can tional ocean-boiling efforts, which continue to good business and technical sense in a city like work and that SA/Es can sell things, pay off today in degrees of social, economic and São Paulo—and across the rest of Latin America, support clients and make money. technical value beyond calculation. where the vast majority of people live in urban Watching successful ocean-boiling efforts areas, especially as both television and comput- We want to show the prototype at encourages ambitions on the same scale. It ing are moving onto the Net and into “the FISL (June 28–July 2nd) in Porto certainly has for me (but we’ll leave that one for cloud”. A cheap connected device with a good Alegre, Brazil, and have it ready to another EOF—after we start seeing results). It also display (also cheap these days) and a capable start being sold by SA/Es shortly after has for Jon ’maddog’ Hall, who has been an server with plenty of low-latency storage is a that. We will probably have intensive accessory to Linux’s success right from the start. bargain for customers and a business for training on Project Cauã in the But, maddog is a modest guy. These days he’s entrepreneurs who like to be highly connected November/December time frame at not trying to boil a whole ocean, but instead (literally and otherwise) to their communities. Latinoware in Foz do Iguassu, Brazil. putting the heat on just one country: Brazil. With its concentrated urban populations, What he’s cooked up is Project Cauã proactive government, cooperative universities, Right now, maddog and friends are evange- (www.projectcaua.org). Although the Web advanced manufacturing facilities and growing lizing the project and raising funds. maddog says, site’s first paragraph describes it as “a Free FOSSH-friendly technical community, Brazil is “We need that funding because volunteers only and Open Source Software and Hardware a large lever on the rest of Latin America and get us so far. Some things require funding, such (FOSSH) project conceived to make it possible the world beyond. “After we get it working in as legal work, working with the government, for people to make a living as a Systems Brazil, we’ll translate the materials into Spanish, paying fees for registration, typing and clerical Administrator/Entrepreneur using FOSSH as then run pilots in other countries. Or help others work, documentation, buying hardware and other the basis”, its ambitions are larger than that. do it for themselves. We’ll all learn by doing.” areas. Once we have the money guaranteed, we maddog wants to create a whole new business For a new entrepreneur, here’s the drill: can start hiring and begin the project in full force. category with lots of jobs in it, while greening It should take six months from that point until we everything up by sucking less power off the grid I Train on FOSSH system administration. start a pilot.” His goal for funding at the outset: and making cities more livable. “Three million US dollars, guaranteed.” I got some hang time with maddog when I Get certified, licensed and bonded. For investors, the “play” isn’t Project Cauã, he spoke at an MIT gathering earlier this year, but the entrepreneurs it bootstraps. For the and I like where he’s going with Project Cauã. He I Get letters of intent from prospective biggest and most aggressive investors, $3 million looks at a city like São Paulo (one of the world’s customers. is an ATM withdrawal. My advice for them— largest) and sees “tall buildings and apartment and for anybody wanting to heat things up— complexes with servers in the basements and I Get underwritten loan commitments is to step up to Project Cauã’s table and start thin clients in the rooms” and “large new business from banks. placing some good bets. opportunities for small entrepreneurs, manufac- Postscript: thanks to maddog for his patience turers, and the economics that both support.” I Buy and install the gear. while I wrote this up and to podcast.de for When people ask, “How can I make money with some of the quotage.I free software?”, his answer is “By renting out I Deliver services. thin clients, server space and services to back Doc Searls is Senior Editor of Linux Journal. He is also both, in large volume and at low costs.” I Get feedback from services. a fellow with the Berkman Center for Internet and Society at On the technical side, “The software is Harvard University and the Center for Information Technology done, and the hardware is close.” The University I Improve documentation and shared knowledge and Society at UC Santa Barbara.

80 | august 2011 www.linuxjournal.com ANYONE INTERESTED IN SAVING MONEY?

Looks like these guys are comfortable overpaying for enterprise storage. Are You? “Hewlett-Packard Co. agreed to buy 3Par Inc. for $2.35 billion” — Bloomberg.com “EMC to Buy Isilon Systems Inc. for $2.25 Billion” — Wall Street Journal “Dell to Buy Compellent for $960 Million” — CNBC

So what “benefit” will you see by this spending spree, other than higher costs? The AberSAN Z-Series scalable unified storage platform, featuring the Intel® Xeon® processor 5600 series, brings the simplicity of network attached storage (NAS) to the SAN environment by utilizing the innovative ZFS file system. The AberSAN Z20 is easily found starting under $20,000.

Who gives you the best bang for the buck? 3Par Compellent Isilon Aberdeen InServ F200 Storage Center Series 30 NL-Series AberSAN Z20 Storage Scale-Out ✓ ✓ ✓ ✓ Thin Provisioning ✓ ✓ ✓ ✓ HA Clustering ✓ ✓ ✓ ✓ VMware® Ready Certified ✓ ✓ ✓ ✓ Async / Synchronous Replication ✓ ✓ ✓ ✓ iSCSI / Fibre Channel Target ✓ ✓ iSCSI Only ✓ Unlimited Snapshots x ✓ ✓ ✓ Native Unified Storage: NFS, CiFS x x ✓ ✓ Virtualized SAN x x x ✓ Deduplication x x x ✓ Native File System none none OneFS ZFS 128-bit RAID Level Support 5 and 6 5 and 6 Up to N+4 5, 6 and Z Raw Array Capacity (max) 128TB 1280TB 2304TB Unlimited Warranty 3 Years 5 Years 3 Years 5 Years Online Configurator with Pricing Not Available Not Available Not Available Available Above specific configurations obtained from the respective websites on Feb. 1, 2011. Intel, Intel Logo, Intel Inside, Intel Inside Logo, Pentium, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. All trademarks are the property of their respective owners. All rights reserved. For terms and conditions, please see www.aberdeeninc.com/abpoly/abterms.htm. lj038 888-297-7409 www.aberdeeninc.com/lj038 Cut Execution Time by >50% with WhisperStation-GPU Delivered ready to run new GPU-enabled applications:

Design Simulation BioTech

3ds Max ANSYS Mechanical MATLAB AMBER Bunkspeed Autodesk Moldﬂow ACUSIM AcuSolve GROMACS Shot Mathematica Tech-X GPULib NAMD, VMD Adobe CS5 TeraChem

Integrating the latest CPUs with NVIDIA Tesla Fermi GPUs, Microway’s WhisperStation-GPU delivers 2x-100x the performance of standard workstations. Providing explosive performance, yet quiet, it’s custom designed for the power hungry applications you use. Take advantage of existing GPU applications or enable high performance with CUDA C/C++, PGI CUDA FORTRAN, or OpenCL compute kernels.

Up to Four Tesla Fermi GPUs, each with: 448 cores, 6 GB GDDR5, 1 TFLOP single and 515 GFLOP double precision performance Up to 24 cores with the newest Intel and AMD Processors, 128 GB memory, 80 PLUS® certified power supply, and eight hard drives Nvidia Quadro for state of the art professional graphics and visualization Ultra-quiet fans, strategically placed baffles, and internal sound-proofing New: Microway CL-IDE™ for OpenCL programming on CPUs and GPUs

WhisperStation with 4 Tesla Fermi GPUs

OctoPuter™ 4U Server with up to 8 GPUs and 144 GB memory Microway’s Latest Servers for Dense Clustering 4P, 1U nodes with 48 CPU cores, 512 GB and QDR InﬁniBand

2P, 1U nodes with 24 CPU cores, 2 Tesla GPUs and QDR InﬁniBand 1U Node with 2U Twin2 with 4 Hot-Swap MBs, each with 2 Processors + 256 GB 2 Tesla Fermi GPUs 1U S2050 servers with 4 Tesla Fermi GPUs

Microway Puts YOU on the Cutting Edge Design your next custom conﬁguration with techs who speak HPC. Rely on our integration expertise for complete and thorough testing of your workstations, turnkey clusters and servers. Whether you need Linux or Windows, CUDA or OpenCL, we’ve been resolving the complicated issues – so you don’t have to – since 1982.

Conﬁgure your next WhisperStation or Cluster today! microway.com/quickquote or call 508-746-7341 2U Twin2 Node with 4 Hot-Swap Motherboards Each with 2 CPUs and 256 GB Sign up for technical newsletters and special GPU promotions at microway.com/newsletter

GSA Schedule Contract Number: GS-35F-0431N

pC4_Microway.indd 1 10/17/10 5:07:51 PM