™ A LOOK AT KDE’s KStars Astronomy Program

Since 1994: The Original Magazine of the Community FEBRUARY 2016 | ISSUE 262 | www.linuxjournal.com + Programming Working with Command How-Tos Arguments in Your Program a Shell Scripts BeagleBone Interview: Katerina Black Barone-Adesi on to Help Brew Beer Developing the Snabb Switch Network Write a Toolkit Short Script to Solve a WATCH: ISSUE Math Puzzle OVERVIEW V

LJ262-February2016.indd 1 1/21/16 5:26 PM NEW! Agile Improve Product Business Development Processes with an Enterprise Practical books Author: Ted Schmidt Job Scheduler for the most technical Sponsor: IBM Author: Mike Diehl Sponsor: people on the planet. Skybot

Finding Your DIY Way: Mapping Commerce Site Your Network Author: to Improve Reuven M. Lerner Manageability GEEK GUIDES Sponsor: GeoTrust Author: Bill Childers Sponsor: InterMapper

Combating Get in the Infrastructure Fast Lane Sprawl with NVMe

Author: Author: Bill Childers Mike Diehl Sponsor: Sponsor: Puppet Labs Silicon Mechanics & Intel

Download books for free with a Take Control Linux in simple one-time registration. of Growing the Time Redis NoSQL of Malware http://geekguide.linuxjournal.com Server Clusters Author: Author: Federico Kereki Reuven M. Lerner Sponsor: Sponsor: IBM Bit9 + Carbon Black

LJ262-February2016.indd 2 1/21/16 5:26 PM NEW! Agile Improve Product Business Development Processes with an Enterprise Practical books Author: Ted Schmidt Job Scheduler for the most technical Sponsor: IBM Author: Mike Diehl Sponsor: people on the planet. Skybot

Finding Your DIY Way: Mapping Commerce Site Your Network Author: to Improve Reuven M. Lerner Manageability GEEK GUIDES Sponsor: GeoTrust Author: Bill Childers Sponsor: InterMapper

Combating Get in the Infrastructure Fast Lane Sprawl with NVMe

Author: Author: Bill Childers Mike Diehl Sponsor: Sponsor: Puppet Labs Silicon Mechanics & Intel

Download books for free with a Take Control Linux in simple one-time registration. of Growing the Time Redis NoSQL of Malware http://geekguide.linuxjournal.com Server Clusters Author: Author: Federico Kereki Reuven M. Lerner Sponsor: Sponsor: IBM Bit9 + Carbon Black

LJ262-February2016.indd 3 1/21/16 5:26 PM FEBRUARY 2016 CONTENTS ISSUE 262 FEATURES 64 Temperature Control in a Homebrewing Tun Using a BeagleBone Black How to program a BeagleBone Black to solve your beer-brewing problems. Klaus Kolle 88 Command-Line Tutorial: Does Every Year Have a Friday the 13th? Solving this puzzle is a great exercise in picking a handful of shell commands from Linux’s rich palette and combining them into a short script. Sol Lederman

ON THE COVER ‹7YVNYHTH)LHNSL)VUL)SHJR[V/LSW)YL^)LLYW ‹>YP[LH:OVY[:JYPW[[V:VS]LH4H[O7\aaSLW ‹(3VVRH[2+,Z2:[HYZ(Z[YVUVT`7YVNYHTW ‹>VYRPUN^P[O*VTTHUK(YN\TLU[ZPU@V\Y:OLSS:JYPW[ZW ‹0U[LY]PL^!2H[LYPUH)HYVUL(KLZPVU +L]LSVWPUN[OL:UHII:^P[JO5L[^VYR;VVSRP[W

4 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 4 1/21/16 5:26 PM COLUMNS 32 Reuven M. Lerner’s At the Forge Tech Book Roundup 40 Dave Taylor’s Work the Shell Working with Command Arguments 22 46 Shawn Powers’ The Open-Source Classroom The Powers That Be 52 Susan Sons’ Under the Sink Fast Network Routing, Meet Userspace 100 Doc Searls’ EOF Giving Silos Their Due 46 IN EVERY ISSUE 8 Current_Issue.tar.gz 10 Letters 14 UPFRONT 30 Editors’ Choice 60 New Products 103 Advertisers Index

64

LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., PO Box 980985, Houston, TX 77098 USA. Subscription rate is $29.50/year. Subscriptions start with the next issue.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 5

LJ262-February2016.indd 5 1/22/16 11:02 AM Executive Editor Jill Franklin [email protected] Senior Editor Doc Searls [email protected] Associate Editor Shawn Powers [email protected] Art Director Garrick Antikajian [email protected] Products Editor James Gray [email protected] Editor Emeritus Don Marti [email protected] Technical Editor Michael Baxter [email protected] Senior Columnist Reuven Lerner [email protected] Security Editor Mick Bauer [email protected] Hack Editor Kyle Rankin lj@greenfly.net Virtual Editor Bill Childers [email protected]

Contributing Editors )BRAHIM (ADDAD s 2OBERT ,OVE s :ACK "ROWN s $AVE 0HILLIPS s -ARCO &IORETTI s ,UDOVIC -ARCOTTE 0AUL "ARRY s 0AUL -+ENNEY s $AVE 4AYLOR s $IRK %LMENDORF s *USTIN 2YAN s !DAM -ONSEN

President Carlie Fairchild [email protected]

Publisher Mark Irgang [email protected]

Associate Publisher John Grogan [email protected]

Director of Digital Experience Katherine Druckman [email protected]

Accountant Candy Beauchamp [email protected]

Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA

Editorial Advisory Panel Nick Baronian Kalyana Krishna Chadalavada "RIAN #ONNER s +EIR $AVIS -ICHAEL %AGER s 6ICTOR 'REGORIO $AVID ! ,ANE s 3TEVE -ARQUEZ $AVE -C!LLISTER s 4HOMAS 1UINLAN #HRIS $ 3TARK s 0ATRICK 3WARTZ

Advertising % -!),: [email protected] URL: www.linuxjournal.com/advertising 0(/.%     EXT 

Subscriptions % -!),: [email protected] URL: www.linuxjournal.com/subscribe MAIL: PO Box 980985, Houston, TX 77098 USA

LINUX IS A REGISTERED TRADEMARK OF ,INUS 4ORVALDS

LJ262-February2016.indd 6 1/21/16 5:27 PM Where every interaction matters.

break down your innovation barriers

power your business to its full potential When you’re presented with new opportunities, you want to focus on turning them into successes, not whether your IT solution can support them.

Peer 1 Hosting powers your business with our wholly owned FastFiber NetworkTM,

solutions that are secure, scalable, and customized for your business.

Unsurpassed performance and reliability help build your business foundation to be rock-solid, ready for high growth, and deliver the fast user experience your customers expect.

Want more on cloud? Call: 844.855.6655 | go.peer1.com/linux | Vew Cloud Webinar:

Public and Private Cloud | Managed Hosting | Dedicated Hosting | Colocation

LJ262-February2016.indd 7 1/21/16 5:27 PM Current_Issue.tar.gz

For the Love SHAWN POWERS of Linux

love my job. I teach Linux by day Reuven M. Lerner provides a and write about Linux at night. BREAKDOWN OF SOME OF HIS TOP PICKS I )TS EASY TO FALL IN LOVE WITH YOUR ON TOPICS FROM PROGRAMMING TO work when the things you do align podcasting. It’s hard to go wrong WITH YOUR PASSIONS !LL OF US HERE IN with a good book, and Reuven will the Linux Journal community have HELP YOU FIND ONE A LOVE FOR ,INUX AND OPEN SOURCE 7ITH THE HELP OF $AVE 4AYLOR YOU but even inside our world, there are will learn to deal with command some topics that are just downright arguments in scripts using getopt FUN 4HIS MONTH IS FULL OF ARTICLES in your code. Dealing with we’re passionate about. arguments doesn’t seem like a big *OEY "ERNARD STARTS OFF WITH A DEAL BUT WHAT IF PEOPLE COMBINE LOOK AT +3TARS )F YOURE A SPACE NUT THEM &OR INSTANCE -rf instead like me, you’ll want to check out OF -r -f 2ATHER THAN WRITE PAGES HIS DETAILED LOOK AT THE +$% NATIVE OF CONDITIONALS $AVE EXPLORES HOW astronomy program. It’s dark enough to use getopt, which does all the to see the only at night, but DIRTY WORK FOR YOU )F YOU NEED TO WITH *OEYS HELP YOU CAN SURF THE write a script that accepts command- NIGHT SKY ANY TIME OF DAY 4HEN line arguments, this month’s column WHAT BETTER WAY TO END A DAY OF is a must-read. LCD gazing than to read a ) TAKE A AWAY FROM THE TECH BOOK ON YOUR FAVORITE SUBJECT keyboard this issue and head around TO THE BACK OF THE ˆ

V VIDEO: SPECIFICALLY TO THE POWER CORD -Y Shawn Powers runs FAMILY RECENTLY MOVED INTO AN OLD through the latest issue. TURN OF THE CENTURY HOUSE AND EVEN

8 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 8 1/21/16 5:27 PM CURRENT_ISSUE.TAR.GZ

with a massive electrical overhaul, the 3OL ,EDERMAN FINISHES OFF THE POWER IS FLAKY AT BEST ) DECIDED TO issue with some awesome problem share my experience this month and SOLVING 4HE BEST USE OF ,INUX OR CLARIFY SOME OF THE VARIOUS HARDWARE technology in general, is to make options available to help regulate QUICK WORK OF SOMETHING THAT WOULD and stabilize the electricity coming take a long time to do on your own. INTO YOUR HOME OR BUSINESS 4HE FACT 3CRIPTING WAS DESIGNED FOR THAT that I’m writing this on a desktop exact purpose. In this command-line computer rather than a laptop means tutorial, Sol describes how to write a MY HARD WORK HAS PAID OFF ONE LINER TO FIND A SPECIFIC DATE AND Susan Sons has an incredible then put that one-liner into a script interview with Katerina Barone-Adesi TO MAKE IT MORE FLEXIBLE -OST GOOD this month where she gets the details programs start with a problem that ON 3NABB )F YOUVE NEVER HEARD OF needs to be solved, and in this case, Snabb, you’ll be glad you read the THE PROBLEM IS TRYING TO FIND THE NEXT interview, because it’s an incredible Friday the 13th. network toolkit that isn’t built in We all have passions that drive us TO THE ,INUX KERNEL )F THE IDEA OF TO LEARN AND EXPLORE &OR SOME OF bypassing the to increase US ME ITS FEEDING THE BIRDS &OR SPEED SEEMS ODD YOU DEFINITELY WANT OTHERS ITS BREWING THE PERFECT BEER TO READ THIS INTERVIEW The best part about being a Linux You might remember Kyle Rankin geek is that we can take advantage WRITING A FEW YEARS AGO ABOUT USING OF ,INUXS OPEN SOURCE NATURE AND a Pogo Linux device (and later a make our passions really come to , I think) to keep his LIFE 7E HOPE YOU SHARE SOME OF beer chilled while it the excitement we brought into this AGED +LAUS +OLLE TAKES A DIFFERENT issue and can’t wait to hear about approach, and using a BeagleBone WHAT SORTS OF PROJECTS YOU IMPROVE "LACK HE CONTROLS THE HEATING OF WITH THE HELP OF OUR FAVORITE /3Q the mash during his beer-making process. Whether you need to keep Shawn Powers is the Associate Editor for Linux Journal. your homebrew hot or cold, it turns He’s also the Gadget Guy for LinuxJournal.com, and he has out Linux can be the answer in either an interesting collection of vintage Garfield coffee mugs. CASE #HECK OUT HIS COOL PROJECT THIS Don’t let his silly hairdo fool you, he’s a pretty ordinary guy month, and potentially improve your and can be reached via e-mail at [email protected]. beer while improving your geek cred. Or, swing by the #linuxjournal IRC channel on Freenode.net.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 9

LJ262-February2016.indd 9 1/21/16 5:27 PM letters

Server Hardening Regarding Greg Bledsoe’s “Server (ARDENINGv ARTICLE IN THE .OVEMBER  ISSUE GREAT ARTICLEˆLOTS OF detailed help in one source. One QUESTION IS IT POSSIBLE TO MENTION SPECIFIC LOGS YOU REFERENCE IN THE ARTICLE ) GET LOST QUICKLY WHEN SEEING THE LARGE NUMBER OF LOGS SCATTERED about my server. —Tom Browder

Bug in Script ) BELIEVE THAT THE FINAL VERSION OF THE Hardware Interrogation script that Dave Taylor came up with I’ve just read Federico Kereki’s in his Work the Shell column titled article about interrogating a h!NALYZING #OMMA 3EPARATED 6ALUES Linux system titled “What’s in the #36 &ILESv IN THE $ECEMBER  ISSUE "OX )NTERROGATE 9OUR (ARDWAREv OF Linux Journal contains an oversight. in the December 2015 issue. I 3PECIFICALLY IT DOES NOT HANDLE THE CASE LOVE THIS KIND OF ARTICLE AND HOPE IN WHICH MORE THAN ONE FIELD CONTAINS TO SEE MORE COMMAS FOR EXAMPLE THE DOLLAR —Brian Clark AMOUNT FIELD AND THE COMMENT FIELD  ) HAVE MODIFIED $AVES SCRIPT TO TAKE Federico Kereki replies: Thanks, THIS INTO ACCOUNT (OPEFULLY THIS WILL Mr Clark, for your kind words. The BE OF SOME HELP ) ALWAYS ENJOY $AVES article grew out of my actual need COLUMN AND HAVE LEARNED A LOT FROM IT to know about the hardware in (ERES THE MODIFIED SCRIPT my own machine, and because of

Linux’s openness, I learned even #! /bin/ -

more than I had expected. I’m glad

you liked my results! # fixcsv

10 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 10 1/21/16 5:27 PM [ LETTERS ]

# fix CSV files with embedded commas # For each line that contains at least one field enclosed

in double quotes, process each such field from left to

# The problem is that some spreadsheet fields may contain right until no fields are enclosed in double quotes and

commas. In the sample case, this includes the dollar all remaining commas are field separators. The steps are:

amount and comment fields. I believe you overlooked the (1) replace the double quotes enclosing the field being

case in which both the dollar amount and comment fields processed with a temporary delimiter to isolate that

contain commas. Your script assumes that there is at specific field, (2) remove any commas embedded in the

most one such instance. isolated field, (3) reconstruct the line without the

temporary delimiters. The temporary delimiter must be

# The simplest solution is to export the spreadsheet a single character (for the cut command) that cannot

contents with some field delimiter that can never appear appear in the input file. I selected an asterisk (*),

in any field, e.g., a tab. Then write the script but other characters can be used. Some characters (such

using this delimiter. as asterisk, colon, hyphen, and equals) work fine, while

others (such as tab and semicolon) do not.

# Original code

td=* # temporary delimiter

# while read inline

# do while read inline

# if [ ! -z "$(echo $inline | grep \")" ] do

# then while [ ! -z "$(echo $inline | grep \")" ]

# f1=$(echo $inline | cut -d\" -f1) do

# f2=$(echo $inline | cut -d\" -f2) inline=$(echo $inline | sed "s/\"/$td/" | sed "s/\"/$td/")

# f3=$(echo $inline | cut -d\" -f3) f1=$(echo $inline | cut -d"$td" -f1)

# echo $f1`echo $f2|sed 's/,//g'`$f3 f2=$(echo $inline | cut -d"$td" -f2)

# else f3=$(echo $inline | cut -d"$td" -f3)

# echo $inline inline=$(echo "$f1$(echo $f2 | sed 's/,//g')$f3")

# fi done

# done echo $inline

# exit 0 done

exit 0

# This works correctly ONLY when there is EXACTLY ONE

field enclosed in double quotes. # Test input file fixcsvtest.txt:

# Revised code $ cat fixcsvtest.txt

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 11

LJ262-February2016.indd 11 1/21/16 5:27 PM [ LETTERS ]

4/7/14,subscriptions,199.99,Ask Dave Taylor Monthly ab,cd,ef,gh

4/10/14,subscriptions,"1,300.99",Linux Journal ab,cd,ef,gh

4/10/14,subscriptions,"1,300.99","Linux Journal, APR 2014" ab,cd,ef,gh

4/10/14,subscriptions,19.99,"Linux Journal, annual" ab,cd,ef,gh

ab,cd,ef,gh

ab,cd,ef,gh ab,cd,ef,gh

ab,cd,ef,"g,h" ab,cd,ef,gh

ab,cd,"e,f",gh ab,cd,ef,gh

ab,cd,"e,f","g,h" ab,cd,ef,gh

ab,"c,d",ef,gh ab,cd,ef,gh

ab,"c,d",ef,"g,h" ab,cd,ef,gh

ab,"c,d","e,f",gh ab,cd,ef,gh

ab,"c,d","e,f","g,h" $

"a,b",cd,ef,gh

"a,b",cd,ef,"g,h" —Jeff Mumma

"a,b",cd,"e,f",gh

"a,b",cd,"e,f","g,h" Dave Taylor replies: Thanks for your

"a,b","c,d",ef,gh note, Jeff, and I do believe you’re

"a,b","c,d",ef,"g,h" correct that I didn’t test the case

"a,b","c,d","e,f",gh where more than a single field of the

"a,b","c,d","e,f","g,h" input data included commas. Bah,

$ pesky debugging! I like your mods,

and yet still have a niggling sense that

# Test Results the entire problem can be sidestepped

with the perfect regular expression. If

$ ./fixcsv < fixcsvtest.txt I only had a few weeks to create it!

4/7/14,subscriptions,199.99,Ask Dave Taylor Monthly

4/10/14,subscriptions,1300.99,Linux Journal Photo of the Month

4/10/14,subscriptions,1300.99,Linux Journal APR 2014 I thought you would like to see an

4/10/14,subscriptions,19.99,Linux Journal annual unusual place where LJ is being read

THIS MONTH  DEGREES NORTH 

ab,cd,ef,gh DEGREES WEST 4HATS THE MIDDLE OF THE

ab,cd,ef,gh Atlantic Ocean at 20 knots heading

ab,cd,ef,gh FOR .9# 4HE SATELLITE )NTERNET COSTS

ab,cd,ef,gh are rather steep on board so I brought

12 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 12 1/21/16 5:27 PM [ LETTERS ]

A FEW ISSUES WITH ME -UST DASH AS THE IS over the yard arm. At Your Service

SUBSCRIPTIONS: Linux Journal is available in a variety of digital formats, including PDF, .epub, .mobi and an on-line digital edition, as well as apps for iOS and Android devices. Renewing your subscription, changing your e-mail address for issue delivery, paying your invoice, viewing your account details or other subscription inquiries can be done instantly on-line: http://www.linuxjournal.com/subs. E-mail us at [email protected] or reach us via postal mail at Linux Journal, PO Box 980985, Houston, TX 77098 USA. Please remember to include your complete name and address when contacting us.

ACCESSING THE DIGITAL ARCHIVE: Your monthly download notifications will have links to the various formats and to the digital archive. To access the digital archive at any time, log in at http://www.linuxjournal.com/digital.

LETTERS TO THE EDITOR: We welcome your letters and encourage you to submit them at http://www.linuxjournal.com/contact or mail them to Linux Journal, PO Box 980985, Houston, TX 77098 USA. Letters may be edited for space and clarity.

WRITING FOR US: We always are looking for contributed articles, tutorials and real-world stories for the magazine. An author’s guide, a list of topics and due dates can be found on-line: http://www.linuxjournal.com/author.

FREE e-NEWSLETTERS: Linux Journal editors publish newsletters on both a weekly and monthly basis. Receive late-breaking news, technical tips and —Roger Greenwood tricks, an inside look at upcoming issues and links to in-depth stories featured on http://www.linuxjournal.com. Subscribe for free today: http://www.linuxjournal.com/ PHOTO OF THE MONTH enewsletters.

Remember, send your Linux-related ADVERTISING: Linux Journal is a great photos to [email protected]! resource for readers and advertisers alike. Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising and marketing opportunities by visiting WRITE LJ A LETTER us on-line: http://ww.linuxjournal.com/ advertising. Contact us directly for further We love hearing from our readers. Please information: [email protected] or send us your comments and feedback via +1 713-344-1956 ext. 2. http://www.linuxjournal.com/contact.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 13

LJ262-February2016.indd 13 1/21/16 5:27 PM UPFRONT NEWS + FUN

diff -u What’s New in Kernel Development The OOM killer is a tough nut to Michal posted a patch to create a crack. How can a system recover new kernel thread that would reclaim when it’s violently thrashing and out THAT EXTRA MEMORY IF IT WENT UNUSED OF 2!- /NCE UPON A TIME YOUD Then the OOM killer could try the just have to reboot. And today, that SAME THING ON A DIFFERENT PROCESS still might be necessary, but less so, AND HOPEFULLY HAVE A DIFFERENT RESULT because the OOM killer attempts to And although there were no major IDENTIFY AND STOP THE PROCESS THAT OBJECTIONS TO -ICHALS PATCH ITSELF seems to be causing the hangup. A VARIETY OF FOLKS OBJECTED TO THE The problem is, it may not choose IDEA OF MAKING ANY KIND OF the right process every time. Another incremental improvement to the problem is that the whole thing is OOM killer, when the Big Problem super tough and complex. had not yet been solved. Michal Hocko recently tried to The Big Problem, as described PEEL OFF A SLIVER TO WORK ON TAKING by Johannes Weiner, was how to THE LEAD FROM Mel Gorman and resolve memory deadlocks in general. Oleg Nesterov. Apparently, the Only by solving that problem could current OOM killer would allocate an THE //- KILLER SUCCESSFULLY KILL THE EXTRA BATCH OF MEMORY JUST FOR THE processes it needed to, even to the process it wanted to kill to actually POINT OF KILLING ALL USER PROCESSES JUST give it enough breathing room to to keep the kernel up. terminate properly. But under some "UT -ICHAL MADE A POINT OF KEEPING circumstances, the process would the discussion clamped down to a accept the extra memory and still CONSIDERATION OF ONLY THE SMALL FIXES hang the system. Then with no more he’d proposed. He acknowledged memory to dole out, the OOM killed THAT HE HAD NO SOLUTION FOR THE "IG couldn’t try again, and it was time to Problem, and he pointed out that hit the reset button. no one else seemed to have a viable

14 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 14 1/21/16 5:27 PM [ UPFRONT ]

SOLUTION FOR THE "IG 0ROBLEM EITHER ALL IF YOU CANT GET THE LOCK And until something viable came up, Michal saw no point in stalling OOM  !VOIDING !""! DEADLOCKS IF KILLER DEVELOPMENT )F SOMETHING COULD you have an A->B locking order, BE DONE TO IMPROVE IT HE FELT THEN IT BUT YOU ALREADY HOLD " INSTEAD OF should be done. “drop B, then take A and B in the By and large everyone went along RIGHT ORDERv YOU MAY DECIDE FIRST with this, but still, it’s clear there’s to trylock(A) AND IF THAT FAILS A LOT OF PRESSURE ON THE //- KILLER YOU THEN FALL BACK ON THE hDROP AND SYSTEM TO COME UP WITH SOME KIND OF RELOCK IN THE RIGHT ORDERv new idea or at least to create a policy- BASED SYSTEM THAT PUTS CONTROL OF THE "UT IF WHAT YOU WANT TO CREATE IS CHOICES OF PROCESSES TO KILL INTO THE A hGET LOCK USING TRYLOCKv YOU HANDS OF SYSTEM ADMINISTRATORS RATHER NEED TO BE VERY AWARE OF THE CACHE than the kernel algorithms themselves. COHERENCY TRAFFIC ISSUE AT LEAST had some advice FOR ANYONE WRITING KERNEL CODE It is possible that we should think that needs to lock resources: it’s about trying to introduce a new probably better to use existing locking PRIMITIVE FOR THAT loop_try_lock() implementations rather than rolling thing. But it’s probably not common YOUR OWNˆAT LEAST UNTIL YOU KNOW ENOUGH TO BE WORTH ITˆWEVE HAD what you’re doing. As he put it: THIS ISSUE BEFORE BUT ) THINK ITS A hONCE EVERY COUPLE OF YEARSv KIND OF People need to realize that locking thing rather than anything that we is harder than they think, and not need to worry about. cook up their own lock primitives using things like trylock without 4HE hLOCKING IS HARDv ISSUE IS VERY really thinking about it a lot. real, though. We’ve traditionally had a lot OF CODE THAT TRIED TO DO ITS OWN Basically, trylock() on its own locking, and not getting the memory should never be used in a loop. ordering right, etc. Things that 4HE MAIN USE FOR TRYLOCK SHOULD HAPPEN TO WORK ON X BUT DONT ON BE ONE OF other architectures, etc.

1) Thing that you can just not do at —ZACK BROWN

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 15

LJ262-February2016.indd 15 1/21/16 5:27 PM [ UPFRONT ]

Android Candy: Quick Games

The biggest problem I have with LEARN BECAUSE hLEARNING TO HAVE FUNv GAMING IS THAT IT TAKES FAR TOO LONG TO ISNT VERY MUCH FUN AT ALL GET hINTOv GAMES )M GENERALLY VERY 4HERE ARE A FEW OLD STANDBYS THAT busy, and my gaming time usually lasts work well: Bejeweled, Angry Birds, AS LONG AS IT TAKES FOR THE DENTIST TO Peggle, Plants vs. Zombies, Candy CALL ME IN FROM THE WAITING ROOM OR Crush and so on. The problem is, possibly how long it takes me to use ALTHOUGH THOSE GAMES ARE FUN ) GROW the bathroom, but eiw, let’s not go TIRED OF THEM FAIRLY QUICKLYˆEITHER THAT THERE  &OR ME THE PERFECT GAME CAN OR THEYRE SO ADDICTIVE ) FEAR MY FAMILY BE FUN EVEN IF ) CAN PLAY ONLY FOR A FEW WILL DISOWN ME FOR IGNORING THEM MINUTES )T ALSO HAS TO BE VERY QUICK TO while I play just one more level.

16 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 16 1/21/16 5:27 PM [ UPFRONT ]

Here are some games I’ve been playing lately: They

Q Swish ITS A PUZZLE GAME BUT ITS JUST DIFFERENT Said It ENOUGH THAT ) FIND IT FUN 4HE PHYSICS SEEM Sometimes we hRIGHTv TO ME WHILE PLAYING AND THE GRAPHICS do a thing in are really great. The premise is that you’re an order to find alien playing basketball in space. You know, like out the reason ALIENS ALWAYS DO 4HERE ARE  LEVELS AND THERES for it. Sometimes ENOUGH OF A CHALLENGE TO MAKE IT FUN our actions are questions Q Dumb Ways to Die 2 ) LOVED THE FIRST Dumb Ways not answers. —John Le Carré to Die GAME )T WAS ABSURD )T WAS FUNNY )T WAS CARTOONISHLY MORBID 0ART  IS MORE OF THE SAME The greatest and that’s a very, very good thing. justice in life is that your Q Asphalt Series THERE ARE A BUNCH OF GAMES IN vision and THE !SPHALT RACING SERIES 3OME OF THEM WORK looks tend to go BETTER THAN OTHERS AND ALL OF THEM ARE FAIRLY simultaneously. large downloads. I like Asphalt because you —Kevin Bacon GET TO RACE REALLY COOL CARS REALLY FAST )F YOU There are some ever played Burnout on the PlayStation, Asphalt things you learn GAMES WILL SEEM PLEASANTLY FAMILIAR 9OU HAVE best in calm, and TO PAY FOR GAS IF YOU PLAY FOR TOO LONG BUT ) some in storm. generally don’t have enough time and play each —Willa Cather session only until my gas is gone. The only true 7HAT ARE YOUR FAVORITE hFIVE MINUTE GAMESv happiness comes FOR THE !NDROID PLATFORM ) NEVER CAN REALLY GET from squandering INTO A BOOK IN THAT SHORT AMOUNT OF TIME SO IT ourselves for a purpose. TENDS TO BE THE ONLY TIME ) PLAY GAMES )F YOU —William Cowper have any suggestions, drop me a message at SHAWN LINUXJOURNALCOM AND )LL TRY TO FOLLOW UP If your ship IN LATER MONTHS WITH THE BEST OF THE BEST doesn’t come in, .OTE YOU CAN FIND ALL THE GAMES MENTIONED HERE swim out to it! in the Google Play Store.—SHAWN POWERS —Jonathan Winters

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 17

LJ262-February2016.indd 17 1/21/16 5:27 PM [ UPFRONT ]

Non-Linux FOSS: Snk

18 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 18 1/21/16 5:27 PM [ UPFRONT ]

I’m apparently in a silly-game mood )F YOURE JUST STARTING WITH /3 8 this month, because I stumbled 3WIFT DEVELOPMENT Snk is a project across an open-source project I YOU CAN TWEAK FOR SOME LEARNING COULDNT KEEP ALL TO MYSELF Snk )F ON THE FLY YOU REMEMBER THE CLASSIC GAME OF -Y FAVORITE PART OF THE PROJECT IS snake, Snk is the same concept, but ACTUALLY THE hREVIEWSv FOR IT (EAD smaller, harder and with music. over to http://www.mowglii.com/snk I actually really like the Snk to grab Snk and its PROGRAM BECAUSE ITS FAIRLY SIMPLE today. And, good luck with level and the developer (Mowglii) three; I run directly into the wall has put the Xcode project on GitHub EVERY TIME FOR FOLKS TO DOWNLOAD —SHAWN POWERS

2015 Linux Journal Archive NOW AVAILABLE as a DVD or Digital Download www.linuxjournal.com/archive

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 19

LJ262-February2016.indd 19 1/21/16 5:27 PM [ UPFRONT ]

Handheld Emulation: Achievement Unlocked!

I love video game emulation. My own the original cartridges, so I’m FAVORITE GAMES WERE PRODUCED IN NOT GOING TO TELL YOU WHERE TO FIND THE S AND S SO IF ) WANT ROMs to download or anything like to play them, I almost always have that. What I am going to share is to emulate the old systems. There MY RECENT DISCOVERY OF THE PERFECT is usually a legal concern about handheld gaming system. Oddly 2/- FILES FOR GAMES EVEN IF YOU enough, it was never intended to be

(Image from http://wololo.net)

20 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 20 1/21/16 5:27 PM [ UPFRONT ]

an . TOUGHEST STEPS )F YOU LIKE THE LARGER The PSP is truly incredible layout, I recommend the PSP 2000 HARDWARE 4HE 030 6ITA IS ITS model. It has an incredible screen BIGGER YOUNGER SIBLING BUT IF AND FEWER BUGGY DESIGN CHOICES you have an old PSP, I urge you THAN THE ORIGINAL )F YOURE LOOKING not to throw it away. With a FOR PORTABILITY )M VERY FOND OF THE SIMPLE HACK ALSO LEGALLY 030 'O ) PURCHASED FROM E"AY 4HE QUESTIONABLE ) SUPPOSE ITS screen is smaller, but it’s still plenty possible to load that LARGE AND HAS BEAUTIFUL QUALITY WILL PLAY !TARI .%3 3.%3 'AME 'OOD LUCK AND HAVE FUN Boy, Genesis, PS1 and most other —SHAWN POWERS CONSOLE GAMES ALMOST FLAWLESSLY I never had a PSP, but I was able to get a PSP Go in mint CONDITION ON E"AY FOR  4HE 030 'O COMES WITH '" OF storage, so you don’t even need LINUX JOURNAL to get its proprietary memory on your e-Reader CARD TO LOAD IT UP WITH GAMES /NE OF THE PROBLEMS WITH THE emulation scene is that sites seem TO COME AND GO FAIRLY REGULARLY ) FOUND ALL THE INFORMATION ) NEEDED to get my PSP Go ready to play Mario by doing some Google SEARCHING FOR 030 EMULATORS e-Reader 3PECIFICALLY THIS PAGE WAS GREAT editions http://wololo.net/emulators-for-the- FREE for Subscribers psp-ps-vita-the-ultimate-download-list. )F YOU ALREADY HAVE A 030 DEVICE Customized Kindle and Nook THE INSTRUCTIONS FOR editions now available INSTALLATION IS SIMPLE )F YOU DONT have one, deciding which version LEARN MORE OF THE 030 TO PURCHASE IS ONE OF THE

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 21

LJ262-February2016.indd 21 1/21/16 5:27 PM [ UPFRONT ]

Astronomy for KDE

Although I have covered a large THIS ARTICLE )M STARTING OFF WITH THE NUMBER OF SCIENCE APPLICATIONS IN THIS KStars astronomy program. space, I haven’t really looked at too )F YOU HAVE THE FULL +$% ENVIRONMENT MANY OPTIONS AVAILABLE WITHIN THE +$% installed, you already should have it desktop environment. This has been AVAILABLE )F YOU DONT YOU SHOULD due to my own biases in using a GTK- be able to install it. For example, you based desktop environment, but now can install KStars on Debian-based )D LIKE TO LOOK AT SOME OF THE PACKAGES distributions with this command: AVAILABLE FOR PEOPLE WHO REALLY LIKE TO USE +$% ON THEIR OWN MACHINES 3O IN sudo apt-get install kstars

Figure 1. When you first start KStars, you need to go through the setup wizard.

22 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 22 1/21/16 5:27 PM [ UPFRONT ]

)F THIS IS THE FIRST +$% BASED command in a terminal window. application that you are installing, 4HE FIRST TIME YOU START +3TARS it also will need to pull in a rather you need to go through the setup LARGE SET OF DEPENDENCIESˆTHATS WIZARD TO CONFIGURE ELEMENTS LIKE JUST THE PRICE OF USING A NEW YOUR LOCATION /NCE +3TARS FINISHES '5) TOOLKIT /F COURSE ANY OTHER starting up, you should see a display packages will be incrementally OF THE SKY FROM THE LOCATION YOU SET SMALLER SINCE ALL OF THE SHARED during the setup. dependencies already will be there. You can pan the display around To launch KStars, you either can simply by clicking and dragging the click on a menu item in your desktop STAR FIELD TO SEE LOCATIONS OF INTEREST environment or enter the kstars The items that are labeled depend

Figure 2. You get a display of the sky from your location when KStars starts.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 23

LJ262-February2016.indd 23 1/21/16 5:27 PM [ UPFRONT ]

Figure 3. You can interact with the objects in the display.

on your zoom level. Two buttons at the display move around until the THE TOP OF THE DISPLAY ALLOW YOU TO OBJECT IS IN THE CENTER OF THE DISPLAY zoom in and out. Beside these is a )F IT IS BELOW THE HORIZON A WARNING third button, labeled Find Object, BOX WILL POP UP ASKING IF YOU STILL that you can click on to get a search want to re-center the display. WINDOW 4HIS WINDOW CAN USE FILTERS Once you have selected your object, TO SEARCH FOR PARTICULAR TYPES OF you can right-click on it to get a objects, like or , or DROP DOWN MENU OF THINGS YOU CAN YOU CAN SEARCH THROUGH ALL OF THE do with it. objects that KStars knows about. 4HE HEADER OF THIS DROP DOWN /NCE YOU FIND AN OBJECT OF CONTAINS THE FULL NAME OF THE OBJECT interest, you can click on it and have along with rising and setting times.

24 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 24 1/21/16 5:27 PM [ UPFRONT ]

Figure 4. The detail window has a lot of extra information available for most objects.

"ELOW THAT THE FIRST OPTION IS TO ACROSS THE FIELD OF VIEW 9OU ALSO center and track the object. This is can calculate the angular distance USEFUL BECAUSE THE DEFAULT DISPLAY to another object or plan out a star- mode is to have the display updated hopping to some other object. in real time. This way, you always For some objects, there may be HAVE A VIEW OF WHAT THE SKY LOOKS IMAGES FROM THE $EEP 3KY 3URVEY like right now. You can select the AVAILABLE )F THEY ARE THERE WILL BE details option to pull up even more options on this drop-down menu INFORMATION ON THE OBJECT LABELED h3HOW $33 )MAGEv OR h3HOW 9OU CAN ADD A FLAG TO MAKE THE 3$33 )MAGEv OBJECT EASIER TO KEEP TRACK OF OR !LTHOUGH +3TARS COMES WITH QUITE even add tracks as the object moves A BIT OF DATA WHEN YOU INSTALL IT THIS

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 25

LJ262-February2016.indd 25 1/21/16 5:27 PM Figure 5. Several extra data sets are available that you can download and install.

isn’t everything that is available. You by clicking on the menu item can add new data sources by clicking DataAUpdatesA and selecting on the menu item DataADownload which catalog to update. You New Data. This will open up a even can import your own data by new dialog window giving you a list GOING TO THE CONFIGURATION WINDOW OF WHAT DATA CATALOGS ARE AVAILABLE and clicking the Import Catalog... to download. button on the Catalogs section. &OR SOME OF THE DATA SETS LIKE 7HEN YOU ARE IN THE CONFIGURATION AND INFORMATION window, you can see that you also THERE IS A CONSTANT UPDATING OF can change settings on how to THE DETAILED INFORMATION AVAILABLE VIEW SEVERAL DIFFERENT CATEGORIES to the astronomical community. such as solar system objects, You can download those updates satellites and supernovae.

26 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 26 1/21/16 5:27 PM [ UPFRONT ]

Figure 6. You can add a telescope as a device under the control of KStars.

4HE GUIDES SECTION DEFINES WHAT item ObservationAObservation EXTRA INFORMATION IS DISPLAYED SUCH Planner will pop up a window where as constellation lines, names and YOU CAN DEFINE A COMPLETE SET OF the Milky Way. The INDI (Instrument OBSERVATIONS YOU WANT TO RUN )F IT IS .EUTRAL $ISTRIBUTED )NTERFACE SECTION A BIT CONFUSING AT FIRST A WIZARD IS controls how KStars talks to your available to help walk you through connected telescope. You can pull up setting up a plan. the telescope wizard by clicking on While you will make your own ToolsADevicesATelescope W izard. observations, you also may want Once the connection is made, to look at observations made by you can pull up the control panel OTHER RESEARCHERS 4HE FILE FORMAT and send instructions to your MOST OFTEN USED IS THE &)43 FORMAT telescope. You can even automate (Flexible Image Transport System). your observations using this KStars includes a FITS viewer, which FUNCTIONALITY #LICKING ON THE MENU HAS A NUMBER OF ANALYSIS TOOLS

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 27

LJ262-February2016.indd 27 1/21/16 5:27 PM Figure 7. Once it is connected to your computer, you can use KStars to control your telescope.

9OU CAN OPEN A &)43 FILE BY CLICKING with the data in the image. You can on FileAOpen FITS. LOOK AT THE BASIC STATISTICS OF THE IMAGE Several tools are available to work including the width, height, maximum,

28 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 28 1/21/16 5:27 PM [ UPFRONT ]

Figure 8. The FITS viewer lets you analyze and manipulate observational images.

MINIMUM AND MEAN OF THE PIXELS 9OU can pull up a histogram, showing the FREQUENCY SPREAD OF THE IMAGE DATA 4HE 6IEW MENU ITEM INCLUDES SEVERAL MORE TOOLS WHERE YOU CAN EQUALIZE THE image, pass it through a high contrast FILTER OR APPLY AN AUTO STRETCH 4HERE IS even an entry to mark any stars that +3TARS CAN IDENTIFY AUTOMATICALLY You now have another astronomy program in your toolkit that you can use WHEN YOUR SCIENTIFIC RESEARCH MOVES IN that direction. In the coming months, I PLAN TO LOOK AT OTHER SCIENTIFIC SOFTWARE packages within this desktop environment and see just how much research can be DONE WITH +$%—JOEY BERNARD

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 29

LJ262-February2016.indd 29 1/21/16 5:27 PM [ EDITORS' CHOICE ]

™

EDITORS’ Poppins CHOICE -Y FRIEND AND FELLOW Linux Journalian programs are ★ Kris Occhipinti recently posted a available, but REMINDER ON &ACEBOOK FOR EVERYONE TO Poppins doesn’t BACK UP REGULARLY IN  !LTHOUGH TRY TO BE A FULL BLOWN SYSTEM RATHER it’s something we already should be ITS A SIMPLE ONE LINER THAT WILL DO FILE DOING IF YOURE NOT A REGULAR BACKER rotation, snapshots and more. It can UPPER YOU SHOULD START TODAY 4HE be automated with cron, or you can METHOD OF BACKUP ISNT NEARLY AS RUN IT MANUALLY FROM THE COMMAND IMPORTANT AS THE ACT ITSELF BUT THIS line. (But you should really, really MONTH ) FOUND A NEW PROJECT THAT MAKE A CRON JOB SIMPLIFIES THE BACKUP PROCESS NICELY Other backup systems have more Poppins is an open-source project ROBUST INTERFACES RESTORATION OPTIONS that builds on the SSH and rsync AND SO ON BUT THE BEAUTY OF 0OPPINS programs to create an incremental is its simplicity. It’s just one step above BACKUP SYSTEM THAT IS SIMPLE FAST MANUALLY RUNNING RSYNC YOURSELF BUT AND RELIABLE 4ONS OF OTHER BACKUP that one step means it’s simpler to do. In my world, simple is about the only way to make sure something gets DONE AT ALL )N FACT EVEN THOUGH ITS A NEW project and still in beta, Poppins gets THIS MONTHS %DITORS #HOICE AWARD It’s simple enough that you might actually get around to using it, and it HAS ENOUGH FEATURES TO REALLY BENEFIT YOU IN THE CASE OF A CATASTROPHIC FAILURE #HECK IT OUT TODAY AT http://poppinsbackups.wordpress.com, or go right to the Bitbucket page: http://bitbucket.org/poppins. —SHAWN POWERS

30 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 30 1/21/16 5:27 PM A Technical Conference Exploring Open Tech and the Open Web

March 16 & 17 Downtown Atlanta In the heart of Technology Square and Georgia Tech University www.greatwideopen.org

SOME OF THE TOP TECHNOLOGISTS IN THE WORLD WILL BE FEATURED:

Kelsey Hightower Danese Cooper Chris Van Tuin Steve Klabnik Erica Stanley Trek Glowacki Developer Advocate Distinguished Member/CTO Chief Technologist/ Developer/Author Founder Core Team Member Google Cloud PayPal/Wikimedia Western Mozilla Acire Studios, Women Ember.js Platform Foundation Who Code - Atlanta

LJ262-February2016.indd 31 1/21/16 5:27 PM COLUMNS AT THE FORGE Tech Book REUVEN M. Roundup LERNER Looking to sharpen your professional skills? Here’s Reuven’s latest roundup of useful books to improve your knowledge and career.

In the computer industry, you books (and other media) are not only DONT HAVE A CHOICEˆYOU MUST SUBJECTIVE IN TERMS OF THE CONTENT constantly keep learning new things, BUT ALSO IN TERMS OF THE TIMING ITS just to stay in place. I spend many QUITE POSSIBLE THAT ) ONLY RECENTLY hours each week reading blogs, discovered a book that came out a WATCHING CONFERENCE VIDEOS AND JUST while ago. I try to mention books exploring technologies on my own, HERE THAT YOU MIGHT FIND INTERESTING BECAUSE IF ) DONT DO THAT )M GOING OR USEFUL IN YOUR WORK MUCH AS )VE TO FIND MYSELF UNABLE TO USE THE LATEST FOUND THEM TO BE INTERESTING AND technologies to help my clients. USEFUL IN MY OWN WORK "UT OF COURSE THERE IS AT LEAST ONE Perhaps the most interesting other tried-and-true way to learn: trend we’re now seeing in the books. Sure, books (or their electronic book market, and especially when EQUIVALENTS HAVE BEEN EULOGIZED it comes to computer books, is the NUMEROUS TIMES DURING THE LAST FEW NUMBER OF SELF PUBLISHED EBOOKS years. But, programming-related being produced. (Heck, I’ve written books continue to be published, TWO OF THEM MYSELF ONE ON 0YTHON some by well known publishers and and another on regular expressions.) others by independent programmers. 4HESE BOOKS RELY ON WORD OF MOUTH %VERY SO OFTEN ) TAKE TIME IN THIS social media and marketing by the COLUMN TO REVIEW THE LATEST CROP OF developers/authors themselves. books that have helped make me There’s no doubt that traditional MORE EFFECTIVE IN MY WORK !S ALWAYS publishers continue to have great THE DESCRIPTIONS AND REVIEWS OF THESE reach and are producing more books

32 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 32 1/21/16 5:27 PM COLUMNS AT THE FORGE

But, if you’re trying to teach children to program and are agnostic on the language used, I’m not sure if Ruby is the right way to go.

(and well edited books, at that) than Perotta and published by the Pragmatic independent developers ever can "OOKSHELF 4HE BOOK HAS BEEN UPDATED HOPE TO MATCH "UT THE NUMBER OF to include Ruby 2, and although SELF PUBLISHED EBOOKS CONTINUES TO Ruby’s underlying object system hasn’t grow, as well as those sold via sites changed in many years, every version like https://leanpub.com. It will be OF 2UBY IS A BIT DIFFERENTˆAND KNOWING interesting to see what happens in HOW TO TAKE ADVANTAGE OF THOSE the coming years as book-creation DIFFERENCES CAN MAKE A HUGE DIFFERENCE systems, such as GitBook and IN YOUR PRODUCTIVITY )F YOU ARE USING 3OFTCOVER MAKE IT EVEN EASIER TO Ruby in your work, you really should create and sell your own books. read this book. So, what have I been reading, and Teaching children to program WHAT WOULD ) SUGGEST YOU READ AS WELL ALWAYS HAS BEEN A PASSION OF MINE so I’m always interested in books Programming Books AND SOFTWARE THAT TRY TO TEACH KIDS I continue to enjoy reading how to program. Several years ago, programming books, partly because I mentioned the book Python for there’s always something new to Kids AND IT SHOULDNT BE MUCH OF A LEARN PARTLY BECAUSE DIFFERENT surprise to discover that there’s now AUTHORS OFFER DIFFERENT PERSPECTIVES A BOOK FOR KIDS TO GET INTO 2UBY AND PARTLY BECAUSE ) OFTEN CAN called Ruby Wizardry. I think I should pick up insights and tricks that I DISTINGUISH BETWEEN TWO DIFFERENT can incorporate into my courses. I GOALS HERE THOUGH )F YOU REALLY WANT CONTINUE TO USE 2UBY ON 2AILS FOR your children to learn Ruby, this book Web development, so I always am is an entertaining way to do so, with happy to look at Ruby-related books. numerous examples and a storyline /NE OF MY MOST FAVORITE 2UBY BOOKS I’d like to think kids would enjoy. is Metaprogramming Ruby by Paolo "UT IF YOURE TRYING TO TEACH CHILDREN

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 33

LJ262-February2016.indd 33 1/21/16 5:27 PM COLUMNS AT THE FORGE

to program and are agnostic on the UNDERSTANDING ON A FEW TOPICS THIS LANGUAGE USED )M NOT SURE IF 2UBY IS book seems like a good place to go. THE RIGHT WAY TO GO 3EE BELOW FOR MY Finally, I continue to work a great RECOMMENDATION OF AN ON LINE SYSTEM DEAL WITH 0OSTGRE31, A DATABASE that seems to be doing that right.) My WHOSE FEATURES AND COMMUNITY limited experience with trying to teach CONTINUE TO GROW IN SIZE AND QUALITY Ruby, or even Python, to children has with every passing year. Hans- BEEN FAIRLY LACKLUSTERˆNOT BECAUSE THE Jurgen Schonig, a very accomplished materials are bad, but because kids are 0OSTGRE31, CONSULTANT AND CONTRIBUTOR less interested in the language than in has written Troubleshooting what they can do with the language. PostgreSQL (published by Packt), a ) HAVE BEEN USING A GREAT DEAL OF BOOK THAT IS SOMETHING LIKE A FIRST Python in recent years and also have AID KIT FOR 0OSTGRE31, DEVELOPERS BEEN TEACHING A HUGE NUMBER OF AND $"!S (E TOUCHES ON ALL OF THE Python classes around the world. I’m sensitive topics, including indexing, ALWAYS LOOKING FOR WAYS TO EXPLAIN transactions, locking, monitoring, ADVANCED ASPECTS OF 0YTHON BETTER REPLICATION AND BACKUPS %VEN IF YOU THUS /BI )KE .WOSUS SELF PUBLISHED HAVE BEEN USING 0OSTGRE31, FOR SOME Intermediate Python ebook caught time, Troubleshooting PostgreSQL is MY ATTENTION ) FOUND THAT MANY an excellent investment, one that will ASPECTS OF THIS BOOK PUT IDEAS INTO MORE THAN PAY FOR ITSELF IN TIME SAVED NICE CONTEXT AND INTO A FOCUS THAT and hair (not) pulled. OTHERWISE MIGHT HAVE BEEN LOST )F you have been working with Python Data Science FOR MORE THAN A FEW MONTHS AND It was probably somewhat inevitable want to deepen your knowledge and as someone who has worked with

Even if you have been using PostgreSQL for some time, Troubleshooting PostgreSQL is an excellent investment, one that will more than pay for itself in time saved and hair (not) pulled.

34 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 34 1/21/16 5:27 PM COLUMNS AT THE FORGE

databases, uses Python and has an it’s likely to use paradigms with interest in statistics that I would which Python developers already slowly but surely slide into data ARE FAMILIAR SUCH AS ITERATION AND science. I’ve thus been spending time COMPREHENSIONS 4HE FORMER BOOK during the last year, and especially the which I have recommended in LAST FEW MONTHS LEARNING MORE AND earlier book-review columns, gives MORE ABOUT THIS FASCINATING SUBJECTˆ a serious introduction to the how both so I can help my clients solve and why, not only including code, data-science problems and so I can but also the mathematical ideas OFFER TRAINING IN THE USE OF 0YTHON behind that code. FOR DATA SCIENCE 4HERE HAS BEEN AN Another book on this topic, EXPLOSION OF BOOKS ABOUT DATA SCIENCE CURRENTLY IN EARLY RELEASE FROM /2EILLY IN THE PAST FEW YEARS AND )VE BEEN is Python Data Science Handbook working my way through no small WRITTEN BY *AKE 6ANDER0LAS )F Doing NUMBER OF THEM Data Science takes a mathematical 4WO HIGH QUALITY BOOKS ABOUT approach, and Data Science data science are Doing Data Science from Scratch TAKES A FUNCTIONAL by Cathy O’Neil and Rachel Schutt, programming approach, then this published by O’Reilly, and Data book takes what I’d call a practical Science from Scratch by Joel Grus, APPROACHˆINTRODUCING THE ELEMENTS also published by O’Reilly. Grus OF 0YTHON THAT A BUDDING DATA uses Python and pushes hard in scientist most needs to understand, THE USE OF FUNCTIONAL PROGRAMMING keeping the programming and TECHNIQUES IN 0YTHON ALONG WITH THE MATHEMATICS AT A REASONABLE LEVEL FOR standard data-science stack available PEOPLE WHO ARE UNFAMILIAR WITH EITHER to Python developers, such as or both. I’m still reading through matplotlib and pandas. But, he also this early-release edition, but as an PROVIDES NO SMALL AMOUNT OF INSIGHTS introduction to learning science with INTO HOWˆAND JUST AS IMPORTANT a relatively easy learning curve, it may WHEN AND WHYˆWE SHOULD USE SUCH soon become my most-recommended TECHNIQUES AS LINEAR REGRESSION VS BOOK FOR PEOPLE NEW TO THE SUBJECT multiple regression vs. k-nearest Ivan Idris has written two books neighbors. The book contains a very FOR 0ACKT 0RESS THAT ARE GOOD OVERALL LARGE NUMBER OF CODE EXAMPLES introductions to data-science tools in and because it’s written in Python, Python, namely Python Data Analysis

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 35

LJ262-February2016.indd 35 1/21/16 5:27 PM COLUMNS AT THE FORGE

and Python Data Analysis Cookbook. Other Subjects 4HE FORMER INTRODUCES THE TOPICS AND ) MIGHT BE A SOFTWARE DEVELOPER BUT (as with the book by Grus, mentioned I also run a small company, so I’m previously) provides a tutorial into ALWAYS LOOKING FOR BOOKS THAT CAN areas like visualization, time series and help me run my company even better machine learning. The cookbook is OR FIND NEW BUSINESS OPPORTUNITIES A COMPLEMENTARY VOLUME INSTEAD OF /NE BOOK PUBLISHED JUST BEFORE teaching based on subjects, it takes a ) WROTE THIS COLUMN COMES FROM problem-solution approach. I’d suggest A GROUP OF CONSULTANTS WITH A THAT READING THE TUTORIAL FIRST IS A GOOD Slack channel. They jointly wrote idea; you then can read through the and published The independent COOKBOOKS TABLE OF CONTENTS ANDOR consulting manual, an ebook meant INDEX TO FIND THE SPECIFIC ISSUES ON FOR PEOPLE WHO ARE CONSULTING OR which you need to work. want to consult, but aren’t sure )F YOURE INTERESTED IN DATA SCIENCE WHERE TO START HOW TO FIND CLIENTS but would rather hear about the how to price themselves or how to insights and perspective than the create products. The book (at http:// mathematics, read Dataclysm: Who independentconsultingmanual.com) We Are (When We Think No One Is HAS MANY FAMOUS CONTRIBUTORS ALL OF Looking) by Christian Rudder, one whom have made the leap to running OF THE FOUNDERS OF /+#UPID )N their own businesses, and the lessons THIS BOOK 2UDDER FINDS FASCINATING they’ve learned, and contributed, make correlations that provide insights FOR EXTREMELY WORTHWHILE READING INTO HUMAN BEHAVIORˆTHANKS TO /NE OF THE AUTHORS OF THE ABOVE THE HUGE NUMBER OF DATA POINTS EBOOK AND A CO PANELIST OF MINE collected by the dating site he runs. on the Freelancers Show podcast, ) WOULD BE REMISS IF ) WERE TO is Philip Morgan. Morgan has done ignore the amazing e-mail list a lot to encourage consultants to PRODUCED BY !NALYTICS 6IDHYA A NAME POSITION THEMSELVESˆTHAT IS TO THAT DOESNT QUITE ROLL OFF OF THE DEFINE A SPECIFIC NICHE IN WHICH THEY TONGUE OF %NGLISH SPEAKING !MERICANS can be the best-known experts. His BUT WHICH OFFERS DAILY INSIGHTS FOR book, The Positioning Manual for anyone interested in the technology, Technical Firms (available at ALGORITHMS AND PRACTICE OF DATA http://ThePositioningManual.com) is science: http://analyticsvidhya.com. FULL OF EXCELLENT ADVICE INCLUDING

36 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 36 1/21/16 5:27 PM COLUMNS AT THE FORGE

In What If, Munroe asks crazy hypothetical questions, but then proceeds to answer them with the perfect balance of science and humor.

many reminders that although book while eating at a restaurant, ADVERTISING YOURSELF IN A SPECIFIC AND ) FEEL BAD FOR THE OTHER PATRONS niche might seem like a way to lose WHO HAD TO ENDURE ME LAUGHING QUITE money, it’s actually the ticket to loudly at the absurdities in this book, making more and to becoming a MIXED TOGETHER WITH OF COURSE GREAT MORE SUCCESSFUL CONSULTANT illustrations and explanations. /F COURSE ) READ ABOUT OTHER Finally, my children are getting SUBJECTS TOO )F LIKE ME YOURE OLDER SO ) HAVE LESS AND LESS OF A INTERESTED IN THE INTERSECTION OF chance to read to them. However, HISTORY SCIENCE AND FOOD YOU SHOULD I did manage to read The Terrible read the book Consider the Fork Two by Mac Barnett and Jory John, by Bee Wilson, which describes the A BEAUTIFUL AND FUNNY STORY ABOUT EVOLUTION AND DIVERSITY OF KITCHEN PRANKS OR hHACKSv AS WE CALLED THEM utensils and gadgets. Wonder why during my undergrad days at MIT), #HINA AND !MERICA HAVE DIFFERENT FRIENDSHIP AND THE ALL IMPORTANT GOAL SHAPED SPOONS OR HOW FORKS AND THE OF MAKING A SCHOOL PRINCIPAL LOOK SHAPE OF OUR TEETH ARE RELATED 4HIS FOOLISH IN FRONT OF THE STUDENTS book describes it all. I was introduced TO THIS BOOK BY THE GREAT h'ASTROPODv Podcasts and Other Resources podcast, which talks about these I continue to listen to a very large subjects in great depth. NUMBER OF PODCASTS EACH WEEK Another great book that I read this partly because I enjoy them so year is What If by Randall Munroe, the much and partly because they’re so AUTHOR OF 8+#$ )N What If, Munroe CONVENIENT FOR SOMEONE LIKE ME ASKS CRAZY HYPOTHETICAL QUESTIONS BUT WHO OFTEN COMMUTES BY BUS OR TRAIN then proceeds to answer them with TO CLIENTS OFFICES THE PERFECT BALANCE OF SCIENCE AND Two new Python-related podcasts HUMOR ) READ THE FIRST PART OF THIS have emerged in the last year: “Talk

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 37

LJ262-February2016.indd 37 1/21/16 5:27 PM COLUMNS AT THE FORGE

0YTHON TO -Ev AND h0ODCAST? ?INIT? ?v MY SONS CASE HIS SCHOOL PAID FOR are both interesting, with great ENTRY INTO ALL OF THE LEVELS BUT ) WOULD interviews and many insights into gladly have paid the $30 that Code technologies that I either use every -ONKEY CHARGES TO DO ALL OF THE LEVELS day or should start to learn. IT OFFERS AS WELL AS BUILD SOME OF Other podcasts that I enjoy tend MY OWN )TS RARE FOR ME TO SAY THAT ) TO BE ON THE SUBJECTS OF POLITICS have seen an educational technology AND ECONOMICSˆFROM 3LATES that lives up to its hype, but this is h0OLITICAL 'ABFESTv AND h3LATE DEFINITELY ONE OF THOSE TIMES -ONEYv TO .02S h0LANET -ONEYv Finally, let me mention a site that TO h0ODCAST FOR !MERICAv AN ) RUN CALLED $AILY 4ECH 6IDEO irreverent look at the US political (http://DailyTechVideo.com), which race. But there are many other great OFFERS A NEW HIGH QUALITY CONFERENCE podcasts to try, such as the BBC’s VIDEO EVERY DAY )F YOURE INTERESTED h-ORE OR ,ESSv ABOUT STATISTICS in open-source technologies, h&UNCTIONAL 'EEKERYv ABOUT programming languages, databases FUNCTIONAL PROGRAMMING h&REELANCER and even some computer history, 4RANSFORMATIONv ABOUT HOW TO I invite you to take a look at my site BECOME A SUCCESSFUL CONSULTANT AND EVEN TO SUGGEST HIGH QUALITY AND h3TARTUPv WHICH IS ALL ABOUT CONFERENCE TALKS ) MIGHT HAVE MISSED what it’s like to run a venture-based 9OU ALSO CAN FOLLOW THE SITES UPDATE startup company. ON 4WITTER AT $AILY4ECH6IDEOQ I mentioned earlier my interest in helping children learn to code, Reuven M. Lerner trains companies around the world in and I’ve discovered (thanks to my Python, PostgreSQL, Git and Ruby. His ebook, “Practice Makes TEN YEAR OLD SON A TERRIFIC SITE THAT Python”, contains 50 of his favorite exercises to sharpen your does just that: Code Monkey (at Python skills. Reuven blogs regularly at http://blog.lerner.co.il http://PlayCodeMonkey.com) uses and tweets as @reuvenmlerner. Reuven has a PhD in #OFFEE3CRIPT OF ALL THINGS TO TEACH Learning Sciences from Northwestern University, and he programming to children. However, lives in Modi’in, Israel, with his wife and three children. it does so using inventive and CREATIVE GRAPHICS HIGH QUALITY ERROR messages and a staged approach that Send comments or feedback via DOES A VERY GOOD JOB OF INTRODUCING http://www.linuxjournal.com/contact programming to kids aged 8–12. In or to [email protected].

38 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 38 1/21/16 5:27 PM March 7!10, 2016 San Francisco, CA

Practical training in JavaScript, HTML5, & web development.

Save 20% Register today. use code PCLinuxJournal

D1541 FLCA16_Linux_Journal_Ad.indd 1 1/12/16 11:15 AM LJ262-February2016.indd 39 1/21/16 5:27 PM COLUMNS WORK THE SHELL

Working with DAVE TAYLOR Command Arguments getopt—the right way to parse starting arguments in your shell script.

In my last article, I went interstellar a conditional statement: and created a multi-planet time , allowing you to learn just if [ "$1" = "-a" ]; then HOW OLD YOUD BE IF YOU LIVED ON flaga=1 Saturn or Pluto. So in this article, fi I want to head back to a more FUNDAMENTAL ASPECT OF SHELL SCRIPTING There are a couple problems with working with command arguments. this approach, however, not the I suspect that most shell scripts LEAST OF WHICH IS THAT IT ENDS UP go through an evolution with TAKING A FAIR AMOUNT OF SPACE IN THE THEIR COMMAND FLAGS ACTUALLY CODE "ECAUSE TO BE PROPER THE FULL starting out with none, then maybe SEQUENCE ALSO WOULD INCLUDE flaga=0 one or two parsed in a sloppy BEFORE THE CONDITIONAL BECAUSE FASHION FINALLY UPGRADED TO A REPEAT AFTER ME YOU NEVER CAN PROPER IMPLEMENTATION OF getopt, assume that the shell will correctly and then, perhaps, even being INSTANTIATE THE VALUE OF A NEWLY COMPLETELY REWRITTEN AS A # OR DEFINED VARIABLE Ruby program as the complexity 4HE OTHER PROBLEM IS THAT AFTER keeps inexorably increasing. THIS CODE SEQUENCE THE COMMAND The really easy way to parse a PARAMETERS ARE OUT OF SYNC $1 is still STARTING FLAG OF COURSE IS JUST TO USE EITHER POSSIBLY THE STARTING FLAG VALUE

40 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 40 1/21/16 5:27 PM COLUMNS WORK THE SHELL

Where I always get tripped up is that $# is the number of arguments, not the total number of words in the command.

(-a) or another argument or value actually it’s 0. THAT THE USER HAS SPECIFIED 4HIS GOES BACK TO THE DAWN OF To make this a bit more logical, UNIX development actually, and it’s let’s imagine a shell script that’s a KNOWN AS THE h INDEX PROBLEMv )N WRAPPER TO SOMETHING LIKE THE TERRIFIC ARRAYS THE FIRST VALUE IS REFERENCED curl: give it a URL, and it’ll grab as being in slot 0, or array[0]. that content and save it to the local For some developers, that makes DIRECTORY AS A FILE !DD THE IMAGINED COMPLETE SENSE AND FOR OTHERS IT -a FLAG AND ITLL ALSO INCLUDE PROGRESS CAN BE CONFUSING )NDEED )VE SEEN INFORMATION 3O A TYPICAL USAGE MIGHT C programs where the writer ignores look like this: THE FIRST SLOT ENTIRELY AND INDEXES starting at 1, not 0.

getpage.sh -a http://www.linuxjournal.com/index.html 7ITH THAT IN MIND AFTER THE CONDITIONAL HAS BEEN USED TO CHECK FOR )N THIS SITUATION A NUMBER OF the -a FLAG IN THE FIRST VARIABLE SPACE command parameter variables are going ($1), what really needs to happen is to be instantiated as the shell script is that all the positional variables above invoked, notably $# = 2, $0 = THIS ONE FOR EXAMPLE   AND SO getpage.sh, $1 = -a and $2 = ON NEED TO SHIFT DOWN A SPOT )DEALLY http://www.linuxjournal.com/ AFTER THE CONDITIONAL  CONTAINS THE index.html. 52, VALUE REGARDLESS OF WHETHER THE Where I always get tripped up is STARTING FLAG WAS SPECIFIED that $# IS THE NUMBER OF ARGUMENTS That way, the next statement block NOT THE TOTAL NUMBER OF WORDS IN THE IN THE SCRIPT SAFELY CAN ASSUME THAT  COMMAND 4HEREFORE IF THIS SCRIPT IS will be the URL and not have to test invoked without any arguments at REDUNDANTLY TO SEE IF ITS STILL -a. all, $# seems like it should be 1 (what This is done with the shift ABOUT THE COMMAND NAME BUT command, and so, here’s the proper

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 41

LJ262-February2016.indd 41 1/21/16 5:27 PM COLUMNS WORK THE SHELL

WAY TO TEST CONDITIONALLY FOR AN $1 = -a optional variable in a shell script: $2 = test.html $3 = SomeURL flaga=0 ----- if [ "$1" = "-a" ]; then $# = 1 flaga=1 $1 = SomeURL shift $2 = fi $3 =

That works exactly as you’d hope, Initially, all three positional variables BUT LEADS TO THE NEXT QUESTION WHAT are set, and the arg count ($#) is 3, HAPPENS IF THE FLAG HAS AN OPTIONAL as makes sense given the command VALUE THAT THE USER CAN SPECIFY )N THE INVOCATION "UT AFTER THE shift 2, CASE OF THIS CURL SCRIPT PERHAPS THE everything’s moved down two slots, FLAG IS SOMETHING LIKE O OUTPUT FILE and the arg count also is decremented That’s actually an easy addition to two, as you can see above. the above code: .OW WHAT IF YOUR DEVELOPMENT OF THE SCRIPT HITS A JUNCTURE WHERE outputspecified=0 YOU REALIZE THAT ITD BE USEFUL if [ "$1" = "-o" ]; then TO HAVE THREE DIFFERENT STARTING outputspecified=1 ARGUMENTS ONE OF WHICH INDEED outputfilename="$2" TAKES AN ARGUMENT shift 2 fi Dealing with Lots of Starting Flags %VEN TWO FLAGS CAN BE A PAIN As you can see, shift takes because it’d be terrible code that a single numeric argument that WOULD FORCE THE USER TO SPECIFY REFLECTS HOW MANY SLOTS YOU WANT them in order, but without that, to have everything cascade. It’s easy A C AND O COULD REQUIRE YOU TO TO SEE IF YOU CONSIDER THE POSITIONAL TEST FOR ALL THREE PARSE AND SHIFT VALUES BEFORE AND AFTER THE ABOVE THEN TEST FOR ALL THREE AGAIN PARSE code block: AND SHIFT THEN TEST FOR ALL THREE one more time. That would be $ sh getpage.sh -0 test.html SomeURL a nightmare, and I haven’t even $# = 3 mentioned how those pesky users

42 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 42 1/21/16 5:27 PM COLUMNS WORK THE SHELL

The getopt command is going to become your best friend if you’re building complex user-facing scripts, no question.

HAVE A TENDENCY TO COMBINE FLAGS Comparing this statement with the too, so would your script properly usage error below, you can see that arg parse -ac -o or -oc -a TOO 1 to getopt IS A LIST OF ALL ACCEPTABLE %NTER getopt. FLAGS WITH  DENOTING THOSE FLAGS THAT The getopt command is going to HAVE A REQUIRED ADDITIONAL ARGUMENT IF BECOME YOUR BEST FRIEND IF YOURE THE FLAG IS SPECIFIEDˆEASY ENOUGH BUILDING COMPLEX USER FACING SCRIPTS Now the status variable $? can be NO QUESTION )T BASICALLY WORKS BY TESTED IF ITS NON ZERO THERE WAS AN EXTRACTING ALL THE OPTIONAL FLAGS AND ERROR IN PARSING THE FLAGS AND IN MOST parameters, then lets you parse SCRIPTS ITS TIME TO FAIL OUT WITH A THROUGH THEM IN A UNIFORM MANNER usage statement: The standard usage is to break

down combined args with an if [ $? != 0] ; then

INVOCATION OF getopt, then use the echo \

set command to replace the existing "Usage: $(basename $0) {-p SFX} {-n} {-t} PTN NEWPTN"

STARTING FLAGS WITH THE NEW NEATER echo " -n sequentially number matching files"

args, then run through a loop, parsing echo " -p use specified suffix SFX for filenames"

them one by one. echo " -t test only - don't execute resultant cmds"

Yes, this is easier just to exit 0

demonstrate, so let me show you the fi COMMAND FLAG PARSING SEGMENT FROM A DIFFERENT PRODUCTION SCRIPT 4HIS 9OULL OFTEN SEE SCRIPTS THAT HAVE particular command has three possible THE USAGE SEQUENCE PUSHED INTO STARTING FLAGS -n, -p and -t. A SEPARATE FUNCTION TO KEEP THE 4HE FIRST STEP IN THE SCRIPT IS TO HAVE CODE CLEAN !LSO NOTE THE USE OF getopt normalize whatever the user $(basename $0) IN THE FIRST ECHO HAS SPECIFIED That’s a handy trick to compensate FOR THE FACT THAT MOST OF THE TIME $0 args=$(getopt np:t $*) IS GOING TO BE THE FULL NAME OF THE

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 43

LJ262-February2016.indd 43 1/21/16 5:27 PM COLUMNS WORK THE SHELL

Now all the positional parameters are neatly organized and ready to parse, something that’s traditionally done with a case statement wrapped in a for loop (no difference from an enigma wrapped in a dilemma, of course).

script, including path. So tapping semicolons in my case statements, BASENAME IS JUST FOR AESTHETICS mostly just to ensure that I use Finally, the statement that does the the needed ;; SEQUENCE TO real work: TERMINATE EACH OF THE INDIVIDUAL conditionals, but otherwise, it set -- $args should be easy to understand. 4HE ONLY THING MISSING FROM THIS Now all the positional parameters CODE FRAGMENT IS SOMETHING ) ALLUDED are neatly organized and ready to TO EARLIER 7HAT IS IT parse, something that’s traditionally !ND THATS IT FOR THIS ARTICLE .OW done with a case statement wrapped go back to your latest shell script, and IN A FOR LOOP NO DIFFERENCE FROM AN JUST FOR PRACTICE GO AHEAD AND ADD ENIGMA WRAPPED IN A DILEMMA OF SOME OPTIONAL STARTING FLAGS AND PARSE course). It looks like this: them with getopt.Q

for i Dave Taylor has been hacking shell scripts since the dawn of the do computer era. Well, not really, but still, 30 years is a long time! case "$i" He’s the author of the popular Wicked Cool Shell Scripts and in Teach Yourself Unix in 24 Hours (new edition just released!). -n ) renumber=1 ; shift ;; He can be found on Twitter as @DaveTaylor and more generally -p ) fixpng=1 ; sfx=$2 ; shift 2 ;; at his tech site: http://www.AskDaveTaylor.com. -t ) doit=0 ; shift ;; esac done Send comments or feedback via http://www.linuxjournal.com/contact I have a particular style with the or to [email protected].

44 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 44 1/21/16 5:27 PM '16HPC.LinuxJnlad.qxp_Layout 1 1/13/16 3:17 PM Page 1

Plan to attend. 13th Annual Conference Program Low-Cost and Free Show. 2016 HPC FOR WALL STREET – CLOUD & DATA CENTERS Show & Conference April 4, 2016 (Monday) The all-star lineup of speakers from HPC 2015 Roosevelt Hotel, NYC Madison Ave and 45th St, next to Grand Central Station

2016 Capital Markets are coming to the 2016 HPC for Wall Street.

Dave Weber Ken Barnes Bernard S Donefer Mike Blalock All-Star Conference program for 2016. Global Financial Services SVP Corp Dev, Options Associate Director, Global Sales Director, Director, Lenovo Information Technology Baruch College Intel Plan to attend the largest meeting of HPC, Cloud, Big Data, Data Cen- ters, Virtualization, Low Latency for the Capital Markets. See the program from 2015. The 2016 program will have the same all-star lineup of speakers.

Andy Bach Jeffrey M. Birnbaum Dino Vitale Harvey Stein Location. Location. Location. The Roosevelt is next to Grand Central Sta- Chief Architect, Founder and CEO, TD Securities Head of Credit Risk Financial Service, 60East Technologies Modeling, tion and within walking distance of JPMorgan Chase, Deutsche Bank, Juniper Networks Bloomberg Morgan Stanley, NASDAQ – all in midtown. Register online today: www.flaggmgmt.com/linux

2015 Sponsors

Fadi Gebara Terry Keene Rob Krugman Lee Fisher Sr Manager, CEO, VP Digital Strategy, VP Marketing, Redline IBM Research iSys Broadridge Fin Sols Trading Solutions

Jeremy Eder Matt Smith David B. Weiss Rick Aiere Engineering, Sol Architect, Sr Analyst, Architect Specialty, Red Hat Red Hat Aite AIG

™

Shagun Bali Jeffrey Scheel Ed Turkel Charles Milo Analyst, Senior Technical Staff, Mgr WW HPC Mkting, Enterprise Technical TABB Group IBM Linux Tech Center Hewlett-Packard Specialist, Intel

www.flaggmgmt.com/linux Alex Tsariounov Ugur Arslan Davor Frank Phil Albinus Principal Architect - Quantative Analyst Sr Solutions Architect, Editor, Traders Maga- Adv. Platforms, Lon- Solarflare zine, SourceMedia Show Hours: Mon, April 4 8:00 - 4:00 don Stock Exchange Conference Hours: Mon, April 4 8:30 - 4:50

Show & Conference: Flagg Management Inc 353 Lexington Avenue, New York 10016

(212) 286 0333 fax: (212) 286 0086 David Malik Russ Kennedy Ryan Eavy Markus Flierl Nick Ciarleglio [email protected] Sr Director, Advanced SVP of Product Exec Dir, Architec- VP Software Dev, Distinguished Syst. En- Services, Cisco Systems Strategy, Cleversafe ture, CME Group Oracle gineer, FSI Product Mgr Arista Networks

LJ262-February2016.indd 45 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

The Powers SHAWN POWERS That Be When the lights go out, do your servers too?

I live in an old house with old ERROR CORRECT BUT AFTER MONTHS OF wires in an old town with a dated struggling, I have only minimal loss. INFRASTRUCTURE )VE NEVER REALLY 5NFORTUNATELY MY DAY JOB REQUIRES ME CONSIDERED THE QUALITY OF THE to have a solid Internet connection, electricity being pumped into my SO )VE BEEN FORCED TO INSTALL TWO house, but since we moved to this separate connections and provide charming house loaded with character FAILOVER SUPPORT IF WHEN ONE OF THATS THE NICE WAY OF SAYING hOLDv them goes down. Although the it’s something I’ve had to deal with Internet connections may be the most OFTEN ) MADE A LOT OF ASSUMPTIONS AND sensitive to electrical issues, I know even made some expensive purchases that unstable voltage and noise can THAT IN HINDSIGHT WERE SILLY 3O FOR THIS BE BAD FOR ALL MY SERVERS AS WELL ARTICLE ) FIGURED ) SHOULD TALK ABOUT 1UITE HONESTLY EVEN IF YOURE NOT POWER BECAUSE IF YOURE READING having Internet issues, there are Linux Journal YOURE FAMILIAR WITH SOME THINGS TO LOOK FOR THAT MIGHT things that plug in to the wall. BE SYMPTOMS OF POWER PROBLEMS Now that I’m paying closer attention, The Symptoms ) NOTICE THAT A FEW TIMES A DAY 4HE BIGGEST RED FLAG FOR ME IN THE NEW OUR LIGHTS FLICKER OR QUICKLY DIM ) house was my Internet connection ASSUMED AT FIRST IT WAS WHEN A LARGE behaving poorly. I’ve literally had the motor kicked on (causing a sag) or technicians out here more than something like that, but now I don’t two-dozen times trying to track down think it is. It appears to be unrelated why I have massive packet loss. At to anything in the house, and it FIRST IT WAS UNUSABLE  PACKET HAPPENS OFTEN ENOUGH THAT ITS NOT loss is something TCP/IP can’t even JUST A FLUKE )F YOU NOTICE TINY GLITCHES

46 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 46 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

I’ve actually never had problems with EMI/RFI on server equipment, but if you have any audio equipment and you hear a buzz, or odd noises in your recordings, a line conditioner might really help.

in the Matrix, it might be a sign that to enter the building. It was a sunny YOUR POWER FLUCTUATES day without lightning, and still the SURGE DESTROYED THOUSANDS OF DOLLARS The Minimum Protection WORTH OF NON PROTECTED EQUIPMENT %VEN IF YOUR HOUSES ELECTRICAL SERVICE is rock-solid, it’s very important to Conditioner, Not Just for Hair HAVE YOUR ELECTRONIC EQUIPMENT ON The local cable technician a surge protector. Although they recommended I purchase a power used to be expensive, you can get a conditioner in order to give my REALLY NICE ONE FOR LESS THAN  AT EQUIPMENT CLEAN POWER 4HIS ACTUALLY most stores. A surge protector’s job started my research on power, and IS TO PROTECT YOUR EQUIPMENT FROM BEFORE ) DID MUCH STUDYING ) ORDERED CURRENT FLUCTUATIONS BUT ONLY extreme a rack-mounted power conditioner FLUCTUATIONS OR SURGES THUS THE NAME  FOR ALL MY SERVERS ) WISH ) HAD DONE A surge protector won’t generally help MORE RESEARCH BEFORE ORDERING IT BUT with minor issues like those in my it does serve a purpose, and what it house, but during a lightning storm, does, it does very well. you’ll be glad you spent the money. A power conditioner does a couple )F YOU LIVE IN A PLACE THAT RARELY GETS THINGS )T PROTECTS YOUR EQUIPMENT LIGHTNING STORMS OR IF YOU JUST DONT FROM SURGES )N FACT A SURGE PROTECTOR think a lightning strike is likely in IS A POWER CONDITIONER OF A SORT ITS your area, I still highly recommend a just the most kind. The other surge protector, or surge suppressor MAIN PURPOSE OF A POWER CONDITIONER AS THEYRE OFTEN CALLED 7HEN ) WORKED IS THAT IT hCLEANSv THE POWER SIGNAL at a local school district, there was a BY FILTERING OUT ELECTROMAGNETIC FAILURE ON ONE LEG OF THE THREE PHASE INTERFERENCE %-) AND RADIO FREQUENCY POWER FEED THAT CAUSED A HUGE SURGE INTERFERENCE 2&) THAT CAN REALLY MESS

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 47

LJ262-February2016.indd 47 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

Figure 1. I love the readout on this line conditioner. I just wish it were a voltage regulator!

Figure 2. I don’t regularly look at the front panel, so really this readout is sufficient. I just prefer the large LCD numbers showing incoming voltage.

WITH SENSITIVE EQUIPMENT )VE ACTUALLY The Big Guns NEVER HAD PROBLEMS WITH %-)2&) ON Although the power line conditioner SERVER EQUIPMENT BUT IF YOU HAVE ANY IS GREAT AT IDENTIFYING VOLTAGE AUDIO EQUIPMENT AND YOU HEAR A BUZZ issues, it doesn’t actually do or odd noises in your recordings, a ANYTHING TO FIX THEM 5NFORTUNATELY line conditioner might really help. the tool to normalize voltage is The model power line conditioner QUITE EXPENSIVE ) PAID AROUND I bought has a really cool display  FOR A RACK MOUNTED VOLTAGE ON THE FRONT THAT SHOWS THE LINE regulator. The good news is that VOLTAGE )N FACT ALTHOUGH IT DOESNT most voltage regulators not only do anything to help with voltage regulate the voltage, but they also FLUCTUATIONS IT IS NICE TO HAVE THE CONDITION THE LINE AND PROTECT FROM readout showing me as voltage surges. (But you really should read comes and goes. Figure 1 shows THE FINE PRINT TO MAKE SURE my power conditioner. It’s in my The voltage regulator on my server OFFICE NOW BUT ) CAN SEE THE RACK FUNCTIONS JUST LIKE A POWER STRIP VOLTAGE VARY ANYWHERE FROM 6 It plugs in to the wall, and then I TO 6 THROUGHOUT THE DAY 4HATS plug my devices in to the back. All a huge variance, and it’s very likely THE DEVICES SEE A PERFECTLY STEADY WHAT HAS BEEN CAUSING PART OF MY 6 POWER SOURCE HOWEVER EVEN IF Internet problem. my actual house voltage gets wacky.

48 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 48 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

It’s not a magic device, and it won’t is that there’s a small cutover time WORK IN THE CASE OF A BROWN OUT BUT when the line power is disrupted it can handle voltage sags down to BEFORE THE INVERTER TAKES OVER )T IS 6 WHICH IS LOWER THAN )VE EVER usually measured in milliseconds, seen it drop in my house. and the more expensive the standby 4HE SPECIFIC VOLTAGE REGULATOR ) 503 THE QUICKER THE CUTOVER TIME purchased is the Furman M-8X AR This is typically not a problem (Figure 2). It supports up to 15 amps FOR MOST COMPUTER HARDWARE BUT OF SERVICE WHICH IS ALL MY SERVER particularly sensitive may RACK POWER LINE IS RATED FOR ANYWAY reboot during switchover. I do wish it had a clearer readout Standby-Ferro UPS: This type FOR INCOMING VOLTAGE LIKE MY LINE OF 503 FUNCTIONS IN THE SAME WAY conditioner does (the line conditioner as a standby UPS, except there is a is the Furman M-8Dx), but that BUILT IN TRANSFORMER THAT REGULATES DOESNT AFFECT ITS PERFORMANCE JUST THE VOLTAGE AND PROVIDES A BIT OF MY DESIRE FOR DATA A hBUFFERv DURING THE CUTOVER TO INVERTER POWER 3INCE TRANSFORMERS Sometimes, the Voltage Is Zero use magnetism while converting That last thing I want to touch on voltage, there is a residual magnetism here is a battery backup. There’s a that usually smooths over the LOT TO BE SAID REGARDING THE TYPE OF transition so that even sensitive battery backup you use, but I just EQUIPMENT DOESNT SUFFER COVER THEM BRIEFLY HERE Since they have built-in voltage Standby UPS: 4HIS IS THE TYPE OF REGULATION THEY HELP WITH FLUCTUATING 503 MOST FOLKS USE EVEN IN SERVER voltage even when there’s not a rooms. There is a battery, which is POWER OUTAGE 5NFORTUNATELY THEY kept charged by the main power are known to overheat with some LINE AND IF THE POWER GOES OUT IT generators, especially those not QUICKLY POWERS AN INVERTER TO SUPPLY generating pure sine waves. Still, with 6 CURRENT 4HIS IS VERY EFFICIENT THE BUFFERING EFFECT OF THE VOLTAGE because the inverter doesn’t regulation circuitry, Standby-Ferro constantly convert the battery power UNITS ARE USUALLY PREFERRED AND USUALLY INTO !# IT ONLY hDOES WORKv WHEN COST SIGNIFICANTLY MORE THAN STANDARD the power goes out. Standby UPS units. 4HE DOWNSIDE OF ALL THIS EFFICIENCY Line Interactive UPS: These types

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 49

LJ262-February2016.indd 49 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

OF BACKUPS PROVIDE THE SMOOTHEST My Cheap, Massive Solution transition during an outage, because I have terrible luck with UPS devices. they constantly use their inverter to It seems like the batteries are always produce electricity. The only switchover dead when our power goes out, is whether the inverter is taking power and I have no idea until it’s too late. FROM THE BATTERIES OR THE WALL CURRENT !ND THEN EVEN THE FAIRLY EXPENSIVE and since both are connected, there’s RACKMOUNT UNITS ARE OFTEN DISPOSABLE NOT EVEN A MILLISECOND OF CUTOVER TIME without the ability to change their to worry about. BATTERIES %VEN IF THE BATTERIES ARE These are the most expensive types replaceable, and you actually know OF 503 DEVICES AND ALTHOUGH THEYRE they’ve gone bad, UPS devices don’t VERY NICE FOR SENSITIVE EQUIPMENT last very long. Usually the best you THEYRE OFTEN OVERKILL FOR PROTECTION CAN HOPE FOR IS TO KEEP YOUR SERVERS FROM THE OCCASIONAL OUTAGE 3TILL IF up long enough that they can shut YOURE LOOKING FOR THE BEST OF THE DOWN PROPERLY INSTEAD OF JUST LOSING best and price is no object, a Line POWER 4HATS A GREAT BENEFIT BUT ITS )NTERACTIVE 503 IS THE 2OLLS 2OYCE OF FRUSTRATING IF YOU WANT TO FUNCTION the UPS world. during an outage.

Figure 3. I really like having a separate inverter/ charger and battery. If the battery dies, I can replace it without replacing the inverter hardware itself. Plus, solar power will be fun to play with!

50 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 50 1/21/16 5:27 PM COLUMNS THE OPEN-SOURCE CLASSROOM

For my home, I decided to Probably More Than You Need SEPARATE THE INVERTER FROM THE 1UITE HONESTLY MY POWER PROBLEMS battery. I bought an AIMS inverter/ are likely more pronounced than most charger unit (Figure 3) that is hard- FOLKS )N OUR LAST HOUSE ) NEVER HAD wired inline with my house wiring. any problem with our line power at 4HE SPECIFIC UNIT ) BOUGHT IS THE ALL AND THE IDEA OF LINE CONDITIONERS 3000-watt pure sine inverter. It gives and voltage regulators seemed absurd. clean voltage and uses external 7ITH OUR NEW HOUSE HOWEVER IF ) batteries. In my case, it uses a huge WANT TO BE ABLE TO WORK FROM HOME 12v marine battery. This will keep and use computers all day, I have to MY LITTLE SERVER RACK RUNNING FOR make sure my power is rock-solid, QUITE SOME TIME BEFORE THE BATTERY and my two Internet connections are is drained, and since it’s a robust reliable. But who knows, maybe I lead acid battery, it seems to last ACTUALLY WILL DECIDE TO GO OFF THE GRID better than the tiny rackmount UPS SOMEDAY )F THAT EVER HAPPENS )LL BE batteries. Time will tell. sure to write about solar panels and I decided to go with 3000 watts battery banks. Until that time comes, because I wanted to make sure ENJOY YOUR CONDITIONED ELECTRICITYQ THE FULL  AMPS OF SERVICE WOULD be supported, even though I don’t Shawn Powers is the Associate Editor for Linux Journal. use nearly that much on my rack. He’s also the Gadget Guy for LinuxJournal.com, and he has an I also like the AIMS power inverter interesting collection of vintage Garfield coffee mugs. Don’t let because it comes with connections his silly hairdo fool you, he’s a pretty ordinary guy and can be that support charging with a solar reached via e-mail at [email protected]. Or, swing by array along with grid power. I have the #linuxjournal IRC channel on Freenode.net. NO INTENTION OF GOING hOFF GRIDv WITH MY UNIT BUT ) LOVE THE IDEA OF OFFSETTING A LITTLE OF MY ELECTRICAL BILL Send comments or feedback via with solar panels. (That will likely be http://www.linuxjournal.com/contact A FUTURE ARTICLE or to [email protected].

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 51

LJ262-February2016.indd 51 1/21/16 5:27 PM COLUMNS UNDER THE SINK Fast Network Routing, Meet SUSAN SONS Userspace Interview with Katerina Barone-Adesi about developing with the Snabb Switch network toolkit, working outside the Linux kernel for cleaner code and faster throughput.

Recently, ) HAD THE PLEASURE OF KB: Snabb should be interesting sitting down with Katerina Barone- to anyone who wants to do high- !DESI A DEVELOPER FROM THE 3NABB SPEED NETWORKING WITH THE FLEXIBILITY Switch community to talk about OF WRITING SOFTWARE AND SERIOUSLY Snabb Switch, writing network needs to consider several gigabits a INFRASTRUCTURE CODE AND MORE )N CASE SECOND OR MORE OF TRAFFIC 3ECONDARILY YOU ARENT FAMILIAR WITH 3NABB 3WITCH it is interesting to hobbyists and IT IS DESCRIBED IN ITS 2%!$-% AS hA researchers. Alexander Gall, at SIMPLE AND FAST PACKET NETWORKING 37)4#( WROTE AN ,60. ON 3NABB TOOLKITv 4HE THING THAT CAUGHT MY BECAUSE THE FEATURES HE WANTED WERE attention was Snabb Switch’s unusual not available elsewhere. It is a really ARCHITECTURE ITS SMALL AND FAST BUT POWERFUL TOOL FOR INDIVIDUALS AND SMALL written largely in LuaJIt, which I didn’t teams who need to do unusual or new expect. It circumvents the kernel things with networks. ALMOST ENTIRELY PREFERRING TO INTERFACE directly with network hardware and SS: So, how did Snabb Switch network code that is slower than that GET STARTED in Snabb Switch. KB: Snabb was started by Luke Gorrie, SS: Can you give me some WHO HAS A FAIRLY DEEP BACKGROUND EXAMPLES OF WHAT 3NABB 3WITCH in networking, along with a CAN BE USED FOR PROFESSIONAL HISTORY USING 3MALLTALK

52 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 52 1/21/16 5:27 PM COLUMNS UNDER THE SINK

&ORTH ETC (E WAS INFLUENCED BY hurdle, assuming some knowledge SOME OF !LAN +AYS IDEAS ABOUT OF NETWORKING IS THAT 3NABB IS A building small, comprehensible FAIRLY YOUNG PROJECT 3OME PARTS ARE systems that individuals can well documented, others are not, UNDERSTANDˆAND ALSO BY THE DESIRE AND THERE ARE NOT MANY EXAMPLES OF TO DO FLEXIBLE HIGH PERFORMANCE how its libraries work, even scattered SOFTWARE DEFINED NETWORKING ON AROUND 'IT(UB 3NABB ITSELF IS PRETTY COMMODITY X HARDWARE SMALL AND WELL FACTORED WHICH HELPS )T STARTED WITH A CODE BUDGETˆTHE a lot. More generally, networking BINARY SHOULD FIT ON A FLOPPY COMPILE is a vast space. When it comes to in less than a second (and less than introducing people to using tools a minute including dependencies), BUILT ON 3NABB THERE IS OFTEN SOME AND BE LESS THAN   LINES OF incredulity about using a garbage- code. It has grown a bit beyond collected high-level language to do THAT %SSENTIALLY IT LETS PEOPLE DO HIGH PERFORMANCE NETWORKING BUT more or less line speed networking LuaJIt has a solid track record in this using 10 gigabit cards, on a single SPACE BY NOW !SIDE FROM THAT MOST CORE WITHOUT THE OVERHEAD OF USING PEOPLE ARE NOT USED TO THINKING OF THE the kernel. The kernel is involved timescales involved. Depending on the PERIPHERALLY FOR INSTANCE IT USES EXPECTED AVERAGE PACKET SIZE IF YOU hugepage support), but Snabb are driving a 10 gigabit card, you have implements its own drivers, and the TENS TO HUNDREDS OF NANOSECONDS PER cards it is handling are not managed packet. With these constraints, you by the kernel. need to think about when you will hit RAM, and things like system calls are SS: ! LOT OF WHATS OUT THERE IN THIS prohibitively expensive. space today is big...when you’re introducing people to these tools SS: Why did you end up choosing (either building them or using them) ,UA*)T INSTEAD OF FOR EXAMPLE do they usually get where you’re STRAIGHT # GOING RIGHT AWAY )F NOT WHAT ARE THE PARTS THAT ARE HARDEST FOR THEM TO KB: Snabb is written in LuaJIt, so the WRAP THEIR HEADS AROUND CHOICE OF LANGUAGE AND LICENSEˆ it’s APLv2) came with the choice to KB: I would say the biggest initial use Snabb. That said, using LuaJIt is

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 53

LJ262-February2016.indd 53 1/21/16 5:27 PM COLUMNS UNDER THE SINK

EXCELLENT FOR PROJECTS LIKE THIS )T IS SS: Have you had to break parts out a just-in-time trace , so the TO # YET actual code being run is shaped by the packets being seen, and there KB: I have not personally had to are no branch mispredictions within a break parts out to C yet. People trace. It also makes it extremely easy OCCASIONALLY DOˆFOR INSTANCE IF THEY to use C when appropriate. It has want to hide an unpredictable branch an extremely nice FFI, good support FROM THE TRACE COMPILER !NOTHER FOR PACKED STRUCTS AND CALLING # ALTERNATIVE THAT A COLLEAGUE OF MINE code seems as easy as it would be used recently is DynASM, which IF THE WHOLE PROJECT WERE IN # 4HE allows you to generate assembly CODE ITSELF TENDS TO BE MORE CONCISE DYNAMICALLY (E USED IT FOR SOME AND QUICKER TO WRITEˆIT IS QUITE !68 INSTRUCTIONS TO ACCESS MEMORY comparable to Python or Ruby in that  BITS AT A TIME /NE OF THE VERY REGARD )T IS QUITE PLEASANT NOT TO HAVE surprising things about LuaJIt is that to read past boilerplate details. IT OFTEN IS AS FAST AS # AND SWAPPING CODE OUT FOR # BLINDLY CAN GENUINELY SS: What tools are you using to MAKE PERFORMANCE WORSE AS YOU LOSE track down bottlenecks and get the THE BENEFITS OF TRACES OPTIMIZATION YOU NEED FOR THE TIGHT TIME CONSTRAINTS SS: How much do you have to KNOW ABOUT THE SPECIFIC HARDWARE KB: The tools that we are using this will be running on to keep to track down bottlenecks include THE SPEED UP )S ARCHITECTURE PLUS PMU counters and a combination network card enough, or are you OF LOGICAL THINKING ABOUT WHAT IS FALLING INTO hOH SORRY YOU HAVE A LIKELY TO BE A BOTTLENECK FOLLOWED BY BRAND OF 2!- WE NEVER TESTEDv measuring changes to see whether LEVEL OF OPTIMIZATION SPECIFICITY they actually make an improvement. Having a small system makes the KB: 3O FAR MICRO ARCHITECTURE latter a surprisingly tractable (Haswell vs. Sky Lake) and raw CPU approach. We also look at LuaJIt’s SPEED DO MAKE A NOTICEABLE DIFFERENCE trace dumps, which include IR IN PERFORMANCE FOR SOME APPLICATIONS (Intermediate Representation) and For others, like the packetblaster X ASSEMBLY app that ships with Snabb, any

54 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 54 1/21/16 5:27 PM COLUMNS UNDER THE SINK

modern CPU Snabb can run on is so the main development branch. We FAR BEYOND WHAT IS NEEDED THAT THE HAVE A VARIETY OF END TO END AND UNIT bottleneck is the network cards. TESTS 0FLUA ALSO HAS PROPERTY BASED tests, which compare its results with SS: So, to recap, Snabb Switch is an and without optimization to libpcap’s open-source userspace application RESULTS ON RANDOMLY GENERATED FILTERS written mostly in LuaJIt that’s pushing We have plans to do more property- NETWORK TRAFFIC ON RUN OF THE MILL BASED AND FUZZ TESTING !DDITIONALLY #/43 X? EASILY AT 'BPS WITH THE SMALL SIZE OF THE SYSTEM MAKES IT A 'BPS TARGET IN BY easier to pay close attention to code BYPASSING THE ,INUX KERNEL FOR FASTER QUALITY &OR SECURITY ALL OF THE ABOVE PERFORMANCEAND YOU SAY YOURE along with thinking hard and paying GETTING COMPARABLE PERFORMANCE TO close attention to RFCs and their # CODE FROM ,UA*)T WITHOUT EXCESSIVE NOTES ON SECURITY /UR FRAGMENTATION SCAFFOLDING FOR PERFORMANCE TESTING reassembly code rejects overlapping FRAGMENTS AS MORE RECENT 2&#S KB: Yep. RECOMMEND FOR EXAMPLE

SS: 3O HOW BIG IS THIS THING AGAIN SS: When you bring on new developers, how do you inculcate KB: The source is less than 5MB; the THEM INTO THIS KIND OF RIGOR GIVEN HOW resulting binary is something like 2MB little it’s present in other programming BEFORE COMPRESSION SHOPS AND HOW ESSENTIAL IT IS FOR DEVELOPING INFRASTRUCTURE SOFTWARE SS: Impressive. However, this isn’t JUST SOME ENDPOINT SOFTWARE ITS KB: I introduced it to my team when INFRASTRUCTUREˆWHAT DOES YOUR ) JOINED IN  7E STARTED USING TEAM DO TO ENSURE CODE QUALITY CONTINUOUS INTEGRATION SOFTWARE AND SECURITY and property-based testing at that POINT BOTH OF WHICH SIGNIFICANTLY KB: &OR CODE QUALITY MY TEAM USES INCREASED THE RELIABILITY OF PFLUA continuous integration, tests and Code reviews also help. We have had code review. All code is reviewed by only one person join our team since; at least one person other than the as we grow more, perhaps I will AUTHOR BEFORE BEING COMMITTED TO HAVE A BETTER ANSWER FOR THAT 4HE

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 55

LJ262-February2016.indd 55 1/21/16 5:27 PM COLUMNS UNDER THE SINK

CODE ALSO HAD A LOT OF CONCEPTUAL ALPHA RELEASE THE WHOLE TEAM FOCUSED rigor to begin with. My colleague ALMOST ENTIRELY ON PERFORMANCE FOR A WHO SHAPED PFLUA MOST DEEPLY !NDY FEW WEEKS !T THE MOMENT ) THINK IT 7INGO SPENDS MOST OF HIS TIME might be around 50/50. working on , which also DEMANDS A FAIRLY HIGH LEVEL OF RIGOR SS: )F YOU WERE TO BREAK IT DOWN and coherent design. TO TWO TO FIVE PRINCIPLES THE 3NABB Switch toolkit is built on, what SS: )F YOU WERE TO GUESS ABOUT WHAT WOULD THEY BE PERCENTAGE OF YOUR PROJECTS EFFORT IS SPENT ON NEW CODEFEATURES VS TESTING KB: I would actually break it down to REFININGOPTIMIZING WRITING TEST CODE just one: systems should be small, and AND SCAFFOLDING FOR FUZZ TESTING VS individuals should be able to hold the DOCUMENTING AND PLANNING TASKS WHOLE SYSTEM IN THEIR HEAD -OST OF THE OTHER PRINCIPLES ARE CONSEQUENCES KB: )T DEPENDS ON THE PHASE OF EACH OF THIS AND THE HIGH SPEED SOFTWARE PROJECT )D SAY 4HEY TEND TO START OFF DEFINED NETWORKINGNETWORK FUNCTION WITH A FLURRY OF FEATURES AND PLANNING VIRTUALIZATION NICHE IT IS DESIGNED FOR and some testing is done in parallel. Later in each project, we optimize and SS: Why is this so important to Snabb TEST MORE 4HE EFFORTS COMPLIMENT 3WITCH IN PARTICULAR each other well, as more extensive tests catch subtle mistakes attempts KB: )T IS AN IMPORTANT PRINCIPLE FOR at optimization can introduce. Some SOFTWARE WHEREVER IT CAN REASONABLY OF THE FEATURES ARE SPECIFICALLY LINKED be applied, and it is one thing that TO PERFORMANCE GOALS WHICH MAKES strongly drew me toward Snabb. I THEM FALL THOROUGHLY INTO BEING CAN ONLY SPECULATE THAT FOR 3NABB BOTH NEW CODEFEATURES AND TESTING IN PARTICULAR IT CAME FROM ,UKES REFININGOPTIMIZING AT THE SAME TIME BACKGROUND WITH 3MALLTALKˆWHERE My colleague Andy Wingo’s recent HE TOOK THE QUOTES DESCRIBING THIS work with dynasm is an example. PRINCIPLE FROMˆAND OTHER MINIMAL %ARLIER IN THE LWAFTR PROJECT ALMOST systems like Forth. ALL OF THE TIME WAS GOING TO NEW CODE AND FEATURES WITH PERHAPS  SS: What do you like best about GOING INTO TESTING THEM "EFORE THE WORKING ON 3NABB 3WITCH

56 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 56 1/21/16 5:27 PM COLUMNS UNDER THE SINK

Between the design principles and being able not only to replace old network functions but also implement entirely new ones on commodity hardware and network cards that cost a few hundred bucks, I think it has a really exciting future.

KB: 3NABB IS A REALLY FUN AND READ A LOT OF BOOKS PLAYED AROUND interesting project to be involved etc. I have liked systems programming with these days. Between the design AND DYNAMIC LANGUAGES FOR ABOUT AS principles and being able not only LONG SO A JOB THAT INVOLVES ALL OF THESE TO REPLACE OLD NETWORK FUNCTIONS BUT ELEMENTS IS PRETTY FUN also implement entirely new ones on COMMODITY X HARDWARE AND NETWORK SS: $ID YOU MAKE A SPECIFIC EFFORT TO CARDS THAT COST A FEW HUNDRED BUCKS ) choose companies where you’d be THINK IT HAS A REALLY EXCITING FUTURE DOING OPEN SOURCE WORK OR WAS IT LUCK

SS: So, what got you interested in KB: )GALIA DOES ONLY TWEAKING THIS TYPE OF NETWORKING CODE AND THIS WAS A MAJOR FACTOR IN MY CHOOSING TO WORK FOR THEM ) DID ALSO KB: This is my day job. I joined Igalia interview at other companies that IN  WHEN THE NETWORKING TEAM DO A MIXTURE OF FREE AND NON FREE WAS WORKING ON PFLUA 0FLUA IS A SOFTWARE ON TOP OF 3NABB THAT USES A SUBSET OF LIBPCAPS FILTERING LANGUAGE SS: $O YOU HAVE ANY ADVICE FOR )T IS SMALLER FASTER AND THE SUBSET coders interested in Snabb Switch that it supports is believed to be IN PARTICULAR OR IN MOVING FROM ENTIRELY COMPATIBLE EXCEPT FOR LIBPCAP places higher up the stack into more optimizer bugs. INFRASTRUCTURE Y AREAS OF PROGRAMMING I got into networking as a hobby WORK LIKE WHAT YOURE DOING back around 2000 though. I ran a small home network on Linux, various BSDs, KB: For coders interested in Snabb

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 57

LJ262-February2016.indd 57 1/21/16 5:27 PM COLUMNS UNDER THE SINK

Switch in particular, it depends on running it on real hardware a simple their background; ones with more MATTER OF CHANGING A COUPLE LINES OF relevant experience can probably jump CONFIGURATION AND RECOMPILING in and implement something Luke brain-dumps about on the mailing list, SS: 4HANKS AGAIN +ATERINA FOR or something else entirely. For those sharing your time and expertise.Q NEWER TO THESE KINDS OF PROGRAMMING I would recommend starting by Susan Sons serves as a Senior Systems Analyst at Indiana writing a Snabb app. These can be University’s Center for Applied Cybersecurity Research as simple as a packet blaster or an (http://cacr.iu.edu), where she divides her time between app that echoes packets between helping NSF-funded science and infrastructure projects INTERFACES OR MORE COMPLEX THAN THE improve their security, helping secure a DHS-funded static l2vpn or lwAFTR, and can be built analysis project, and various attempts to save the world UP INCREMENTALLY 3OME OF THE TESTS from poor information security practices in general. Susan can be run, and app development also volunteers as Director of the Internet Civil Engineering CAN BE DONE ON ANY MODERN X Institute (http://icei.org), a nonprofit dedicated to supporting machine running Linux. I prototyped and securing the common software infrastructure on which the lwAFTR on my development we all depend. In her free time, she raises an amazing LAPTOP WHICH HAS NO %THERNET CARDS mini-hacker, writes, codes, researches, practices martial arts, 3NABB HAS A MECHANISM FOR PLUGGING lifts heavy things and volunteers as a search-and-rescue and apps into each other that makes disaster relief worker.

Resources

Snabb Switch: https://github.com/SnabbCo/snabbswitch

“Snabb Switch Deep Dive on Software Gone Wild” by Ivan Pepeinjak: http://blog.ipspace.net/2014/09/snabb-switch-deep-dive-on-software-gone.html

“L2VPN over IPv6 with Snabb Switch on Software Gone Wild” by Ivan Pepeinjak: http://blog.ipspace.net/2014/12/l2vpn-over-ipv6-with-snabb-switch-on.html

“High-performance packet filtering with Pflua”: https://archive.fosdem.org/2015/schedule/event/packet_filtering_pflua

“Deutsche Telekom TeraStream: Designed for Simplicity” by Ivan Pepeinjak: http://blog.ipspace.net/2013/11/deutsche-telekom-terastream-designed.html

58 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 58 1/21/16 5:27 PM Linux Journal eBook Series GEEK GUIDES Practical books for the most technical people on the planet. FREE Download NOW! NEW! Agile Product Development

Author: Ted Schmidt Sponsor: IBM

Every day, a new Internet-connected gadget hits the market, promising to make everyone’s lives incrementally more convenient. Devices like Fitbit and Sen.se Mother help monitor your health. Trackdot and Bikn let you track your possessions through GPS. Connect, Nest, SkyBell and LIFX all make it possible to control home appliances from anywhere. By 2020, Gartner forecasts that there will be more than 25 billion of these “things” connected to the Internet. And every day, consumers are demanding more from these things: more features, more interconnectivity, more convenience.

Improve Business Processes with an Enterprise Job Scheduler

Author: Mike Diehl Sponsor: Skybot

Modern IT shops run a whole lot more than just a few file and print servers like they did in the old days. Today’s enterprises are vastly more complex, often with servers in different data centers and even scattered all over the globe. Additionally, they typically have Windows servers, Linux servers, mainframe servers and various other flavors of UNIX servers, and all of these servers, all over the network, produce and process data.

Go to http://geekguide.linuxjournal.com

LJ262-February2016.indd 59 1/21/16 5:27 PM NEW PRODUCTS

SoftMaker Office

&OUNDED IN  THE 'ERMANY BASED SOFTWARE VENDOR 3OFT-AKER HAS SPENT THE MODERN 0# ERA DEVELOPING AND IMPROVING ITS OWN OFFICE PROGRAM 3OFT-AKER /FFICE 3OFT-AKER /FFICE  FOR ,INUX IS 3OFT-AKERS LATEST RELEASE OF THE OFFICE SUITE WHICH CONSISTS OF THE WORD PROCESSOR TextMaker, the spreadsheet program PlanMaker and the 3OFT-AKER 0RESENTATIONS 3OFT-AKERS VALUE PROPOSITION VS OTHER OFFICE SUITES INCLUDES A COMPACT FOOTPRINT STABILITY AVAILABILITY IN 3TANDARD AND 0ROFESSIONAL VERSIONS ADD ONS FOR THE 4HUNDERBIRD E MAIL CLIENT AS WELL AS COMPATIBILITY WITH AND A FIVE FOLD SPEED ADVANTAGE OVER -ICROSOFT /FFICE 4HE NEW 3OFT-AKER /FFICE  BOASTS MORE THAN  NEW FEATURES SUCH AS OPTIMIZED SOFTWARE ARCHITECTURE ACCELERATED BY THE USE OF /PEN', FOR SCREEN DISPLAY AND MULTIPLE #05 CORES FOR GRAPHIC RENDERING IMPROVED COMPATIBILITY FEATURES WITH -3 /FFICE BUILT IN VERSION MANAGEMENT FOR DOCUMENTS SUPPORT FOR   LABEL FORMATS IMPROVED PIVOT TABLE FUNCTIONALITY SMART GUIDES FOR ALIGNMENT OF OBJECTS IN PRESENTATIONS AND MORE 4HE 0ROFESSIONAL EDITION ADDS FOUR HIGH QUALITY DICTIONARIES BY "ERLITZ FOR EASY TRANSLATION BETWEEN %NGLISH 3PANISH )TALIAN 'ERMAN AND &RENCH !LL MODERN ,INUX DISTRIBUTIONS ARE SUPPORTED http://softmaker.com

Ziften ZFlow

!S ENTERPRISES HAVE MIGRATED TO THE CLOUD FOR ECONOMIC AND operational purposes, traditional network visibility has been lost DUE TO A LACK OF ACCESS AT THE INFRASTRUCTURE LAYER 4O HELP ORGANIZATIONS AVOID OPERATING IN THE DARK IN THE PENGUIN ZONE SECURITY SPECIALIST :IFTEN HAS RELEASED A NEW VERSION OF ITS FLAGSHIP :IFTEN :&LOW FOR THE ,INUX  :IFTEN :&LOW DELIVERS GREATER NETWORK VISIBILITY BY PROVIDING FULL VISIBILITY CONTEXTUAL INTELLIGENCE USER BEHAVIORAL ANALYSIS AND INTEGRATION into previously deployed security tools. This new integration with Linux provides previously NON EXISTENT VISIBILITY INTO THE PUBLIC CLOUD INFRASTRUCTURE AND ENABLES :IFTENS NEW #LOUD 6ISIBILITY )NITIATIVE 4HE INITIATIVE HELPS SECURE CLOUD OPERATIONS WITH THE VISIBILITY THAT SECURITY PROFESSIONALS NEED TO IDENTIFY AND RESPOND TO POTENTIAL THREATS AND ATTACKS QUICKLY :IFTEN ADDS THAT :&LOW IS LIGHTWEIGHT MEETS )0&)8 STANDARDS AND ENABLES BETTER EAST WEST VISIBILITY TO IDENTIFY LATERAL MOVEMENT OF AN ATTACK WITHIN THE DATA CENTER http://ziften.com

60 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 60 1/21/16 5:27 PM NEW PRODUCTS

Thomas Erl, Wajid Khattak and Paul Buhler’s Big Data Fundamentals (Prentice-Hall)

"ECAUSE "IG $ATA WILL ONLY GET BIGGER THE CASE FOR READING Big Data Fundamentals: Concepts, Drivers & Techniques will only get stronger. #O AUTHORED BY BEST SELLING )4 AUTHOR 4HOMAS %RL AND COLLABORATORS Wajid Khattak and Paul Buhler, Big Data Fundamentals is a pragmatic, no-nonsense introduction TO "IG $ATA FOR BUSINESS AND TECHNOLOGY PROFESSIONALS 4HE AUTHORS CLEARLY EXPLAIN KEY "IG $ATA CONCEPTS THEORY AND TERMINOLOGY AS WELL AS FUNDAMENTAL TECHNOLOGIES AND TECHNIQUES !LL coverage is supported with case-study examples and numerous simple diagrams. The authors COMMENCE BY EXPLAINING HOW "IG $ATA CAN MOVE AN ORGANIZATION FORWARD BY SOLVING A WIDE RANGE OF PREVIOUSLY INTRACTABLE BUSINESS PROBLEMS .EXT THEY DEMYSTIFY KEY ANALYSIS TECHNIQUES AND TECHNOLOGIES AND SHOW HOW A "IG $ATA SOLUTION ENVIRONMENT CAN BE BUILT AND INTEGRATED TO OFFER COMPETITIVE ADVANTAGES /THER KEY TOPICS INCLUDE DIFFERENCES BETWEEN "IG $ATA AND PREVIOUS FORMS OF DATA ANALYSIS AND SCIENCE BUSINESS MOTIVATIONS AND DRIVERS BEHIND "IG $ATA ADOPTION AND RECOGNIZING THE FIVE h6v CHARACTERISTICS OF DATASETS IN "IG $ATA environments: volume, velocity, variety, veracity and value. http://informit.com

Varnish Software’s Zipnish

'AINING INSIGHTS INTO HOW QUICKLY SERVICES ARE running or whether they are adding latency is a DIFFICULT TASK IN DISTRIBUTED ARCHITECTURES SUCH AS MICROSERVICES 4O SIMPLIFY THIS TASK 6ARNISH 3OFTWARE LAUNCHED :IPNISH A NEW OPEN SOURCE ARCHITECTURE AGNOSTIC TOOL THAT TRACKS PERFORMANCE AND HELPS RESOLVE LATENCY ISSUES IN MICROSERVICES ARCHITECTURES :IPNISH USES THE 6ARNISH LOGGING !0) FROM 6ARNISH #ACHE  TO MONITOR TRANSACTIONS AND USES 0YTHON AND THE EVENT LIBRARY 4WISTED TO TRANSPORT THE DATA -Y31, IS USED AS THE DATABASE FOR STORAGE 4HE PRESENTATION BACK END IS DONE IN 0YTHON WHEREAS A SLIGHTLY MODIFIED VERSION OF :IPKIN IS USED AS FRONT END http://varnish-software.com

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 61

LJ262-February2016.indd 61 1/21/16 5:27 PM NEW PRODUCTS

Red Hat CloudForms

To meet customer demands in today’s hybrid, heterogeneous world, Red Hat established an ALLIANCE WITH -ICROSOFT TO RELEASE 2ED (AT #LOUD&ORMS  THE LATEST VERSION OF THE COMPANYS OPEN PLATFORM HYBRID CLOUD MANAGEMENT SOLUTION THAT NOW FEATURES -ICROSOFT !ZURE SUPPORT 2ED (AT DECLARES THAT 2ED (AT #LOUD&ORMS  GOES BEYOND SELF SERVICE OFFERING A CONSISTENT EXPERIENCE AND COMPREHENSIVE LIFECYCLE MANAGEMENT ACROSS PLATFORMS COVERING VIRTUALIZATION private cloud, public cloud and containers. In addition to enabling Azure customers to manage THOSE WORKLOADS AND RESOURCES WITHIN #LOUD&ORMS RELEASE  ADDS MANAGEMENT FOR CONTAINER ARCHITECTURES AND IMPROVED SELF SERVICE DASHBOARDS AND CHARTS TO BETTER ANALYZE THE RELATIONSHIPS BETWEEN DIFFERENT CLOUD PLATFORMS AND CONTAINER HOSTS /THER PLATFORMS BESIDES !ZURE INTO WHICH 2ED (AT #LOUD&ORMS  PROVIDES OPERATIONAL INSIGHT INCLUDE !MAZON 7EB 3ERVICES (YPER 6 /PEN3HIFT BY 2ED (AT /PEN3TACK 2ED (AT %NTERPRISE 6IRTUALIZATION AND 6-WARE http://redhat.com

Imagination Technologies’ Creator Ci40

h7HERE DO ) START v IS A COMMON QUESTION FOR THOSE OF US EXPLORING THE )NTERNET OF 4HINGS )O4 AND EMBEDDED COMPUTING MARKETS )MAGINATION 4ECHNOLOGIES ANSWERS THAT QUESTION IN THE FORM OF A PRODUCT NAMELY THE FIRMS NEW #REATOR #I )O4 KIT WHICH HELPS ONE MAKE SENSE OF THE OH SO MANY BOARDS AND ACCESSORIES HARDWARE AND SOFTWARE INTERFACES CONNECTIVITY STANDARDS AND !0)S IN )O4 4HE #REATOR #I INCLUDES NOT ONLY THE HARDWARE BUILDING BLOCKS NEEDED TO PROTOTYPE A WIRELESS )O4 SYSTEM FROM SCRATCH QUICKLY BUT ALSO THE OPEN SOURCE SOFTWARE FRAMEWORKS THE NETWORK STACKS AND THE CLOUD CONNECTIVITY CAPABILITIES REQUIRED TO CONNECT AND AUTHENTICATE DEVICES TO THE CLOUD SECURELY 4HE INCLUDED #I MICROCOMPUTER HAS BEEN SPECIFICALLY DESIGNED FOR SMART HOME )O4 AND OTHER CONNECTED DEVICES DRONES ROBOTS AND SO ON AND FEATURES THE HARDWARE REQUIREMENTS AC X 7I &I AN %THERNET PORT 3$ AND 53" STORAGE FOR USE AS A HIGH SPEED WIRELESS ROUTER 4HE BOARD RUNS ,INUX $EBIAN /PEN7RT AND IS PART OF 'OOGLES GOLDEN REFERENCE PROGRAM FOR ITS "RILLO /3 FOR )O4 2OUNDING OUT THE KIT ARE ADD ONS FROM -IKROELECTRONICA TWO BATTERY POWERED #LICKER BOARD DEVELOPMENT KITS AND THREE #LICK BOARDS FOR MEASURING TEMPERATURE DETECTING MOTION AND CONTROLLING A RELAY http://imgtec.com

62 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 62 1/21/16 5:27 PM NEW PRODUCTS

Opera Max

Cheapo data hogs (like you) with limited data plans can stretch their BUDGETS AND DOUBLE THEIR STREAMING OF VIDEOˆAND NOW MUSICˆUSING /PERA -AX /PERA !3!S UPGRADED DATA MANAGEMENT AND DATA SAVING APP FOR !NDROID 4HE LATEST RELEASE ADDS STREAMING AUDIO OPTIMIZATION TO THE EXISTING VIDEO OPTIMIZATION FUNCTIONALITY /PERA NOTES THAT NINE HOURS OF STREAMING MUSIC OR ON LINE RADIO REQUIRE '" OF THROUGHPUT WHICH EASILY CAN BURN THROUGH A data plan when using mobile networks. Complementing Opera Max’s existing ability to optimize STREAMING VIDEO SERVICES LIKE 9OU4UBE AND .ETFLIX COMES NEW FUNCTIONALITY THAT OPTIMIZES streaming music apps, such as YouTube Music, Pandora, Slacker Radio, Gaana and Saavn, WITH OTHERS ON THE WAY /PERAS OPTIMIZATION TECHNIQUE FOR AUDIO INVOLVES APPLYING TECHNOLOGY FROM 2OCKET /PTIMIZER FOR CONVERTING -0 AND -0 AUDIO STREAMS TO THE MORE EFFICIENT !!# CODEC WHICH DELIVERS HIGH AUDIO QUALITY OVER A LOW BITRATE CONNECTION TO ANY COMPATIBLE DEVICE http://opera.com

Sander van Vugt’s Beginning the Linux Command Line, 2nd Edition (Apress)

)4 AUTHOR 3ANDER VAN 6UGT SAYS THAT HIS BOOK Beginning the Linux Command Line NOW IN IT SECOND EDITION IS FOR EVERYONE WHO USES ,INUXˆNO EXCEPTIONS 6AN 6UGTS BOOK IS A GUIDE TO ,INUX IN WHICH the user digs deeply into the system, wielding the shell as a shovel. Key overarching goals include understanding how Linux is organized AND HOW TO hTHINK ,INUXv /THER LEARNING OBJECTIVES INCLUDE FINDING THE RIGHT COMMAND FOR THE TASK AT HAND WORKING WITH TEXT EDITORS AND INTELLIGENT FILTERS SHELL PROGRAMMING CONFIGURING ACCESS TO HARDWARE DEVICES AND MORE Beginning the Linux Command Line is CHECKED AGAINST ALL OF THE MOST IMPORTANT ,INUX DISTRIBUTIONS AND FOLLOWS A TASK ORIENTED APPROACH THAT IS DISTRIBUTION AGNOSTIC 4HIS NEW SECOND EDITION OF THE BOOK COVERS FEATURES OF THE VERY LATEST VERSIONS OF THE ,INUX OPERATING SYSTEM INCLUDING THE NEW "TRFS FILESYSTEM THE SYSTEMD BOOT PROCEDURE AND FIREWALL MANAGEMENT WITH FIREWALLD http://www.apress

Please send information about releases of Linux-related products to [email protected] or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 63

LJ262-February2016.indd 63 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

TEMPERATURE CONTROL IN A HOMEBREWING TUN Using a BeagleBone Black

By using a thermometer, a PID controller and a PWM output on the BeagleBone, you can control your brew tun when brewing your beer.

KLAUS KOLLE

64 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 64 1/21/16 5:27 PM LOVE BEERˆGOOD BEER NOT in several temperature stages. This industrial beers. The ones results in a more complex beer with a ) BREW MYSELF WHERE ) CAN more malty body. And, we now have control the ingredients and the BETTER EFFICIENCYˆTHAT IS MORE SUGARS I process, are very good. FOR THE YEAST TO PRODUCE ALCOHOL FROM "EER BREWING IS A SIMPLE PROCESSˆIT meaning less malt will provide the USED TO BE A DUTY OF HOUSEWIVES ON SAME AMOUNT OF ALCOHOL IN THE BEER IF THE FARMS HERE IN MY COUNTRY LONG OUR EFFICIENCY IS HIGHER THAN NORMAL ago. The process I and my two brew But, some problems arise when mates use is a bit more complicated trying to brew using our simple than the process used in ancient brew tun. The tun is a big pot with a country houses though. One thing we 29-liter capacity. The pot has a built-in can do nowadays is mash the malt heating element with a rated power

Figure 1. Simplified Homebrew Tun

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 65

LJ262-February2016.indd 65 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

OF  7 &IGURE   (ELPED BY A RELAY ON AND OFF IN CONTROLLED INTERVALS CIRCULATING PUMP THE MASH FLUID THEREBY CONTROLLING THE AMOUNT OF CIRCULATES FROM THE BOTTOM OF THE TUN POWER FED INTO THE HEATER )F THE RELAY TO THE TOP OVERFLOWING THE MALT IS CLOSED FOR  SECONDS AND OFF FOR  -ANY OF OUR BEERS HAVE A STARTING SECONDS   7  7 IS FED POINT BETWEEN ª#nª# )N THIS into the heater. Well, to be honest, this temperature range, we are preparing problem could be solved using a simple THE PROTEINS IN THE MALT FOR THE LATER 555 chip and a potentiometer, but PROCESS OF EXTRACTING SUGARS OUT OF since we are programmers, we tend to the malt. The optimal temperature pull out the tools we love, even when FOR THE BETA AND ALPHA ENZYMES THAT it’s overkill. convert starch to sugars is between ! SIDE BENEFIT FROM SETTING UP ª#nª# 3O AFTER THE PROTEIN a BeagleBone is the ability to log STOP IN THE ª# RANGE WE HAVE TO temperatures during the whole brew raise the temperature in the mash. PROCESS 4HIS IS HANDY FOR EVALUATING "UT UNFORTUNATELY TOO MUCH HEAT WILL the beer when adjustments to the BURN SOME OF THE PARTICLES FLOATING recipe, which include the processing, in the mash onto the heater in the are discussed. Previously, we noted BOTTOM OF THE TUN RESULTING IN A BAD temperatures and times on a piece TASTE AND DIFFICULTY GETTING THE HEAT OF PAPER A FEW TIMES DURING THE FROM THE HEATING ELEMENT INTO THE BREW PROCESSˆWHEN WE REMEMBERED mash. We have determined empirically TO DO IT !NOTHER BENEFIT IS THAT THE THAT FROM ª#nª# WE CAN APPLY WHOLE THING CAN BE CONTROLLED FROM A FULL POWER 4O FINISH THE PROCESS THE Web page. TEMPERATURE IS RAISED TO ª# TO STOP THE ENZYMES FROM WORKING System Overview So, we need a way to control the I’ve already mentioned the AMOUNT OF POWER FED INTO THE HEATING REQUIREMENTS AND FROM THERE YOU element. What is more natural than to can extract that we need a controlling use a BeagleBone Black to control the FUNCTIONALITY THAT WILL READ THE RATE OF ENERGY BURNED IN THE HEATER temperature in the mash when it leaves through a Pulse Width Modulated THE BOTTOM OF THE TUN "ASED ON THE (PWM) controller already built in to the temperature read, it will set the period "EAGLE"ONE 4HE 07- WILL PRODUCE TIME OF THE OUTPUT 07- !LSO WE A PULSE OF VARYING WIDTH TURNING A need a database enabling us to log the

66 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 66 1/21/16 5:27 PM collected data during the brew process. embedded a Websocket server into Now, a database also could archive THE APPLICATION 4HIS LED AFTER A FEW THE RECIPES OR AT LEAST THE PART OF THE ITERATIONS TO THE SOFTWARE ARCHITECTURE recipes that is about temperatures diagram shown in Figure 2. Add to and time. Finally, I also mentioned the architecture diagram a Web client HAVING A 7EB PAGE $ATA COMES FROM PROGRAMˆFOR EXAMPLE &IREFOX THAT the database, but how do we serve loads and executes the JavaScript code THE DATA TO THE 7EB PAGE "ECAUSE embedded in the Web page. However, ) ALSO WANTED TO SHOW THE STATE OF the database and Web page are the controller on the Web page, I BEYOND THE SCOPE OF THIS ARTICLE ) MAY

Figure 2. Brew Controller Software Architecture

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 67

LJ262-February2016.indd 67 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

COVER THEM IN A FUTURE ARTICLE Q A log utility. ) DECIDED TO DEVELOP IN # BUT I could have used any programming Q Threading. LANGUAGE )N FACT ) DID START developing a prototype in BoneScript, Cross Compilation: ) PREFER TO A DIALECT OF *AVA3CRIPT THAT BUILDS DEVELOP USING %CLIPSE WHEN IT IS A on Node.js, which is programmed in larger project that runs over some *AVA3CRIPT )T WORKED QUITE WELL BUT time. There is no problem when I was unsure whether I could trust it DEVELOPING SMALL AND QUICK SOLUTIONS to be stable in the long run. I noticed directly on the BeagleBone, but when during my tests that the BoneScript complexity increases, the help that an PROGRAM HAD SOME DELAYS A FEW integrated development environment arbitrary times and hiccups I couldn’t PROVIDES LEVELS OUT SOME OF THE explain. A program, where I know all COMPLEXITY #AN YOU INSTALL %CLIPSE DETAILS AND THAT IS COMPILED IS BY FAR ON THE "EAGLE"ONE 9ES IT SHOULD BE THE MOST EFFICIENT AND ONCE TESTED possible, but since the storage on the and debugged, I can trust this one "EAGLE"ONE IS &LASH BASED ) PREFER TO RUN FOR LONG TIME !ND NOW ) ALSO TO RUN %CLIPSE ON MY LAPTOP ! LOT OF HAVE SEVERAL DIFFERENT EXAMPLES OF THE WRITING TO FILES CAN WEAR OUT THE &LASH same solution to show my students memory on the BeagleBone. and discuss with them. The BeagleBone is built using an ARM CPU architecture, and most Preparing for Development desktops and laptops are Intel CPU )N ORDER TO PREPARE FOR DEVELOPING architectures. This means we need THE SOFTWARE THAT FULFILLS THE to install a cross compiler on the REQUIREMENTS A FEW THINGS NEEDED laptop. A cross compiler is a compiler TO BE SET UP AND CONFIGURED THAT PRODUCES CODE FOR another CPU architecture than what it Q Cross compilation. executes on. In this case, the compiler executes on an Intel CPU but produces Q Remote debugging. CODE FOR AN !2- #05 I run Fedora Linux and have been Q Setting up a temperature sensor. doing so ever since the Fedora Project WAS LAUNCHED ) COULDNT FIND A SUITABLE Q Setting up the PWM output. cross compiler in the repositories

68 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 68 1/21/16 5:27 PM FOR &EDORA HOWEVER 3O AFTER SOME ANYTHING IN %CLIPSE SEARCHING EVENTUALLY ) FOUND THAT )N %CLIPSE YOU HAVE TO SELECT Linaro.org maintains a suitable compiler the Cross GCC choice in the SUITE THAT ) COULD INSTALL )F YOURE SECTION OF THE .EW 0ROJECT FOLLOWING ALONG YOU SHOULD DOWNLOAD dialog. In the next dialog, enter A STABLE RELEASE FROM THE REPOSITORY hARM LINUX GNUEABIHF v IN THE #ROSS This is the one I downloaded: #OMPILER 0REFIX ENTRY )F YOU LOOK http://releases.linaro.org/14.11/ into the directory where you stored components/toolchain/binaries/ THE CROSS COMPILER YOU WILL FIND arm-linux-gnueabihf/gcc-linaro-4.9- that almost ALL BINARIES ARE PREFIXED 2014.11-x86_64_arm-linux-gnueabihf. with this. So the gcc is really tar.xz 4HERE MIGHT BE FRESHER COPIES ARM LINUX GNUEABIHF GCC )N THE by the time you read this. Take a look cross compiler path, enter the at http://releases.linaro.org/14.11/ path to where you installed the components/toolchain/binaries, CROSS COMPILERˆFOR EXAMPLE AND BE SURE TO DOWNLOAD A GNUEABIHF OPTGNUEABIHFBIN VERSIONˆTHE COMPILER THAT PRODUCES .OW %CLIPSE KNOWS HOW TO PRODUCE CODE FOR A HARD FLOATING POINT UNIT that will run on the BBB. I keep my downloaded tools, like Remote Debugging: Regardless COMPILERS AND %CLIPSES IN THE OPT OF HOW EXPERIENCED WE ARE BUGS directory, so I unpacked the tarball in tend to creep into our code making a directory I named toolchains. In this programs not work as expected. So directory, you can make a symbolic THE ABILITY TO DEBUG IS MANDATORYˆ link to the very long directory name WELL FOR ME AT LEAST BUT ) ALSO DO YOU WILL GET OUT OF THE TARBALL 4HIS HAVE ONLY  YEARS OF PROGRAMMING MAKES IT EASIER WHEN CONFIGURING experience. Luckily, it is possible to A PROJECT IN %CLIPSE ) MADE A SET UP %CLIPSE SO YOU CAN EXECUTE THE SYMBOLIC LINK CALLED GNUEABIHF FRESHLY COMPILED PROGRAM OVER ON THE WHICH IS FAR EASIER TO REMEMBER THAN BeagleBone under debugger control. GCC LINARO   X??ARM )NCLUDED WITH THE GNUEABIHF LINUX GNUEABIHF !ND IF ) UPDATE package is a gdbserver that can the compiler at a later time, I can COOPERATE WITH %CLIPSE OR MORE just remove the link and create a precisely, with gdb the debugger. new one pointing to the newer The gdbserver that comes with compiler. I don’t have to change THE "EAGLE"ONE BY DEFAULT DOES

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 69

LJ262-February2016.indd 69 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

NOT COOPERATE WELL WITH %CLIPSE similar sites. So, you have to copy the gdbserver Getting the Temperature: I’ve to the BeagleBone: never worked with 1-Wire devices (I just knew they existed), so I had to

scp /opt/gnueabihf/bin/gdbserver @:~/ do some reading to understand the DEVICE BUT IT IS RATHER SIMPLE %ACH )F YOU JUST USE THE 53" CREATED DEVICE ON THE  7IRE BUS HAS A UNIQUE NETWORK INTERFACE THE ADDRESS IS ADDRESS .OW FOR THE PURPOSE OF THIS  ) HAVE SET UP A 7I &I article and the project, you don’t need dongle on my BBB, so my address TO HAVE A FULL UNDERSTANDING OF THE IS DIFFERENT 1-Wire devices, because the Debian 4HE NEXT TASK IS TO CONFIGURE Linux that comes with the BeagleBone %CLIPSE TO PERFORM REMOTE HAS DRIVERS FOR  7IRE DEVICES 3O debugging. Because this is a rather there’s no need to develop a device long procedure, I have collected it on driver ourselves. a page at http://klaus.ede.hih.au.dk/ The BeagleBone uses the Flattened index.php/BBB_Remote_Debugging. I Device Tree (FDT), like many modern MADE IT FOR MY STUDENTS TO FOLLOW SO ,INUX DISTROS FOR EMBEDDED SYSTEMS READERS OF Linux Journal also should running on an ARM architecture. The BE ABLE TO FOLLOW THESE INSTRUCTIONS ARM CPU that is on the BeagleBone has an electronic pin multiplexer (a Measuring and Controlling KIND OF SWITCHBOARD BUILT IN ENABLING Now we’re ready to start YOU TO CONNECT PINS FROM THE OUTSIDE programming. My typical practise, WORLD TO A SPECIFIC INTERNAL DEVICEˆ when attacking something I don’t FOR EXAMPLE '0)/ OR 07- 4HE &$4 know anything or at least not much will help you set this up, and it tells about, is to create small projects THE KERNEL WHICH TYPE OF INTERFACE IT where I isolate the particular is enabling, so the kernel can use a PROBLEM ) AM DEALING WITH /NE OF suitable driver to service the pin. For THE FIRST THINGS WAS MEASURING THE MORE INFORMATION ON THE &$4 SEE temperature. I had a DS18B20 1-Wire http://elinux.org/Device_Tree. temperature sensor. The sensor is Basically, the FDT works by having EMBEDDED IN A METAL CAP SO IT IS FINE A TEXT FILE WITH THE SPECIFICATION IN FOOD PRODUCTION ENVIRONMENTS OF THE CONNECTION YOU WANT TO 9OU EASILY CAN FIND THEM ON E"AY OR ENABLE 4HIS SPECIFICATION FILE IS

70 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 70 1/21/16 5:27 PM COMPILED INTO BINARY FORMAT 9OU _ _overlay_ _ {

MAY BE INTERESTED IN A GENERATOR FOR ds1820_pins: pinmux_ds1820_pins {

$43 FILES BECAUSE IT TAKES A LOT OF pinctrl-single,pins = <0x78 0x37>;

research and thought to put together };

a DTS. I stumbled upon this site: };

http://kilobaser.com/blog/2014-07-28- };

beaglebone-black-devicetreeoverlay-

generator#dtogenerator, which fragment@1 {

) FIND HELPFUL IN COOPERATION WITH target = <&ocp>;

THE TECHNICAL SPECIFICATION OF _ _overlay_ _ {

the BeagleBone (which is at onewire@0 {

https://github.com/CircuitCo/ status = "okay";

BeagleBone-Black/blob/master/ compatible = "w1-gpio";

BBB_SRM.pdf?raw=true). pinctrl-names = "default";

By looking in the technical pinctrl-0 = <&ds1820_pins>;

SPECIFICATION IN TABLE  ) COULD gpios = <&gpio2 28 0>;

see that pin 12 on the P9 header };

could be connected to a GPIO pin. };

!FTER SOME TESTING ) MANAGED TO };

PUT TOGETHER A DEVICE TREE FILE THAT }; enabled me to connect the DS18B20 to P9 pin 12: 3AVE THE SPECIFICATION IN A FILE called DS18B20-00A0.dts, and

/dts-v1/; COMPILE IT INTO THE BINARY FORMAT

/plugin/; using this command:

/ { root@beaglebone:~# dtc -O dtb -o /lib/firmware/DS1820-00A0.dtbo

compatible = "ti,beaglebone", "ti,beaglebone-black"; ´-b 0 -@ DS1820-00A0.dts

part-number = "DS1820";

version = "00A0"; dtc is a Device Tree Compiler. In

the above command, it’s instructed to

exclusive-use = "P9.12"; OUTPUT IN THE DTB FORMAT -O) into a FILE IN THE LIBFIRMWARE DIRECTORY -o). fragment@0 { The -b sets the boot CPU (here 0) and target = <&am33xx_pinmux>; the -@ means to use symbols, and

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 71

LJ262-February2016.indd 71 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

FINALLY THE INPUT FILE IS SPECIFIED

4O ENABLE THIS PART OF THE DEVICE while True:

tree, issue this command: raw = open(w1, "r").read()

print "Temperature is "+str(float(raw.split("t=")

root@beaglebone:~# echo DS1820 > ´[-1])/1000)+" degrees"

´/sys/devices/bone_capemgr.*/slots time.sleep(1)

)F YOU DONT GET ANY ERRORS RUN ls )F YOU WANT TO HAVE THE -la in the /sys/bus/w1/devices directory: DS18B20 enabled at every boot, PREPARE A FILE IN ETCINITD AND

root@beaglebone:~# ls -la /sys/bus/w1/devices call it enable-DS18B20 or something

total 0 SIMILAR 0UT THIS INTO THE FILE

drwxr-xr-x 2 root root 0 Jan 1 2000 .

drwxr-xr-x 4 root root 0 Jan 1 2000 .. #! /bin/sh

lrwxrwxrwx 1 root root 0 Aug 30 17:32 28-000005a7ce64 ->

´../../../devices/w1_bus_master1/28-000005a7ce64 ### BEGIN INIT INFO

lrwxrwxrwx 1 root root 0 Aug 30 17:32 w1_bus_master1 -> # Provides: enable-DS18B20

´../../../devices/w1_bus_master1 # Required-Start: $all

root@beaglebone:/lib/firmware# # Required-Stop: $all

# Default-Start: 2 3 4 5

4HE  ACE FILE A # Default-Stop: 0 1 6

symbolic link) is the thermometer. # Short-Description: Enables the DS18B20 1-wire

%ACH  7IRE DEVICE HAS A UNIQUE )$ ´on P9 pin 12

SO IT WILL BE IDENTIFIED BY A PATTERN # Description: Connecting the 1-wire driver to

like 28-00000nnnnnnn, where ´the P9 pin 12

NNNNNNN IS THE UNIQUE ADDRESS ### END INIT INFO

! QUICK TEST TO SEE IF THE case "$1" in

DS18B20 is working correctly can start)

be done by changing the device echo "Enabling 1-Wire DS18B20 on P9 Pin 12"

address in this Python script to echo DS1820 > /sys/devices/bone_capemgr.9/slots

SUIT YOUR CONFIGURATION ;;

stop)

import time #no-op

;;

w1="/sys/bus/w1/devices/28-000005a7ce64/w1_slave" *)

72 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 72 1/21/16 5:27 PM #no-op supplied to the heater is to use a

;; 07- SIGNAL TO CONTROL A RELAY )F

esac YOU TURN IT ON FOR ONE THIRD OF THE

TIME AND OFF FOR TWO THIRDS THE

exit 0 APPLIED HEAT IS ONLY ONE THIRD OF THE USUAL AMOUNT OF HEAT WHEN RUNNING !FTER CREATING THE FILE EXECUTE switched on all the time. This can prevent having particles in the brew

root@beaglebone:/etc/init.d# chmod 755 enable-DS18B20 burn onto the heater. See Figure 3

root@beaglebone:/etc/init.d# update-rc.d enable-DS18B20 FOR AN EXAMPLE 07- SIGNAL OUTPUT

´defaults Here I have set the period time to be approximately one second and the Try to reboot your system and check hONv TIME TO BE APPROXIMATELY  that the device is enabled as expected. OF THE PERIOD TIME Setting Up the PWM Controller: !S FOR THE  7IRE YOU ALSO NEED !S ) MENTIONED EARLIER A WAY OF A CONFIGURATION ON THE DEVICE TREE CONTROLLING THE AMOUNT OF POWER in order to gain access to the PWM

Figure 3. PWM Signal Output

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 73

LJ262-February2016.indd 73 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

controllers on the board and to /* the pin header uses */

CONNECT IT TO A SUITABLE PIN )F YOU "P9.14",

consult the BeagleBone’s technical /* the hardware IP uses */

SPECIFICATION YOU WILL FIND THAT "ehrpwm1A";

0  CONNECTS TO A 07- CONTROLLER

So, jump to the on-line device tree fragment@0 {

GENERATOR AND SELECT 0? IN THE target = <&am33xx_pinmux>;

h3ELECT 0INv BOX 4HEN SELECT h&AST _ _overlay_ _ {

3LEWv IN THE h3LEWv BOX h0ULLUPv bs_pwm_P9_14_0x16: pinmux_bs_pwm_P9_14_0x16 {

IN h0ULLUP$OWNv AND FINALLY IN pinctrl-single,pins = <0x048 0x16>;

-UX-ODE h-ODE EHRPWM!v AND };

YOU WILL GET A DEVICE TREE FILE LIKE THIS };

};

*

* Copyright (C) 2013 CircuitCo fragment@1 {

* Copyright (C) 2013 Texas Instruments target = <&ocp>;

* _ _overlay_ _ {

* This program is free software; you can redistribute bs_pwm_test_P9_14 {

* it and/or modify it under the terms of the GNU compatible = "pwm_test";

* General Public License version 2 as pwms = <&ehrpwm1 0 1000000000 1>;

* published by the Free Software Foundation. pwm-names = "PWM_P9_14";

*

* This is a template-generated file from BoneScript pinctrl-names = "default";

*/ pinctrl-0 = <&bs_pwm_P9_14_0x16>;

/dts-v1/;

/plugin/; enabled = <1>;

duty = <0>;

/ { status = "okay";

compatible = "ti,beaglebone", "ti,beaglebone-black"; };

};

/* identification */ };

part-number = "BS_PWM_P9_14_0x16"; };

/* state the resources this cape uses */ )F YOU WANT THE POLARITY INITIALLY SET

exclusive-use = to 0, change the 1 to 0 in the pwms

74 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 74 1/21/16 5:27 PM line so it looks like this: -rw------1 root root 4096 Sep 19 11:40 run

lrwxrwxrwx 1 root root 0 Jan 1 2000 subsystem ->

pwms = <&ehrpwm1 0 500000 0>; ´../../../bus/platform

-rw-r--r-- 1 root root 4096 Jan 1 2000 uevent !GAIN YOU NEED TO COMPILE IT FROM TEXT FORMAT TO THE BINARY FORMAT 4HE FILES OF INTEREST ARE hPOLARITYv readable by the kernel: hPERIODv AND hDUTYv 4HE POLARITY controls whether the controller root@beaglebone:~# dtc -O dtb -o OUTPUTS  OR  WHEN TURNED OFF 4HE ´/lib/firmware/bspwm_P9_14_16-00A0.dtbo period, as the name indicates, controls ´-b 0 -@ bspwm_P9_14_16-00A0.dts THE PERIOD TIMEˆTHAT IS FOR WHAT LENGTH OF TIME IS A COMPLETE CYCLE IN To check that you can create a PWM the controller. The period time is set in DEVICE FILE EXECUTE THESE COMMANDS nanoseconds. Finally, the duty controls THE AMOUNT OF TIME THAT THE SIGNAL root@beaglebone~# echo "am33xx_pwm" > FROM THE 07- CONTROLLER IS ACTIVE ´/sys/devices/bone_capemgr.9/slots 4HE DUTY IS SET IN PERCENTAGESˆFOR root@beaglebone~# echo bspwm_P9_14_16 > EXAMPLE FOR  YOU WOULD WRITE ´/sys/devices/bone_capemgr.9/slots  TO THE DUTY FILE !ND AS FOR THE TEMPERATURE SENSOR )F NO ERRORS ARE SHOWN TAKE A LOOK an automatic start at boot would be in the device directory: HANDY SO CREATE THIS FILE IN ETCINITD enable-pwm with this content:

root@beaglebone:~# ls -al

´/sys/devices/ocp.3/bs_pwm_test_P9_14.16/ #! /bin/sh

total 0

drwxr-xr-x 3 root root 0 Jan 1 2000 . ### BEGIN INIT INFO

drwxr-xr-x 42 root root 0 Jan 1 2000 .. # Provides: enable-pwm

lrwxrwxrwx 1 root root 0 Sep 19 11:40 driver -> # Required-Start: $all

´../../../bus/platform/drivers/pwm_test # Required-Stop: $all

-rw-rw-rw- 1 root root 4096 Mar 1 2015 duty # Default-Start: 2 3 4 5

-r--r--r-- 1 root root 4096 Sep 19 11:40 modalias # Default-Stop: 0 1 6

-rw-rw-rw- 1 root root 4096 Mar 1 2015 period # Short-Description: Enables the PWM chips and

-rw-rw-rw- 1 root root 4096 Mar 1 2015 polarity # connects it through the pinmux

drwxr-xr-x 2 root root 0 Sep 19 11:40 power # Description: Connecting the pwm output through

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 75

LJ262-February2016.indd 75 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

# the pinmux and enables the PWM chip on board 4HEREFORE ) OFTEN PRINT OUT TO THE

### END INIT INFO CONSOLE OR LOG TO A FILE WHICH LETS ME

case "$1" in FOLLOW THE PROGRESS OF THE PROGRAM

start) In this project, I started printing

echo "Enabling PWM on P9 Pin 14" to the console but switched to

grep -q am33xx_pwm /sys/devices/bone_capemgr.9/slots || LOGGING TO A LOGFILE ) SEARCHED THE

´echo "am33xx_pwm" \ Internet and ran across this site:

> /sys/devices/bone_capemgr.9/slots http://www.infernodevelopment.

echo bspwm_P9_14_16 > /sys/devices/bone_capemgr.9/slots com/c-log-file-class-forget-debuggers.

;; The header looks like this:

stop)

#no-op #include

;;

*) using namespace std;

#no-op

;; class Log

esac {

public:

exit 0 Log (const char* filename); ~Log ( ); Reboot your BeagleBone and check void Write (const char* logline, ...); THAT YOU GET THE 07- DEVICE FILES private: created as shown above. ofstream m_stream; Now you’re ready to develop }; a program that will measure the temperature, and based on a recipe, And the implementation is simple: calculate the necessary power to apply to the heater. #include "Log.h" A Simple Log Utility: When #include DEVELOPING A PROGRAM ITS OFTEN NECESSARY TO LOG DIFFERENT VALUES FROM Log::Log (const char* filename) the program, especially when you { have several threads running. Nothing m_stream.open (filename); happens at the time you would expect } IT TOˆTHIS IS THE NATURE OF THREADING

76 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 76 1/21/16 5:27 PM Log::~Log ( ) WHERE YOU OF COURSE NEED TO { CHANGE LOGFILENAME TO MATCH m_stream.close (); YOUR CURRENT LOG FILE } Threading: ) KNOW THAT # )3/ STANDARD  OR JUST #  ALLOWS void Log::Write (const char* logline, ...) YOU TO CREATE THREADS IN # DIRECTLY { but prior to that, I needed a way to va_list argList; MAKE CLASSES OR A FUNCTION IN A CLASS char cbuffer[1024]; into a thread. va_start(argList, logline); ) FOUND AN EXAMPLE CLASS ON vsnprintf (cbuffer, 1024, logline, argList); 3TACK/VERFLOW THAT ) HAVE BEEN USING va_end(argList); ever since. So why learn something m_stream << cbuffer << endl; NEW WHEN YOU KNOW HOW TO DO IT } We are all a bit lazy now and then, ARENT WE (ERE IT IS This version allows me to log

USING THE SPRINTF CONVERSION #include

SPECIFIERSˆFOR EXAMPLE %f FOR #include

FLOATS ) USE IT LIKE THIS

class Threadable

log->Write("PowerController: INFUSION POWER: \ {

PWM set to %f\n", maxPWM); public:

Threadable()

Notice here, that I use %f and pass {

A FLOAT VALUE WHICH WILL BE LOGGED _thread = (pthread_t)NULL;

JUST LIKE S PRINTF WOULD CONVERT THE }

FLOAT TO A STRING OF DIGITS 4HIS IS DUE virtual ~Threadable()

TO THE VARADIC DECLARATION OF THE {/* empty */

Write FUNCTION }

When you want to see the log

while the program runs, open /** Returns true if the thread was successfully

another SSH connection to the started, false if there was an error starting

BeagleBone and run: the thread */

bool StartInternalThread()

root@beaglebone:~# tail -f {

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 77

LJ262-February2016.indd 77 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

return (pthread_create(&_thread, NULL, public:

´InternalThreadEntryFunc, this) == 0); BrewController ( );

} virtual ~BrewController ( );

void InternalThreadEntry ( );

/** Will not return until the internal thread has exited. */ ...

void WaitForInternalThreadToExit()

{ The InternalThreadEntry ( ) is

(void) pthread_join(_thread, NULL); THE THREAD FUNCTION FOR THIS CLASS 3O

} YOU WILL JUST HAVE TO FILL IN THE CODE

pthread_t GetThreadID () { return _thread; } THAT COMPOSES THE THREAD 4HE REST OF

the methods in the class are helper

protected: FUNCTIONS IN ONE WAY OR ANOTHER

/** Implement this method in your subclass with

the code you want your thread to run. */ Controlling the Heater

virtual void InternalThreadEntry() = 0; In order to control the heater, we

need to measure the temperature in

private: the mash. Having a representation

static void * InternalThreadEntryFunc(void * This) OF THE TEMPERATURE IT IS A MATTER OF

{ comparing the current temperature

((Threadable *) This)->InternalThreadEntry(); WITH THE DESIRED TEMPERATURE FOR THE

return NULL; current step in the recipe. We use a

} PID controller to calculate the amount

OF POWER NEEDED BUT DUE TO OUR

pthread_t _thread; experience with particles in the mash

}; burning onto the heater, it may be necessary to limit the power. So the This is a virtual base class that 0)$ CONTROLLER MAY CALL FOR FULL POWER YOU CANT CREATE INSTANCES OF but the applied power is limited. DIRECTLY BUT YOU WILL INHERIT FROM Measuring the Temperature: it in a new sub-class. 4HE FIRST THING TO DO IN THE PROGRAM 3O FOR INSTANCE MY "REW#ONTROLLER is to locate the thermometer or the class begins like this: temperature sensor. I want the program to locate any DS18B20 attached to class BrewController: public Threadable the BeagleBone. As mentioned earlier, { THE  7IRE DEVICES ALL HAVE UNIQUE

78 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 78 1/21/16 5:27 PM ADDRESSES ) CURRENTLY HAVE FIVE OR SIX return 0;

DS18B20 thermometers on hand, so }

) DESIGNED THE SOFTWARE SO THAT ) CAN }

ATTACH ANY OF MY THERMOMETERS TO exit(EXIT_FAILURE);

the BeagleBone, and the program will } locate it and start using it. This code snippet does the trick oneWireDir is a class variable where IT IS FROM A CLASS ) CALL 4HERMOMETER  ) KEEP THE DIRECTORY AND FILE FROM WHICH I can read the temperature sensor.

int Thermometer::locateThermometer() 4HE OUTPUT FROM A $3" IS MADE

{ UP OF DIFFERENT INFORMATION 4AKE A

string initialDir = "/sys/bus/w1/devices/"; look at the output below:

string regExpr = "28-00000";

string dir; root@beaglebone:/sys/bus/w1/devices/28-000005a7ce64# cat w1_slave

85 01 4b 46 7f ff 0b 10 5f : crc=5f YES

// Open directory 85 01 4b 46 7f ff 0b 10 5f t=24312

DIR *dp = opendir(initialDir.c_str()); root@beaglebone:/sys/bus/w1/devices/28-000005a7ce64#

if (!dp)

{ 4HE FIRST LINE IS OF NO INTEREST TO

exit (EXIT_FAILURE); us. The second line shows that the

} TEMPERATURE T IS ª# &ROM THIS

struct dirent *dirp; WE LEARN THAT IT IS JUST A MATTER OF

READING THE W?SLAVE FILE AND GRABBING

// Loop through the directory entries THE TEMPERATURE FROM THE OUTPUT ) HAVE

while ((dirp = readdir(dp)) != NULL) DESIGNED THIS FUNCTION TO DO THE TASK

{

std::size_t found = \ int Thermometer::readTemperature()

string(dirp->d_name).find(regExpr); {

if (found != std::string::npos) int fd;

{ int res;

// We found one entry that matches

oneWireDir = initialDir + string(dirp->d_name)\ // Open the OneWire thermometer file

+ string("/w1_slave"); fd = open(oneWireDir.c_str(), O_RDONLY);

// Nicely close the directory again if(!fd)

(void) closedir(dp); {

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 79

LJ262-February2016.indd 79 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

return -1; BUFFER FOR  7HEN THE  IS LOCATED ) } CONVERT THE !3#)) TEXT TO A FLOAT USING char buf[256]; atof and scale it with 1,000 since // Read from it there are no decimal points in the res = read(fd, buf, sizeof(buf)); READOUT FROM THE SENSOR if (res < 0) The next thing to do is to calculate { the average temperature. I keep the close (fd); LAST FIVE SAMPLES IN AN ARRAY IN ORDER return -1; to calculate an average temperature. } 4HE REASON FOR THIS IS THAT ) OCCASIONALLY // Retrieve current temperature from device saw some spurious measurements if (res > 0) WHERE THE TEMPERATURE WAS A BIT OFF { The averaging will smooth this out: std::size_t found = string(buf).rfind('=');

if (found != std::string::npos) int Thermometer::calcAvgTemp()

{ {

curTemp = atof((const char *) \ float t = avgTemp - curTemp;

&buf[found+1]) / 1000; if ((t < -20 || t > 20) && tempSamples.size() > 4)

} {

else // Skip this measurement - it's way out of range.

{ log->Write("Skipped this temperature: %f, \

close(fd); t = %d\n", curTemp, t);

return -1; return -1;

} }

} if (tempSamples.size() > maxSamples)

close (fd); {

// Call calc average temperature // Get rid of first element

return calcAvgTemp(); tempSamples.erase(tempSamples.begin());

} }

tempSamples.push_back(curTemp);

4HE FILE IS OPENED AND READ AND t = 0;

because there are two = characters in for (int i = 0; i < (int)tempSamples.size(); i++)

THE BUFFER ) PERFORM A REVERSE SEARCH {

FROM THE END OF THE BUFFER RATHER t += tempSamples[i];

THAN FROM THE BEGINNING IN THE }

80 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 80 1/21/16 5:27 PM Figure 4. PID en updated feedback by TravTigerEE—Own work. Licensed under CC BY-SA 3.0 via Commons.

avgTemp = t / tempSamples.size();

return 0; From this point, we just need to set

} up, somewhere else, a timed job that calls the readTemperature() FUNCTION maxSamples currently is set to 5. As I have set it up to read every tenth YOU CAN SEE IF THE VECTOR THAT HOLDS THE SECOND 7ITH A MASS OF APPROXIMATELY SAMPLES IS FILLED ) DROP MEASUREMENTS 30kg, it is a relatively slow process. THAT ARE ª# OFF THE AVERAGE )F THE A PID Controller: What is a PID VECTOR IS NOT FILLED ) ADD THE SAMPLE TO CONTROLLER )T IS A PROPORTIONAL BE INCLUDEDˆIT IS NEEDED WHEN STARTING integral-derivative controller. From the measurements. that, you can see that it is a controller

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 81

LJ262-February2016.indd 81 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

that reacts proportionally on the apply to the heater is controlled measured input compared to the BY THE ERROR SIGNAL )F THE ERROR IS set point (the desired value), and it SMALL A SMALL AMOUNT OF POWER IS accumulates the historic development APPLIED AND IF IT IS LARGE WE APPLY OF THE CONTROLLED USING AN INTEGRAL more power to the heater. By having FUNCTION AND FINALLY IT TRIES TO PREDICT A FACTOR +P TO MULTIPLY WITH THE THE FUTURE IN THE DERIVATIVE PART 4HAT ERROR WE CAN AMPLIFY THE ERROR OR IF WAS A BIG MOUTHFUL ,ETS TAKE THIS ONE THE FACTOR IS BELOW  ATTENUATE THE step at a time. IMPACT OF THE ERROR 4HE PROPORTIONAL 3EE THE DIAGRAM OF A 0)$ CONTROLLER PART OF THE 0)$ CONTROLLER TENDS TO IN &IGURE  OR EXPRESSED AS A FORMULA IN OVERREACTˆTHAT IS THE TEMPERATURE Figure 5. overshoots the desired temperature /UR hPLANTv AS SHOWN IN &IGURE  BY QUITE AN AMOUNT 4HIS IS NOT is the brew tun. We measure, like desired in a process where we are in this project, a temperature, seeking to give our enzymes ideal and elsewhere, we have a desired TEMPERATURES TO WORK IN !ND IF TEMPERATURE WE WANT TO KEEP FOR A enzymes get too warm, they will WHILE 4HE DIFFERENCE BETWEEN THE DENATUREˆTHAT IS DIE 4HE DERIVATIVE measured and desired temperature is PART WILL COMPENSATE FOR THIS TO CALLED THE ERRORˆET IN THE &IGURE some degree. 4HE ERROR SIGNAL IS FED INTO THE The Integral Controller: The proportional, integral and derivative INTEGRAL PART OF THE CONTROLLER SUMS OR PART OF THE CONTROLLER accumulates the error over time. This The Proportional Controller: )F THE part can suppress the proportional measured temperature is lower than PART BUT WITH THE COST OF BEING SLOWER THE DESIRED WE CAN FIRE UP THE HEATER to reach the desired set point. TO RAISE THE TEMPERATURE )F IT IS HIGHER The Derivative Controller: This we can do nothing in this setup but PART LOOKS INTO THE FUTURE SO TO WAIT FOR IT TO COOL DOWN A LITTLE speak. It compares the previous 4HE AMOUNT OF POWER THAT WELL measurement with the current one

Figure 5. PID Formula

82 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 82 1/21/16 5:27 PM IN ORDER TO PREDICT THE FUTURE STATE pGain = 5.0; // Kp OF THE PLANT 4HIS RESULTS IN A QUICKER iGain = 0.025; // Ki raise to the desired set point, but it dGain = 30.0; // Kd ALSO WILL BACK OFF WHEN NEARING THE SET POINT LEAVING THE FINE TUNING TO I let the proportional and derivative the P and I parts. PARTS HAVE QUITE A LARGE IMPACT AND THE &OR FURTHER DETAILS TAKE A LOOK INTEGRAL PART IS DAMPENED QUITE A BIT AT THIS FINE ARTICLE BY 7ESS 3COTT 3O AFTER GETTING THE 0)$ CONTROLLER http://m.eet.com/media/1112634/ set and working as desired (although f-wescot.pdf, or see this page on SOME FINE TUNING CAN BE DONE ON THE Wikipedia: https://en.wikipedia.org/ +N PARAMETERS IN ORDER TO FIND THE wiki/PID_controller. OPTIMAL FOR MY SET UP ITS TIME TO The PID Code: It’s rather easy to turn to the PowerController. IMPLEMENT A 0)$ CONTROLLER IN SOFTWARE Controlling the Heater: The output FROM THE 4HERMOMETER CLASS IS FED // The PID Controller into the PowerController. This class float processVal = curTemp; uses the PID controller to calculate the AMOUNT OF POWER TO APPLY // The Proportional part !S MENTIONED BEFORE ) HAD TO LIMIT float error = setPoint - processVal; THE AMOUNT OF POWER APPLIED EVEN float pTerm = pGain * error; IF THE 0)$ CONTROLLER CALLED FOR MORE 4HE REASON IS THAT BELOW ª#nª# // The Integral part WE HAVE DETERMINED THAT IF WE APPLY iState = iState + error; ONE THIRD OF THE AVAILABLE POWER WE float iTerm = iState * iGain; will not cause anything to burn onto THE HEATER 4HEREFORE ) HAVE SET SOME // The Derivative part LIMITS REGARDLESS OF HOW MUCH POWER float dTerm = (dState - processVal) * dGain; THE 0)$ CONTROLLER CALLS FOR dState = processVal; "UT HEY WHAT ABOUT INFUSION )NFUSION is the point in time where we add the // Scaling to fit malt to the preheated water. It’s also float pwr = (pTerm + iTerm + dTerm) / 10; called the strike temperature. The colder malt mixes with the water and cools it a 4HE FACTORS +P +I AND +D THAT ) LITTLE 4HEREFORE THE INFUSION TEMPERATURE currently use are set to: CAN BE CALCULATED FROM THE MASS OF WATER

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 83

LJ262-February2016.indd 83 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

Figure 6. Strike Temperature Calculation

AND MALT USING THE AMOUNT OF THEM tempPwrRange[1][0] = 60; // Below this temp. use

BOTH AND THE TEMPERATURE OF THE MALT tempPwrRange[1][1] = 0.666; // this amount of pwr

&IGURE   3O IF THE DESIRED INFUSION OR tempPwrRange[2][0] = 102; // Below this temp. use

STRIKE TEMPERATURE IS ª# THE AMOUNT tempPwrRange[2][1] = 1.0; // this amount of pwr OF WATER IS  LITERS AND THE KG OF MALT IS ª# WE CAN CALCULATE THAT THE Then, it’s simple to compare WATER SHOULD BE PREHEATED TO ª# against the current temperature and BEFORE ADDING THE MALT !FTER A SHORT FIND THE MAXIMUM AMOUNT OF POWER WHILE ITLL FIND ITS REST AT ª# that may be applied: )N ORDER TO GET READY FOR BREWING AS QUICKLY AS POSSIBLE WE CAN HEAT THE // Locate the temperature range to operate within WATER WITH FULL POWER UNTIL WE ADD THE // Use the current temperature measured to retrieve MALT 3O WE NEED TO KNOW THE DIFFERENT // the maxPwr STEPS OF THE MASHING PROCESS WHICH for (int i = 0; i < 3; i++) is kept in a database (more about that { later). From that, we get the set point. limitTemp = tempPwrRange[i][0]; I have put the maximum maxPwr = tempPwrRange[i][1]; temperature ranges into a if (curTemp > limitTemp) TWO DIMENSIONAL ARRAY OF FLOATS {

continue;

tempPwrRange[0][0] = 57; // Below this temp. use }

tempPwrRange[0][1] = 0.333; // this amount of pwr }

84 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 84 1/21/16 5:27 PM !FTER HAVING THE 0)$ CONTROLLER // If we are in the infusion step it is allowed to use max pwr

CALCULATE THE DESIRED POWER IT IS LEFT if (infusion)

to the setPwrLvl FUNCTION TO FIND {

what can be set according to the if (pwm.setDutyCycle (maxPWM) == 0)

CURRENT STEP IN THE RECIPE FROM WHERE {

we have derived the maxPwr: log->Write("PowerController: INFUSION POWER:\

PWM set to %f\n", maxPWM);

float PowerController::setPwrLvl (float pwr, }

float maxPwr) else

{ {

// The PWM operates with a high granularity, log->Write("PowerController: ERROR: PWM not\

// hence we have to scale the pwr set! value: %f\n", maxPWM);

}

// Corrects the duty cycle when applied to the PWM return 1.0;

const long long Factor = 999000000; }

log->Write("PowerController: maxPwr is set to %f",\

maxPwr); // If the pwr is less than the maxPwr allowed

// use the requested pwr

// If the requested pwr is less that 0 if (pwr < maxPwr)

// we set it to 0 {

if (pwr < 0) // Set PWM to pwr

{ if (pwm.setDutyCycle (pwr * Factor) == 0)

// Set PWM to 0 {

if (pwm.setDutyCycle (0) == 0) log->Write("PowerController: PWM set to \

{ %f\n", pwr * Factor);

log->Write("PowerController: PWM set to 0\n"); }

} else

else {

{ log->Write("PowerController: ERROR: PWM not \ set!\n");

log->Write("PowerController: ERROR: \ }

PWM not set! (pwr<0)\n" ); return pwr;

} }

return 0.0;

} // Otherwise use the maximum power, i.e. 100%

else

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 85

LJ262-February2016.indd 85 1/21/16 5:27 PM FEATURE Temperature Control in a Homebrewing Tun Using a BeagleBone Black

{ return -1;

// Set PWM to the maxPwr }

if (pwm.setDutyCycle (maxPwr * Factor) == 0)

{ // Write the requested duty cycle to the file

log->Write("PowerController: PWM set to \ char str[100];

%f\b", maxPwr * Factor); sprintf(str, "%lld", duty);

} size_t res = write (f, str, strlen(str));

else close (f);

{

log->Write("PowerController: ERROR: PWM not \ // Handle errors

set!\n"); if (res != strlen (str))

} {

return maxPwr; log->Write("ERROR Opening file %s with this \

} error: %s\n",

return 0.0; dir.c_str(),

} strerror( errno ));

return -1; The setDutyCycle FUNCTION ACTUALLY } WRITES TO THE 07- FILE THAT REPRESENTS return 0; the PWM controller on board: }

int PWMController::setDutyCycle(long long duty) 9ES ) KNOWˆ) AM A # HACKER MIS { USING THE FINE # LANGUAGE WITH FILE string dir; operations in plain C code. But, so it IS 7HAT COMES EASIEST TO THE FINGERS dir = initialDir + "/duty"; over the keyboard is what goes in the CODEˆSOMETIMES ) THINK THERE MAY // Open the file representing the PWM controller not be a brain involved at all. But int f = open(dir.c_str(), O_WRONLY); OTHERWISE ITS STRAIGHTFORWARD CODE if (f < 0) 4HIS CONCLUDES THE MEASURING OF { the temperature and controlling the log->Write("ERROR Opening file %s with this \ POWER OUTPUT )N A FUTURE ARTICLE )LL error: %s\n", LOOK INTO ADDING AN 31,ITE DATABASE dir.c_str(), to the brewController program. In strerror( errno )); order to service a Web page, we also

86 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 86 1/21/16 5:27 PM need a WebSocket server, so I will make it start automatically when the introduce that as well. BeagleBone is powered. The database has two purposes: All the code will be open-sourced STORING RECIPE DETAILS FOR THE MASH WHEN IT HAS PROVEN TO CONTROL A FEW PROCESS AND CONTINUOUS LOGGING OF BREWS THIS AUTUMN )F YOU ARE IN A data produced during a brew. hurry, send me an e-mail and I will The WebSocket server is the send you the code.Q INTERFACE TO THE 7EB PAGE FROM where we can monitor and control Klaus Kolle is currently teaching electronic engineering the brewing process. So I also will students software development and development dive into a relatively complex Web in the Linux kernel at Aarhus University. Klaus has been PAGE WITH A LOT OF *AVA3CRIPT CODE working and developing on UNIX and Linux since 1988. to control the graphs and retrieval Klaus loves well brewed beer, especially if it is home-brewed. OF DATA FROM THE 7EB3OCKET SERVER WHICH WILL LOOK UP IN THE DATABASE FOR the correct data to serve. Send comments or feedback via Finally, I’ll write about how to http://www.linuxjournal.com/contact dæmonise the brewController and or to [email protected].

Resources

You can retrieve a cross compiler for the ARM processor at http://releases.linaro.org/14.11/components/toolchain/binaries/arm-linux-gnueabihf/ gcc-linaro-4.9-2014.11-x86_64_arm-linux-gnueabihf.tar.xz.

My instructions to set up the remote debugging in Eclipse: http://klaus.ede.hih.au.dk/index.php/BBB_Remote_Debugging

About the Flattened Device Tree (FDT): http://elinux.org/Device_Tree

A Web page for setting up the Device Tree specification files: http://kilobaser.com/ blog/2014-07-28-beaglebone-black-devicetreeoverlay-generator#dtogenerator

A simple log class: http://www.infernodevelopment.com/c-log-file-class-forget-debuggers

The technical specifications for the BeagleBone Black: https://github.com/CircuitCo/BeagleBone-Black/blob/master/BBB_SRM.pdf?raw=true

PID without a PhD: http://m.eet.com/media/1112634/f-wescot.pdf

Wikipedia on PID controllers: https://en.wikipedia.org/wiki/PID_controller

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 87

LJ262-February2016.indd 87 1/21/16 5:27 PM FEATURE Command-Line Tutorial: Does Every Year Have a Friday the 13th?

Command-Line Tutorial: Does Every Year Have a Friday the 13th? Can you write a one-liner that answers this question for a given year? Can you write a short script around your one-liner and solve this puzzle for all time? SOL LEDERMAN

88 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 88 1/21/16 5:27 PM ERES A FUN LITTLE MATH PROBLEM YEARS 0ONDER THAT FOR A BIT THAT A HANDFUL OF ,INUX ,ETS DIVE IN (OW MANY UNIQUE H COMMANDS A BIT OF THOUGHT CALENDARS ARE THERE 4WO CALENDARS ARE and a short shell script will help solve: hUNIQUEv IF THEY LOOK DIFFERENTˆTHAT does every year have a Friday the IS IF SOME MONTHS START ON DIFFERENT TH cal, cut, grep, sed and cksum days when you compare the two. WILL BE YOUR FRIENDS IN THIS EXPLORATION 4WO CALENDARS ARE THE SAME IF YOU "EFORE ENGAGING THE COMPUTER could hang either one on the wall, let’s consider how you might solve AND EXCEPT FOR THE YEAR PRINTED AT this problem without one; you could THE TOP OF THE CALENDAR YOU COULDNT EXAMINE SOME NUMBER OF CALENDARS TELL THE DIFFERENCE  STARTED ON )F YOU DISCOVER A YEARS CALENDAR WITH a Wednesday, and 2015 started on no Friday the 13th, you’re done. A 4HURSDAY 4HOSE ARE TWO UNIQUE Otherwise, you eventually will come CALENDARS  AND  BOTH to believe that every year does have started on a Sunday. Are they the A &RIDAY THE TH h%VENTUALLYv IS A SAME CALENDAR #OULD YOU SWAP ONE long time away though. So, how many FOR THE OTHER .O BECAUSE  WAS CALENDARS DO YOU NEED TO LOOK AT 9OU A LEAP YEAR BUT  WASNT 3O may want to pause to consider this ALTHOUGH *ANUARY OF BOTH CALENDARS QUESTION BEFORE READING FURTHER !ND WAS THE SAME FOR THOSE YEARS THE to support your exploration, you may other months weren’t. want to use the Linux cal command Considering that some years are leap to display calendars to your terminal. years and that others aren’t leads to 7ELCOME BACK $ID YOU COUNT  one approach to counting calendars. DIFFERENT CALENDARS THAT YOU NEED TO %VERY YEAR STARTS ON ONE OF THE SEVEN CONSIDER $ID YOU COUNT SEVEN "OTH DAYS )F YOU DONT CONSIDER LEAP YEARS answers are correct, depending on (and there is an approach where you YOUR APPROACH )F YOU DONT BELIEVE don’t need to consider them), then that looking at seven calendars THERE ARE SEVEN DIFFERENT CALENDARS )F is enough, see the suggested you also consider leap years, you have EXPLORATIONS AT THE END OF THE ARTICLE SEVEN MORE CALENDARS TO CONSIDER 3O IF ,ETS ALSO CONSIDER LEAP YEARS FOR THIS YOU EXAMINE  UNIQUE CALENDARS YOU exploration. And, there’s another will have the answer. You either will consideration: is it enough to inspect FIND A YEAR THAT DOESNT HAVE A &RIDAY THE CALENDARS FOR  CONSECUTIVE the 13th or you won’t.

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 89

LJ262-February2016.indd 89 1/21/16 5:27 PM FEATURE Command-Line Tutorial: Does Every Year Have a Friday the 13th?

(ERE IS THE PART OF THE ANALYSIS WHERE phone line. You want some assurance the computer is going to help. You THAT THE FILE WASNT CORRUPTED DURING NEED TO LOOK AT  UNIQUE CALENDARS the transmission. What could you Will it be enough to look at the DO 9OU COULD COMPARE THE SIZES OF CALENDARS FOR  THROUGH  THE FILES ON BOTH ENDS )F THE SIZES 4HATS A  YEAR SPAN )T TURNS OUT THAT ARE DIFFERENT YOU WOULD KNOW THE THAT SPAN DOES NOT INCLUDE  UNIQUE FILE COPY IS CORRUPT "UT IF THE SIZES calendars. In particular, 2001 and 2007 are the same, you certainly can’t be are the same calendar. Rather than CONFIDENT THAT THE CONTENT OF THE FILES trying to think through how many is the same. You could transmit the calendars you need to look at, let’s FILE TWICE AND COMPARE THE TWO COPIES TAKE A DIFFERENT APPROACH ,ETS WRITE byte by byte, but that would double Checksums solve the problem of verifying that two files are identical without incurring a large computational or transmission cost.

a shell script that will review calendars the transmission time. UNTIL IT HAS LOOKED AT  UNIQUE ONES Checksums solve the problem How can you make the computer OF VERIFYING THAT TWO FILES ARE TELL YOU IF CALENDARS FOR TWO YEARS ARE identical without incurring a large THE SAME ,ETS TAKE A DETOUR AWAY computational or transmission cost. FROM THAT QUESTION FOR A SHORT WHILE The Linux cksum utility computes and consider this command: cksum. A SINGLE NUMBERˆTHE CHECKSUM cksum DISPLAYS THE CHECKSUM OF A FILE FOR A FILE 4HE PROCESS FOR USING OR OF STDIN  7HATS A CHECKSUM ! the checksum is this: compute the checksum is a number associated with CHECKSUM TRANSMIT THE FILE COMPUTE A FILE THAT IS COMMONLY USED TO DETECT THE CHECKSUM FOR THE RECEIVED FILE transmission errors. AND COMPARE THE CHECKSUMS )F THE )MAGINE THAT YOU SEND A FILE FROM checksums are identical, there is a one computer to another via a noisy NEAR  PROBABILITY THAT THE TWO

90 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 90 1/21/16 5:27 PM FILES ARE IDENTICAL .EAR  IS QUITE commands. I mentioned earlier that GOOD ENOUGH FOR MOST PURPOSES 2001 and 2007 are the same calendar. Now, let’s return to the calendar #OMPARE THE OUTPUT OF THESE TWO PROBLEM 9OU WANT TO FIND  UNIQUE commands: CALENDARS AND CHECK EACH TO SEE IF they have a Friday the 13th. Here’s cal 2001 some pseudo-code that illustrates cal 2007 this approach: .OW COMPUTE THE CHECKSUM OF Year = 2000 # Year increases until we EACH CALENDAR BY PIPING THE OUTPUT OF # have seen 14 unique cal to cksum: # calendars ChecksumCount = 0 # This counts how many unique cal 2001 | cksum # calendars we've seen 3673415557 2014 While ChecksumCount < 14 { Compute the checksum for Year cal 2007 | cksum If we have not seen this checksum yet { 2655244645 2014 Add this checksum to the list of checksums we've seen Hmmm. The two checksums are Add 1 to ChecksumCount NOT THE SAME 7HY IS THAT .OTE THAT } hv IS THE NUMBER OF CHARACTERS Add 1 to Year IN THE OUTPUT OF cal. Ignore that and } JUST LOOK AT THE FIRST NUMBER )TS THE TITLE IN THE OUTPUT OF cal (the year) Why do you use checksums to that makes the two calendars appear COMPARE CALENDARS 7HY DONT YOU JUST TO BE DIFFERENT !LL YOU NEED TO DO IS CREATE A FILE FOR EACH UNIQUE CALENDAR REMOVE THE FIRST LINE AND YOULL BE and compare each calendar to every good. sed IS ONE OF A NUMBER OF ,INUX SAVED FILE 9OU COULD DO THAT "UT TOOLS THAT EASILY CAN REMOVE THE FIRST once you work through the checksum LINE OF THE OUTPUT approach, I think it will become clear THAT COMPARING FILES WITH THE diff cal 2001 | sed '1d' | cksum command or some other approach, is 2408573533 1980 clunkier. You decide. cal 2007 | sed '1d' | cksum It’s time to dive in to some shell 2408573533 1980

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 91

LJ262-February2016.indd 91 1/21/16 5:27 PM FEATURE Command-Line Tutorial: Does Every Year Have a Friday the 13th?

Note that cksum may generate DATA PROCESSING ,INUX TOOL )F YOUVE A DIFFERENT CHECKSUM FOR YOUR cal never used awk, it’s worth reading a output. That’s because cal has slightly tutorial about it and exploring it. For DIFFERENT LAYOUTS ON SOME FLAVORS OF THE PURPOSES OF THIS ARTICLE ALL YOURE Linux. As long as you’re comparing doing with awk is telling it to print cal output on the same machine, THE FIRST WHITESPACE SEPARATED FIELD everything will work correctly. OF THE INPUT  FROM THE PIPE cksum HAS CONFIRMED THAT  With the new checksum, and 2007 are the same calendar.  USE grep to search the 4HE NEXT TASK IS TO BUILD A LIST OF THE LIST OF CHECKSUMS YOUVE SEEN FOR THIS UNIQUE CHECKSUMS THAT YOUVE SEEN SO new one: that you can compare new calendar checksums to those on the list. Let’s echo "$checksums" | grep -w 2408573533 use a string to hold that list and initialize it to be empty: Note that it’s a good idea to use the -w FLAG WITH grep to match only checksums=""  AS A WORD WITH SPACES tabs and punctuation separating 4HEN ADD THE CHECKSUM FOR  WORDS 9OU DONT FOR EXAMPLE WANT to the list: TO MATCH  You also will want to use grep with checksums="$checksums 2408573533" the -q option when you write your shell script to count Friday the 13ths. In order to add just the checksum The option tells grep TO SEARCH QUIETLY TO THE LIST AND NOT THE NUMBER OF and not to display any output. You then CHARACTERS IN THE OUTPUT OF cal, you WILL USE THE RETURN STATUS FROM grep to WILL NEED TO KEEP JUST THE FIRST PART know whether it matched anything: OF cksum’s output and discard the second part. There are many ways to echo $checksums | grep $newChecksum do this in bash. Here’s one way: if [ $? -eq 0 ] # if return value is 0 then cal 2001 | sed '1d' | cksum | awk '{print $1}' # we matched something 2408573533 fi

!WK IS A VERY POWERFUL TEXT AND 7HENEVER YOU SEE A UNIQUE

92 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 92 1/21/16 5:27 PM calendar, increment a counter: man page and play with it a little BEFORE READING FURTHER numUniqueCalendars=$((numUniqueCalendars+1)) cut allows you to select one or more columns. Sunday dates are $((..)) is a way you can do simple in columns 1 and 2; column 3 is arithmetic in bash. Note that inside blank; Monday dates are in columns the $((..)), you don’t put dollar  AND  AND IF YOU KEEP COUNTING SIGNS IN FRONT OF VARIABLE NAMES columns, you see that Friday dates 7HEN THE COUNTER REACHES  ARE IN COLUMNS  AND  0IPE THE YOUVE SEEN ALL UNIQUE CALENDARS OUTPUT OF cal to cut, and you get Now that you have a general idea all the Friday dates: about how to use checksums, strings and grep to determine whether cal 4 2001 | cut -c16-17 A CALENDAR IS UNIQUE AND TO COUNT Fr UNIQUE CALENDARS THERE IS ONE MORE 6 SIGNIFICANT TASK 'IVEN A CALENDAR 13 HOW DO YOU KNOW IF IT HAS A &RIDAY THE 20 TH ,ETS LOOK AT THE OUTPUT FROM 27 cal FOR JUST ONE MONTH All you need now is to grep these cal 4 2001 &RIDAY DATES FOR THE NUMBER  April 2001 Su Mo Tu We Th Fr Sa cal 4 2001 | cut -c16-17 | grep 13 1 2 3 4 5 6 7 13 8 9 10 11 12 13 14 15 16 17 18 19 20 21 So, you now have a Linux pipeline 22 23 24 25 26 27 28 THAT WILL TELL YOU IF A GIVEN YEAR AND 29 30 month have a Friday the 13th. How can you generalize this approach to April 13, 2001, is a Friday. It’s easy SCAN AN ENTIRE YEAR .OTE THAT cal ENOUGH FOR US HUMANS TO SCAN THE DISPLAYS ENTIRE YEARS IN A X LAYOUT &RIDAY COLUMN LOOKING FOR  (OW OF THREE MONTHS ACROSS AND FOUR MIGHT A COMPUTER DO THE SAME )F months down. You can use cut again YOURE NOT FAMILIAR WITH THE ,INUX AND HAVE IT DISPLAY THREE COLUMNS OF cut command, take a look at its Fridays. Note that you will need to

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 93

LJ262-February2016.indd 93 1/21/16 5:27 PM FEATURE Command-Line Tutorial: Does Every Year Have a Friday the 13th?

include a column with a space, either You may be wondering why BEFORE OR AFTER THE DATES OTHERWISE you need the -o option to grep. the column values will run together: Consider this:

cal 2001 | cut -c16-18,38-40,60-62 echo 12 13 14 13 | grep 13 12 13 14 13 Note that the column ranges may be DIFFERENT IN YOUR VERSION OF ,INUX ) HAD The number 13 appears twice in to change column numbers to get the the list that you echo, but grep is OUTPUT TO BE CORRECT ON ONE OF MY ,INUX matching the entire line, and when MACHINES (ERE IS SOME OF THE OUTPUT YOU PIPE THE OUTPUT OF grep to wc -l to count matches, you get this: Fr Fr Fr 6 4 1 echo 12 13 14 13 | grep 13 | wc -l 13 11 8 1 20 18 15 27 25 22 It would be nice to know how many 29 Friday the 13ths there are in a year. This is where the -o OPTION IS USEFUL

Fr Fr Fr echo 12 13 14 13 | grep -o 13 6 3 13 13 10 7 13 20 17 14 27 24 21 Adding -o tells grep to print only 31 28 THE MATCHING PART OF LINES AND TO PRINT matches on multiple lines. Now you Notice that there are two Friday EASILY CAN COUNT &RIDAY THE THS FOR the 13ths in 2001. Let’s add grep to any year. How many Friday the 13ths THE PIPELINE TO FILTER OUT JUST THE LINES WERE THERE IN  ,ETS SEE with 13 in them and use wc to count

those lines: cal 2014 | cut -c16-18,38-40,60-62 | grep -o 13 | wc -l

1

cal 2001 | cut -c16-18,38-40,60-62 | grep -o 13 | wc -l

2 #ONGRATULATIONS 9OUVE LAID ALL OF

94 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 94 1/21/16 5:27 PM the groundwork and should now be not have a space, tab or any other able to write a shell script to answer CHARACTER AFTER THEM THIS ARTICLES BURNING QUESTION

Let’s summarize the programming #!/bin/bash

approach used here. This should # Starting with the year 2000, we

look very similar to the pseudo- # compute how many times

code earlier in this article with one # Friday the 13th occurs in every

addition: you are going to count the # year. We stop when we have

NUMBER OF &RIDAY THE THS IN EACH # looked at 14 different calendars.

UNIQUE CALENDAR year=1999

checksums=""

1. Pick a starting year. Does it matter numUniqueCalendars=0

WHAT THAT YEAR IS echo "Year Number of Friday the 13ths"

while [ $numUniqueCalendars -lt 14 ]

2. For this year, compute its do

calendar checksum. year=$((year+1))

sum=$(cal ${year} | sed '1d' | cksum |\

 )F THE CHECKSUM IS NOT ON THE awk '{print $1}')

UNIQUE CHECKSUM LIST THEN YOU echo "$checksums" | grep -q $sum

are looking at a calendar that if [ $? -ne 0 ]

you’ve not looked at previously. then # It's a new calendar

!DD ITS CHECKSUM TO YOUR UNIQUE checksums="$checksums $sum"

checksum list, and add one to numUniqueCalendars=$((numUniqueCalendars+1))

THE NUMBER OF UNIQUE CALENDARS echo -n "${year} "

you’ve seen. Count the number cal ${year} | cut -c16-18,38-40,60-62 |\

OF TIMES &RIDAY THE TH APPEARS grep -o 13 | wc -l

in this year. fi

done  )F YOUVE SEEN  UNIQUE CALENDARS YOURE DONE )F NOT ADD ONE TO THE Here’s the output: year and loop back to step 2. Year Number of Friday the 13ths Here is a bash script that puts 2000 1 ALL OF THESE IDEAS TOGETHER .OTE 2001 2 that the lines ending in \ need to 2002 2

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 95

LJ262-February2016.indd 95 1/21/16 5:27 PM FEATURE Command-Line Tutorial: Does Every Year Have a Friday the 13th?

2003 1 TH 7HATS THE MOST ANY YEAR CAN 2004 2 HAVE &OR THIS EXPLORATION AND FOR 2005 1  YOULL NEED TO GENERALIZE THE 2006 2 CONCEPT OF PULLING OUT ONE OR MORE 2008 1 COLUMNS FROM A CALENDAR TO BE ABLE 2009 3 TO SELECT ANY DAY OF THE WEEK 2010 1 2012 3  )F YOU EXCLUDE *ANUARY AND 2016 1 February, does every year still have 2020 2 A &RIDAY THE TH 2024 2   HAS THREE &RIDAY THE THS Notice that you had to examine 7HEN WILL THAT HAPPEN NEXT 25 consecutive calendars (between  AND  TO FIND  UNIQUE  (OW LIKELY IS THE TH OF A MONTH ones. And, notice...drum roll... TO FALL ON A &RIDAY VS THE OTHER DAYS Friday the 13th occurs once, twice OF THE WEEK (AVE THE COMPUTER or three times in every year. SCAN EVERY MONTH FOR  YEARS OF I chose to start checking calendars CALENDARS TO FIND OUT starting in 2000. What would have HAPPENED IF ) HAD CHOSEN SOME  7HICH DAYDATE PAIRS OCCUR EVERY OTHER YEAR YEAR &OR EXAMPLE DOES EVERY I hope you appreciate the power YEAR HAVE A -ONDAY THE ST 4HIS OF A HANDFUL OF ,INUX COMMANDS exploration generalizes #2.Q some logical thinking and a short SHELL SCRIPT TO SOLVE A FUN MATH Sol Lederman is a math geek and general techie. As a CHALLENGE )F YOU ENJOYED THIS writer and programmer, he especially loves to communicate problem, consider exploring these technical things to a broad audience. Read more about related problems: Sol at http://sollederman.com.

1. How would using ncal INSTEAD OF cal IF ITS AVAILABLE ON YOUR SYSTEM SIMPLIFY YOUR CODE Send comments or feedback via http://www.linuxjournal.com/contact 2. Does every year have a Monday the or to [email protected].

96 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 96 1/21/16 5:27 PM Instant Access to Premium Online Drupal Training

Instant access to hundreds of hours of Drupal training with new videos added every week!

Learn from industry experts with real world H[SHULHQFHEXLOGLQJKLJKSURȴOHVLWHV

Learn on the go wherever you are with apps for iOS, Android & Roku

We also offer group accounts. Give your whole team access at a discounted rate!

Learn about our latest video releases and RIIHUVȴUVWE\IROORZLQJXVRQ)DFHERRNDQG 7ZLWWHU #GUXSDOL]HPH 

Go to http://drupalize.me and get Drupalized today!

LJ262-February2016.indd 97 1/21/16 5:27 PM FREE DOWNLOADS

WEBCASTS Maximizing NoSQL Clusters for Large Data Sets Sponsor: IBM

4HIS FOLLOW ON WEBCAST TO 2EUVEN - ,ERNERgS WELL RECEIVED AND WIDELY ACCLAIMED 'EEK 'UIDE 4AKE #ONTROL OF 'ROWING 2EDIS .O31, 3ERVER #LUSTERS WILL EXTEND THE DISCUSSION AND GET INTO THE NUTS AND BOLTS OF OPTIMALLY MAXIMIZING YOUR .O31, CLUSTERS WORKING WITH LARGE DATA SETS 2EUVENgS DEEP KNOWLEDGE OF DEVELOPMENT AND .O31, CLUSTERS WILL COMBINE WITH "RAD "RECHgS INTIMATE UNDERSTANDING OF THE INTRICACIES OF )"-gS 0OWER 3YSTEMS AND LARGE DATA SETS IN A FREE WHEELING DISCUSSION THAT WILL ANSWER ALL YOUR QUESTIONS ON THIS COMPLEX SUBJECT

> http://geekguide.linuxjournal.com/content/maximizing-nosql-clusters-large-data-sets

How to Build High-Performing IT Teams — Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report Sponsor: Puppet Labs

$EV/PS REPRESENTS A PROFOUND CHANGE FROM THE WAY MOST )4 DEPARTMENTS HAVE TRADITIONALLY WORKED FROM SILOED TEAMS AND HIGH ANXIETY RELEASES TO EVERYONE COLLABORATING ON UNEVENTFUL AND MORE FREQUENT RELEASES OF HIGHER QUALITY CODE )T DOESNgT MATTER HOW LARGE OR SMALL AN ORGANIZATION IS OR EVEN WHETHER ITgS HISTORICALLY SLOW MOVING OR RISK AVERSE ˆ THERE ARE WAYS TO adopt DevOps sanely, and get measurable results in just weeks.

> http://geekguide.linuxjournal.com/content/how-build-high-performing-it-teams-including-new-data- it-performance-puppet-labs-2015-state

WHITE PAPERS Comparing NoSQL Solutions In a Real-World Scenario Sponsor: RedisLabs | Topic: Web Development | Author: Avalon Consulting

3PECIALIZING IN CLOUD ARCHITECTURE %MIND #LOUD %XPERTS IS AN !73 !DVANCED #ONSULTING 0ARTNER AND A 'OOGLE #LOUD 0LATFORM 0REMIER 0ARTNER THAT ASSISTS ENTERPRISES AND STARTUPS IN ESTABLISHING SECURE AND SCALABLE )4 OPERATIONS 4HE FOLLOWING BENCHMARK EMPLOYED A REAL WORLD USE CASE FROM AN %MIND CUSTOMER 4HE %MIND TEAM WAS TASKED WITH THE FOLLOWING HIGH LEVEL REQUIREMENTS

s Support a real-time voting process during massive live events EG TELEVISED ELECTION SURVEYS OR h!MERICA 6OTESv TYPE GAME SHOWS  s +EEP VOTERS DATA ANONYMOUS BUT UNIQUE s %NSURE SCALABILITY TO SUPPORT SURGES IN REQUESTS

> http://geekguide.linuxjournal.com/content/comparing-nosql-solutions-real-world-scenario

98 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 98 1/21/16 5:27 PM FREE DOWNLOADS

NEW Forrester Study! Linux Management with Red Hat Satellite: Measuring Business Impact and ROI !CHIEVING !PPLICATION $ELIVERY 6ELOCITY WITH A  2/)

)"- COMMISSIONED &ORRESTER #ONSULTING TO CONDUCT ITS 4OTAL %CONOMIC )MPACT© 4%) STUDY THAT EXAMINES AND QUANTIFIES POTENTIAL RETURN ON INVESTMENT 2/) FOR )"- 5RBAN#ODE $EPLOY WITHIN AN ENTERPRISE $EV/PS ENVIRONMENT 4HE STUDY DETERMINED THAT A COMPOSITE ORGANIZATION BASED ON THE CUSTOMERS INTERVIEWED EXPERIENCED AN 2/) OF 

Read the Forrester Consulting study and learn learn how these enterprise organizations achieved:

s  REDUCTION IN THE COST OF RELEASES s 2EDUCTION IN THE RISK OF FAILED DEPLOYMENTS s  FASTER DEPLOYMENT TIMES

See how IBM UrbanCode brings deployment velocity while reducing release costs.

> http://devops.linuxjournal.com/devops/total-economic-impacttm-ibm-urbancode

Mobile to Mainframe DevOps for Dummies )N TODAYS ERA OF DIGITAL DISRUPTION EMPOWERED BY CLOUD MOBILE AND ANALYTICS ITS IMPERATIVE FOR ENTERPRISE ORGANIZATIONS TO DRIVE FASTER INNOVATION WHILE ENSURING THE STABILITY OF CORE BUSINESS SYSTEMS 7HILE INNOVATIVE SYSTEMS OF ENGAGEMENT DEMAND SPEED AGILITY AND EXPERIMENTATION EXISTING SYSTEMS OF RECORD REQUIRE SIMILAR ATTRIBUTES WITH ADDITIONAL AND UNCOMPROMISING REQUIREMENTS FOR GOVERNANCE AND PREDICTABILITY )N THIS NEW BOOK BY 2OSALIND 2ADCLIFFE )"- $ISTINGUISHED %NGINEER YOU WILL LEARN ABOUT

s 2ESPONDING TO THE CHALLENGES OF VARIABLE SPEED )4 s 7HY THE MAINFRAME IS A UNIQUE AND IDEAL PLATFORM FOR DEVELOPING HYBRID CLOUD APPLICATIONS s (OW MOBILE FRONT ENDS CAN REJUVENATE BACK END SYSTEMS TO REACH NEW CUSTOMERS s !ND SPECIAL CONSIDERATIONS FOR USING A $EV/PS APPROACH TO ACCELERATE MAINFRAME SOFTWARE DELIVERY

> http://devops.linuxjournal.com/devops/mobile-mainframe-devops-dummies

BRAND-NEW EDITION! DevOps For Dummies - New Edition with SAFe® )N THIS .%7 ND EDITION LEARN WHY $EV/PS IS ESSENTIAL FOR ANY BUSINESS ASPIRING TO BE LEAN AGILE AND CAPABLE OF RESPONDING rapidly to changing customers and marketplace.

$OWNLOAD THE % BOOK TO LEARN ABOUT

s 4HE BUSINESS NEED AND VALUE OF $EV/PS s DevOps capabilities and adoption paths. s How cloud accelerates DevOps. s The Ten DevOps myths. s And more.

> http://devops.linuxjournal.com/devops/devops-dummies-new-edition-safe

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 99

LJ262-February2016.indd 99 1/21/16 5:27 PM EOF Giving Silos DOC SEARLS Their Due We’re not wrong in our principles or our work. But, we may have to admit that silos have proven right in some big ways.

wo things I got way wrong, 0AUL 3AFFO FAMOUSLY SAID WE way back. overestimate in the short term and T One was Linux on the Desktop underestimate in the long. But I’m (LOTD, http://www.linuxjournal.com/ going to go out on a limb and say googlesearch?s=lotd). Around the long term is mostly behind us. THE TURN OF THE -ILLENNIUM ) (How many years have we been PREDICTED BIG SUCCESSES FOR ,/4$ WAITING FOR ,/4$ TO TAKE OFF and Linux on the Laptop (LOTL, Somebody is going to break the ice http://www.linuxjournal.com/ this year. My own instinct says it’ll article/7464 ˆAND CONTINUED TO be IBM, mostly because there must do the same, annually, until I BE SERIOUS DEMAND FOR ,/4$ AS gave up. Here’s how my optimism well as improvised solutions with looked in January 2003 it) inside the company. You can’t (http://www.linuxjournal.com/ have that many engineers walking article/6548): AROUND WITH ,INUXIFIED 4HINK0ADS and not get around to selling them Is 2003 the year we see at some point. big-name hardware companies selling Linux as aggressively as I’m betting Dell will be next. Then THEY SELL 7INDOWSˆBY WHICH Gateway. Then HP.... I mean Linux PCs show up in STORES AND ON THE FRONT PAGES Didn’t happen. Sure, some OF 7EB SITES OF THE BIGS SOLD PERSONAL ,INUX

100 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 100 1/21/16 5:27 PM EOF

Which brings me to the second thing I got wrong: expecting the world to prefer flat, distributed, open and free technology ecosystems over silo’d centralized, closed and proprietary ones.

hardware (and still do), but not “as MORE VALUABLE THAN FREE ONESˆAND AGGRESSIVELY AS THEY SELL 7INDOWSv everybody agrees, at least tacitly, )F YOURE NOT A ,INUX GEEK TODAY THAT hFREE MARKETv MEANS hYOUR YOUR CHOICES OF DESKTOPS AND CHOICE OF CAPTORv laptops are contained by two silo’d Which brings me to the second OPERATING SYSTEMS FROM JUST TWO thing I got wrong: expecting the COMPANIES !PPLE AND -ICROSOFT WORLD TO PREFER FLAT DISTRIBUTED Hardware choices are also narrower OPEN AND FREE TECHNOLOGY ECOSYSTEMS SINCE -ICROSOFT DECIDED TO FOLLOW over silo’d centralized, closed and Apple into the hardware business. proprietary ones. One example is 4HE CENTER OF DEVICE USE GRAVITY HAS XMPP (https://en.wikipedia.org/ also moved over to mobile, where wiki/XMPP), originally called Jabber. MOST OF US ARE USING GEAR RUNNING It was meant to bridge or replace JUST TWO OPERATING SYSTEM PLATFORMS all the competing proprietary Apple’s iOS and Google’s Android. instant-messaging systems in use True, Android is based on Linux (a AT THAT TIME !/,S !)- AND )#1 victory we gladly claim), and the range -ICROSOFTS -3. 9AHOOS -ESSENGER OF DEVICES RUNNING !NDROID IS HUGE Apple’s whatever-it-was (now called (http://techcrunch.com/2014/08/21/ iMessage) and so on. opensignal-2014-android- $IDNT HAPPEN ,OOK UP h*ABBERv ecosystem-report), but Android is still today at LinuxJournal.com and 'OOGLES SHOW 4HE NETWORK FRONTIER you’ll get a lot OF RESULTS ALSO IS ALSO MOSTLY FENCED OFF BY MOBILE FROM BACK AROUND THE TURN OF THE carriers and device suppliers who Millennium, mostly written by me. believe, almost across the board, )N THOSE DAYS MANY OF US HAD FULL that captive customers and users are CONFIDENCE THAT *ABBER8-00 WOULD

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 101

LJ262-February2016.indd 101 1/21/16 5:27 PM EOF

Wikipedia currently lists 55 sites, services, protocols and programs that do chat. There is no underlying protocol for interoperation among them.

DO FOR INSTANT MESSAGING AKA CHAT 1ZONE A SOCIAL SITE WITH MESSAGING WHAT 3-400/0)-!0 DID FOR E MAIL built in). Some interoperate, but and HTTP/HTML and its successors MOST DONT SINCE THAT WOULD FREE DID FOR PUBLISHING AND ALL THE OTHER THEIR USERS FROM CAPTIVITY things one can do on the World Wikipedia currently lists 55 sites, Wide Web. We would have a nice services, protocols and programs FLAT DISTRIBUTED AND UNIVERSAL that do chat. There is no underlying standard that people could employ PROTOCOL FOR INTEROPERATION AMONG any way they wanted, including them. Nor do any makers seem on their own personal hardware especially interested. Google Talk AND SOFTWARE WITH COUNTLESS (https://developers.google.com/talk) interoperable systems and no natural interoperated with other XMPP BARRIERS TO MOVING DATA EASILY FROM systems, but then it was any one system to any other. replaced with Google Hangouts Didn’t happen. Yes, XMPP turned (http://windowspbx.blogspot. into a widely used standard and com/2013/05/hangouts-wont- either supplied or inspired lots hangout-with-other.html), OF MESSAGING SYSTEMS "UT TODAY WHICH LACKS SUPPORT FOR 8-00 instant messaging is almost entirely So, while XMPP.org says “it’s not silo’d, and it’s more popular THE END OF 8-00 FOR 'OOGLE 4ALKv than ever. Facebook (including (https://xmpp.org/2015/03/no-its-not- WhatsApp), Twitter, Google and the-end-of-xmpp-for-google-talk), LinkedIn have their own messaging IT HARDLY MATTERS IF 'OOGLE 4ALK IS SYSTEMS INSIDE THEIR OWN hSOCIALv ITSELF DEAD SILOS 3O DO !PPLE AND -ICROSOFT 4HE LAST NAIL IN 8-00S COFFIN (including Skype). Tencent Holdings, PERHAPS IS THIS FROM &ACEBOOK the giant Web portal company in (https://developers.facebook.com/ #HINA HAS 4ENCENT 11 7E#HAT AND docs/chat):

102 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 102 1/21/16 5:27 PM /N !PRIL   WE ANNOUNCED THE DEPRECATION OF THE 8-00 ADVERTISER #HAT !0) AS PART OF THE RELEASE OF 0LATFORM !0) V INDEX Thank you as always for supporting our !FTER !PRIL TH  APPS WILL advertisers by buying their products! no longer be able to access the service or API. This includes both ACCESS TO CHATFACEBOOKCOM AND ADVERTISER URL PAGE # THE XMPP?LOGIN PERMISSION

Drupalize.me http://drupalize.me 97 We recommend people access Great Wide Open http://www.greatwideopen.org/ 31 Facebook Messages on the (0# 7ALLSTREET HTTPWWWFLAGGMGMTCOMLINUX  desktop via Facebook.com or Messenger.com. /g2EILLY &LUENT HTTPCONFERENCESOREILLYCOM  FLUENTJAVASCRIPT HTML US

On the other hand (or on another Peer 1 Hosting http://go.peer1.com/linux 7 tentacle), the number and variety OF CHAT PLATFORMS AND APPS IS HUGE and growing. Apple’s App Store currently lists 171 chat apps, while 'OOGLES 0LAY 3TORE FOR !NDROID LISTS ATTENTION ADVERTISERS

 https://play.google.com/store/ The Linux Journal brand’s search?q=chat&hl=en). Nearly all following has grown to a OF THEM ARE PROPRIETARY AND MANY monthly readership nearly also depend on (or work inside) large one million strong. Encompassing the magazine, Web site, SILOD PROPRIETARY SERVICES ,ISTEN FOR newsletters and much more, complaints about all this and you’ll Linux Journal offers the hear crickets. ideal content environment are also silos. As more to help you reach your marketing objectives. AND MORE OF THE CONNECTED WORLD For more information, please visit comes to depend on Whatever as http://www.linuxjournal.com/advertising. A 3ERVICE hTHE CLOUDv AND OTHER remote and centralized services, the VERY IDEA OF DISTRIBUTED CAPACITIES

WWW.LINUXJOURNAL.COM / FEBRUARY 2016 / 103

LJ262-February2016.indd 103 1/21/16 5:27 PM EOF

So now the chat ecosystem is a forest of silos standing on free and open-source geology, but not built with the same rock.

on standalone open-source (or way to make the world work in the open-source-based) hardware long run. And I’ll always believe AND SOFTWARE THAT INDIVIDUALS THAT THE FLAT DISTRIBUTED WORLD BUILT OR ORGANIZATIONS FULLY CONTROL AS ON FREE AND OPEN STUFF IS THE MOST independent and sovereign entities SUPPORTIVE AND FERTILE BASE ON WHICH in the world, seems terribly retro, to build the best and broadest utopian, or both. RANGE OF GOODS AND SERVICES "UT )M "RIAN "EHLENDORF SAYS WISELY past believing that this will ever be that the ideal is “minimum viable OBVIOUS TO THE WORLD OF DEVELOPERS CENTRALIZATIONv ) THINK THAT SHOULD businesses and governing bodies. be a transcendent design principle. Muggles have always outnumbered But the centralized thing is too wizards and always will. OFTEN THE EASIEST TO MAKE OR TO RELY "EING OUT OF SIGHT AND MIND FOR on (as is the case with APIs and big MOST OF THE WORLD DOESNT MAKE US SILOD PLATFORMS  3O NOW THE CHAT WRONG )N FACT IT MIGHT JUST MAKE ECOSYSTEM IS A FOREST OF SILOS STANDING us right in ways most will never see, ON FREE AND OPEN SOURCE GEOLOGY BUT no matter how much they depend not built with the same rock. on what we do.Q 7HAT MAY BE HARDEST FOR old-skool types like me to admit is that Doc Searls is Senior Editor of Linux Journal. He is also a SILOS ARE NOW THE PREFERRED METHOD fellow with the Berkman Center for Internet and Society at FOR INVENTING AND PROMULGATING Harvard University and the Center for Information Technology technologies supporting uses and Society at UC Santa Barbara. that scale and invite incompatible competitors to do the same, while the marketplace as a whole doesn’t Send comments or feedback via HAVE A BIG PROBLEM WITH ANY OF IT http://www.linuxjournal.com/contact I’ll never believe silos are the best or to [email protected].

104 / FEBRUARY 2016 / WWW.LINUXJOURNAL.COM

LJ262-February2016.indd 104 1/21/16 5:27 PM