Linux Journal Ebook Series

™ HOW TO: Home Automation with Raspberry Pi

Since 1994: The Original Magazine of the Linux Community MAY 2015 | ISSUE 253 | www.linuxjournal.com COOL PROJECTS

BUILD A Vehicle Monitoring and Control System CREATE A Safe to Store Your Sensitive Data PLUS Understanding Linux Permissions

Send SMS Working Modify the WATCH: Notiﬁcations with Django Boot Menu— ISSUE OVERVIEW

to Your Models and Libreboot V Smart Watch Migrations on an X60

LJ253-May2015bu.indd 1 4/24/15 12:18 PM NEW! Linux Journal eBook Series

GEEK GUIDES FREE Download The DevOps Toolbox: NOW! Tools and Technologies for Scale and Reliability By Bill Childers Introducing The DevOps Toolbox: Tools and Technologies for Scale and Reliability by Linux Journal Virtual Editor Bill Childers.

When I was growing up, my father always said, “Work smarter, not harder.” Now that I’m an adult, I’ve found that to be a core concept in my career as a DevOps engineer and manager. In order to work smarter, you’ve got to have good tools and technology in your corner doing a lot of the repetitive work, so you and your team can handle any exceptions that occur. More important, your tools need to have the ability to evolve and grow over time according to the changing needs of your business and organization.

In this eBook, I discuss a few of the most important tools in the DevOps toolbox, the beneﬁts of using them and some examples of each tool. It’s important to not consider this a review of each tool, but rather a guide to foster thinking about what’s appropriate for your own organization’s needs.

Beyond Cron How to Know When You’ve Outgrown Cron Scheduling— and What to Do Next By Mike Diehl

If you’ve spent any time around UNIX, you’ve no doubt learned to use and appreciate cron, the ubiquitous job scheduler that comes with almost every version of UNIX that exists. Cron is simple and easy to use, and most important, it just works. It sure beats having to remember to run your backups by hand, for example.

But cron does have its limits. Today’s enterprises are larger, more interdependent, and more interconnected than ever before, and cron just hasn’t kept up. These days, virtual servers can spring into existence on demand. There are accounting jobs that have to run after billing jobs have completed, but before the backups run. And, there are enterprises that connect Web servers, databases, and ﬁle servers. These enterprises may be in one server room, or they may span several data centers.

http://linuxjournal.com/geekguides

LJ253-May2015bu.indd 2 4/24/15 12:18 PM LJ253-May2015bu.indd 3 4/24/15 12:18 PM MAY 2015 CONTENTS ISSUE 253 COOL PROJECTS FEATURES 54 Home Automation 68 Embed Linux in with Raspberry Pi Monitoring and Have a home with a life of its own. Control Systems Bharath Bhushan Lohray Linux touches the real world. Rick Brown

ON THE COVER /V^;V!/VTL(\[VTH[PVU^P[O9HZWILYY`7PW )\PSKH=LOPJSL4VUP[VYPUNHUK*VU[YVS:`Z[LTW *YLH[LH:HML[V:[VYL@V\Y:LUZP[P]L+H[HW :LUK:4:5V[PMPJH[PVUZ[V@V\Y:THY[>H[JOW >VYRPUN^P[O+QHUNV4VKLSZHUK4PNYH[PVUZW 4VKPM`[OL)VV[4LU\·3PIYLIVV[VUHU?W 7S\Z!

4 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 4 4/24/15 12:18 PM INDEPTH 80 Hacking a Safe with Bash How to script a safe to keep your sensitive data secure. Adam Kosmin COLUMNS 24 30 Reuven M. Lerner’s At the Forge Django Models and Migrations 38 Dave Taylor’s Work the Shell System Status as SMS Text Messages 44 Kyle Rankin’s Hack and / Libreboot on an X60, Part III: Modify the Boot Menu 90 Doc Searls’ EOF 28 An Easy Way to Pay for Journalism, Music and Everything Else We Like IN EVERY ISSUE 8 Current_Issue.tar.gz 10 Letters 16 UPFRONT 28 Editors’ Choice 50 New Products 93 Advertisers Index 54

LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., 2121 Sage Road, Ste. 395, Houston, TX 77056 USA. Subscription rate is $29.50/year. Subscriptions start with the next issue.

WWW.LINUXJOURNAL.COM / MAY 2015 / 5

LJ253-May2015bu.indd 5 4/24/15 12:18 PM Executive Editor Jill Franklin [email protected] Senior Editor Doc Searls [email protected] Associate Editor Shawn Powers [email protected] Art Director Garrick Antikajian [email protected] Products Editor James Gray [email protected] Editor Emeritus Don Marti [email protected] Technical Editor Michael Baxter [email protected] Senior Columnist Reuven Lerner [email protected] Security Editor Mick Bauer [email protected] Hack Editor Kyle Rankin lj@greenﬂy.net Virtual Editor Bill Childers [email protected]

Contributing Editors )BRAHIM (ADDAD s 2OBERT ,OVE s :ACK "ROWN s $AVE 0HILLIPS s -ARCO &IORETTI s ,UDOVIC -ARCOTTE 0AUL "ARRY s 0AUL -C+ENNEY s $AVE 4AYLOR s $IRK %LMENDORF s *USTIN 2YAN s !DAM -ONSEN

President Carlie Fairchild [email protected]

Publisher Mark Irgang [email protected]

Associate Publisher John Grogan [email protected]

Director of Digital Experience Katherine Druckman [email protected]

Accountant Candy Beauchamp [email protected]

Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA

Editorial Advisory Panel Nick Baronian Kalyana Krishna Chadalavada "RIAN #ONNER s +EIR $AVIS -ICHAEL %AGER s 6ICTOR 'REGORIO $AVID ! ,ANE s 3TEVE -ARQUEZ $AVE -C!LLISTER s 4HOMAS 1UINLAN #HRIS $ 3TARK s 0ATRICK 3WARTZ

Advertising E-MAIL: [email protected] URL: www.linuxjournal.com/advertising PHONE: +1 713-344-1956 ext. 2

Subscriptions E-MAIL: [email protected] URL: www.linuxjournal.com/subscribe MAIL: PO Box 980985, Houston, TX 77098 USA

LINUX is a registered trademark of Linus Torvalds.

LJ253-May2015bu.indd 6 4/24/15 12:18 PM LJ253-May2015bu.indd 7 4/24/15 12:18 PM Current_Issue.tar.gz

Robotic SHAWN POWERS Sharks with Laser Eyes

love the Cool Projects issue. with a new topic this month (you Don’t get me wrong, most might remember Dave was working I issues of Linux Journal are on a word search project in his last full of cool things to do, but this column). In this issue, he takes month, we do it just because of the on the topic of how to make your cool factor. As you can imagine, shell scripts send text messages. no Cool Projects issue is complete It’s a great way to get instant without a Raspberry Pi article, and notifications to users, which isn’t this one is particularly awesome. usually possible from inside a script. But let me start off with a bit I describe a couple cool programs about our columns. in this month’s upfront section, Reuven M. Lerner continues his starting with Budgie. If you like the series on Django, and this month, simplicity of the Chrome desktop he covers migrations and updating interface, but prefer a full-blown databases. If you’re a developer Linux system underneath, Budgie looking for a framework to start is perfect. I also talk about the with, or if you’re already using intricacies of the Linux permissions Django and want to learn more, system and even a few Bitcoin Reuven’s series is a great way to clients. It’s hard to beat the cool begin. Dave Taylor follows Reuven factor of Kyle Rankin’s column this month, however, as he continues

V VIDEO: his series on Libreboot. People Shawn Powers runs have been installing open source through the latest issue. on hard drives for years, but with

8 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 8 4/24/15 12:18 PM CURRENT_ISSUE.TAR.GZ

Kyle’s assistance, you will learn Adam Kosmin goes much further to install the open-source BIOS and describes his complete system replacement as well! for keeping data secure. Using Be sure to check out Bharath freely available tools and a handful Bhushan Lohray’s article for an of scripts and methods, he shows incredible home automation how to integrate secure encryption project. Starting from scratch into your daily routine. If you want with a Raspberry Pi, some relays to encrypt your data, but find it and some wiring, Bharath walks complex and frustrating, be sure to through the steps of using the read Adam’s article. GPIO pins to manage multiple The Cool Projects issue is a systems. Although it’s certainly favorite of mine year after year. Not possible to buy one of the many only is it a chance to start working embedded home automation on those ideas you’ve been putting kits available, starting from off for months, but it’s also a great scratch allows for some serious way to learn while playing. I learned customization and infinite more about how keyboards function programability. If you’ve been while making my MAME cabinet struggling to choose a brand of than ever before or since. As a kid home automation systems to try, who took apart everything I got my perhaps after reading this article, hands on, the Cool Projects issue THAT QUESTION WILL BECOME MOOT is an awesome way to learn how Rick Brown describes another to put a few things back together! awesome project, but this time it Whether you love projects or just integrates with existing systems. want some tech tips, product Specifically, he explains how he announcements and programming connected a Linux system to a lessons, this issue of Linux Journal vehicle to get real-time operation should provide lots of entertainment data. Rick also shows how to design and education.Q a display for the information, so that you’re not grepping log files Shawn Powers is the Associate Editor for Linux Journal. while driving! He’s also the Gadget Guy for LinuxJournal.com, and he has In past issues, you have learned an interesting collection of vintage Garfield coffee mugs. how to do basic encryption with Don’t let his silly hairdo fool you, he’s a pretty ordinary guy Linux tools in order to keep your and can be reached via e-mail at [email protected]. sensitive data safe. This month, Or, swing by the #linuxjournal IRC channel on Freenode.net.

WWW.LINUXJOURNAL.COM / MAY 2015 / 9

LJ253-May2015bu.indd 9 4/24/15 12:18 PM letters

Sometimes the oldies but goodies are a pretty good choice, methinks. —Thomas

Thomas, yes, GKrellM is awesome, but there’s not a readily downloadable version for OS X. (There is for Windows, however.) Honestly, the “Non-Linux FOSS” piece is one of the toughest for me to write, because most FOSS software that is available for Windows and/or OS X is also available for Linux. Finding non-Linux FOSS is...challenging!—Shawn Powers Non-Linux Foss: MenuMeters? Regarding Shawn Powers’ Non-Linux Runtime “Stuff Happens”—Linux, FOSS Upfront piece in the March Tell Me about It 2015 issue: MenuMeters? Hmm... I was pleased and interested to that’s pretty much the same thing as read about GNOME’s pending GKrellM (which has been around since enhancements to its desktop forever-ish). GKrellM not only displays notification component(s). With all the stuff on your working system so many things going on behind but also on any of your networked the scenes of a contemporary servers with little, if any, fooling workstation, it is important to have around. I’ve used it for years to keep effective (aka, “the right things”) an eye on the headless database and efficient (aka, “things done servers sitting in the closet mumbling well”) ways to tell the end user that to themselves. I’ve also installed it on something happened. One might FRIENDS 7INDOWS MACHINES FOR QUICK also hope that any such “telling” and dirty troubleshooting and cores might also offer clues about what at 100% (ain’t Windows wonderful, an end user might do as a corrective eh?). I don’t know for sure, but I’ll bet action or workaround for whichever there’s a version for Mac? “stuff happened”.

10 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 10 4/24/15 12:18 PM [ LETTERS ]

That’s nice—where is the admin and Scale13x end-user dispatch table that says, I was listening to Kyle Rankin’s IN PART hWHEN %6%.4X HAPPENS “Tails” talk at Scale13x. Sitting do ACTIONx”, etc.? Why isn’t this in the third row, I heard a call for dispatch table as clean and obvious assistance on my two-way radio as /etc/anacrontab? IN ANOTHER ROOM ) WAS AN !6 volunteer. I wanted to get up and “Another tool suite” called systemd sneak into the back of the room is causing a lot of chatter trying to do to exit so I didn’t disturb the great mostly the same thing—“do ACTION presentation. Wow! When I turned WHEN %6%.4 HAPPENSv around, the room was jam-packed with people sitting all over the aisle Q FancyPrinter just dropped off-line. floors. I had to walk right in front Show a dialog box to announce of the room to exit. I hope Kyle that fact. didn’t think I was bored with his presentation. I did have a shirt on Q eSATA or USB drive connected. THAT HAD h!6 6OLUNTEERv WRITTEN “Do a little dance ...” make the on it in big letters. I came back content available. after a few minutes and Kyle had finished, and there was a crowd of Decades ago, I worked on TOPS-10/ PEOPLE ASKING HIM QUESTIONS ) HOPE TOPS-20 mainframes. There was a he made it out of there okay. Most utility suite that 1) handled all of the Linux folks are friendly though, and notifications; 2) routed notifications we did have a sense of community to end users, operators or others at the conference. who were on the lists to care about —Roman those notices; and 3) provided a per-user portal for trying to recover, Quizzes correct or resolve whatever was I thought it would be interesting to going on. I’m all for innovation, but SHARE THESE QUIZZES WITH YOU AND isn’t it time we re-solved (sic) this your readers. At http://dcjtech.info/ feature set and move on? quizzes, I made some interactive —Dan QUIZZES ABOUT ,INUX THAT PEOPLE

WWW.LINUXJOURNAL.COM / MAY 2015 / 11

LJ253-May2015bu.indd 11 4/24/15 12:18 PM [ LETTERS ]

can take for fun, practice, school or Initializing and Managing interviews. My Web site is ad-free, so Services in Linux it should not annoy visitors. Thank you for the very informative —Devyn Collier Johnson and timely “Managing Services in Linux: Past, Present and Future” Cool, thanks for the link, Devyn! article by Jonas Gorauskas in the —Shawn Powers March 2015 issue of LJ. The following is a URL of a comparison chart of Linux Newbie Request different init systems (yes, it’s Gentoo- I really appreciate Shawn Powers’ centric): http://wiki.gentoo.org/wiki/ enthusiasm and his approach Comparison_of_init_systems. to teaching Linux in the CBT —Richard Nuggets videos he’s done for the LPIC 1 certification. Individual Contribution to Open-Source Project Currently, I’m in a software support Have you ever chronicled the role in LDN with descent bash experience of a software developer knowledge, and my next step contributing to an open-source would be to move on to a system project for the first time? Some of administrator career. Can you the highlights could be finding a recommend any Web sites with project, finding something to work LPI practice exams? on in the project, submitting work, responding to rejected work and first The Internet is packed with Web accepted work. Then as the developer SITES BUT FORUM REVIEWS ARE QUITE becomes experienced, contrast concerning, as many are scams, getting involved in more projects, outdated, crippled with errors or so as to describe how valuable the have no support. lessons learned were going forward. —Patrick —Jon Redinger

I’ve found the folks at the LPI to be That might be an interesting story. great to work with. I’d recommend If you’re offering, submit a pitch to contacting them directly (even [email protected]. See via Twitter) for advice on practice http://linuxjournal.com/author for exams.—Shawn Powers more information.—Shawn Powers

12 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 12 4/24/15 12:18 PM [ LETTERS ]

systemd Question for Dave Taylor I always hated the init solution—System I was trying to put together a simple 6 )NIT IS DEFINITELY SOMETHING OF AN script to recurse through a directory tree: old technology. Therefore, I welcome systemd. It promises to auto-restart; #/bin/bash support “new” technologies like USB, Bluetooth, storage devices and so on; function recurse_dir() and to support future technologies as { well. It’s a modern approach to tackle for f in * ; do the current and future problems in #do stuff ; services. There always are initial start if [ -d "${f}" ] ; then problems with new technologies, pushd "${f}" ; but we shouldn’t hold on to past recurse_dir ; technologies and lose the technology popd ; advantage we have with GNU/Linux. fi ; done ; “Managing Services in Linux: Past, } Present and Future” by Jonas Gorauskas in the March 2015 issue pushd ~/dir ; was a good article! I advise Linus recurse_dir ; Torvalds to implement systemd as popd ; soon as possible in the mainstream kernel as the default service manager When running this script, I got errors and drop the older technologies in about invalid options. It turned out favor of systemd. You could add a that some of the subdirectories had compatibility modus with the older leading dashes (-) that were being one if it is necessary, but the way interpreted as options. I remembered forward is systemd. The fact that it’s that it is possible to add a dash, implemented in Debian and Red Hat or double dash, to turn off further is already proof for me that it is good option processing. Have you any technology. Maybe this is a good time other tips for dealing with difficult to change to a higher version—for filenames? I think it is possible to have example, 3.4 or 4.0—to indicate a QUOTES IN SOME AND THAT HAS CAUSED major improvement in the kernel? me problems in the past. —Patrick Op de Beeck —Jeremy

WWW.LINUXJOURNAL.COM / MAY 2015 / 13

LJ253-May2015bu.indd 13 4/24/15 12:18 PM [ LETTERS ]

Dave Taylor replies: That’s an interesting script you’re trying to build there, Jeremy. At Your Service I’m not clear why you’re using push/pop as you traverse the directories too. Why not just SUBSCRIPTIONS: Linux Journal is available in a variety of digital formats, including PDF, have cd ${f} followed by cd .. to get back .epub, .mobi and an on-line digital edition, as well as apps for iOS and Android devices. up a level and simplify things? Renewing your subscription, changing your e-mail address for issue delivery, paying your invoice, viewing your account details or other In terms of difficult filenames, yes, Linux subscription inquiries can be done instantly on-line: http://www.linuxjournal.com/subs. wasn’t really written to deal with filenames E-mail us at [email protected] or reach us via postal mail at Linux Journal, PO Box that start with a dash, have a space or other 980985, Houston, TX 77098 USA. Please remember to include your complete name punctuation, etc. The best you can do is and address when contacting us.

experiment to see if the commands you’re ACCESSING THE DIGITAL ARCHIVE: using accept -- as a way to delineate that Your monthly download notiﬁcations will have links to the various formats you’re done with command arguments, and to the digital archive. To access the digital archive at any time, log in at and quote the directory names themselves, http://www.linuxjournal.com/digital.

as you’ve done. LETTERS TO THE EDITOR: We welcome your letters and encourage you to submit them at http://www.linuxjournal.com/contact or mail them to Linux Journal, PO Box 980985, Good luck with this! Houston, TX 77098 USA. Letters may be edited for space and clarity.

WRITING FOR US: We always are looking for contributed articles, tutorials and real-world stories for the magazine. WRITE LJ A LETTER An author’s guide, a list of topics and due dates can be found on-line: We love hearing from our readers. Please http://www.linuxjournal.com/author. send us your comments and feedback via FREE e-NEWSLETTERS: Linux Journal http://www.linuxjournal.com/contact. editors publish newsletters on both a weekly and monthly basis. Receive late-breaking news, technical tips and tricks, an inside look at upcoming issues and links to in-depth stories featured on http://www.linuxjournal.com. Subscribe PHOTO OF THE MONTH for free today: http://www.linuxjournal.com/ Remember, send your Linux-related photos to enewsletters. [email protected]! ADVERTISING: Linux Journal is a great resource for readers and advertisers alike. Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising and marketing opportunities by visiting us on-line: http://ww.linuxjournal.com/ advertising. Contact us directly for further information: [email protected] or +1 713-344-1956 ext. 2.

14 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 14 4/24/15 12:18 PM LJ253-May2015bu.indd 15 4/24/15 12:18 PM UPFRONT NEWS + FUN diff -u WHAT’S NEW IN KERNEL DEVELOPMENT

/NE ONGOING QUESTION KERNEL were associated with which files, he developers face is the best way to reasoned, zeroing out all relevant data delete data so no one else can recover would be a trivial operation. it. Typically there are simple tools There were various objections. to undelete files that are deleted Alan Cox pointed out that hard accidentally, although some filesystems drives have become so smart these make this easier than others. days that it’s hard to know exactly Alexander Holler wanted to make what they’re doing in response to a it much harder for anyone to recover given command. As he put it, “If you deleted data. He didn’t necessarily want zero a sector [the disk is] perfectly to outwit the limitless resources of our entitled to set a bit in a master index governmental overlords, but he wanted of zeroed sectors, and not bother to make data recovery harder for the actually zeroing the data at all.” Alan average hostile attacker. The problem said that the disk simply had to accept as he saw it was that filesystems often user inputs and return the correct would not actually bother to delete outputs, and everything happening data, so much as they would just behind the curtain was entirely up to decouple the data from the file and the hardware manufacturer. make that part of the disk available for Russ Dill pointed out that a lot use by other files. But the data would of user programs also made it more still be there, at least for a while, for difficult to know exactly where a file’s anyone to recouple into a file again. data was on disk. The vim program, Alexander posted some patches for example, created temporary backup to implement a new system call that files, as did many other programs. first would overwrite all the data There was not much support for associated with a given file before Alexander’s patch. But I imagine the making that disk space available for ability to delete files permanently will use by other files. Since the filesystem come up again at some point. For knew which blocks on the disk kernel features though, the goal always

16 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 16 4/24/15 12:18 PM [ UPFRONT ]

tends to be doing a thorough job that, add a cool and useful feature limiting in this case at least, would indeed the number of open processes allowed outwit the government overlords’ in a given virtual machine. This would efforts to recover the data. prevent certain types of denial-of- There’s an ongoing debate about service attacks. The problem, as pointed cgroups, between the group of people out by Tejun Heo, was that an open who want to implement cool features process limit doesn’t correspond to and the group of people who want to any actual limit on a real system. And, ensure security. The security people there’s a strong reluctance to put limits always win, but the debate is rarely on anything that’s not a true resource, simple, and sometimes a cool feature like RAM, disk space, number of CPUs just needs to be rotated a little in order and so on. TO MATCH THE SECURITY REQUIREMENTS On the other hand, as Austin For example, Aleksa Sarai wanted to Hemmelgarn said, process IDs (PIDs)

LJ253-May2015bu.indd 17 4/24/15 12:18 PM [ UPFRONT ]

were an actual limit on a real system, and Tejun agreed it might make sense to allow them to They Said It be limited within a cgroup. And because that could be used to limit the number of open If you want to be processes, everyone could end up happy. But free, there is but the feature had to be presented as limiting an one way; it is to actual system resource, rather than limiting a guarantee an equally relatively arbitrary characteristic of a system. full measure of liberty to all your neighbors. The tracing system has been showing There is no other. signs of outgrowing its infrastructure lately, —Charles Schurz and Steven Rostedt recently posted some patches to fix that. Up until now, the tracing It’s not the things directory used DebugFS. But as Steven we do in life that said, tracing needed to create and remove we regret on our directories, and DebugFS didn’t support that. death bed, it is the So, tracing had been using various hacks to things we do not. get around it. Steven’s solution was to create —Randy Pausch a new filesystem called TraceFS, specifically for the tracing system. Getting fired is There were no major objections, but there nature’s way to were some technical obstacles to get past. telling you that you had the wrong job In particular, Steven discovered that the perf in the first place. system was hard-coded to assume that the —Hal Lancaster tracing system used DebugFS, so that had to be fixed before TraceFS could go into the kernel. Little by little, Other issues came up; for example, Greg one travels far. Kroah-Hartman suggested basing TraceFS —J.R.R. Tolkien on KernFS, and Steven considered that for a while. But it turned out that KernFS had a lot You can’t deny of cgroup-related complexity that TraceFS didn’t laughter. When it need, and Al Viro remarked, “It’s not a good comes, it plops down model for anything, other than an anti-hard- in your favorite chair drugs poster (’don’t shoot that shit, or you and stays as long as might end up hallucinating this’).” Ultimately, it wants. —Stephen King Steven decided against KernFS.—ZACK BROWN

18 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 18 4/24/15 12:18 PM LJ253-May2015bu.indd 19 4/24/15 12:18 PM [ UPFRONT ]

Android Candy: Every Hero Needs a Sidekick

I’ve touted the awesomeness of Calibre in the past (http://www.linuxjournal.com/ content/calibre-cloud). And although the Web- based calibre2opds still is an awesome way to access your eBook library, using a native Android app is even smoother. If you have your Calibre library on your local network, using Calibre Companion ($3.99 in the Google Play Store), your Android device connects to your library like a device connected via USB. It’s possible to load books directly onto your device without syncing your entire collection into the cloud! I admittedly still use calibre2opds in combination with Dropbox to make my library accessible remotely. But, if you’re concerned about your books being on the Web, (Image from the Google Play Store) Calibre Companion is a local network solution. Check it out today at https://play.google.com/store/ apps/details?id=com.multipie.calibreandroid. —SHAWN POWERS

20 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 20 4/24/15 12:18 PM Where every interaction matters.

break down your innovation barriers

power your business to its full potential When you’re presented with new opportunities, you want to focus on turning them into successes, not whether your IT solution can support them.

Peer 1 Hosting powers your business with our wholly owned FastFiber NetworkTM,

solutions that are secure, scalable, and customized for your business.

Unsurpassed performance and reliability help build your business foundation to be rock-solid, ready for high growth, and deliver the fast user experience your customers expect.

Want more on cloud? Call: 844.855.6655 | go.peer1.com/linux | Vew Cloud Webinar:

Public and Private Cloud | Managed Hosting | Dedicated Hosting | Colocation

LJ253-May2015bu.indd 21 4/24/15 12:18 PM [ UPFRONT ]

Non-Linux FOSS: All the Bitcoin, None of the Bloat

I love Bitcoin. Ever since I first discovered it in 2010 and mined thousands of them, I’ve been hooked on the technology, the concept and even the software. (Sadly, I sold most of those thousands of Bitcoin when they were less than a dollar. I’m still kicking standard Bitcoin client does. Because myself.) One of the frustrations with it reads the Bitcoin network in real using Bitcoin, however, is that the time, it takes seconds to sync up blockchain has gotten so large. It as opposed to days. Sending and currently weighs in at a little less receiving Bitcoin with MultiBit is than 20GB, and it takes about a week fast, efficient and secure. Oh, and to download the first time you do it’s open-source, multiplatform and so. There are ways to jumpstart the under constant development! If you download with a bootstrap file, but want to run a local Bitcoin client, but still, it’s a huge undertaking to run the don’t want to download the entire standard Bitcoin client. blockchain, check out MultiBit at Enter MultiBit. http://multibit.org. (Also check out Although it doesn’t have the entire Electrum, a Python-based alternative blockchain, MultiBit does have all at http://electrum.org.) of the security and encryption the —SHAWN POWERS

22 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 22 4/24/15 12:18 PM [ UPFRONT ]

My Humble Little Game Collection

I currently have the flu. Not the hSORTA QUEASYv STOMACH FLU BUT THE full out Influenza with fever, aches and delirium-ridden nightmares. Bouts of crippling illness tend to be my only chance to play games. Thankfully, since I’m such a terrible gamer, being sick doesn’t really hurt my skills very much! Today I was playing Torchlight II from Runic Games. I realized it was just as simple for me to run this new, awesome game on Linux using Steam as it is for any Windows user. Thanks to the Humble Bundle sales, I have dozens and dozens of games that aren’t cheesy knockoffs, but are actual versions of real games. I think Steam plays a big part in getting more and more games released natively on your Linux system. If for Linux, but whatever the reason, it’s you’re a gamer, or just have the flu, a great time to be a gamer in an open- go download some games: source operating system! The screenshot here is just a section Q http://www.humblebundle.com of my past Humble Bundle game purchases. The Humble Bundle (which Q http://www.steampowered.com I’ve mentioned before) is a great way to get inexpensive games. When you Q http://www.runicgames.com add Steam, you have yet another way to play and buy games that work —SHAWN POWERS

WWW.LINUXJOURNAL.COM / MAY 2015 / 23

LJ253-May2015bu.indd 23 4/24/15 12:18 PM [ UPFRONT ]

It’s Easier to Ask Forgiveness...

...than to understand Linux current directory. POSIX permissions permissions! Honestly though, that’s allow you to control read, write and not really true. Linux permissions are execute abilities for the user, group simple and elegant, and once you and a third set of users, “other”. understand them, they’re easy to work Using those three aspects on three with. Octal notation gets a little funky, different sets of users allows for but even that makes sense once you some fairly complex “who can do understand why it exists. what” on a filesystem. Users and Groups: First I need Figure 1 shows an example of what to address that Linux does have a file listing looks like. I’ve separated ACL support for file and folder the different sections showing which permissions. It’s not how things fields are which. Note the first field work by default, however, and ACLs is usually either a “d” or a “-”; the were added to address the fine former appears on directories, and controls needed for some situations. the latter appears on regular files. Most Linux systems rely on standard For files, the permissions make POSIX permissions. That’s what I’m straightforward sense. If the “read” covering here. bit is turned on, it means that user Every file has an owner and a (user, group or other) can read the group membership. If you type ls contents of the file. If the “write” bit -l, you’ll see the owner and group is set, it can be written to, and if the on all the files and folders in your “execute” bit is set, the file can be

Figure 1. Example File Listing

24 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 24 4/24/15 12:18 PM [ UPFRONT ]

executed. Execute bits are usually set It’s also possible to leave “ugo” on binary files or on scripts you want off if you want to change user, to run directly. group and other at the same time. On folders, the permissions work So the following examples all do almost the same: read means the the same thing: ability to see what files are in the folder, and write is the ability to chmod ugo-rw file.txt save a file into the folder. Execute chmod -rw file.txt is a little less obvious. When the chmod u-rw,g-rw,o-rw file.txt “execute” bit is set, it means the user or group can change into the Although the “ugo” format is directory. So the cd command will fairly robust, it’s sometimes very work as long as you have execute complicated to craft the string of access to a folder. pluses and minuses to get the exact Changing Permissions: Although permissions string you want. That’s Linux systems usually come with where octal notation comes into play. fairly sane standards for what It’s a little more confusing, but far permissions files are created with, more convenient once understood. it’s often necessary to change them. Octal Notation: In Figure 2, you For that, you use the chmod tool. can see I’ve shown the numeric The format is to use: value of each permission bit. By simply adding the numbers, it’s chmod u+rw,g-r,o+rwx file.txt possible to create any possibility of permissions with three numbers. Granted, that’s a fairly complex Figure 3 shows a few examples of example, but let’s break it down. The how those can be figured out. letters before the + or - are u for user, g for group and o for other. Then you either add or take away (+ or -) whichever aspects you desire. It’s NOT REQUIRED TO SPECIFY ALL THREE USER definitions, and they can be lumped together like this: Figure 2. Numeric Value of Each chmod ugo+rw file.txt Permission Bit

WWW.LINUXJOURNAL.COM / MAY 2015 / 25

LJ253-May2015bu.indd 25 4/24/15 12:18 PM [ UPFRONT ]

Figure 3. Creating Permissions

So with octal notation, the functionally the same: following two chmod statements are chmod ug+rw,ug-x,o+r,o-wx file.txt chmod 662 file.txt

LINUX JOURNAL !LTHOUGH ITS NOT A REQUIREMENT TO USE on your octal notation for setting permissions, device it’s usually the way it’s done. Android I urge you to play with chmod a bit Download the until you get the results you expect app now on when using octal notation. Then, just the Google for fun, see if you can string together Play Store a bunch of pluses and minuses to get the same results! Linux permissions are simple, elegant and allow for some very flexible file- sharing options on a filesystem. We use them most often when installing Web applications, because the Web server runs as a particular user, and that user (or group) needs to have access to the files www.linuxjournal.com/android it’s attempting to serve.—SHAWN POWERS

26 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 26 4/24/15 12:18 PM LJ253-May2015bu.indd 27 4/24/15 12:18 PM [ EDITORS' CHOICE ]

™

EDITORS’ Chrome-Colored CHOICE Parakeets ★

I personally like Google’s Chrome like the lack of ability to install interface. It’s simple, fast, Chrome easily on generic hardware. elegant and did I mention fast? Thankfully, Budgie is here to help. Unfortunately, I don’t like how If you like the simplicity and speed locked down the Chrome OS is on a of the Chrome interface, but want a Chromebook, nor do I like its total full-blown system underneath that dependence on Google. I also don’t deceptively simple GUI, I urge you

28 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 28 4/24/15 12:18 PM to give Budgie a try. You either instead of Unity. You’ll find a very can download the Evolve-OS Chrome-like interface but on top (http://evolve-os.com), or just of a full-blown Linux system instead install the PPA into a standard of Chrome! The preferences are Ubuntu system. I simply typed: fairly simplistic, but the entire interface is designed to get out of sudo apt-add-repository ppa:evolve-os/ppa the way and let you work. Due to sudo apt-get update its blazing-fast speed and ease of sudo apt-get install budgie-desktop use, the Budgie Desktop is this month’s Editors’ Choice. Give it a Then log out, and when logging try today! in, choose the Budgie desktop —SHAWN POWERS

LINUX JOURNAL now available for the iPad and iPhone at the App Store.

linuxjournal.com/ios

For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or [email protected].

LJ253-May2015bu.indd 29 4/24/15 12:18 PM COLUMNS AT THE FORGE

REUVEN M. Django LERNER Models and Migrations Django’s migrations make it easy to define and update your database schema.

In my last two articles, I looked applications also use a database, at the Django Web application and in many cases, that means a framework, written in Python. Django’s relational database. Indeed, it’s a rare DOCUMENTATION DESCRIBES IT AS AN -46 Web application that doesn’t use a framework, in which the acronym database of some sort. stands for model, template and views. For many years, Web applications 7HEN A REQUEST COMES IN TO A $JANGO typically spoke directly with the application, the application’s URL DATABASE SENDING 31, VIA TEXT STRINGS patterns determine which view method Thus, you would say something like: will be invoked. The view method

can then, as I mentioned in previous s = "SELECT first_name, last_name FROM Users where id = 1" articles, directly return content to the user or send the contents of a template. 9OU THEN WOULD SEND THAT 31, The template typically contains not to the server via a database client ONLY (4-, BUT ALSO DIRECTIVES UNIQUE library and retrieve the results using to Django, which allow you to pass that library. Although this approach along variable values, execute loops and does allow you to harness the power display text conditionally. OF 31, DIRECTLY IT MEANS THAT YOUR You can create lots of interesting application code now contains text Web applications with just views strings with another language. This and templates. However, most Web MIX OF FOR EXAMPLE 0YTHON AND 31,

30 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 30 4/24/15 12:18 PM COLUMNS AT THE FORGE

can become difficult to maintain Appointment. Each of these models and work with. Besides, in Python, is defined in the models.py file inside you’re used to working with objects, your application. attributes and methods. Why can’t This is a good place to point you access the database that way? out that models are specific to a The answer, of course, is that you particular Django application. Each can’t, because relational databases Django project contains one or more EVENTUALLY DO NEED TO RECEIVE 31, applications, and it is assumed that in order to function correctly. Thus, you can and will reuse applications many programs use an ORM (object- within different projects. relational mapper), which translates In the Django project I have created method calls and object attributes for this article (“atfproject”), I have INTO 31, 4HERE IS A WELL ESTABLISHED a single application (“atfapp”). Thus, ORM in the Python world known as I can define my model classes in 31,!LCHEMY (OWEVER $JANGO HAS atfproject/atfapp/models.py. That file, opted to use its own ORM, with which by default, contains a single line: you define your database tables, as well as insert, update and retrieve from django.db import models information in those tables. So in this article, I cover how you Given the example of creating an create models in Django, how you can appointment calendar, let’s start by create and apply migrations based on defining your Appointment model: those model definitions, and how you

can interact with your models from from django.db import models

within a Django application.

class Appointment(models.Model):

Models starts_at = models.DateTimeField()

A “model” in the Django world is a ends_at = models.DateTimeField()

Python class that represents a table in meeting_with = models.TextField()

the database. If you are creating an notes = models.TextField()

appointment calendar, your database def _ _str_ _(self):

likely will have at least two different return "{} - {}: Meeting with {} ({})".format(self.starts_at,

tables: People and Appointments. self.ends_at,

To represent these in Django, you self.meeting_with,

create two Python classes: Person and self.notes)

WWW.LINUXJOURNAL.COM / MAY 2015 / 31

LJ253-May2015bu.indd 31 4/24/15 12:18 PM COLUMNS AT THE FORGE

Django provides a large number of field types that you can use in your models, matching (to a large degree) the column types available in most popular databases.

Notice that in Django models, IN 31, 4HESE FIELD DEFINITIONS NOT you define the columns as class only determine what type of column attributes, using a Python object is defined in the database, but also known as a descriptor. Descriptors the way in which Django’s admin allow you to work with attributes interface and forms allow users to (such as appointment.starts_at), but enter data. In addition to TextField, for methods to be fired in the back. you can have BooleanFields, EmailFields In the case of database models, (for e-mail addresses), FileFields Django uses the descriptors to (for uploading files) and even retrieve, save, update and delete GenericIPAddressField, among others. your data in the database. Beyond choosing a field type that’s The one actual instance method appropriate for your data, you also can in the above code is _ _str_ _, which pass one or more options that modify every Python object can use to define how the field behaves. For example, how it gets turned into a string. DateField and DateTimeField allow Django uses the _ _str_ _ method to you to pass an “auto_now” keyword present your models. argument. If passed and set to True, Django provides a large number Django automatically will set the field of field types that you can use in to the current time when a new record your models, matching (to a large is stored. This isn’t necessarily behavior degree) the column types available that you always will want, but it is in most popular databases. For NEEDED FREQUENTLY ENOUGH THAT $JANGO example, the above model uses two provides it. That’s true for the other DateTimeFields and two TextFields. As fields, as well—they provide options you can imagine, these are mapped to that you might not always need, but the DATETIME and TEXT columns that really can come in handy.

32 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 32 4/24/15 12:18 PM COLUMNS AT THE FORGE

Migrations be serving the general public. For So, now you have a model! How such cases, you’ll want something can you start to use it? Well, first more powerful, such as my favorite you somehow need to translate DATABASE 0OSTGRE31, .EVERTHELESS YOUR MODEL INTO 31, THAT YOUR for the purposes of this little database can use. This means, before EXPERIMENT HERE YOU CAN USE 31,ITE continuing any further, you need to One of the many advantages of tell Django what database you’re 31,ITE IS THAT IT USES ONE FILE FOR using. This is done in your project’s each database; if the file exists, configuration file; in my case, that 31,ITE READS THE DATA FROM THERE would be atfproject/atfproject/ And if the file doesn’t yet exist, it settings.py. That file defines a is created upon first use. Thus, by number of variables that are used USING 31,ITE YOURE ABLE TO AVOID throughout Django. One of them is any configuration. DATABASES, a dictionary that defines However, you still somehow need the databases used in your project. TO CONVERT YOUR 0YTHON CODE TO 31, (Yes, it is possible to use more than DEFINITIONS THAT 31,ITE CAN USE 4HIS one, although I’m not sure if that’s is done with “migrations”. normally such a good idea.) Now, if you’re coming from the By default, the definition of world of Ruby on Rails, you are DATABASES is: familiar with the idea of migrations— they describe the changes made to DATABASES = { the database, such that you easily can 'default': { move from an older version of the 'ENGINE': 'django.db.backends.sqlite3', database to a newer one. I remember 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), the days before migrations, and they } were significantly less enjoyable— } their invention really has made Web development easier. In other words, Django comes, out Migrations are latecomers to the OF THE BOX DEFINED TO USE 31,ITE world of Django. There long have 31,ITE IS A WONDERFUL DATABASE FOR been external libraries, such as South, most purposes, but it is woefully but migrations in Django itself are underpowered for a real, production- relatively new. Rails users might be ready database application that will surprised to find that in Django,

WWW.LINUXJOURNAL.COM / MAY 2015 / 33

LJ253-May2015bu.indd 33 4/24/15 12:18 PM COLUMNS AT THE FORGE

developers don’t create migrations 'django.contrib.messages', directly. Rather, you tell Django to 'django.contrib.staticfiles', examine your model definitions, to 'atfapp' compare those definitions with the ) current state of the database and then to generate an appropriate migration. The output of makemigrations on Given that I just created a model, my system looks like this: I go back into the project’s root directory, and I execute: Migrations for 'atfapp': 0001_initial.py: django-admin.py makemigrations - Create model Appointment

This command, which you execute In other words, Django now has in the project’s root directory, tells described the difference between Django to look at the “atfapp” the current state of the database (in application, to compare its models which “Appointment” doesn’t exist) with the database and then to and the final state, in which there will generate migrations. be an “Appointment” table. If you’re Now, if you encounter an error curious to see what this migration at this point (and I often do!), you looks like, you can always look in the should double-check to make sure atfapp/migrations directory, in which your application has been added you’ll see Python code. to the project. It’s not sufficient Didn’t I say that the migration to have your app in the Django will describe the needed database project’s directory. You also must UPDATES IN 31, 9ES BUT THE add it to INSTALLED_APPS, a tuple description originally is written in in the project’s settings.py. For Python. This allows you, at least example, in my case, the definition in theory, to migrate to a different looks like this: database server, if and when you want to do so. INSTALLED_APPS = ( Now that you have the migrations, 'django.contrib.admin', it’s time to apply them. In the project’s 'django.contrib.auth', root directory, I now write: 'django.contrib.contenttypes', 'django.contrib.sessions', django-admin.py migrate

34 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 34 4/24/15 12:18 PM COLUMNS AT THE FORGE

And then see: If and when you retrieve the latest changes from Git, you’ll get all of the

Operations to perform: migrations from your coworkers and

Apply all migrations: admin, contenttypes, auth, atfapp, sessions then can apply them to your app.

Running migrations:

Applying contenttypes.0001_initial... OK Migrating Further

Applying auth.0001_initial... OK Let’s say that you modify your model.

Applying admin.0001_initial... OK How do you create and apply a new

Applying atfapp.0001_initial... OK migration? The answer actually is

Applying sessions.0001_initial... OK fairly straightforward. Modify the model and ask Django to create an The above shows that the “atfapp” appropriate migration. Then you can initial migration was run. But where run the newly created migration. did all of these other migrations come So, let’s add a new field to the from? The answer is simple. Django’s Appointment model, “minutes”, to user model and other built-in models keep track of what happened during also are described using migrations the meeting. I add a single line to and, thus, are applied along with the model, such that the file now mine, if that hasn’t yet happened in looks like this: my Django project.

You might have noticed that each from django.db import models

migration is given a number. This

allows Django to keep track of the class Appointment(models.Model):

history of the migrations and also to starts_at = models.DateTimeField()

apply more than one, if necessary. You ends_at = models.DateTimeField()

can create a migration, then create a meeting_with = models.TextField()

new migration and then apply both notes = models.TextField()

of them together, if you want to keep minutes = models.TextField() # New line here!

the changes separate. def _ _str_ _(self):

Or, perhaps more practically, you return "{} - {}: Meeting with {} ({})".format(self.starts_at,

can work with other people on a self.ends_at,

project, each of whom is updating the self.meeting_with,

database. Each of them can create self.notes) their own migrations and commit them into the shared Git repository. Now I once again run

WWW.LINUXJOURNAL.COM / MAY 2015 / 35

LJ253-May2015bu.indd 35 4/24/15 12:18 PM COLUMNS AT THE FORGE

makemigrations, but this time, Sure enough, the new migration Django is comparing the current has been applied! definition of the model with the Of course, Django could have current state of the database. It seems guessed as to my intentions. However, like a no-brainer for Django to deal in this case and in most others, with, and it should be, except for one Django follows the Python rule of thing: Django defines columns, by thumb in that it’s better to be explicit default, to forbid NULL values. If I add than implicit and to avoid guessing. the “minutes” column, which doesn’t allow NULL values, I’ll be in trouble Conclusion for existing rows. Django thus asks me Django’s models allow you to create whether I want to choose a default a variety of different fields in a value to put in this field or if I’d prefer database-independent way. Moreover, to stop the migration before it begins Django creates migrations between and to adjust my definitions. different versions of your database, One of the things I love about making it easy to iterate database migrations is that they help you definitions as a project moves avoid stupid mistakes like this one. forward, even if there are multiple I’m going to choose the first option, developers working on it. indicating that “whatever” is the In my next article, I plan to look (oh-so-helpful) default value. Once I at how you can use models that have done that, Django finishes with you have defined from within your the migration’s definition and writes it Django application.Q to disk. Now I can, once again, apply the pending migrations: Reuven M. Lerner is a Web developer, consultant and trainer. He recently completed his PhD in Learning Sciences from django-admin.py migrate Northwestern University. You can read his blog, Twitter feed and newsletter at http://lerner.co.il. Reuven lives with his wife And I see: and three children in Modi’in, Israel.

Operations to perform:

Apply all migrations: admin, contenttypes, auth, atfapp, sessions

Running migrations: Send comments or feedback via

Applying atfapp.0002_appointment_minutes... OK http://www.linuxjournal.com/contact or to [email protected].

36 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 36 4/24/15 12:18 PM How to succeed in a post-screen world: Find out at Solid.

The Internet of Things is changing everything. The O’Reilly Solid Conference is coming to San Not long ago, if you wanted to work with machines, Francisco’s waterfront June 23!25. It’s a unique event: you needed specialized knowledge of things like a mash-up of MIT and Disneyland for the IoT—deep, electrical engineering or assembly language. But with intelligent conversations about the vital issues like tools like node.js for embedded systems or Spark.io, security, business models, data, and standards; along programming physical objects has become as easy as with demos of some of the coolest devices, drones, programming a website. robots, and wearables that exist—or are imagined—today.

Solidcon.com Save 20% on your ticket @oreillysolid Use code LINUXJ

“The future has a funny way of sneaking up on you. You don’t notice it until you’re soaking in it. That was the feeling at O’Reilly’s Solid Conference.” –Wired JUNE 23 ! 25, 2015 SAN FRANCISO, CA

LJ253-May2015bu.indd 37 4/24/15 12:18 PM COLUMNS WORK THE SHELL

System Status DAVE TAYLOR as SMS Text Messages Loadwatch—intrepid shell script columnist Dave Taylor admits that it’s not the tool for all jobs and shows how to create a smart load alert script that’s perfect for sending SMS notifications to your new smart watch.

If you’re paying really close of being the shell script programming attention, you’ll remember that in columnist who is bailing on a script, my last article, I was exploring the I will point out that there’s a lot to rudiments of a script that would learn from this experience actually. accept a list of words as input and Most specifically, although it’s nice to create a word search grid, suitable imagine that the Linux environment for printing. It turns out that’s crazy is completely egalitarian, and that hard to do as a shell script—it just every script, every language and doesn’t have the muscle to implement every program is as powerful and any sort of functional algorithm in an well designed as every other, it’s clear elegant fashion. So, I’m going to bail that’s not the case. on it, at least until I can find someone Take Perl versus Awk, for example. else’s open-source code I can explore !WK IS POWERFUL AND ) USE IT FREQUENTLY for inspiration. but although there are major software Or, of course, if you’re motivated programs written in Perl, you’d be and have some time to experiment, go hard-pressed to find any significant back to my April 2015 column, read software, functions, applications or through it, then try your own hand at utilities programmed directly in Awk. implementing something. The same goes for C++ versus PHP, for Before I get letters about the oddity example, or any modern structured

38 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 38 4/24/15 12:18 PM COLUMNS WORK THE SHELL

language versus, well, the Bourne access to the power and capabilities of Again Shell. There, I said it. Shell script the entire Linux environment and, by programming can take you only so far, extension, the entire Internet. and then you realize that you’ve hit What’s your take? You read this the edges of the environment and its column, so it’s reasonable for me to capabilities, and it’s time to jump to conclude that you are interested in another language. learning more about programming Indeed, when I wrote my popular within the Linux shell environment. How book Wicked Cool Shell Scripts, there often do you bump into the bleeding was a tiny C program that snuck in by edge of your shell and realize you have necessity: it was a few lines of C to do to flip into Perl, Ruby, C, Cobol (just a certain date calculation that would kidding!) or another more sophisticated have been dozens, if not hundreds, of development environment to solve the lines of shell script. problem properly? Having said that, I will rush back to defend the shell as a powerful, Let’s Talk about Text Messages lightweight programming and I was watching the Apple prototyping environment perfect for a introduction of its new Apple Watch variety of tasks because of its super-easy and was struck by the fact that

Table 1. E-mail Addresses of SMS Gateways for Cellular Carriers

WIRELESS CARRIER DOMAIN NAME

At&T @txt.att.net

Cricket @mms.mycricket.com

Nextel @messaging.nextel.com

Qwest @qwestmp.com

Sprint @messaging.sprintpcs.com

T-Mobile @tmomail.net

US Cellular @email.uscc.net

Verizon @vtext.com

Virgin @vmobl.com

WWW.LINUXJOURNAL.COM / MAY 2015 / 39

LJ253-May2015bu.indd 39 4/24/15 12:18 PM COLUMNS WORK THE SHELL

like a few of the high-end Android The last three figures are the load smart watches, it will show you average over the last 1, 5 and 15 the entirety of e-mail and text minutes. In this case, the system barely messages on the tiny watch screen. is being tapped at all. But what if it This means it’s a great device for jumped up to 10, or 35 or more than sysadmins and Linux IT folk to keep 100? Then everything would slow tabs on the status of their machine down. Here’s how you could write a or set of machines. simple script to test for that condition: Sure, you could do this by having the system send an e-mail, but let’s go #!/bin/sh a bit further and tap into one of the # loadwatch.sh - send an alert if uptime > MAXOK e-mail-to-SMS gateways instead. Table MAXOK=10 1 shows a list of gateway addresses loadavg=$(uptime | cut -d\ -f11 | cut -d. -f1) for the most common cellular carriers if [ $loadavg -gt $MAXOK ] ; then in the United States. echo "Alert: Load avg $(uptime | cut -d\ -f11)" For example, I can send a text fi message to someone on the exit 0 AT&T network with the number (303) 555-1234 by formatting the Armed with the information about e-mail like this: the various SMS gateways, it’s easy to hard code a recipient address, [email protected] which changes just the echo line within the conditional: Armed with this information,

there are a lot of different statuses mail -s "Alert: Load avg $(uptime|cut -d\ -f11)" $recipient that you can monitor and get a succinct text message if something’s where earlier in the script “recipient” messed up. is formatted similar to: Worried about load averages becoming excessive? That’s a figure [email protected] easily accessible through the one-line output of uptime: or as appropriate for your own smart watch or, um, other device.

$ uptime For this script to be useful, you’d

11:20 up 4 days, 22:44, 3 users, load averages: 1.08 1.40 1.46 likely want to run it every few minutes

40 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 40 4/24/15 12:18 PM COLUMNS WORK THE SHELL

so that when there is a spike in usage, useful to know at this point is you’re alerted as soon as possible. This lockfile(1). This manages the atomic most easily would be a cron job, and creation of the semaphore so that if you haven’t explored how your own you never hit what’s called a “race custom cron jobs can make your life condition” where two instantiations as even the most rudimentary of Linux of the script might collide on who is users better, well, you’re missing out! creating the file. To make the script run every ten Here’s how it’ll work with the minutes, here’s how it might look in the addition of the semaphore: root or even just your user crontab file:

statefile=/home/taylor/bin/.loadavg

0,10,20,30,40,50 * * * * /home/taylor/bin/loadwatch.sh if [ -f "$statefile" ] ; then

# statefile already exists, we're in a high load situation

Modern crontabs have a more if [ $loadavg -gt $MAXOK ] ; then

sophisticated notational language that # still in high load situation

can make this a wee bit more succinct: echo "nothing to do, still in high load situation"

else

*/10 * * * * /home/taylor/bin/loadwatch.sh # high load situation has ended

/bin/rm -f $statefile

For this really to be useful, it might be mail -s "Alert: load average back to normal" $recipient \

better to have the script monitor state < /dev/null > /dev/null 2>&1

changes, so it’d notify you when the fi

load rose above a specified threshold else

but not notify you again until it then # statefile doesn't exist, let's create it.

went back down below that threshold. if [ $loadavg -gt $MAXOK ] ; then

This is done with what we old- # load average has jumped above OK level

school programmers call a semaphore, lockfile $statefile

a state variable that remembers what’s load=$(uptime | cut -d\ -f11)

happening. Because a shell script is mail -s "Alert: load average is $load" $recipient \

transient in nature, the semaphore needs < /dev/null > /dev/null 2>&1

to be a file. Typically these are located in else

a protected directory of some sort, but # load average was okay and still is.

let’s just drop it in your home directory echo "nothing to do, load average still ok."

for the purposes of this demo script. fi

The command-line function that’s fi

WWW.LINUXJOURNAL.COM / MAY 2015 / 41

LJ253-May2015bu.indd 41 4/24/15 12:18 PM COLUMNS WORK THE SHELL

Of course, there are two of the resultant “null message body” warning four possible scenarios where you’d message. That’s what this does: really want to remove the debugging code, clean up the if-then-else chain < /dev/null > /dev/null 2>&1 and shorten the script, because if this is going to run every ten minutes, Otherwise, hopefully you can read you most assuredly do not want “no through and see what it does. change” messages generated! With that in mind, here’s the more What Else Could You Monitor? succinct code block: Tracking load average is rather trivial when you think about all the many

if [ -f "$statefile" ] ; then things that can go wrong on a Linux

# statefile already exists, we're in a high load situation system, including processes that get

if [ $loadavg -le $MAXOK ] ; then wedged and use an inordinate amount

# high load situation has ended of CPU time, disk space that could be

/bin/rm -f $statefile close to filling up, RAM that’s tapped

mail -s "Alert: load average back to normal?" $recipient \ out and causing excessive swapping,

< /dev/null > /dev/null 2>&1 or even unauthorized users logging in.

fi All of those situations can be

else analyzed, and alerts can be sent to you

# statefile doesn't exist, let's create it. via e-mail or SMS text message, even to

if [ $loadavg -gt $MAXOK ] ; then your shiny gold $17,000 Apple Watch.

# load average has jumped above OK level Now, you tell me, what do you think is

lockfile $statefile worth monitoring on your system?Q

load=$(uptime | cut -d\ -f11)

mail -s "Alert: load average is $load?" $recipient \ Dave Taylor has been hacking shell scripts for more than 30

< /dev/null > /dev/null 2>&1 years—really. He’s the author of the popular Wicked Cool Shell

fi Scripts (10th anniversary update coming very soon from O’Reilly

fi and NoStarch Press). He can be found on Twitter as @DaveTaylor and more generally at his tech site http://www.AskDaveTaylor.com. Note the extra work involved in using the command-line Mail program, where you have to redirect input so Send comments or feedback via that it’s not waiting for a message http://www.linuxjournal.com/contact from stdin and redirecting the or to [email protected].

42 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 42 4/24/15 12:18 PM LINUX JOURNAL on your Android device

Download the app now on the Google Play Store

www.linuxjournal.com/android For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or [email protected].

LJ253-May2015bu.indd 43 4/24/15 12:18 PM COLUMNS HACK AND /

Libreboot KYLE RANKIN on an X60, Part III: Modify the Boot Menu Your BIOS is freer than ever, but now what? It’s time to change that boot menu.

In the first two articles of this the default order. With Libreboot, all series, I explained the fundamentals of the device settings are set inside behind the Libreboot free software the ROM itself, and you use a GRUB BIOS project and why you might menu to select a boot device. want to replace your BIOS with it. The existing GRUB menu provides a I followed up by describing how to number of common boot options that install Libreboot on a ThinkPad X60. In hopefully should work on your system. this final article of the series, I explain The default menu item attempts how to perform one major task that to boot off the first partition, and so far I’ve left unexplained: how to after that, there are options to modify the default GRUB boot menu. boot removable devices and finally A traditional BIOS provides users an option to search for and load with a configuration menu where any local GRUB configuration that they can change boot orders and might be on a hard drive. Ideally this enable and disable devices. Typically default menu would be sufficient, there is an assigned key on the but there are some cases (such as keyboard (Esc, F11 and F12 are booting the Tails USB disk) that common), so that you can select your MIGHT REQUIRE SOME TWEAKS boot device instead of going with On the one hand, if you are

44 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 44 4/24/15 12:18 PM COLUMNS HACK AND /

familiar with GRUB commands, binaries, ROMs and supporting you can boot more less any device scripts. You already should have you want on the fly with the right installed the Libreboot build incantation. On the other hand, dependencies when setting up it can be a pain to type in GRUB Libreboot, but if not, first run either commands every time you want THE DEPS TRISQUEL OR DEPS PARABOLA to boot something, so if you find script if you are on a Debian- yourself tweaking the default menu based or Arch-based distribution, items to boot a special device, you respectively. If you are using probably will want to modify the another distribution, inspect the GRUB menu more permanently. packages those scripts install and What I suggest is that you map them to the package names for experiment with sample GRUB your distribution. configuration changes directly from The Libreboot ROM actually contains the GRUB boot menu, because it a small filesystem called CBFS, so to edit allows you to edit the configuration it, you need to install the cbfstool binary. of any menu item directly. This way, Within the libreboot_bin directory is YOU QUICKLY CAN TEST ANY SAMPLE a script called builddeps-cbfstool, so changes without having to go run that script, and you should see a through the full process of writing cbfstool and rmodtool binary appear to and flashing a new ROM. Once under libreboot_bin: you know what changes you’d like to make, you are ready to move on to $ ./builddeps-cbfstool make them permanent. Modify the ROM The Setup Once cbfstool is installed, the next If you have followed the previous step is to choose the ROM to modify two articles in this series, you so you can view the files within it already should have downloaded and extract a copy of the GRUB and validated the Libreboot binary configuration. For this example, and installed Libreboot on an X60. I’m going to use one of the ROMs Let’s pick up from that point by provided by Libreboot itself. First, run opening a terminal and changing cbfstool along with the path to the to the libreboot_bin directory ROM and the print argument. The that contains all of the Libreboot print argument will then list all of

WWW.LINUXJOURNAL.COM / MAY 2015 / 45

LJ253-May2015bu.indd 45 4/24/15 12:18 PM COLUMNS HACK AND /

the files within the ROM: first, update the ROM and flash your BIOS with it, then select the option

$ ./cbfstool bin/x60/libreboot_usqwerty_vesafb.rom print in the GRUB menu to load grubtest.cfg.

libreboot_usqwerty_vesafb.rom: 2048 kB, bootblocksize 1424, Then you can validate that your

´romsize 2097152, offset 0x0 config works before you copy the

alignment: 64 bytes, architecture: x86 same change to grub.cfg.

With that in mind, start by

Name Offset Type Size extracting the grubtest.cfg file

cmos_layout.bin 0x0 cmos_layout 1788 using cbfstool:

cmos.default 0x740 cmos_default 256

fallback/romstage 0x880 stage 50924 $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb.rom extract

fallback/ramstage 0xcfc0 stage 81695 ´-n grubtest.cfg -f grubtest.cfg

fallback/payload 0x20f40 payload 541644

etc/ps2-keyboard-spinup 0xa5380 raw 8 Here, instead of print, you are

config 0xa53c0 raw 4504 passing the extract command

background.jpg 0xa6580 raw 67907 to cbfstool along with two new

dejavusansmono.pf2 0xb6f00 raw 100513 arguments. The -n option specifies

grub.cfg 0xcf800 raw 1637 the name of the file within the

grubtest.cfg 0xcfec0 raw 1629 CBFS filesystem to extract, and the

(empty) 0xd0580 null 1242264 -f option specifies what to name the copy of the file on the local As you can see, there are two filesystem. Since the grub.cfg file different GRUB config files: grub.cfg references this specific filename, it’s and grubtest.cfg. The former is the best to keep it the same. default GRUB config that is loaded, The grubtest.cfg will contain a and the second can be loaded by number of GRUB settings at the top the first for testing new configs. of the file, but the more interesting The fact is, if you make some major settings will be found down in the mistake in your GRUB config, you menuentry sections: potentially could lock yourself out

of booting your system (or at menuentry 'Load Operating System' {

least make it very difficult), so it’s set root='ahci0,msdos1'

important to validate your changes linux /vmlinuz root=/dev/sda1

in a safe way. The recommended initrd /initrd.img

workflow is to modify grubtest.cfg }

46 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 46 4/24/15 12:18 PM COLUMNS HACK AND /

menuentry 'Parse ISOLINUX menu (USB)' { your changes. This is important,

set root='usb0' because the default Libreboot ROMs

syslinux_configfile -i (usb0)/isolinux/isolinux.cfg are created with particular sections

} of the ROM blank to work well

menuentry 'Parse ISOLINUX menu (CD)' { with initial flashing. I’ve personally

set root='ata0' bricked an X60 by attempting the

syslinux_configfile -i (ata0)/isolinux/isolinux.cfg first flash with one of my custom

} ROMs, so it’s worth keeping the original ROMs intact: For instance, the above three sections are for menu items to boot $ cp bin/x60/libreboot_usqwerty_vesafb.rom a Linux kernel from the first disk, a bin/x60/libreboot_usqwerty_vesafb-custom.rom USB disk and a CD, respectively. If you find, for example, that your root Now remove the old grubtest.cfg partition isn’t /dev/sda1 but instead from your custom ROM and use the /dev/sda2, you would edit the first print command to confirm that it menuentry section to reflect that. no longer exists: In my case, I noticed that the Tails

live USB disks created prior to version $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom remove

REQUIRED A SPECIAL SET OF BOOT ´-n grubtest.cfg

options. After some experimentation, $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom print

I came up with the following addition libreboot_usqwerty_vesafb-custom.rom: 2048 kB,

for GRUB: ´bootblocksize 1424, romsize 2097152, offset 0x0

alignment: 64 bytes, architecture: x86

menuentry 'Tails (USB)' {

set root='usb0,gpt1' Name Offset Type Size

syslinux_configfile -i (usb0,gpt1)/syslinux/live486.cfg cmos_layout.bin 0x0 cmos_layout 1788

} cmos.default 0x740 cmos_default 256

fallback/romstage 0x880 stage 50924

A Quick Warning fallback/ramstage 0xcfc0 stage 81695

Once you have made changes, it’s fallback/payload 0x20f40 payload 541644

time to copy the modified grubtest.cfg etc/ps2-keyboard-spinup 0xa5380 raw 8

to your ROM. If you are using one config 0xa53c0 raw 4504

of the standard Libreboot ROMs, I background.jpg 0xa6580 raw 67907

recommend first making a copy for dejavusansmono.pf2 0xb6f00 raw 100513

WWW.LINUXJOURNAL.COM / MAY 2015 / 47

LJ253-May2015bu.indd 47 4/24/15 12:18 PM COLUMNS HACK AND /

grub.cfg 0xcf800 raw 1637 system you used to install Libreboot

(empty) 0xcfec0 deleted 1688 in the first place, you already should

(empty) 0xd0580 null 1242264 have flashrom built and available. Otherwise, if you are running this Now you are ready to add your from a system like Tails, or if you custom version and use the print haven’t yet installed flashrom, first run command to confirm it exists: the builddeps-flashrom script from the base of the libreboot_bin directory

$ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom as root. When you are ready to flash

´add -n grubtest.cfg -f grubtest.cfg -t raw your BIOS, make sure you are in the

$ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom print libreboot_bin directory and run:

libreboot_usqwerty_vesafb-custom.rom: 2048 kB, bootblocksize 1424,

´romsize 2097152, offset 0x0 $ sudo ./flashrom/flashrom -p internal -w

alignment: 64 bytes, architecture: x86 bin/x60/libreboot_usqwerty_vesafb-custom.rom

flashrom v0.9.7-unknown on Linux 3.16.0-4-586 (i686)

Name Offset Type Size flashrom is free software, get the source code at

cmos_layout.bin 0x0 cmos_layout 1788 ´http://www.flashrom.org

cmos.default 0x740 cmos_default 256

fallback/romstage 0x880 stage 50924 Calibrating delay loop... delay loop is unreliable, trying

fallback/ramstage 0xcfc0 stage 81695 ´to continue OK.

fallback/payload 0x20f40 payload 541644 coreboot table found at 0xcf6bd000.

etc/ps2-keyboard-spinup 0xa5380 raw 8 Found chipset "Intel ICH7M". Enabling flash write... OK.

config 0xa53c0 raw 4504 Found Macronix flash chip "MX25L1605D/MX25L1608D/MX25L1673E"

background.jpg 0xa6580 raw 67907 ´(2048 kB, SPI) mapped at physical address 0xffe00000.

dejavusansmono.pf2 0xb6f00 raw 100513 Reading old flash chip contents... done.

grub.cfg 0xcf800 raw 1637 Erasing and writing flash chip...

grubtest.cfg 0xcfec0 raw 1629 Erase/write done.

(empty) 0xd0580 null 1242264 Of course, replace the above ROM Flash the BIOS with the full path to your custom Now you can flash your BIOS with ROM. Once the flash succeeds, reboot the modified ROM. You can use the your machine and at the boot menu, flashrom utility that’s included inside select the menu item that switches your Libreboot binary directory. If you to your custom grubtest.cfg. You you are running this from the same then should see whatever changes

48 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 48 4/24/15 12:18 PM COLUMNS HACK AND /

you made, and you can attempt to Now you can repeat the steps boot from them. If everything works you performed to delete and re-add as expected, you are ready to make it grubtest.cfg from the ROM, only this the default. If not, especially if your time with grub.cfg: changes made GRUB not work at all,

just be glad it’s the test file. You’ve $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom

been given a second chance to iterate ´remove -n grub.cfg

through grubtest.cfg until it does $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom print

work, and then you can move on. $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom add

Warning: make sure before ´-n grub.cfg -f grub.cfg -t raw

you move on from here that $ ./cbfstool bin/x60/libreboot_usqwerty_vesafb-custom.rom print you have completely tested your grubtest.cfg changes and everything Confirm that grub.cfg has been added works as expected. properly to your ROM, and then flash your BIOS with the new custom ROM: Edit the Default Menu Boot back in to your system and go $ sudo ./flashrom/flashrom -p internal -w back to your working directory. Since bin/x60/libreboot_usqwerty_vesafb-custom.rom grubtest.cfg works, the next step is to create a copy of it named grub.cfg Once you reboot, you should be able that you will use as the default to use your new modified GRUB menu. GRUB config. The official Libreboot Just be sure to take the extra steps of documentation for the GRUB menu validating changes with grubtest.cfg first lists this following sed script that will each time you do this—you wouldn’t do all of the work of creating a grub.cfg want to get locked out of your system!Q based on your grubtest.cfg, but it will change the menu entries to make Kyle Rankin is a Sr. Systems Administrator in the San Francisco sure they still reference grubtest.cfg Bay Area and the author of a number of books, including The and grub.cfg where appropriate (be Official Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks. sure to run this in the directory that He is currently the president of the North Bay Linux Users’ Group. contains your custom grubtest.cfg):

$ sed -e 's:(cbfsdisk)/grub.cfg:(cbfsdisk)/grubtest.cfg:g' Send comments or feedback via

´-e 's:Switch to grub.cfg:Switch to grubtest.cfg:g' http://www.linuxjournal.com/contact

´< grubtest.cfg > grub.cfg or to [email protected].

WWW.LINUXJOURNAL.COM / MAY 2015 / 49

LJ253-May2015bu.indd 49 4/24/15 12:18 PM NEW PRODUCTS

Wolfram Research’s Wolfram Data Drop

It’s always enlightening to learn what inventor, scientist and MacArthur Genius Grant recipient Dr Stephen Wolfram is sharing with the world. The latest dispatch from his company, Wolfram Research, is the Wolfram Data Drop, a solution for handling data from the emergent Internet of Things. Dr Wolfram notes THE POWER OF THE 7OLFRAM ,ANGUAGE FOR INTERPRETING VISUALIZING ANALYZING QUERYING AND OTHERWISE DOING INTERESTING THINGS WITH THIS DATA 4HE QUESTION HOWEVER NOTES $R 7OLFRAM is this: how should the data from all those connected devices and everything else actually get to where good things can be done with it? The Wolfram Data Drop is Wolfram Research’s next-step contribution toward making the world computable. The Wolfram Data Drop not only gathers and stores information from connected devices and the Internet of Things, but once data is in, it also becomes both universally interpretable and universally accessible to the Wolfram Language and any system that uses it. Dr Wolfram adds that the Wolfram Data Drop will be of great value to organizations or individuals that create connected devices, enabling them to store their data in the Wolfram Cloud, or a private version of it, where it will be READILY ACCESSIBLE TO ANALYZE VISUALIZE QUERY OR DEPLOY http://www.wolfram.com

Jolla Ltd.’s Sailfish Secure

Mobile developer Jolla shows how our tribe shines brightest when options are few. Chafing at the vanilla-chocolate, iOS-Android dominance, the team at Jolla has released the secure—and first “truly open”— mobile phone platform called Sailfish Secure. Based on Sailfish OS and utilizing SSH Communications Security’s SSH communication encryption and key management platform, Sailfish Secure is a secure, adaptable mobile phone solution for government officials, corporations and consumers. With Sailfish Secure, Jolla and partner SSH seek to satiate the increased demand for privacy in mobile communications as well heed the call for a secure, transparent and open mobile solution alternative that is not controlled by any country or major industry player. Sailfish Secure also enables significant adaptation to local needs and hardware configurations. Jolla and SSH welcome other industry players to join the initiative. http://jolla.com

50 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 50 4/24/15 12:18 PM NEW PRODUCTS

Ryft Systems Inc.’s Ryft ONE

To illustrate the power of its new Ryft ONE analytics platform, Ryft Systems claims that a single device, using less power than a hair dryer, can store and ANALYZE THE EQUIVALENT OF THE CONTENTS OF 7IKIPEDIA IN 4.5 seconds. Ryft ONE, says its producer, is the first commercially available 1U analytics platform capable of an unprecedented 10+ GB/second performance without any data indexing, preprocessing, tuning or partitioning. The company further shared that the Ryft ONE, powered by a new massively parallel, hardware-accelerated architecture, analyzes historical and streaming data together at speeds 200X faster than conventional hardware, enabling data scientists and business analysts to slash operational costs by 70%. The new architecture, called Ryft Analytics Cortex, is a platform built for real-time analysis, optimizing compute, storage and I/O in tandem. The Ryft ONE, adds Ryft, “is open and compact like a Linux server but executes like a high-performance computer”. http://ryft.com

EarthLCD.com’s Pi-Raq

%ARTH,#$S NEW 0I 2AQ AN open-source Raspberry-Pi- based 1U rackmount Internet appliance, was inspired by the potential of an earlier company innovation, the “world’s first” 10" x 1" (25.4cm x 2.5cm) TFT LCD. That display is integrated directly INTO THE 0I 2AQ 4HIS WAS POSSIBLE THANKS TO %ARTH,#$ COMPLETING A COMPREHENSIVE open-source reference design for the new product, which includes packaging, software and firmware to build a 1U rackmount appliance. EarthLCD notes the importance of Raspberry Pi for allowing it to get the TFT LCD into customers’ hands QUICKLY AND ALLOWING THEM TO DESIGN HIGH VALUE )NTERNET APPLIANCES RAPIDLY MERELY BY adding their software- and application-specific I/O via USB or I2C interfaces. Running $EBIAN ,INUX OPENS UP THE 0I 2AQ STANDARD NETWORK ANALYZER SOFTWARE ALLOWING MUSIC SERVERS AND NUMEROUS OPEN SOURCE APPLICATIONS TO BE PORTED TO THE 0I 2AQ http://earthlcd.com

WWW.LINUXJOURNAL.COM / MAY 2015 / 51

LJ253-May2015bu.indd 51 4/24/15 12:18 PM NEW PRODUCTS

Red Hat’s Certified Container Ecosystem Program

Red Hat’s newly announced Certified Container Ecosystem Program is really a three- “product” solution set for delivering secure, reliable and verified Docker-based CONTAINERS TO THE ENTERPRISE WORLD 4HE FIRST ELEMENT OF THE PROGRAM FOR )36 PARTNERS consists of access to the Red Hat Container Development Kit, a collection of tools and resources to create enterprise-ready containers. The second element is the toolset for delivering the Red Hat Container Certification, which verifies that a container’s contents are secure, unmodified, free of vulnerabilities and supported on Red Hat infrastructure. And finally, the third element is the distribution mechanism, the fully supported Red Hat Container Registry. This inaugural registry eventually will be part of a network of federated, standardized container registries hosted by partners and )36S 4HE NEW PRODUCT IS BASED ON 2ED (ATS OUTLOOK OF ,INUX CONTAINERS AS THE NEXT wave of enterprise application architecture. Containers facilitate the creation of an efficient, composable fabric of lightweight “microservices” that can be woven into more complex applications, yet still are flexible enough to adapt to changing IT needs. http://redhat.com

SUSE OpenStack Cloud

Formerly known as SUSE Cloud, the new SUSE OpenStack Cloud 5 is available—the latest edition of the company’s OpenStack distribution for building Infrastructure-as-a-Service private clouds. SUSE OpenStack Cloud 5 is based on the newest OpenStack Juno release and provides increased networking flexibility and improved operational efficiency to simplify private cloud infrastructure management. 6ERSION ALSO PROVIDES hAS A SERVICEv CAPABILITIES TO ENABLE DEVELOPMENT AND BIG data analytic teams to deliver business solutions that integrate with SUSE Enterprise Storage and SUSE Linux Enterprise Server 12 data-center solutions rapidly. Additional version 5 benefits include greater networking functionality and support for third-party OpenStack networking plugins, seamless incorporation of existing servers running outside the private cloud and centralized log collection and search. http://suse.com

52 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 52 4/24/15 12:18 PM NEW PRODUCTS

NVIDIA SHIELD

.6)$)! IS ESTABLISHING ITSELF FURTHER OUTSIDE THE CONFINES OF THE 0# with the newest member of the SHIELD family of gaming devices. .6)$)! 3()%,$ IS .6)$)!S FIRST LIVING ROOM ENTERTAINMENT DEVICE AND THE COMPANY CALLS IT hTHE WORLDS FIRST !NDROID 46 CONSOLE TO DELIVER MUSIC APPS CONSOLE QUALITY GAMES AND VIDEOv 3()%,$ IS BUILT ON !NDROID 46 AND POWERED BY .6)$)!S POWERFUL 4EGRA 8 PROCESSOR 4HE ARTISTICALLY DESIGNED 3()%,$ CONSOLE STREAMS HIGH QUALITY + VIDEO CONTENT AND ENABLES NATIVE AND STREAMING GAMEPLAY AT P RESOLUTIONFPS 4HE CONSOLE PROVIDES COMPLETE ACCESS TO !NDROID 46S RICH APP AND GAMES ecosystem, with more than 50 games optimized for the platform. Other games, such as Crysis 3, Doom 3: BFG Edition and Borderlands: The Pre-Sequel!, are under development. SHIELD ALSO IS THE GATEWAY TO THE .6)$)! '2)$ GAME STREAMING SERVICE WHICH THE COMPANY SELLS AS hTHE .ETFLIX FOR 'AMESv 3()%,$ COMES PRE PACKAGED WITH .6)$)!S CONSOLE GRADE CONTROLLER and optional accessories include a remote control, additional controllers and a vertical stand. http://shield.nvidia.com

Jeffrey Haemer’s Git Under the Hood, LiveLessons Video (Addison-Wesley Professional)

Since Linus Torvalds invented Git in 2005, it rapidly has become the standard distributed version control system in existence. If you need or want to stop stumbling around in Git and truly understand what you’re doing, a professionally developed training resource is available. Jeffrey Haemer’s Git Under the Hood contains more than six hours of video instruction covering four in-depth lessons to help developers gain a deeper understanding of Git so that they can use it more effectively. Intermediate to advanced developers comfortable using a terminal window/command line will learn the varied pieces of Git’s repository, basic Git commands and how to use them, design parallels between Git and Linux, how to experiment when Git does something unexpected and how to extend Git with new commands. http://informit.com

Please send information about releases of Linux-related products to [email protected] or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content.

WWW.LINUXJOURNAL.COM / MAY 2015 / 53

LJ253-May2015bu.indd 53 4/24/15 12:18 PM FEATURE Home Automation with Raspberry Pi

Home Automation with Raspberry Pi

How-to project: time the lights in your home and control them remotely over the Internet.

BHARATH BHUSHAN LOHRAY

54 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 54 4/24/15 12:18 PM he Raspberry Pi has been very for the Raspberry Pi 2 through its popular among hobbyists and IoT developer program for no charge T educators ever since its launch (https://dev.windows.com/en-us/ in 2011. The Raspberry Pi is a credit- featured/raspberrypi2support). This, card-sized single-board computer with in addition to its versatile features, a Broadcom BCM 2835 SoC, 256MB has caused fans like me to upgrade to 512MB of RAM, USB ports, GPIO to the Raspberry Pi 2. With a few pins, Ethernet, HDMI out, camera new Raspberry Pi 2 boards in hand, I header and an SD card slot. The most set out to find some useful ways to attractive aspects of the Raspberry Pi employ my older Pi boards. are its low cost of $35 and large user In this article, I briefly describe community following. THE REQUIREMENTS OF THE PROJECT The Pi has several pre-built that I outlined, and I explain the images for various applications various tools I decided to use to (http://www.raspberrypi.org/ build it. I then cover the hardware I downloads), such as the Debian-based chose and the way to assemble the Raspbian, XBMC-based (now known parts to realize the system. Next, I as Kodi) RASPBMC, OpenELEC-based continue setting up the development Plex Player, Ubuntu Core, RISC OS and environment on the Raspbian image, more. The NOOBS (New Out Of the Box and I walk through the code and Setup) image presents a user-friendly bring everything together to form the menu to select and install any of the complete system. Finally, I conclude SEVERAL DISTRIBUTIONS AND SUBSEQUENTLY with possible improvements and hacks boot into any of the installed OSes. The that would extend the usefulness of a Raspbian image comes with the Wolfram Pi home automation system. language as part of the setup. Since its initial launch in February The Internet of Things 2011, the Raspberry Pi has been An ongoing trend in embedded devices revised four times, each time is to have all embedded devices receiving upgrades but maintaining connected to the Internet. The Internet the steady price of $35. The newest was developed as a fail-safe network release of the Pi (the Raspberry Pi 2) that could survive the destruction of BOASTS A -(Z QUAD CORE CORTEX several nodes. The Internet of Things A7 and 1GB of RAM. Moreover, (IoT) leverages the same redundancy. Microsoft announced Windows 10 With the move to migrate to IPv6,

WWW.LINUXJOURNAL.COM / MAY 2015 / 55

LJ253-May2015bu.indd 55 4/24/15 12:18 PM FEATURE Home Automation with Raspberry Pi

A multitude of IoT-connected devices in a home has the potential to act as a living entity that exhibits response to stimuli.

the IP address space would be large UI. The UI that I had in mind would enough for several trillion devices to support multiple users logged in to stay connected. A connected device the same Pi server. The UI state had also makes it very convenient to to keep up with the actual state of control it from anywhere, receive the system in real time indicating inputs from various sensors and which lights actually were on when respond to events. A multitude of multiple users operated the system IoT-connected devices in a home has simultaneously. Apart from this, the the potential to act as a living entity lights may toggle on or off when that exhibits response to stimuli. triggered by the timer. A UI running on a device, such as a phone or a Raspberry Pi Home Automation tablet, may be subject to random Inspired by the idea of having a home connection drops. The UI is expected that has a life of its own, I settled to handle this and attempt to on a home automation project to reconnect to the Pi server. control the lights in my living room. The goal of my project was to be able Hardware to time the lights in my living room (AVING OUTLINED THE REQUIREMENTS ) and control them remotely over the began to build the hardware. Table Internet using a Web browser. I also 1 shows the bill of materials that I wanted to expose an API that could used to build the hardware part of the be used to control the device from system, and Figure 1 shows a block other devices programatically. diagram of the hardware system. The interesting part of this project Wiring this is time-consuming but is not the hardware, which is fairly easy. First, wire the SMPS to the wall simple and easy to construct, but the outlet by cutting off an extension cord

56 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 56 4/24/15 12:18 PM Table 1. Bill of Materials

COMPONENT QUANTITY APPROXIMATE PRICE PROCURED FROM FUNCTION

Raspberry Pi 1 $35 Newark The CPU

SD card 1 $25 amazon.com To boot the RPi

Edimax WiFi 1 $10 amazon.com (http://www.amazon.com/ To give the Edimax-EW-7811Un-150Mbps-Raspberry- RPi wireless Supports/dp/B003MTTJOY) connectivity

Relay 1 $10 amazon.com (http://www.amazon.com/ Used for module JBtek-Channel-Relay-Arduino-Raspberry/ switching dp/B00KTELP3I)

Ribbon cable 1 $7 amazon.com (http://www.amazon.com/ To connect Veewon-Flexible-Multicolored-Breadboard- the RPi header Jumper/dp/B00N7XWXRK) to the relay module

Power 1 $8 amazon.com (http://www.amazon.com/ To power the supply gp/product/B00HF3G7NO) RPi and the relay module

Extension 9 $54 Walmart (http://www.walmart.com/ip/ To power the cord Qvs-PC3PX-10-10ft-3-Outlet-3-Prong- SMPS and to Power-Cabl-Extension-Cord-Ac-Male- provide a plug To-Female/41440394) interface to the relays

Pencil box 1 $2 Walmart To house the entire setup

USB cable 1 $5 amazon.com (http://www.amazon.com/ To power AmazonBasics-USB-Cable-Micro-Meters/ the RPi dp/B003ES5ZSW)

14 gauge 1 6 Home Depot To wire the relay wire terminals to the live wire from the wall outlet

Cable clamp 1 $2 Home Depot As a strain relief

at the socket end. Strip the wires and of the USB cable and wiring it to the screw them into the screw terminals of wire ends of the SMPS and the micro B the SMPS. Next, wire the Raspberry Pi to end to the RPi. Strip out two strands of the SMPS by cutting off the type A end wires from the ribbon cable, and wire

WWW.LINUXJOURNAL.COM / MAY 2015 / 57

LJ253-May2015bu.indd 57 4/24/15 12:18 PM FEATURE Home Automation with Raspberry Pi

Figure 1. Block Diagram of the Hardware System

the appropriate terminals to GND and The RPi’s GPIO pins are shown in *$6CC 2EMOVE THE JUMPER THAT CONNECTS Figure 2. The RPi’s IO ports operate at THE *$6CC AND 6CC .OT REMOVING THIS 3.3v, and the relay module works at 5v. jumper will feed back 5v to the 3.3v pins However, the relays are isolated from of the Pi and damage it. the RPi’s GPIO pins using optocouplers. Now that all the terminals are wired The optocouplers may be supplied for power, connect the IN1-IN8 lines V OVER THE 6CC PIN 4HE 6CC PIN OF of the relay module to the appropriate the relay module may be supplied 3.3v GPIO pins of the RPi using more of from the GPIO header of the Pi. Make the ribbon cable as shown in Figure sure you have removed the jumper that 2. The code I present here is written bridges the Vcc and JDVcc on the relay for the case where I wire IN1-IN8 to module board. 4HE *$6CC PIN SHOULD GPIO1-GPIO7. Should you decide to be supplied 5v for proper operation of wire them differently, you will need to the relay. The relay module is designed modify your code accordingly. to be active low. This means that you

58 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 58 4/24/15 12:18 PM have to ground the terminals IN1-IN8 to switch on a relay. Warning: handle all wiring with caution. Getting a shock from the line can be fatal! Cut the remaining extension cables at the plug end, and screw in the wire end to the relay. Also daisy-chain the live wire from the wall outlet to the relay terminals. The entire setup can be housed in a pencil box or something similar. Plan this out in advance to avoid having to unwire and rewire the terminals. Additionally, I added a few screw cable clamps to the holes I made in my housing to act Figure 2. The RPi’s GPIO Pins as a strain relief (Figure 3).

Figure 3. The Hardware Setup

WWW.LINUXJOURNAL.COM / MAY 2015 / 59

LJ253-May2015bu.indd 59 4/24/15 12:19 PM FEATURE Home Automation with Raspberry Pi

Environment Next, you need to set up the I built my environment starting Wi-Fi connection. You can find with a fresh install of Raspbian. For detailed instructions for this at the initial installation, you need http://www.raspberrypi.org/ an HDMI-capable display, a USB documentation/configuration/ keyboard, mouse and a wired Ethernet wireless. I recommend the wicd-curses connection. You also optionally may option. At this point, you can make connect a Wi-Fi adapter. Build the changes to the RPi setup using the SD card for the first boot by sudo raspi-config command. following the instructions given This will bring up a GUI that lets at http://www.raspberrypi.org/ you choose options like the amount documentation/installation/ of RAM you share with the GPU, installing-image. During the first overclocking, GUI Boot and so on. boot, the installer sets up the OS Another useful tool is the Cloud and expands the image to fill the 9 IDE (https://github.com/c9/core). entire card. After the first boot, you The Cloud9 IDE allows you to edit should be able to log in using the your code on the RPi using a Web default credentials (user “pi” and browser. It also gives you a shell password “raspberry”). interface in the browser. You can Once you successfully log in, it’s develop and execute all your code good practice to update the OS. without leaving the Web browser. The The Raspbian image is based on #LOUD )$% REQUIRES A SPECIFIC VERSION Debian and uses the aptitude of NodeJS. Using the wrong version package manager. You also will WILL CAUSE FREQUENT CRASHES OF THE need python, pip and git. I also Cloud 9 server, resulting in constant recommend installing Webmin frustration. Instructions for installing to ease administration processes. NodeJS on the Raspberry Pi are Instructions for installing Webmin outlined at http://weworkweplay.com/ are at http://www.webmin.com/ play/raspberry-pi-nodejs. deb.html (follow the directions in the “Using the Webmin APT Software repository” section): I decided to build my front-end UI using HTML5, CSS3 and JavaScript. sudo apt-get update && sudo apt-get dist-upgrade The combination of these three sudo apt-get install python python-pip git git-core form a powerful tool for building

60 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 60 4/24/15 12:19 PM The back-end server on the Raspberry Pi needs to control the GPIO pins on the Raspberry Pi board. It also needs an HTTP interface to serve the UI and a WebSocket interface to pass command and status messages.

UIs. JavaScript provides easy Libraries communication APIs to servers. The back-end server on the There also are a lot of JavaScript Raspberry Pi needs to control LIBRARIES LIKE *1UERY "OOTSTRAP the GPIO pins on the Raspberry and so on from which to choose. Pi board. It also needs an HTTP HTML5 supports the WebSocket interface to serve the UI and API that allows the browser to a WebSocket interface to pass keep a connection alive and receive command and status messages. communication over this connection. Such a specific server did not exist This makes WebSocket useful for for off-the-shelf deployment, so implementing live and streaming I decided to write my own using apps, such as for games and chat Python. Python has prebuilt modules interfaces. CSS is useful for styling for the Raspberry Pi GPIO, HTTP the various HTML elements. When server and WebSockets. Since these used properly, it lets one build modules are specialized, minimum dynamic UIs by switching the styles CODING WAS REQUIRED ON MY PART on an element in response to events. However, these modules are &OR THIS PROJECT ) CHOSE *1UERY not a part of Python and need to handle events, Bootstrap CSS to be installed separately. First, (http://getbootstrap.com/css) to you need to be able to control lay out the buttons in a grid form the RPi’s GPIO pins. The easiest and pure JavaScript to handle way to do this from Python is WebSocket communications. by using the RPi.GPIO library

WWW.LINUXJOURNAL.COM / MAY 2015 / 61

LJ253-May2015bu.indd 61 4/24/15 12:19 PM FEATURE Home Automation with Raspberry Pi

from https://pypi.python.org/pypi/ for Python (http://www.cherrypy.org). RPi.GPIO. Install this module with: It is easily extendible to support WebSocket using the ws4py module sudo pip install RPi.GPIO (https://github.com/Lawouach/ WebSocket-for-Python). CherryPy and Using the RPi.GPIO module is very ws4py also can be installed using pip: simple. You can find examples of its usage at http://sourceforge.net/p/ pip install cherrypy raspberry-gpio-python/wiki/ pip install ws4py Examples. The first step in using the module is to import it into the code. Examples of using the CherryPy Next, you need to select the mode. framework and the ws4py plugin The mode can be either GPIO.BOARD can be found in the CherryPy docs or GPIO.BCM. The mode decides (https://cherrypy.readthedocs.org/ whether the pin number references en/latest) and the ws4py docs IN THE SUBSEQUENT COMMANDS WILL (http://ws4py.readthedocs.org/en/ be based on the BCM chip or the IO latest). A basic CherryPy server can pins on the board. This is followed be spawned using the code shown by setting pins as either input or in Listing 2. output. Now you can use the IO Slightly more advanced code would PINS AS REQUIRED &INALLY YOU NEED pass the quickstart method an to clean up to release the GPIO object with configuration. The partial pins. Listing 1 shows examples of code in Listing 3 illustrates this. This using the RPi.GPIO module. CODE SERVES REQUESTS TO JS FROM THE CherryPy is a Web framework module js folder. The js folder resides in the

Listing 1. Using the RPi.GPIO Module

import RPi.GPIO as GPIO # import module GPIO.setmode(GPIO.BOARD) # use board pin numbering GPIO.setup(0, GPIO.IN) # set ch0 as input GPIO.setup(1, GPIO.OUT) # set ch1 as output var1=GPIO.input(0) # read ch0 GPIO.output(1, GPIO.HIGH) # take ch1 to high state GPIO.cleanup() # release GPIO.

62 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 62 4/24/15 12:19 PM home directory of the server code. of explaining the major functional To add WebSocket support to the parts of the code; hence, it does CherryPy server, modify the code as not actually do anything. shown in Listing 4. The WebSocket On the client side, the HTML handler class needs to implement needs to implement a function to three methods: opened, closed connect to a WebSocket and handle and received_message. Listing incoming messages. Listing 5 shows 4 is a basic WebSocket server that simple HTML that would do that. has been kept small for the purpose This code uses the jQuery.ready()

Listing 2. Spawning a Basic CherryPy Server

# From the CherryPy Docs at # https://cherrypy.readthedocs.org/en/latest/tutorials.html

import cherrypy # import the cherrypy module

class HelloWorld(object): # @cherrypy.expose # Make the function available def index(self): # Create a function for each request return "Hello world!" # Returned value is sent to the browser

if _ _name_ _ == '_ _main_ _': cherrypy.quickstart(HelloWorld()) # start the CherryPy server # and pass the class handle # to handle request

Listing 3. Passing the quickstart Method

cherrypy.quickstart(HelloWorld(), '', config={ '/js': { # Configure how to serve requests for /js 'tools.staticdir.on': True, # Serve content statically # from a directory 'tools.staticdir.dir': 'js' # Directory with respect to # server home. } });

WWW.LINUXJOURNAL.COM / MAY 2015 / 63

LJ253-May2015bu.indd 63 4/24/15 12:19 PM FEATURE Home Automation with Raspberry Pi

event to start connecting to the onerror() and onmessage(). WebSocket server. The code in this To extend this example, add code Listing implements methods to handle to the onmessage() method to all events: onopen(), onclose(), handle messages.

Listing 4. Basic WebSocket Server

import cherrypy # Import CherryPy server module # Import plugin modules for CherryPy from ws4py.server.cherrypyserver import WebSocketPlugin, WebSocketTool from ws4py.websocket import WebSocket # Import modules for # the ws4py plugin. from ws4py.messaging import TextMessage

class ChatWebSocketHandler(WebSocket): def received_message(self, m): msg=m.data.decode("utf-8") print msg cherrypy.engine.publish('websocket-broadcast', ´"Broadcast Message: Received a message")

def closed(self, code, reason="A client left the room ´without a proper explanation."): cherrypy.engine.publish('websocket-broadcast', ´TextMessage(reason))

class Root(object): @cherrypy.expose def index(self): return "index"

@cherrypy.expose def ws(self): print "Handler created: %s" % repr(cherrypy.request.ws_handler)

if _ _name_ _ == '_ _main_ _': WebSocketPlugin(cherrypy.engine).subscribe() # initialize websocket # plugin cherrypy.tools.websocket = WebSocketTool() # cherrypy.config.update({'server.socket_host': '0.0.0.0', 'server.socket_port': 9003, 'tools.staticdir.root': '/home/pi'}) cherrypy.quickstart(Root(), '', config={ '/ws': { 'tools.websocket.on': True, 'tools.websocket.handler_cls': ChatWebSocketHandler } });

64 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 64 4/24/15 12:19 PM Listing 5. Connecting to WebSocket and Handling Incoming Messages

WWW.LINUXJOURNAL.COM / MAY 2015 / 65

LJ253-May2015bu.indd 65 4/24/15 12:19 PM FEATURE Home Automation with Raspberry Pi

Pi Home Automation { Now that you’ve seen the basics of "c":"", WebSockets, CherryPy and the HTML "r": front end, let’s get to the actual code. } You can get the code from the Git repository at https://bitbucket.org/ The update and updateLabels lordloh/pi-home-automation. You commands do not take a relay number. can clone this repository locally on Apart from relay.py and relayLabel.json, your RPi, and execute it out of the THE ONLY OTHER FILE REQUIRED IS INDEXHTML box using the command: The relay.py script reads this file and SERVES IT IN RESPONSE TO (440 REQUESTS

git clone https://bitbucket.org/lordloh/pi-home-automation.git The index.html file contains the HTML,

git fetch && git checkout LinuxJournal2015May CSS and JavaScript to render the UI.

cd pi-home-automation Once the system is up and running,

python relay.py you’ll want to access it from over the Internet. To do this, you need to set a The relayLabel.json file holds the permanent MAC address and reserved REQUIRED CONFIGURATION SUCH AS IP address for the Raspberry Pi on your labels for relays, times for lights to go on and off and so on. Listing Listing 6. Basic Schema of the Configuration 6 shows the basic schema of the { configuration. Repeat this pattern "relay1": { for each relay. The dow property is "times": [ { formed by using one bit for each day "start": [ , of the week starting from Monday for , the LSB to Sunday for the MSB. ], "end": [ Figure 4 shows the block diagram , of the system displaying the major , functional parts. Table 2 enumerates ], "dow": all the commands the client may send to the server and the action that } the server is expected to take. These ], "id": 1, commands are sent from the browser "label": "" } to the server in JSON format. The } command schema is as follows:

66 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 66 4/24/15 12:19 PM Figure 4. Block Diagram of the System

local network, and set up port forwarding be extended to add new features, such as on your router. The process for doing detecting your phone connected to your this varies according to router, and your Wi-Fi and switching on lights. You also router manual is the best reference for could integrate this with applications, it. Additionally, you can use a dynamic such as OnX and Android Tasker. domain name service so that you do not Adding password protection for out-of- need to type your IP address to access network access is beneficial. Feel free your Pi every time. Some routers include to mention any issues, bugs and feature support for certain dynamic DNS services. REQUESTS AT http://code.lohray.com/ pi-home-automation/issues.Q Conclusion I hope this article helps you to build this Bharath Bhushan Lohray is a PhD student working on or other similar projects. This project can his dissertation on image compression techniques at the Department of Electrical & Computer Engineering, Texas Tech Table 2. Commands University. He is interested in machine learning.

COMMAND DESCRIPTION on Switch a relay on off Switch a relay off Send comments or feedback via update Send status of GPIO pins and relay labels http://www.linuxjournal.com/contact updateLabels Save new labels to JSON files or to [email protected].

WWW.LINUXJOURNAL.COM / MAY 2015 / 67

LJ253-May2015bu.indd 67 4/24/15 12:19 PM FEATURE Embed Linux in Monitoring and Control Systems

EMBED LINUX IN MONITORING AND CONTROL SYSTEMS

How to use a standard Linux distribution platform to make a user interface embedded into a vehicle monitoring and control system.

RICK BROWN

68 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 68 4/24/15 12:19 PM he target vehicle for this connects up-network (toward the project is a vintage intercity host) and the other down-network T transport bus (think Greyhound) (toward the other HIPs). whose instrument panel was sparse The way this application works is and mostly nonfunctional. The that a message is originated by the speedometer cable was twisted host processor and transmitted down- off some place back in 40 feet of loop to the first HIP. There it may be cable, and the fuel sensor had sunk modified under HIP program control long ago. What I wanted was an and relayed on down-loop to the instrument panel more in line with next HIP. The last HIP in the “loop” modern practice. transmits its message up-loop under To bridge the gap, I used a laptop physical jumper control. Processors computer running the Fedora 20-KDE closer to the Host simply pass on distribution of Linux as a host, three up what is coming from below in digital signal processor boards as the “loop”. The Host is the ultimate hardware interface processors (HIPs), receiver of the messages it originates. a USB/RS422 converter that connects A message consists of an SOM byte, to an RS422 loop linking the HIPs an address byte with acknowledge together and some software that bit, a command byte, four data ) CALL THE 6EHICLE -ONITORING AND bytes and two CRC bytes. Going Control application. down loop, the HIPs relay a message on a character-by-character basis System Architecture with a one-character delay per HIP. The HIPs are based on a signal The addressee of a message sets processor chip, programmed in C the acknowledge bit and inserts or and with no user interface except extracts data on the fly. So in a short a heartbeat LED to show that the loop like the one here, the host begins processor is working to some degree. receiving the response from the The HIPs provide signal conditioning network before it has finished sending circuitry for analog input scaling and the original message. For this loop, optical isolation for control signals the communication rate was selected, plus a few specials like thermocouple arbitrarily, as 57,600 baud, so the converters and a pressure transducer loop message time is (9 + 3)/57600 or chip. There also are two RS422 208 microseconds. The left portion of receiver/transmitter pairs. One pair Figure 1 depicts the loop topology.

WWW.LINUXJOURNAL.COM / MAY 2015 / 69

LJ253-May2015bu.indd 69 4/24/15 12:19 PM FEATURE Embed Linux in Monitoring and Control Systems

Figure 1. System Architecture

4HE 6EHICLE -ONITORING AND #ONTROL AND FOR A TOOLKIT IT IS 1T FROM THE 1T 6-# APPLICATION WILL ORIGINATE Project. The first step is to get the messages like “HIP1 set or get register development environment and then whatever”. First, I show how to set up build a “Hello World” application. So, a development environment on a Linux here it is in the Fedora world: box, and then I talk about how to use the tools made available there to weave yum install kdevelop together a Linux real-time application yum install qt THAT PERFORMS THE 6-# APPLICATION yum install gcc

Set Up the Development Environment ...and lots of other stuff. Expect My choice for a development to spend some time if you are not environment is KDevelop from KDE, ALREADY UP AND RUNNING WITH 1T

70 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 70 4/24/15 12:19 PM Qt is an excellent toolkit that is robust and well documented—except for a few quirks.

When you get KDevelop to load, reside. As you add source files and click SessionAStart New SessionANew libraries, you must keep ~/projects/ 0ROJECT 4HIS WILL BE h1Tv AND 6-##MAKE,ISTSTXT CURRENT h'RAPHICALv -AKE UP A NAME 6-# for example), accept the defaults, Build the Application and soon you will be presented the Here is how to use tools available opportunity to “Build” and then in the Linux environment to create “Execute”. On “Execute”, a Launch THE 6-# APPLICATION &IRST UP IS Configurations dialog box will enable communications. To your application, you to select your project name, “Add the communication loop looks like a New”, and your project. A click or file stream created like this: two later, and you should see a basic “Hello World” window appear on your int hNet = open("/dev/ttyUSB0", O_RDWR); screen. This you may expand into your real-time application. or /dev/ttyUSBwhatever, depending upon 4HE h(ELLO 7ORLDv YOU SEE IS A 1T what else is going on in your system. APPLICATION 1T IS AN EXCELLENT TOOLKIT Now you can read() and write() that is robust and well documented— hNet and the USBCRS422 converter EXCEPT FOR A FEW QUIRKS 4HE NEW will connect you to the loop. Writing project build process creates a is no issue up to loop speed (in this case directory structure that includes 57600/9 = 6400 messages/second), ^PROJECTS6-#BUILD 4O MINIMIZE so that is your write (hNet,...) speed HARD TO DIAGNOSE 1T BUILD ERRORS KEEP limit. Reading is a different deal as all of your source and header files in read(hNet,...) is a blocking operation. ^PROJECTS6-# UNTIL YOU KNOW HOW A process that makes that call remains TO DO OTHERWISE 4HE ^PROJECTS6-# stuck there until some data arrives. Thus, build directory is the execute directory you want to make your read(hNet,...) for the purposes of KDevelop. It is calls from a process (thread) whose only here that run directory files should task is to catch characters as they come

WWW.LINUXJOURNAL.COM / MAY 2015 / 71

LJ253-May2015bu.indd 71 4/24/15 12:19 PM FEATURE Embed Linux in Monitoring and Control Systems

in and then make them available in a STATEMENTS IN THE 6-# CONSTRUCTOR ARE buffer to other processes as they need them—most briefly, in abbreviated code: COMthread *gCOMgo = new COMthread; gCOMgo->start();

//A thread to perform the read(hNet,...) function class COMthread : public Qthread The complement to that loop data { fetch is a character fetch routine Q_OBJECT //Notice use of the Qt tools running under some other process. protected: That routine, using its own mutexes, //Start point when myThread->start(); is called extracts data from the buffer sourced void run() by the thread above. { Now that you can send and while (1) receive data via the loop, let’s look { at how the application may interact pthread_mutex_t mutex1\ with the hardware. = PTHREAD_MUTEX_INITIALIZER; Figure 2 shows the Instrument Panel //Lock out other processes while working display as seen on a video display pthread_mutex_lock( &mutex1 ); mounted in the driver’s view. -manipulate shared variables here- The Tach and Speed display data are //unlock for other processes during read(hNet,... sourced from timer registers in an HIP pthread_mutex_unlock( &mutex1 ); that is timing the period between shaft rotations. The five indicators below are //This is where this thread spends sourced by A/D registers in various HIPs. read(hNet, Buf, BUF_SIZE);///////// These seven data items are collected //99.99 (+/-) percent of its time by sending seven nine-character data REQUEST MESSAGES TO THE LOOP AND //Now lock while updating for new data decoding the returned 63 characters pthread_mutex_lock( &mutex1 ); (7X9). Below that is a representation of a -buffer data and update pointers- partially populated map of a 4X4 keypad pthread_mutex_unlock( &mutex1 ); that is serviced by one of the HIPs. Each } of the represented keys on that map } ISSUES A QUERY FOR THE ()0 RESPONSIBLE FOR }; the physical keypad to see if its key was the last pressed. It gets back yes or no. To activate that code, your When you use KDevelop to create a

72 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 72 4/24/15 12:19 PM Figure 2. Instrument Panel Display

6-# PROJECT SOME FILES OF INTEREST TO VMC::VMC() : QMainWindow() you now were created. Look in directory { ^PROJECTS6-# AND THERE YOU WILL FIND //declare a central widget to host our screen: MAINCPP AND 6-#CPP &ILE MAINCPP QWidget* gCentralWidget = new QWidget(this); is fine as is. It simply declares and runs setCentralWidget(gCentralWidget); the application described by the code in //Set fonts, colors, geometry, etc 6-#CPP .ONE OF THE SAMPLE CODE - - - - IN 6-#CPP WITHIN THE CURLY BRACES IS //Declare an object to hold the screen features: useful for you, so let’s replace it with the ScreenC cScreenLayout = new ScreenC(); CONSTRUCTOR FOR THE 6-# APPLICATION !S //Lastly, breathe life into the application I mentioned previously, this application cHeartBeat = new QTimer (this); RELIES UPON 1T SO AN IMPORTANT RESOURCE connect(cHeartBeat, SIGNAL(timeout()), this,\ for you is http://qt-project.org/doc/ SLOT(slotPaintScreen())); qt-4.8/classes.html. cHeartBeat->setSingleShot(false); 9OUR 6-# CLASS WILL INHERIT FROM cHeartBeat->start(50); //milliseconds/20Hz 1MAIN7INDOW SO YOUR CONSTRUCTOR WILL } BE DEFINED IN 6-#CPP AS SHOWN HERE

WWW.LINUXJOURNAL.COM / MAY 2015 / 73

LJ253-May2015bu.indd 73 4/24/15 12:19 PM FEATURE Embed Linux in Monitoring and Control Systems

It is here in the ScreenItemC class code where you can fancy it up.

That is an abridged view of the Here you define the location and constructor, but the actual code isn’t size and name the object type of much longer. The connected routine each on-screen object. At update slotPaintScreen() will be activated time “update” is simply relayed to its on a 50 millisecond interval by the children like this: timer overflow. It too is brief: //Update screen features //Fetch loop characters gathered by COMthread for (i=0; iUpdate(); cScreenLayout->Update(); } //Redraw the screen update(); The ScreenItemC class constructor is responsible for the look of items (Again abridged because this is a on the screen. In this application, story about how to do it rather than a ScreenItemC item consists of two how to code it.) 1,ABEL OBJECTS PLACED ONE ABOVE THE The central portion of Figure 1 other so as to appear to be a single shows the cascade of object creation INSTRUMENT 4HE FORM OF A 1,ABEL THAT WILL EMBODY THE 6-# APPLICATION declaration is: Notice the declaration of a ScreenC

OBJECT BY THE 6-# CONSTRUCTOR AND THE QLabel cReading = new QLabel(gCentralWidget, Qt::FramelessWindowHint); update of that object at a 20Hz rate. The ScreenC class constructor The instrument displays of Figure simply declares a ScreenItemC object 2 are pretty “plain Jane”. It is here for each entity that appears on the in the ScreenItemC class code where screen. A typical declaration is: you can fancy it up. The ScreenItemC constructor also declares a MeasureC pF[i] = new ScreenItemC(xOff,yOff,xSiz,ySiz,\ object. That object’s update routine MEAS_TACHOMETER, 0, "Tach"); returns the data that the ScreenItemC

74 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 74 4/24/15 12:19 PM object places on the screen: object here. The ControlC object will use its own SensorC object to MeasureC cMeasure = new INQUIRE OF THE LOOP IF ITS KEY IS THE MeasureC(MEAS_TACHOMETER); most recently pressed. ControlC objects also run device-specific code The MeasureC class is where the (like timing a blinker, for example). hardware interface is described. HIP The ControlC object may place address, register numbers and scale commands on the loop as necessary. factors are defined. For example: The ControlC update routine will return 1 or 0 depending on whether case MEAS_TACHOMETER: its control target has changed state { or not. That return flows back up fScale = 27648000.0; //29.75Hz -> 1788rpm the cascade to its grandparent // RPM = fScale / binary from loop + fOffset ScreenItemC object and then is fOffset = 0; reflected on the display. rule = MEAS_RULE_RECIPROCAL_TACHO; This cascade of object creation ends DeviceId = NODE_E; //Loop device id with SensorC objects that return the DevicePort = P_IC_PERIOD_2; //Sensor on device RESULT OF THEIR PREVIOUS REQUEST TO THE //Create a sensor for the measurement LOOP AND ISSUE A NEW DATA REQUEST pSens = new SensorC(MEAS_TACHOMETER,\ at each update time. As ControlC DeviceId, DevicePort, fScale, Offset, rule); objects may place commands on the break; loop at their whim, the loop will have } a mixture of independent commands circulating that must be resolved back Notice the declaration above of to their originator. When a command a SensorC object. At update time, is issued to the loop, the issuer of that SensorC object will fetch its that command also inserts into a class most recent raw reading from the visible circular buffer a pointer to itself. loop buffer, scale that and return As mentioned above, slotPaintScreen() the result to its MeasureC parent, will call SensorLoopService() at each which will relay that back to its update time. SensorLoopService() ScreenItemC parent, which will extracts characters that have been display that result on the screen. placed into the loop receive buffer by The MeasureC items that represent a the gCOMgo thread. Mutexes are used keypad key will declare a ContrtolC here to prevent interference by other

WWW.LINUXJOURNAL.COM / MAY 2015 / 75

LJ253-May2015bu.indd 75 4/24/15 12:19 PM FEATURE Embed Linux in Monitoring and Control Systems

Some kinks that Linux throws in include the screen saver that defaults active, but is bad news in a monitoring application.

threads. SensorLoopService() parses complete valid message, it places the the characters as it fetches data from four data bytes into a location pointed the buffer, and when it has detected a to by the pointer mentioned above. This data will be returned up the cascade at the next update time. Here it is in fewer words: the update event cascades down from LINUX JOURNAL the ScreenC object to multiple ARCHIVE DVD SensorC objects that bounce parameter states back up to ScreenItemC objects that paint those states on the screen. The left panel of Figure 1 depicts this.

Linux Environment Considerations Some kinks that Linux throws in include the screen saver that defaults active, but is bad news in a monitoring application. To turn it off, go to System SettingsAPower Management and disable all Screen Energy Saving options. Another issue NOW AVAILABLE is automatic software updates. It is my consideration that if something works, www.linuxjournal.com/dvd don’t screw around with the operating

76 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 76 4/24/15 12:19 PM environment, as software updates do. Serendipitously, this will not The safest way to suppress updating is bring up multiple instances of the by staying off the Internet while your application if it was active when you application is active. Another way is last powered down, and you have to disable updates by software control. your system set to restore the previous To do so, go to the Application Launcher session at power up. (lower left on the desktop), start the 7ITH A MAN IN THE LOOP THE 6-# System Settings from Favorites, go to application is not time-critical at all Software Management and left-click and may take its share of CPU time the wrench icon at the upper-right whenever it is offered. There is a edge. Select Settings from its menu. lot of other stuff in a Linux system In the General Settings page, set the that also wants CPU time (look at Check for updates menu to Never, and ps -A). If your application is time- “Apply” that. Also, go to /etc/yum/ critical with predetermined response pluginconf.d/refresh-packagekit.conf times at close tolerances between and set enabled to 0 (disable update). events, this scheme will not work For me it was just too easy to switch for you. However, if you have a few off the Wi-Fi when I wanted a stable milliseconds here and there to spare, environment, so I can’t give you other Linux will host your monitoring advice here. and control applications with a To claim credit as being an reasonably small level of effort and “embedded” application, this system good reliability.Q should come up with the power—that is, without login or any other user Rick Brown is a US Navy veteran, holds a BSEE granted in INPUT REQUIRED TO MAKE IT GO 4O KILL 1970 by the University of Florida, developed atmospheric the login, go to /etc/kde/kdm/kdmrc research instruments for many years as a faculty member and set AutoLoginAgain=true and of the University of Nevada System, consulted in the AutoLoginUser=YourUserName. To private sector as a developer of electronic instruments and bring up your application with system manufacturing test systems and now lives happily ever after start up, go to ~/.kde/Autostart and on his little spread north of Reno, Nevada. place an executable script there like this:

#!/bin/bash Send comments or feedback via cd /home/YourUserName/projects/VMC/build http://www.linuxjournal.com/contact ./VMC or to [email protected].

WWW.LINUXJOURNAL.COM / MAY 2015 / 77

LJ253-May2015bu.indd 77 4/24/15 12:19 PM KNOWLEDGE HUB

WEBCASTS Learn the 5 Critical Success Factors to Accelerate IT Service Delivery in a Cloud-Enabled Data Center Today's organizations face an unparalleled rate of change. Cloud-enabled data centers are increasingly seen as a way to accelerate IT service delivery and increase utilization of resources while reducing operating expenses. Building a cloud starts with virtualizing your IT environment, but an end-to-end cloud orchestration solution is key to optimizing the cloud to drive real productivity gains.

> http://lnxjr.nl/IBM5factors

Modernizing SAP Environments with Minimum Risk—a Path to Big Data Sponsor: SAP | Topic: Big Data Is the data explosion in today’s world a liability or a competitive advantage for your business? Exploiting massive amounts of data to make sound business decisions is a business imperative for success and a high priority for many firms. With rapid advances in x86 processing power and storage, enterprise application and database workloads are increasingly being moved from UNIX to Linux as part of IT modernization efforts. Modernizing application environments has numerous TCO and ROI benefits but the transformation needs to be managed carefully and performed with minimal downtime. Join this webinar to HEAR FROM TOP )$# ANALYST 2ICHARD 6ILLARS ABOUT THE PATH YOU CAN START TAKING NOW TO ENABLE YOUR ORGANIZATION TO GET THE benefits of turning data into actionable insights with exciting x86 technology.

> http://lnxjr.nl/modsap

WHITE PAPERS White Paper: JBoss Enterprise Application Platform for OpenShift Enterprise Sponsor: DLT Solutions Red Hat’s® JBoss Enterprise Application Platform for OpenShift Enterprise offering provides IT organizations with a simple and straightforward way to deploy and manage Java applications. This optional OpenShift Enterprise component further extends the developer and manageability benefits inherent in JBoss Enterprise Application Platform for on-premise cloud environments.

Unlike other multi-product offerings, this is not a bundling of two separate products. JBoss Enterprise Middleware has been hosted on the OpenShift public offering for more than 18 months. And many capabilities and features of JBoss Enterprise Application Platform 6 and JBoss Developer Studio 5 (which is also included in this offering) are based upon that experience.

This real-world understanding of how application servers operate and function in cloud environments is now available in this single on-premise offering, JBoss Enterprise Application Platform for OpenShift Enterprise, for enterprises looking for cloud benefits within their own datacenters.

> http://lnxjr.nl/jbossapp

78 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 78 4/24/15 12:19 PM KNOWLEDGE HUB

WHITE PAPERS Linux Management with Red Hat Satellite: Measuring Business Impact and ROI Sponsor: Red Hat | Topic: Linux Management

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows IN IMPORTANCE IN TERMS OF VALUE TO THE BUSINESS MANAGING ,INUX ENVIRONMENTS TO HIGH STANDARDS OF SERVICE QUALITY AVAILABILITY SECURITY AND PERFORMANCE BECOMES AN ESSENTIAL REQUIREMENT FOR BUSINESS SUCCESS

> http://lnxjr.nl/RHS-ROI

Standardized Operating Environments for IT Efficiency Sponsor: Red Hat

The Red Hat® Standard Operating Environment SOE helps you define, deploy, and maintain Red Hat Enterprise Linux® AND THIRD PARTY APPLICATIONS AS AN 3/% 4HE 3/% IS FULLY ALIGNED WITH YOUR REQUIREMENTS AS AN EFFECTIVE AND MANAGED process, and fully integrated with your IT environment and processes.

Benefits of an SOE:

SOE is a specification for a tested, standard selection of computer hardware, software, and their configuration for use on computers within an organization. The modular nature of the Red Hat SOE lets you select the most appropriate solutions to address your business' IT needs.

SOE leads to:

s $RAMATICALLY REDUCED DEPLOYMENT TIME

s 3OFTWARE DEPLOYED AND CONFIGURED IN A STANDARDIZED MANNER

s 3IMPLIFIED MAINTENANCE DUE TO STANDARDIZATION

s )NCREASED STABILITY AND REDUCED SUPPORT AND MANAGEMENT COSTS

s 4HERE ARE MANY BENEFITS TO HAVING AN 3/% WITHIN LARGER ENVIRONMENTS SUCH AS

s ,ESS TOTAL COST OF OWNERSHIP 4#/ FOR THE )4 ENVIRONMENT

s -ORE EFFECTIVE SUPPORT

s &ASTER DEPLOYMENT TIMES

s 3TANDARDIZATION

> http://lnxjr.nl/RH-SOE

WWW.LINUXJOURNAL.COM / MAY 2015 / 79

LJ253-May2015bu.indd 79 4/24/15 12:19 PM INDEPTH Hacking a Safe with Bash By combining simple tools, you can build an effective safe to store your sensitive data.

ADAM KOSMIN

Through the years, I have settled configuration. Many powerful tools on maintaining my sensitive data and utilities exist in this space, but in plain-text files that I then some introduce unacceptable levels of encrypt asymmetrically. Although “bloat” in one way or another. Being I take care to harden my system a minimalist, I have little interest and encrypt partitions with LUKS in dealing with GUI applications wherever possible, I want to secure that slow down my work flow or my most important data using application-specific solutions (such higher-level tools, thereby lessening as browser password vaults) that are dependence on the underlying system applicable only toward a subset of my

Asymmetric Encryption Asymmetric encryption, or public-key cryptography, relies on the use of two keys: one of which is held private, while the other is published freely. This model offers greater security over the symmetric approach, which is based on a single key that must be shared between the sender and receiver. GnuPG is a free software implementation of the OpenPGP standard as defined by RFC4880. GnuPG supports both asymmetric and symmetric algorithms. Refer to https://gnupg.org for additional information.

80 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 80 4/24/15 12:19 PM INDEPTH

GPG

This article makes extensive use of GPG to interact with files stored in your safe. Many tutorials and HOWTOs exist that will walk you through how to set up and manage your keys properly (https://www.gnupg.org/documentation/index.html). It is highly recommended to configure gpg-agent in order to avoid having to type your passphrase each time you interact with your private key. One popular approach used for this job is Keychain (http://www.funtoo.org/Keychain), because it also is capable of managing ssh-agent.

sensitive data. Working with text files Where “resource” is something affords greater flexibility over how my MNEMONIC SUCH AS AN &1$. OR EVEN data is structured and provides the a hardware device like a router that ability to leverage standard tools I can is limited to providing telnet access. expect to find most anywhere. Both userid and password fields Let’s take the classic example are represented as hints. This hinting of managing credentials. This is a approach works nicely given my necessary evil and while both pass conscious effort to limit the number (http://www.passwordstore.org) and of user IDs and passwords I routinely KeePassC (http://raymontag.github.io/ use. This means a hint is all that is keepassc) look interesting, I am not yet needed for muscle memory to kick convinced they would fit into my work in. If a particular resource uses some flow. Also, I am definitely not lulled EXOTIC COMPLEXITY RULES ) QUICKLY CAN by any “copy to clipboard” feature. understand the slight variation by You’ve all seen the inevitable clipboard modifying the hint accordingly. For spills on IRC and such—no thanks! For example, a hint of “fo” might end up the time being, let’s fold this job into as “!fo” or “fO”. Another example a “safe” concept by managing this of achieving this balance between data in a file. Each line in the file will security and usability comes up when conform to a simple format of: you need to use an especially long password. One practical solution resource:userid:password would be to combine familiar

WWW.LINUXJOURNAL.COM / MAY 2015 / 81

LJ253-May2015bu.indd 81 4/24/15 12:19 PM INDEPTH

passwords and document the hint write a script to act as a wrapper. accordingly. For example, a hint -Y REQUIREMENTS WERE SIMPLE representing a combination of “fo” and “ba” could be represented 1. Leverage common tools, such as as “fo..ba”. Finally, the hinting GPG, shred and bash built-ins. approach provides reasonable fall-back protection since the limited 2. Reduce typing for common information would be of little use operations (encrypt, decrypt and to an intruder. so on). Despite the obscurity, leaving this data in the clear would be silly 3. Keep things clean and readable and irresponsible. Having GnuPG in order to accommodate future configured provides an opportunity growth. to encrypt the data using your private key. After creating the file, my work 4. Accommodate plain-text files but flow was looking something like this: avoid having to micro-manage them. $ gpg --ear $ shred -u )NTERESTINGLY THE VIM GNUPG 6IM plugin (https://github.com/jamessan/ Updating the file would involve vim-gnupg) easily can handle these decrypting, editing and repeating the REQUIREMENTS BECAUSE IT INTEGRATES steps above. This was tolerable for seamlessly with files ending in .asc, a while since, practically speaking, .gpg or .pgp extensions. Despite its I’m not establishing credentials on abilities, I wanted to avoid having a daily basis. However, I knew the to manage multiple encrypted files day would eventually come when and instead work with a higher- the tedious routine would become level “vault” of sorts. With that goal too much of a burden. As expected, in mind, the initial scaffolding was that day came when I found myself cobbled together: keeping insurance-related notes that I then considered encrypting #!/bin/bash USING THE SAME TECHNIQUE .OW ) am talking about managing multiple CONF=${HOME}/.saferc files—a clear sign that it is time to [ -f $CONF ] && . $CONF

82 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 82 4/24/15 12:19 PM INDEPTH

[ -z "$SOURCE_DIR" ] && SOURCE_ esac DIR=${HOME}/safe done SOURCE_BASE=$(basename $SOURCE_DIR) TAR_ENC=$HOME/${SOURCE_BASE}.tar.gz.asc This framework is simple enough to TAR="tar -C $(dirname $SOURCE_DIR)" build from and establishes some ground rules. For starters, you’re going to avoid usage() { micro-managing files by maintaining cat <

extract_safe create_safe() {

;; [ -d $SOURCE_DIR ] || { "Missing directory: $SOURCE_DIR"; exit 1; }

*) $TAR -cz $SOURCE_BASE | gpg -ear $(whoami) --yes -o $TAR_ENC

usage find $SOURCE_DIR -type f | xargs shred -u

exit 1 rm -fr $SOURCE_DIR

;; }

WWW.LINUXJOURNAL.COM / MAY 2015 / 83

LJ253-May2015bu.indd 83 4/24/15 12:19 PM INDEPTH

The create_safe() function is ´safe/file${i}.txt; done looking pretty good at this point, $ safe.sh -c since it automates a number of tedious steps. First, you ensure that You now should have a file the archive’s base directory exists. If named safe.tar.gz.asc in your home so, you compress the directory into directory. This is an encrypted tarball a tar archive and pipe the output containing the five files previously straight into GPG in order to encrypt written to the ~/safe directory. You the end result. Notice how the result then cleaned things up by shredding of whoami is used for GPG’s -r each file and finally removing the option. This assumes the private GPG ~/safe directory. This is probably a key can be referenced using the same good time to recognize you are ID that is logged in to the system. basing the design around an This is strictly a convenience, as I have expectation to manage a single taken care to keep these elements in directory of files. For my purposes, sync, but it will need to be modified if this is acceptable. If subdirectories are your setup is different. In fact, I could needed, the code would need to be see eventually supporting an override refactored accordingly. of sorts with the ~/.saferc approach. Now that you are able to create your For now though, let’s put that idea on safe, let’s focus on being able to open the back burner. Finally, the function it. The following extract_safe() calls the shred binary on all files function will do the trick nicely: within the base directory. This solves

the annoying “Do I have a plain-text extract_safe() {

version laying around?” dilemma by [ -f $TAR_ENC ] || { "Missing file: $TAR_ENC"; exit 1; }

automating the cleanup. gpg --batch -q -d $TAR_ENC | $TAR -zx

Now you should be able to create } the safe. Assuming no ~/.saferc exists and the $PATH environment variable Essentially, you are just using GPG and contains the directory containing safe.sh, tar in the opposite order. After opening you can begin to test this script: the safe by running the script with -x, you should see the ~/safe directory. $ cd Things seem to be moving along, $ mkdir safe but you easily can see the need to list $ for i in $(seq 5); do echo "this is secret #$i" > the contents of your safe, because you

84 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 84 4/24/15 12:19 PM INDEPTH

Listing 1. safe.sh

#!/bin/bash OPTS=" -zx" # [ $# -eq 1 ] && OPTS+=" $SOURCE_BASE/$1 -O" # safe.sh - wrapper to interact with my encrypted file archive gpg --batch -q -d $TAR_ENC | $TAR $OPTS } CONF=${HOME}/.saferc [ -f $CONF ] && . $CONF create_safe() { [ -z "$SOURCE_DIR" ] && SOURCE_DIR=${HOME}/safe is_or_die $SOURCE_DIR SOURCE_BASE=$(basename $SOURCE_DIR) $TAR -cz $SOURCE_BASE | gpg -ear $(whoami) --yes -o $TAR_ENC TAR_ENC=$HOME/${SOURCE_BASE}.tar.gz.asc shred_source_dir TAR="tar -C $(dirname $SOURCE_DIR)" }

usage() { [ $# -ge 1 ] || { usage; exit 1; } cat < /dev/null [ $? -eq 0 ] && echo OK || echo Failed extract_safe() { done is_or_die

do not want to have to open it each are here, you can start DRYing this up time in order to know what is inside. a bit by consolidating all the file and Let’s add a list_safe() function: directory tests into a single function. You even can add a handy little

list_safe() { backup feature to scp your archive to

[ -f $TAR_ENC ] || { "Missing file: $TAR_ENC"; exit 1; } a remote host. Listing 1 is an updated

gpg --batch -q -d $TAR_ENC | tar -zt version of the script up to this point.

} The new -b OPTION REQUIRES A hostname passed as an argument. No big deal there, as you are just When used, the archive will be scp’d using tar’s ability to list contents accordingly. As a bonus, you can rather than extract them. While you use the -b option multiple times in

WWW.LINUXJOURNAL.COM / MAY 2015 / 85

LJ253-May2015bu.indd 85 4/24/15 12:19 PM INDEPTH

order to back up to multiple hosts. appropriately. In fact, you have an This means you have the option opportunity to avoid re-inventing to configure a routine cron job to the wheel by simply refactoring automate your backups while still your extract_safe() function. being able to run a “one off” at The updated function will operate any point. Of course, you will want on a single file if called accordingly. to manage your SSH keys and Otherwise, it will operate on the configure ssh-agent if you plan to entire archive. Worth noting is automate your backups. Recently, the extra step to provide a bit of I have converted over to pam_ssh user-friendliness. Using the default (https://wiki.archlinux.org/index.php/ $SOURCE_DIR of ~/safe, the user SSH_keys#pam_ssh) in order to fire can pass either safe/my_file or just up my ssh-agent, but that’s a my_file to the -o option: different discussion. Back to the code, there is a small list_safe() { is_or_die() function that accepts is_or_die an argument but falls back to the gpg --batch -q -d $TAR_ENC | tar -zt | sort archive specified in $TAR_ENC. } This will help keep the script lean and mean since, depending on the search_safe() { option(s) used, you know you are is_or_die going to want to check for one or FILE=${1#*/} more files and/or directories before for f in $(list_safe); do taking action. ARCHIVE_FILE=${f#$SOURCE_BASE/} For the remainder of this article, [ "$ARCHIVE_FILE" == "$FILE" ] && return I’m going to avoid writing out the done updated script in its entirety. Instead, false I simply provide small snippets as new } functionality is added. For starters, how about adding extract_safe() { the ability to output the contents is_or_die of a single file being stored in OPTS=" -zx" your safe? All you would need to [ $# -eq 1 ] && OPTS+=" $SOURCE_BASE/${1#*/} -O" do is check for the file’s presence gpg --batch -q -d $TAR_ENC | $TAR $OPTS and modify your tar options }

86 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 86 4/24/15 12:19 PM INDEPTH

The final version of safe.sh is the Arch package maintainer appears maintained at https://github.com/ to have included only the gpg-zip windowsrefund/safe. It supports man page. In short, I prefer the a few more use cases, such as the “keep things as simple as possible; ability to add and remove files. but no simpler” approach. If anyone When adding these features, I tried is interested in improving the to avoid actually having to extract methods used to add and remove the archive to disk as a precursor files, feel free to submit your pull to modifying its contents. I was REQUESTS 4HIS ALSO APPLIES TO THE unsuccessful due to GNU tar’s refusal edit_safe() function, although I to read from STDIN when -r is used. foresee refactoring that at some point A nice alternative to connecting given some recent activity with the GPG with tar via pipes might exist vim-gnupg plugin (https://github.com/ in GnuPG’s gpg-zip binary. However, jamessan/vim-gnupg/issues/24).

LINUX JOURNAL on your e-Reader

Customized e-Reader editions Kindle and Nook editions FREE for Subscribers now available

LEARN MORE

LJ253-May2015bu.indd 87 4/24/15 12:19 PM INDEPTH

In general, these credentials simply could be hard-coded in one or more configuration files but that would lead to shame, regret and terrible things.

Integrating with Mutt variable can be used in other My MUA of choice is mutt. Like many areas of mutt’s configuration. For people, I have configured my mail example, another area of your mutt client to interact with multiple IMAP configuration can use these lines: ACCOUNTS EACH REQUIRING AUTHENTICATION In general, these credentials simply set imap_user = "my_user_id" could be hard-coded in one or more set imap_pass = $my_pass_imap configuration files but that would lead set folder = "imaps://example.com" to shame, regret and terrible things. set smtp_url = smtp://$imap_user:[email protected] Instead, let’s use a slight variation of Aaron Toponce’s clever approach By combining appropriately named (https://pthree.org/2012/01/07/ variables with mutt’s ability to support encrypted-mutt-imap-smtp- multiple accounts, it is possible to passwords) that empowers mutt USE THIS TECHNIQUE TO MANAGE ALL OF with the ability to decrypt and your mail-related credentials securely source sensitive data: while never needing to store plain-text copies on your hard drive.Q

$ echo "set my_pass_imap = l@mepassw0rd" > /tmp/pass_mail

$ safe.sh -a /tmp/pass_mail Adam Kosmin works as a Sr. Systems Engineer for Sailthru where he focuses on automation and configuration management. Now that your safe contains the He has presented at PuppetConf on two occasions and is an pass_mail file; you have mutt read it avid supporter of the Free Software initiative. When not coding, with this line in your ~/.muttrc: tweaking or building something, he hangs out with his cat buddies: Evil and Handsome. source "safe.sh -o pass_mail |"

By reading the file, mutt Send comments or feedback via initializes a variable you have http://www.linuxjournal.com/contact named my_pass_imap. That or to [email protected].

88 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 88 4/24/15 12:19 PM Instant Access to Premium Online Drupal Training

Instant access to hundreds of hours of Drupal training with new videos added every week!

Learn from industry experts with real world H[SHULHQFHEXLOGLQJKLJKSURȴOHVLWHV

Learn on the go wherever you are with apps for iOS, Android & Roku

We also offer group accounts. Give your whole team access at a discounted rate!

Learn about our latest video releases and RIIHUVȴUVWE\IROORZLQJXVRQ)DFHERRNDQG 7ZLWWHU #GUXSDOL]HPH

Go to http://drupalize.me and get Drupalized today!

LJ253-May2015bu.indd 89 4/24/15 12:19 PM EOF An Easy Way DOC SEARLS to Pay for Journalism, Music and Everything Else We Like All we need is to unbox business as usual.

ome of us work for money. The same goes for nearly every Some of us work for love. journalist I knew back in the heyday S Some of us work for both, of the trade, before everything got or just because we feel compelled sucked into the Net. or obliged. Now, thanks to the Net, there is As a journalist, I work for all more news than ever, from more those reasons, except money— sources than ever, coming through meaning I write as much as I ever more channels than ever. Could did, but I rarely get paid for it. So I be there’s more money than ever make up the difference with other paying for it too. I don’t know. I kinds of work for which writing and also don’t know what percentage of talking are still marketable skills. the total flow is news or journalism.

90 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 90 4/24/15 12:19 PM EOF

We hate the term “content”. It sounds like packing material, or container cargo. But hey, that’s the world we live in now.

Now it’s all just “content”. free (as in beer) artistic goods as a .O JOURNALIST OF ANY QUALITY OTHER blank slate. than bad) thinks what he or she I know: we can’t ignore copyright produces is “content”. Nor does any laws, rights-clearing norms and musician, or artist of any kind. We industries (publishing, Hollywood) hate the term “content”. It sounds that call folks like us “pirates” like packing material, or container and hold captive regulators by the cargo. But hey, that’s the world we short hairs. None of that will matter live in now. if we make all of them happy by It’s a good thing the Net we finding a way to get them—and have today is only about 20 years the artists they represent—more old (1995 was when the NSFNET money than they’ve been getting was decommissioned, opening the by coercive means. If the answer is whole Net to commercial activity), MONEY THE ONLY QUESTION IS HOW and all businesses and business Let’s start with this thesis: models are provisional anyway. Information wants to be free, but They may do for now, but none of value wants to be paid for. them foreclose future possibilities— If that’s true, the trick is to make just as no breed of commercial it as easy as possible for anybody UNIX (launched by AT&T in 1973) to pay for anything they like, foreclosed the possibility of free wherever they find it, even if that and open alternatives, such as thing is as small as a like, a tweet, Linux. (Historical note: Linux a post, a graffito or a tune heard Journal was launched alongside just one time—and even if all Linux 1.0 in 1994. And we know anybody wants to pay for that thing what’s happened to UNIX and Linux is a penny. since then.) We do have proof that lots of So let’s look at business models for people will pay for things they can

WWW.LINUXJOURNAL.COM / MAY 2015 / 91

LJ253-May2015bu.indd 91 4/24/15 12:19 PM EOF

get for free. Apple, for example, 5. It should make use of APIs. That proved that people were willing way, for example, one might be to pay 99 cents for a song they able to throw a penny (or more) could also get for $0 on Napster at every tune one tags with the or Limewire. I’ve also conducted Shazam app. tests of value by asking audiences at my talks in the US if any of Those are just off the top of them listen to public radio. Usually my head. Add more of your own. most hands go up. Then I ask how It’s easy if you think outside the many of those listeners pay for the boxes of business-as-usual and privilege. About 10% of the hands context-as-usual (such as Facebook, stay up. Then I ask how many would Twitter, Apple and other silo’d sites pay if it was real easy. The number and services—or thinking “there of hands doubles. needs to be an app for that”). Here’s what I suggest for an approach: It also helps to think inside the boxes that have produced millions 1. It needs be free-as-in-freedom as of free and open-source code well as free-as-in-beer. bases, and useful standards and protocols. Those are the boxes 2. It should make it easy for anybody where the base imperative is to pay any amount for anything making stuff as useful as possible they like. Even if it’s not a thing (a for every purpose to which that song, for example). stuff can possibly be put. As it happens, I proposed one of 3. It should be programmable. these a few years back. It’s called So, for example, one could EmanciPay, currently described this set things up so it’s as easy to WAY ON THE 0ROJECT62- WIKI throw a penny (or more) toward anything, anywhere, as it is to EmanciPay is a payment framework like something on Facebook. for customers operating with full agency in the open marketplace. 4. It should be a capability that It operates on open protocols and can be added to lots of different standards, so it can be used by any things, rather than a standalone buyer, seller or intermediary. Simply thing, such as an app. put, EmanciPay makes it easy for

92 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 92 4/24/15 12:19 PM Advertiser anybody to pay (or offer to pay) — Index Q as much as they like Thank you as always for supporting our advertisers by buying their products! Q however they like

Q for whatever they like ADVERTISER URL PAGE #

Q Drupalcon https://events.drupal.org/ 19 on their own terms Los Angeles losangeles2015

— or at least to start with that full Drupalize.me http://drupalize.me 89

set of options, and to work out EmperorLinux http://www.emperorlinux.com 17 differences with sellers easily and NetGate http://www.netgate.com 7 with minimal friction. /g2EILLY 6ELOCITY HTTPVELOCITYCONFCOM web-mobile-business-conf-2015 EmanciPay turns consumers (aka O'Reilly Solid http://solidcon.com/ 37 users) into customers by giving internet-of-things-2015 them a pricing gun (something Peer 1 Hosting http://go.peer1.com/linux 21 which in the past only sellers used) and their own means to make SUSE http://suse.com 3

offers, to pay outright, and to WITI SUMMIT http://www.witi.com/summit 27 escrow the intention to pay when PRICE AND OTHER REQUIREMENTS ARE met. Payments themselves can also be escrowed.

While EmanciPay was first conceived BY 0ROJECT62- AS A WAY TO MAKE ATTENTION ADVERTISERS live payments to nonprofits The Linux Journal brand’s following has and online publishers, it is also grown to a monthly readership nearly positioned as a counterpart to sellers’ one million strong. Encompassing the magazine, Web site, newsletters and SUBSCRIPTION SYSTEMS IN WHAT :UORA much more, Linux Journal offers the ideal content environment to help you calls the “subscription economy” reach your marketing objectives. For (https://www.zuora.com/ more information, please visit http://www.linuxjournal.com/advertising. what-is-zuora), which it says “is built

WWW.LINUXJOURNAL.COM / MAY 2015 / 93

LJ253-May2015bu.indd 93 4/24/15 12:19 PM EOF

on ever changing relationships with are beyond their control anyway. your customers”. Since relationships Think of it as a way that “free are two-way by nature, EmanciPay is market” can mean more than one way that customers can manage “your choice of captor”. Think their end, while sell-side systems such of it as a way that “customer AS :UORAS MANAGE THE OTHER relationships” can be worthy of the label because both sides In “EmanciPay: A Content are carrying their ends of the Monetization Plan for Newspapers”, relationship burden—rather than I say: the sellers’ side carrying the whole thing (as CRM systems do today). Think of EmanciPay as a way to unburden sellers of the need to I think EmanciPay can work as what keep trying to control markets that Bruce Sterling calls a design fiction:

A formal definition exists: “Design fiction is the deliberate use of diegetic prototypes to The suspend disbelief about change.” White Paper There’s heavy freight in that sentence, but most can be Library disposed of promptly. “Deliberate on use” means that design fiction LinuxJournal.com is something that people do with a purpose.

“Diegetic” is from film and theatre studies. A movie has a story, but it also has all the commentary, scene-setting, props, sets and gizmos to support that story. Design fiction doesn’t tell stories— instead, it designs prototypes that www.linuxjournal.com/whitepapers imply a changed world.

94 / MAY 2015 / WWW.LINUXJOURNAL.COM

LJ253-May2015bu.indd 94 4/24/15 12:19 PM EOF

It’s nearly impossible for any better position to imagine out the threatened businesses to imagine possibilities, and actually assemble what will disrupt them—especially THE CODE REQUIRED FOR IT 4HATS WHY when they are busy fighting I’m talking about it here.Q disruptions full-time already. This is why publishing, entertainment Doc Searls is Senior Editor of Linux Journal. He is also a and other content-pumping fellow with the Berkman Center for Internet and Society at businesses can’t grok the likes Harvard University and the Center for Information Technology of EmanciPay—at least not until and Society at UC Santa Barbara. they see a prototype such as might be produced in a design fiction exercise. Send comments or feedback via But geeks who have already http://www.linuxjournal.com/contact changed the world are in a much or to [email protected].

Resources

EmanciPay: http://cyber.law.harvard.edu/projectvrm/EmanciPay

Project VRM Wiki: http://cyber.law.harvard.edu/projectvrm/Main_Page

Zuora: https://www.zuora.com

Subscription Economy: https://www.zuora.com/what-is-zuora

“EmanciPay: A Content Monetization Plan for Newspapers”: http://blogs.law.harvard.edu/ vrm/2009/05/28/emancipay-a-content-monetization-plan-for-newspapers

CRM: http://en.wikipedia.org/w/index.php?title=Customer_relationship_management

Bruce Sterling: http://en.wikipedia.org/wiki/Bruce_Sterling

“Patently untrue: fleshy defibrillators and synchronized baseball are changing the future”: http://www.wired.co.uk/magazine/archive/2013/10/play/patently-untrue

NFSNET: http://en.wikipedia.org/wiki/National_Science_Foundation_Network#Commercial_traffic

History of the Internet: http://en.wikipedia.org/wiki/History_of_the_Internet

WWW.LINUXJOURNAL.COM / MAY 2015 / 95

LJ253-May2015bu.indd 95 4/24/15 12:19 PM