EPUB, Kindle,SUBSCRIBERS Android, iPhone & iPad editions SQLAlchemy | ConVirt | Fabric | Azure | RaspberryFREE PiTO
™ Make Your Android Device Play with Your Linux Box
A Look at SQLAlchemy’s Since 1994: The Original Magazine of the Linux Community Features SYSTEM FEBRUARY 2013 | ISSUE 226 | www.linuxjournal.com ADMINISTRATION Manage Your HOW TO: Virtual Handling Deployment R Packages with ConVirt Spin up Linux VMs on Azure
Use Fabric for Sysadmin Tasks on Remote Machines
PLUS: Use a Raspberry Pi as a Colocated Server
Cover226-Final-banner.indd 1 1/24/13 11:08 AM LJ226-Feb013-bu.indd 2 1/23/13 1:06 PM visit us at www.siliconmechanics.com or call us toll free at 888-352-1173 RACKMOUNT SERVERS STORAGE SOLUTIONS HIGH-PERFORMANCE COMPUTING
““ Just Just becausebecause it’s it’s badass,badass, doesn’t doesn’t meanmean it’s it’s aa game.”game.”
Pierre, our new Operations Manager, is always looking for the right tools to get more work done in less time. That’s why he respects NVIDIA ® Tesla ® GPUs: he sees customers return again and again for more server products featuring hybrid CPU / GPU computing, like the Silicon Mechanics Hyperform HPCg R2504.v3. When you partner with We start with your choice of two state-of- Silicon Mechanics, you the-art processors, for fast, reliable, energy- get more than stellar efficient processing.T hen we add four NVIDIA ® technology - you get an Tesla® GPUs, to dramatically accelerate parallel Expert like Pierre. processing for applications like ray tracing and finite element analysis. Load it up with DDR3 memory, and you have herculean capabilities and an 80 PLUS Platinum Certified power supply, all in the space of a 4U server.
Silicon Mechanics and Silicon Mechanics logo are registered trademarks of Silicon Mechanics, Inc. NVIDIA, the NVIDIA logo, and Tesla, are trademarks or registered trademarks of NVIDIA Corporation in the US and other countries.
LJ226-Feb013-bu.indd 3 1/23/13 1:06 PM FEBRUARY 2013 CONTENTS ISSUE 226 SYSTEM ADMINISTRATION FEATURES 68 ConVirt: the New Tool in Your Virtual Toolbox Take a look at a great, open-source tool for managing multiple types of hypervisors. Jeramiah Bowling 82 Linux on Azure— a Strange Place to Find a Penguin On the heels of Amazon and Google, Microsoft has jumped into the Infrastructure- as-a-Service world, offering cloud-based Linux virtual machines. Andrew Fabbro 94 Fabric: a System Administrator’s Best Friend No sysadmin should be unaware of this tool for managing machines simultaneously. Adrian Hannah COVER IMAGE: © Can Stock Photo Inc. / buchachon
4 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 4 1/23/13 1:06 PM INDEPTH 104 Making Linux and Android Get Along (It’s Not as Hard as It Sounds) How to make Android devices play with your Linux systems. Aaron Peters COLUMNS 36 Reuven M. Lerner’s At the Forge SQLAlchemy 44 Dave Taylor’s Work the Shell Cribbage: Sorting Your Hand 34 PLEX 48 Kyle Rankin’s Hack and / Raspberry Strudel: My Raspberry Pi in Austria 54 Shawn Powers’ The Open-Source Classroom Trying to Tame the Tablet 120 Doc Searls’ EOF Leaving the Land of the Giants IN EVERY ISSUE 8 Current_Issue.tar.gz 10 Letters 68 CONVIRT 16 UPFRONT 34 Editors’ Choice 64 New Products 129 Advertisers Index
ON THE COVER • Make Your Android Device Play with Your Linux Box, p. 104 • A Look at SQLAlchemy's Features, p. 36 • Manage Your Virtual Deployment with ConVirt, p. 68 • Spin up Linux VMs on Azure, p. 82 • Use Fabric for Sysadmin Tasks on Remote Machines, p. 94 • How To: Handling R Packages, p. 24 • Plus: Use a Raspberry Pi as a Colocated Server, p. 48 104 BOX ANDROID APP
LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., 2121 Sage Road, Ste. 310, Houston, TX 77056 USA. Subscription rate is $29.50/year. Subscriptions start with the next issue.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 5
LJ226-Feb013-bu.indd 5 1/23/13 1:06 PM Executive Editor Jill Franklin [email protected] Senior Editor Doc Searls [email protected] Associate Editor Shawn Powers [email protected] Art Director Garrick Antikajian [email protected] Products Editor James Gray [email protected] Editor Emeritus Don Marti [email protected] Technical Editor Michael Baxter [email protected] Senior Columnist Reuven Lerner [email protected] Security Editor Mick Bauer [email protected] Hack Editor Kyle Rankin [email protected] Virtual Editor Bill Childers [email protected]
Contributing Editors Ibrahim Haddad • Robert Love • Zack Brown • Dave Phillips • Marco Fioretti • Ludovic Marcotte Paul Barry • Paul McKenney • Dave Taylor • Dirk Elmendorf • Justin Ryan
Publisher Carlie Fairchild [email protected]
Director of Sales John Grogan [email protected]
Associate Publisher Mark Irgang [email protected]
Webmistress Katherine Druckman [email protected]
Accountant Candy Beauchamp [email protected]
Linux Journal is published by, and is a registered trade name of, Belltown Media, Inc. PO Box 980985, Houston, TX 77098 USA
Editorial Advisory Panel Brad Abram Baillio • Nick Baronian • Hari Boukis • Steve Case Kalyana Krishna Chadalavada • Brian Conner • Caleb S. Cullen • Keir Davis Michael Eager • Nick Faltys • Dennis Franklin Frey • Alicia Gibb Victor Gregorio • Philip Jacob • Jay Kruizenga • David A. Lane Steve Marquez • Dave McAllister • Carson McDonald • Craig Oda Jeffrey D. Parent • Charnell Pugsley • Thomas Quinlan • Mike Roberts Kristin Shoemaker • Chris D. Stark • Patrick Swartz • James Walker
Advertising E-MAIL: [email protected] URL: www.linuxjournal.com/advertising PHONE: +1 713-344-1956 ext. 2
Subscriptions E-MAIL: [email protected] URL: www.linuxjournal.com/subscribe MAIL: PO Box 980985, Houston, TX 77098 USA
LINUX is a registered trademark of Linus Torvalds.
LJ226-Feb013-bu.indd 6 1/23/13 1:06 PM TrueNAS™ Storage Appliances Harness the Cloud
Unified. Scalable. Flexible.
Thanks to the Intel® Xeon® Processor 5600 series and high- performance flash, every TrueNAS Storage appliance delivers the utmost in throughput and IOPS.
As IT infrastructure becomes increasingly virtualized, effective storage has become a critical requirement. iXsystems’ TrueNAS Storage appliances offer high-throughput, low-latency backing for popular virtualization programs such as Hyper-V, VMWare®, Supports iSCSI and NFS exports and Xen®. TrueNAS hybrid storage technology combines simultaneously memory, NAND flash, and traditional hard disks to dramatically Compatible with popular reduce the cost of operating a high performance storage Virtualization programs such infrastructure. Each TrueNAS appliance can also serve multiple as Hyper-V, VMware, and Xen types of clients simultaneously over both iSCSI and NFS, making TrueNAS a flexible solution for your enterprise needs. 128-bit ZFS file system with up For growing businesses that are consolidating infrastructure, to triple parity software RAID the TrueNAS Pro is a powerful, flexible entry-level storage appliance. iXsystems also offers the TrueNAS Enterprise, which provides increased bandwidth, IOPS and storage capacity for resource-intensive applications.
Call 1-855-GREP-4-IX, or go to www.iXsystems.com
TrueNAS Pro Features TrueNAS Enterprise Features • One Six-Core Intel® Xeon® Processor • Two Six-Core Intel® Xeon® Processors 5600 Series 5600 Series • High Performance Write Cache • Extreme Performance Write Cache • Up to 480GB MLC SSD Cache • Up to 1.2TB High Performance ioMemory • Up to 220 TB SATA Capacity • Up to 500TB SATA or 320TB SAS Capacity • Quad Gigabit Ethernet • Dual Ten Gigabit Ethernet • 48GB ECC Memory • 96GB ECC Memory
Intel, the Intel logo, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and other countries.
LJ222-Oct2012.indd 7 9/19/12 10:28 AM Current_Issue.tar.gz
SHAWN POWERS Digital Duct Tape
’ve had enough system Python module, and if you’re using administration jobs to know that Python with an SQL back end, you’ll want I companies tend to take drastically to check it out. Dave Taylor, on the other different approaches to how they hand, continues his series on creating a handle technology. Some companies shell script to play Cribbage. Dave has a budget extensively for their server great way of tricking us all into learning infrastructures, and others have old things by using fun objectives. We workstations with box fans cooling certainly don’t mind. them for servers. Whatever the server Remember when I said that Kyle room looks like, things inevitably go Rankin got me started with Raspberry wrong, and it’s the job of the sysadmin Pi hacking? This month he covers to save the day. Sometimes that means setting up the smallest colocated server a quick hack to get things going you’ll probably ever see. Kyle has a temporarily, and sometimes it means Raspberry Pi sitting in a data center elaborate planning and scheduling for rack in Austria, and he walks through maintenance and replacement. That’s preparing the little server for remote- the thing about system administration— only administration. Because the RPi you have to think on your feet and lacks many of the features server- come up with solutions on the fly. It class machines usually have, a lot of can be exciting, terrifying, stressful and planning goes into the preparation. rewarding, all at the same time. Even if you don’t plan to set up a This is our system administration issue, Raspberry Pi server, it’s a great article. which always is one of my favorites. My column this issue hits much Rather than diving right into the sysadmin closer to home. If you have a fancy new stuff though, Reuven M. Lerner starts Android tablet, but you’re struggling things off with SQLAlchemy, which acts to use is as much as you’d like, you’re as a bridge for your Python objects to not alone. This month I tackle my Nexus “talk” to an SQL database. It’s a powerful 7. At first I struggled to do much more
8 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 8 1/23/13 1:06 PM CURRENT_ISSUE.TAR.GZ
than play Angry Birds with mine, but same reason geeks do many things: after a lot of effort, my tablet is a useful because they can. Aaron, however, tool at work as well as a fancy toy. Those solves a problem we’re all a little of you struggling to find your tablet’s more familiar with, and that is how to niche may benefit from my experiences. connect your Android tablet with your Jeramiah Bowling addresses Linux system. Although many tablets virtualization this month. Using ConVirt, are unable to plug in to a system with he shows how to manage multiple USB for file access, there are many, virtualization architectures with a single many ways to connect with Android, tool. If you want to manage Xen and and Aaron explores a bunch of them. KVM side by side, it’s worth checking We tried to cover a wide variety of out. In fact, with the paid version of the system administration topics this month, program, it’s even possible to manage not just the traditional geek-in-the- VMware hosts! Having the ability to server-room scenarios. As technology look at different virtualization back ends infiltrates every aspect of our lives, even with the same client makes comparing those folks without the slightest desire performance much easier. to manage a data center must have at System administrators like things to least rudimentary administration skills in be easier, and so this month, Adrian order to function. For those hard-core Hannah teaches how to use Fabric, sysadmins out there, if you’re anything which is a tool for administering like me, your bag of tricks is like Mary dozens of machines simultaneously. Poppins’ bag—there’s always room for Whether you want to remove files more. We hope this issue will be as from your entire server farm or install useful for you as it has been enjoyable a package with its dependencies on a for us to create. For now, I need to whole rack of servers, Fabric can make leave—there’s a server somewhere it a one-step process. that needs to be turned off and Andrew Fabbro and Aaron Peters turned back on....■ both describe how to make Linux play well with others. They have drastically Shawn Powers is the Associate Editor for Linux Journal. different takes on the subject, He’s also the Gadget Guy for LinuxJournal.com, and he has however. Andrew walks through the an interesting collection of vintage Garfield coffee mugs. steps of getting Linux up and running Don’t let his silly hairdo fool you, he’s a pretty ordinary guy in Microsoft’s Azure cloud. Why would and can be reached via e-mail at [email protected]. a person want to do that? Well, for the Or, swing by the #linuxjournal IRC channel on Freenode.net.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 9
LJ226-Feb013-bu.indd 9 1/23/13 1:06 PM letters
Spaces Thanks, and keep up the good work! in F77 —Toshiro Fred and Dave Taylor are both Great suggestion Toshiro, thanks! right (see the We’ll see what we can do.—Ed. December 2012 issue’s Digital Version Letters I liked the print version but was section). The forced to change. The NOOK 7" F77 standard— display was too small, and I didn’t ANSI X3.9-1978 want to spend an outrageous amount ISO 1539-1980 (E)—says 1–6 characters of money for a 10" tablet. But, I using a–z and 0–9; however, it also found a cheap Android tablet from says that “Blanks are used to improve China on eBay, and it works great. readability, but unless otherwise noted I have changed all of my magazine have no significance.” In other words, subscriptions to digital. The only it is generally legal F77 to use spaces disadvantage is not being able to tear in names and keywords to decrease out the pages of interesting articles. readability! Try p r int * , 6 6 Now I am waiting for an issue on (yes, with spaces liberally interspersed)! how to hack it and convert it to a —dandeljx Linux distro of my liking. —Jon GrosJean Notebook Installation Articles One topic that would be very helpful Great to hear Jon. I find the PDF if it were discussed in an article is version a bit too small on my 7" the installation procedure in modern tablet too. I might have to look for an notebooks. The new UEFI stuff and inexpensive 10" model for the same Windows 8 are making it very difficult reason. Thanks for the idea!—Ed. to install a dual-boot on a notebook with Windows pre-installed. There are About Fortran Variables no clear or detailed solutions on the In the Letters section of the Web, so a good article explaining this December 2012 issue, Dave Taylor would help us a lot. and reader Fred comment on
10 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 10 1/23/13 1:06 PM [ LETTERS ]
FORTRAN and F77 variable names. it increasingly stressful in Maine to Although what they write may watch as we spend millions repairing have been correct for FORTRAN Windows and Macintosh systems 77 or F77, the present Fortran along with iPad tablets. It is truthfully standard (Fortran 2003) is much frustrating to advocate systems I’ve more flexible. The name of an been using for the past six years— entity may consist of between 1 Ubuntu, OpenSUSE, VectorLinux and 31 alphanumeric characters and countless others—only to have (letters, underscores and numerals) them waived off as “not being user- of which the first must be a letter. friendly”. I’m not a coder. I’m a poet For example, time_of_flight is a and a writer. So, if I can use these valid name. This is just one of the many enhancements that make the present Fortran standard a modern language. As a final comment, since Embedded Server the 95 standard, the official name • Fanless x86 1GHz CPU Standard SIB is with a capital (F) followed by • 1 GB DDR2 RAM On Board • 4 GB Compact Flash Disk lowercase letters (ortran). • 10/100/1000 Base-T Ethernet • Two RS-232 Ports —Nuno Pinhao • Four USB 2.0 Ports • Mini-PCIe • Audio In / Out • Power Supply Included Dave Taylor replies: Indeed. I’ll • Analog SVGA 3D Video • VESA Hole Pattern have to brush up my Algol-68 too, • Optional Wireless LAN • Locked Compact Flash Access 2.6 KERNEL at this rate. • No Moving Parts • XPE or Linux with Eclipse IDE • Dimensions: 4.9” x 4.7” x 1.7” (125 x 120 x 44mm)
Discouraging The EMAC Server-In-a-Box (SIB) is a low cost, small footprint, yet powerful server. Like all EMAC SIBs, the Standard SIB has no I find it discouraging to have several moving parts and features a rugged enclosure design making it an ideal choice for most industrial applications. The Standard SIB great options for Linux in public has a secure locking cover for securing the flash media, while still offering easy removal for updates and backing up the system. school classrooms only to have it http://www.emacinc.com/servers/Standard_sib.htm dashed by both sides of the aisles. I have long advocated for Linux to Since 1985 OVER come into play in the States and help 28 YEARS OF SINGLE BOARD build repair facilities to facilitate SOLUTIONS EQUIPMENT MONITOR AND CONTROL incomes. More and more, I am finding Phone: (618) 529-4525 · Fax: (618) 457-0110 · Web: www.emacinc.com
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 11
LJ226-Feb013-bu.indd 11 1/23/13 1:06 PM [ LETTERS ]
systems, a school can use these systems users on a budget. I want people to as well. They can assign different realize that they can speak up, that users, different groups and different they can bring the change, and that networking settings. I’ve also been they can bring jobs using Linux. witnessing the ignorance of my own —Joseph Ziehmer party it develops a draconian philosophy of removing computers completely. Joseph, as someone who has worked in education for almost 20 years It’s the brush off that is the most now, I feel your pain. Thankfully, in stressful, however—to advocate, voice my last position, I was able to use it and e-mail the governor of my state LTSP and Linux thin clients to save and be given blank responses. There significant money while providing is truthfully nothing that gets done as a user-friendly experience for our far as bringing Linux to the classroom. students. Sadly, that’s the exception People look at the budgets, I look at rather than the rule. I think as a the budgets, and we see a magic line community we need to continue of slashing budgets. touting the benefits while at the same time avoiding “trash talking” I say, instead of the magic line of the opposition. I’ve found the slashing budgets, advocate for Linux negative campaign method seems to companies to come to the state make people defensive and less likely or area where you live. People say to try Linux at all. Good luck, and Linux is too hard to use or to use keep fighting the good fight.—Ed. systems from the Windows 98-era. That is not how it works. What we Wunderlist and Wunderlist 2 are doing with Apple and Microsoft I was looking at implementing it, but is hamstringing and confining I see that Wunderlist2 does not have parents, teachers and our state a native application for Linux, so that budgets to a massive monopoly. goes in the round can. It’s funny that The game board is rigged, and they can do it for iOS and Android, ever since NVIDIA became a silent both of which descend from either partner to Microsoft, the rules BSD or Linux, but they cannot do a have been changing continually for new one for Linux. Oh yes, they have
12 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 12 1/23/13 1:06 PM [ LETTERS ]
a Windows app too. If the Web app is Backup Software Fully so good, how come they need native Cross-Platform apps for the other platforms? Regarding Doc Searls’ article “Heavy —Chuck Hast Backup Weather” in the October 2012 issue, I’ve been using CrashPlan This seems to be how things go (http://www.crashplan.com) for the for me. I recently wrote about past three years, both for onsite and Wunderlist and its native Linux offsite backups. Aside from its Java client, and then they release version requirement, it’s been great. 2 with no Linux client. My only hope —Gerry Normandin is that the Linux version eventually will come out. As it is now, I have Doc Searls replies: Sounds good. significant egg on my face.—Ed. I’ll give it a try.
LJ226-Feb013-bu.indd 13 1/23/13 1:06 PM [ LETTERS ]
At Your Service Shawn Powers replies: I completely agree! I don’t even mind the Java-based front end, but SUBSCRIPTIONS: Linux Journal is available in a variety of digital formats, including PDF, I certainly wish the dæmon itself was running .epub, .mobi and an on-line digital edition, as well as apps for iOS and Android devices. something other than Java. It’s one of those Renewing your subscription, changing your e-mail address for issue delivery, paying your programs that works so well, I tolerate Java. invoice, viewing your account details or other subscription inquiries can be done instantly on-line: http://www.linuxjournal.com/subs. Advanced Articles E-mail us at [email protected] or reach us via postal mail at Linux Journal, PO Box This is in response to Doug’s letter in the 980985, Houston, TX 77098 USA. Please remember to include your complete name January 2013 issue’s Letters section titled and address when contacting us.
“More-Advanced Articles”. ACCESSING THE DIGITAL ARCHIVE: Your monthly download notifications will have links to the various formats First, I echo Doug’s praise. Linux Journal keeps and to the digital archive. To access the digital archive at any time, log in at me up to date. http://www.linuxjournal.com/digital.
LETTERS TO THE EDITOR: We welcome your letters and encourage you to submit them Second, perhaps instead of trying to balance your at http://www.linuxjournal.com/contact or mail them to Linux Journal, PO Box 980985, articles between beginner or novice-level articles Houston, TX 77098 USA. Letters may be and more-advanced articles in one magazine, you edited for space and clarity. could have a second magazine. “Advanced Linux WRITING FOR US: We always are looking for contributed articles, tutorials and Journal”sounds good to me. I would pay for a real-world stories for the magazine. An author’s guide, a list of topics and subscription to a second magazine. due dates can be found on-line: —harleypig http://www.linuxjournal.com/author. FREE e-NEWSLETTERS: Linux Journal editors publish newsletters on both It’s definitely something to consider. If the a weekly and monthly basis. Receive late-breaking news, technical tips and demand is high enough, perhaps it could happen tricks, an inside look at upcoming issues and links to in-depth stories featured on someday!—Ed. http://www.linuxjournal.com. Subscribe for free today: http://www.linuxjournal.com/ enewsletters.
WRITE LJ A LETTER We love hearing from our readers. Please send us ADVERTISING: Linux Journal is a great your comments and feedback via http://www.linuxjournal.com/contact. resource for readers and advertisers alike. Request a media kit, view our current editorial calendar and advertising due dates, or learn more about other advertising PHOTO OF THE MONTH and marketing opportunities by visiting us on-line: http://ww.linuxjournal.com/ Remember, send your Linux-related photos advertising. Contact us directly for further information: [email protected] or to [email protected]! +1 713-344-1956 ext. 2.
14 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 14 1/23/13 1:06 PM LJ226-Feb013-bu.indd 15 1/23/13 1:06 PM UPFRONT NEWS + FUN diff -u WHAT’S NEW IN KERNEL DEVELOPMENT
The udev project appears to be in the patch. crisis. Kay Sievers has come under This hearkens back to the fire for failing to fix problems that days when kernel folks blamed have cropped up in the system, and GCC for producing bad machine it looks as though top kernel folks code, while the GCC folks blamed like Al Viro, not to mention Linus the kernel for using bad C code. Torvalds, have been calling for One key difference is that unlike someone else to take over the project. GCC, the udev code is actually The main issue is that user part of the kernel and isn’t an systems have been hanging. independent project. According to Kay, this is partly It seems clear that if Kay can’t due to udev having a mysterious fix the problem, or at least adopt slowdown that he hasn’t been able better development practices, to fix yet. The slowdown results someone else will be asked to in certain driver requests being maintain udev. Greg Kroah-Hartman, delayed until they time out, which one of the original udev authors, apparently causes the appearance would be an obvious candidate, at of a crash. least for the short term. But, he’s But Kay feels that the real pretty busy these days doing tons problem is with the kernel’s of other kernel work. behavior, not with udev, and that Recently, Linus Torvalds decided the main kernel code should deal to simplify the cryptographic with it. Al and Linus (and the rest signature code for kernel of the people complaining) argue modules. His initial motivation was that udev previously had been to speed things up by migrating working, and that it was a patch some of the time-consuming to udev that resulted in the system signing issues from compile time to crashes; therefore, udev either install time where they would end needed to fix the issue or revert up being faster.
16 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 16 1/23/13 1:06 PM [ UPFRONT ]
This turned out to be slightly In that case, the sign-off doesn’t controversial. David Howells get included. Al felt this was lost suggested that Linus should go data, and he suggested changing even further and take out all the the process, so that sign-offs could module-signing code and just let be added after the fact. users do it manually. But, this There actually was quite a bit ended up causing some unexpected of support for this idea, and it blowback from Linus. turned out that the latest versions The issue Linus is concerned of git already support it, via the with is the ordinary user who git notes add command. But, wants to protect the system although Linus Torvalds is fine from root kits and other attacks. with people using that sort of Requiring modules to be signed thing for local development, he by a secure key is a good way to said he wouldn’t include after- address that. But, he felt that the-fact sign-offs in the main David was concerned with allowing tree. He just felt it wasn’t that distribution vendors to keep a important. As long as someone cryptographic stranglehold over signs off on the code, especially what kind of software ordinary the author of the given patch, users could run on their systems. he’s fine with not having the There was a brief attempt maximum number of sign-offs recently to change the way that he could get. “signed-off-by” reviews are Considering that the signed-off- submitted. Typically, whenever a by process was created in direct patch gets sent into the kernel, response to the SCO lawsuits it passes through a gauntlet of (http://en.wikipedia.org/wiki/ reviewers who confirm that the SCO%E2%80%93Linux_controversies), patch looks good, contains no he must be pretty confident that proprietary code and so on. But, it’s not an important issue. I believe Al Viro pointed out that in a lot at the time Linus was particularly of cases, reviews show up in the inconvenienced, having to account mailing list, after the patch already for the origins and licenses of many has been accepted into the kernel. kernel patches.—ZACK BROWN
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 17
LJ226-Feb013-bu.indd 17 1/23/13 1:06 PM [ UPFRONT ] Non-Linux FOSS: Dive Deep with Wireshark
Before you say anything, yes, I know this program looks a lot like Ethereal, Wireshark is available for Linux. This you’re absolutely correct. It’s the same time, however, Windows and OS X program, but six or so years ago the users get to play too. Wireshark is name changed. an open-source network analysis Wireshark is strictly a wired-ethernet tool that is really amazing for inspection tool, but if you’re trying to troubleshooting a network. solve a network issue, it’s the de facto Running Wireshark on OS X does standard tool. It’s not a new tool by require an X11 server (see my Non-Linux any means, but if you’re on a foreign FOSS article in the December 2012 issue operating system (that is, not Linux), of LJ on XQuartz). It also looks a bit it’s nice to know some old standbys dated once it’s up and running, but rest are available. Check it out today assured, the latest version is functioning at http://www.wireshark.org. behind the scenes. If you’re thinking —SHAWN POWERS
18 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 18 1/23/13 1:06 PM It’s Getting Steamy in Here!
After months of me promising Steam would be coming to Linux, the public beta is finally here. The early verdict: it’s pretty great! The installer is a simple pre-packaged .deb file for Ubuntu (or Xubuntu in my case), and the user portion of the install looks very much like Duke Nukem Forever, I’m happy to Windows or Macintosh. In my limited say that you (and I) were wrong. testing, I’ve found the Steam beta to be Steam is finally coming to Linux, at least as stable as Desura. I also was which has the potential to change the impressed with the large number of my way Linux users play games. It also Steam games that have Linux versions means fewer reboots into Windows just ready to download and play. to shoot a few zombies! Check it out If you were under the impression at http://www.steamforlinux.com. that Steam was going to be the next —SHAWN POWERS
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 19
LJ226-Feb013-bu.indd 19 1/23/13 1:06 PM [ UPFRONT ] System Administration Poll
System Administration is one of the and Red Hat is your second favorite most popular topics at LinuxJournal.com, (24%), while 7% are running and many of our readers have Windows. The other 8% of you are loads of experience in the field. We running a variety of other operating recently polled our on-line readers systems including other flavors of about their system administration Linux, Solaris, AIX or FreeBSD. habits, and we received some Security updates are a regular and interesting answers, as usual. necessary process, and 43% of you We were surprised to learn that do them at least annually, while 12% an almost equal number of you use apply security updates daily. We’re a GUI or Web-based tool versus the relieved to know so many of you are command line, with 51% using the on top of things. Non-security updates latter. And, on the command line, are also frequent with the majority of your preferred protocol is SSH by a readers updating at least quarterly. wide margin with 87%. Telnet and The full survey results are listed below remote serial console each received for your perusal. Thanks again for always 6%, with 1% of you using something being willing to share with the class! else entirely. 45% of you manage one server, while 15% manage more 1) Do you do the majority of your than 20, and more than a few of you system administration work from: are employed by hosting companies n the command line: 51% or companies with similar needs, so n a GUI/Web-based tool: 49% those numbers get pretty high. We were not very surprised to learn 2) When accessing your servers via that vim was your favorite command- command line, do you use: line text editor by far, with 74% of n SSH: 87% the votes, compared to nano/pico n Telnet: 6% with 14% and emacs with 8%. The n remote serial console: 6% remaining 4% of you use something n other: 1% else, and among the other options was naturally “all of the above”. 3) How many servers do you manage? 61% of you are mostly running n 1: 45% Ubuntu or Debian-based servers, n 2–5: 20%
20 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 20 1/23/13 1:06 PM n 6–10: 10% n annually: 46% n 11–20: 10% n more than 20: 15% 9) Have you ever delayed a kernel update in order to preserve your coveted uptime? 4) Which command-line text editor n yes: 30% is best? n no: 70% n vim: 74% n nano/pico: 14% 10) Do you work on your server farm n emacs: 8% from home? n other: 4% n yes: 44% n no: 56% 5) Do you use a configuration management tool like puppet? 11) If so, do you use a VPN? n yes: 16% n yes: 65% n no: 84% n no: 35%
6) Are most of your servers: 12) Does your server infrastructure n Ubuntu-/Debian-based: 61% include a DMZ? n Red Hat-based: 24% n yes: 52% n Windows: 7% n no: 48% n other: 8% 13) What percentage of your servers 7) How often do you apply security are virtualized? updates to your systems? n 0–25%: 43% n daily: 12% n 26–50%: 20% n weekly: 21% n 51–75%: 17% n monthly: 15% n 76–100%: 20% n quarterly: 9% n annually: 43% 14) If you use virtualization, what is your host environment? 8) How often do you apply non- n VMware: 42% security updates to your system? n Xen: 13% n daily: 7% n KVM: 18% n weekly: 18% n Hyper-V: 3% n monthly: 17% n n/a: 12% n quarterly: 12% n other: 12%
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 21
LJ226-Feb013-bu.indd 21 1/23/13 1:06 PM [ UPFRONT ]
15) Do you host e-mail: 22) Have you ever had a system n locally: 55% compromised? n with a cloud host: 19% n yes: 37% n we don’t provide e-mail: 26% n no: 63%
16) Do you allow users VPN access 23) Do you use: into your network? n a router/firewall appliance n yes: 54% (Cisco, etc.): 62% n no: 46% n a software-based router/firewall solution: 38% 17) Do you have Wi-Fi coverage at your workplace? 24) Does your husband/wife/significant n yes: 84% other know your password(s)? n no: 16% n yes: 7% n no: 93% 18) If yes, do you allow guest access to Wi-Fi? 25) Do you use a password program n yes: 40% like LastPass or KeePassX? n no: 49% n yes: 37% n n/a: 11% n no: 63%
19) Is your network and server layout 26) How often do you change your well-documented? passwords? n yes: 57% n daily: 1% n no: 43% n weekly: 3% n monthly: 19% 20) Are you the lone system n quarterly: 31% administrator at your workplace? n rarely: 46% n yes: 46% n no: 54% 27) Do you force your users to change their passwords? 21) Do you have to support platforms n yes: 50% other than Linux? n no: 50% n yes: 71% n no: 29% —KATHERINE DRUCKMAN and SHAWN POWERS
22 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 22 1/23/13 1:06 PM LINUX JOURNAL ARCHIVE DVD
NOW AVAILABLE
Save $10.00 by using discount code DVDFEB at checkout. Coupon code expires 2/16/2013 www.linuxjournal.com/dvd
LJ226-Feb013-bu.indd 23 1/23/13 1:06 PM [ UPFRONT ]
Handling R Packages
One of the R statistics program’s and how to keep them up to date. great features is its modular Then, I finish with a quick look at nature. As people develop new how to create your own. functionality, R is designed so that The first step is to check and it’s relatively easy to package up see what libraries already are the new functionality and share it installed on your system (Figure with other R users. In fact, there 1). You can do this by running is an entire repository of such library() from within R. This packages, offering all sorts of provides a list of all the libraries goodies for your statistical needs. installed in the various locations In this article, I look at how to visible to R. If you find the library find out what libraries already are you’re interested in, your work is installed, how to install new ones almost done.
Figure 1. The library() command gives a list of currently installed libraries.
24 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 24 1/23/13 1:06 PM [ UPFRONT ]
In order to make R load the library is to use the special character of interest into your workspace, you ?. For example, you could type need to call library with the name ?multicore to get the same of the library in brackets. Let’s say result. A related command that you want to do parallel code with is good to know is ??. It does a the multicore library. You would call search through the library names library("multicore"). and descriptions based on the text If you want to learn more about given. For example, ??plot pulls a library, R includes a help system up entries related to the word plot that is modeled after the man (Figure 3). page system used in Linux. There But, what if the library you are are two ways to access it. The first interested in isn’t already on your is to use the help() command. system? Then you need to install So in this case, you would run it somehow. Luckily, R has a full help("multicore") (Figure package management system built 2). The shortest way to get help in. Installing a package is as easy
Figure 2. Getting Help on a Library
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 25
LJ226-Feb013-bu.indd 25 1/23/13 1:06 PM [ UPFRONT ]
Figure 3. Looking for Help on Plots
Figure 4. Trying to install a library in the system location won’t work.
26 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 26 1/23/13 1:06 PM [ UPFRONT ]
as running install.packages(), a CRAN mirror for downloading where you hand in a list of package the package. This mirror will be names. But, how do you know used for all future downloads. what packages are available for By default, R also will download installation? The R project has a and install any dependencies the full repository of packages ready requested package needs. So in this for you to use. You can find them sense, it really is a proper package at http://cran.r-project.org. On management system. the left-hand menu, you will see an For many packages, all that is entry called “Packages”, which will involved is strictly R code. But bring you to list of packages. You in some cases, the author may can search alphabetically by name have written part of the code in or by category. some other language, like C or Say you’re interested in doing FORTRAN, and wrapped it in R linear programming. On CRAN, code. In those types of packages, you will find the linprog package, the other code needs to be which you can install with compiled into binary code before install.packages("linprog"). it can be used. How can you do When you first run this command, that? Well, the R package system it should come back with an error actually can handle compiling (Figure 4). By default, R tries to external code as part of the install packages into the system installation process. In some library location. But, unless you cases, this external code may are running as root (and you aren’t need other third-party libraries doing that, right?), you won’t in order to be compiled. To hand have the proper permissions to in locations for those, you need do so. Therefore, R will ask if you to add some options to the want to install the new package install.packages function call. into a personal library storage Checking the help page (with location in your home directory. ?install.packages) shows that After you agree to this, it will go you can include installation options ahead and try to download the as INSTALL_opts. source for this package. If this is Now that you have your collection the first time you have installed a of packages all installed and package, R will ask you to select configured on your system, what
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 27
LJ226-Feb013-bu.indd 27 1/23/13 1:06 PM [ UPFRONT ]
Figure 5. Checking Whether Any Packages Have Updates Available
do you do if a bug gets fixed in This command goes through each one of them? Or, what happens available update and asks you if a new version comes out with whether you want to install the new a better algorithm? Well, R’s version. package management system can Many packages include either handle this rather well. You can demos, data files or both. The demos check to see whether any packages walk you through some examples of need to be updated by running how to use the functions provided packageStatus() (Figure 5). If you by the package in question. To see see that updates are available, you what demos are available, you can can install the updates by using the call demo() (Figure 6). To run a command update.packages(). particular demo, for example, the nlm
28 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 28 1/23/13 1:06 PM [ UPFRONT ]
Figure 6. The List of Demos Available in R
demo, you would run demo(nlm). can get more information on the Many packages also include data, including a description and sample data files that you can use a list of the variables available, by when you are learning to use the running ?LakeHuron (Figure 8). new functions. To see what data files So far, I’ve been looking at are available, you would call data() dealing with individual packages, (Figure 7). To load a particular data but sometimes you need functions file, you need to call data with the provided by several different data file you are interested in. For packages. In R parlance, this is example, if you want to play with called task views. These are groups water levels in Lake Huron, you of packages that are all useful for a would call data(LakeHuron). You particular area of research. If you are
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 29
LJ226-Feb013-bu.indd 29 1/23/13 1:06 PM [ UPFRONT ]
Figure 7. The List of Data Files Available in R
interested in using task views, start whole with the update.views() by installing the ctv package. In R, command. These task views, like run install.packages("ctv") to all of the packages, are written install the main task view package. and maintained by other users like Once that’s done, you can load yourself. So, if you have some area the library with library("ctv"). of research that isn’t being served Now, you will have new functions right now, you can step in and included in the install and update organize a new view yourself. packages. To install a view, like the Up to this point, I’ve been Graphics view, you simply can run discussing how to use packages that install.views("Graphics"). have been written and provided by You can update these views as a other people. But, if you are doing
30 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 30 1/23/13 1:06 PM [ UPFRONT ]
Figure 8. The Help Page for the LakeHuron Data
original research, you may end up example of this file looks like: developing totally new techniques
and algorithms. Science and Package: pkgname Version: 0.5-1 Date: 2011-01-01 Title: My first package
knowledge advance when we share Author@R: c(person("Joe", "Developer", email = "[email protected]"),
with others, so R tries to make it person("A.", "User", role="ctb", email="[email protected]"))
easy to create your own packages Author: Joe Developer
and share them with others through User
CRAN. There is a fixed directory Maintainer: Joe Developer
layout where you can put all of Suggests: MASS Description: A short (one paragraph) description
your code. You also need to include License:
a file called “DESCRIPTION”, and GPL (>= 2) URL: http://www.r-project.org, http://www.somesite.com
a writeup of your package. An BugReports: http://bugtracker.com
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 31
LJ226-Feb013-bu.indd 31 1/23/13 1:06 PM [ UPFRONT ]
Once you have all of your code and data files written and packaged, you They Said It can go ahead and run a check on your new package by running the command It does not do to leave R CMD check /path/to/package on a live dragon out of your the command line. This runs through calculations, if you live some standard checks to make sure near him.—J. R. R. Tolkien, everything is where R expects things. The Hobbit Once your package passes the checks, you can run R CMD build /path/ A goal without a plan is to/package to see if R can build your just a wish.—Antoine de package properly. This is especially Saint-Exupéry important if you have external code in another programming language. Once In preparing for battle I your package passes the checks and have always found that builds correctly, you can bundle it plans are useless, but up as a tarball and send it up to planning is indispensable. http://CRAN.R-project.org/incoming —Dwight D. Eisenhower as anonymous, and then send an e-mail to [email protected]. Once Someone’s sitting in your package has been checked by the shade today because someone at CRAN to verify that it someone planted a builds correctly, your newly created tree a long time ago. package will be added to the —Warren Buffett repository. Fame and fortune will be soon to follow. Everybody has a plan Hopefully this article has provided until they get punched enough information to help you in the face.—Mike Tyson get even more work done in R. And remember, we all progress when we share, so don’t hesitate to add to the functionality available to the whole community. —JOEY BERNARD
32 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 32 1/23/13 1:06 PM attend flourish! inspiring talks, illuminating workshops
open source, open future.
march 1-2 uic forum 725 w Roosevelt Rd. (mc 126) chicago‚ IL
a conference promoting the adoption of free, sponsored by: libre, open source software in the midwest.
get updates at http://flourishconf.com/
LJ226-Feb013-bu.indd 33 1/23/13 1:06 PM [ EDITORS' CHOICE ]
™
EDITORS’ Android Candy: CHOICE Plex ★
Anyone with an iPhone probably is familiar with the AirVideo application. Basically, it’s the combination of a server app that runs on your Windows or OS X machine, and it serves video over the network to an AirVideo application on your phone. It’s extremely popular, and for a good reason—it works amazingly well. For a long time, there wasn’t a good solution for the Android world, largely due to the way Android streamed video. Now, however, there is an incredible application for doing the exact same thing iOS users do with AirVideo. You’ve probably heard of Plex, but you may not know about the server/client combination it can do with Android. Figure 1. Plex shows your home video collection in much Once you install the the same way as Hulu or Netflix. server application,
34 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 34 1/23/13 1:06 PM [ EDITORS' CHOICE ]
Figure 2. The video quality adjusts for your current bandwidth and renders crisp video even on a large tablet display.
which runs perfectly fine on a formats and converting to an Linux server, you install the Plex appropriate bandwidth on the fly. application from the Google Play Plex may have started out as a store, and your video collection Macintosh-compatible competitor follows you anywhere you have to XBMC, but it’s evolved into an connectivity. The content is, of incredible video-streaming system. course, dependent on the content With Plex, you can become your you have on your server, but the own Netflix! Due to its Linux format in which your content is compatibility and incredible video stored doesn’t matter very much. streaming ability, Plex is this Plex’s server application does a month’s Editors’ Choice! great job of streaming most video —SHAWN POWERS
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 35
LJ226-Feb013-bu.indd 35 1/23/13 1:06 PM COLUMNS AT THE FORGE SQLAlchemy REUVEN M. LERNER Talk to your SQL database via Python objects with SQLAlchemy.
Although it sometimes might program you have written. The SQL seem as if relational databases have is kept as a text string within the gone the way of the dinosaur, making application and is then sent, via a way for non-relational (NoSQL) network socket, to the server. databases, such as MongoDB and There are several problems with Cassandra, a very large number of this. First, it means you have to mix systems still depend on a relational two different languages within the database. And, although there is same program. Inside your Web no requirement that a relational application, which you’ve worked database use SQL as its query hard to write, and which you try to language, it’s a rare database product ensure is maintainable, you have that does not do so. code in a totally separate language, The good news is that SQL inside strings, which you cannot test is relatively easy to work with, or maintain directly. particularly when the queries are Even if the SQL queries weren’t straightforward. It’s fast and easy to written inside strings, you still would create tables, insert data into them, be faced with the fact that the update that data and write queries majority of your Web application that retrieve some or all parts of the is written in one language, but data. SQL also makes it fairly easy to your data-manipulation routines combine (“join”) information from are written in another language. A multiple tables, letting you normalize Web application contains, no matter the data, while keeping speed and how you slice it, components in flexibility at a maximum. HTML, CSS and JavaScript, as well as SQL might not be difficult to work whatever server-side language you’re with on its own, but you rarely using. Adding SQL to this can only work with it in a vacuum. Usually, complicate things further. your SQL statements reside within a Even if SQL and a typical
36 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 36 1/23/13 1:06 PM COLUMNS AT THE FORGE
server-side language were on equal is responsible for ensuring that the footing in terms of the syntax data is saved to the database. In of a Web application, there’s a other words, Active Record requires fundamental mismatch between that you create a single class, and the ways in which they handle that it handles both sides of the data. SQL operates with rows and object-relational divide. The Active columns within tables; everything Record class in Ruby on Rails is (not in a relational database has to surprisingly) an implementation of fit into this table-centric view of this paradigm and provides a great the world. By contrast, modern deal of power and flexibility. programming languages have a A second paradigm is known as rich variety of data structures and Data Mapper, and it requires the typically are object-oriented to some use of three different object classes: degree or another. a class that represents the data Libraries that bridge the gap itself at the object level, a class that between procedural code and SQL represents the database table and are known as object-relational a “mapper” object that acts as a mappers, or ORMs. ORMs typically go-between, ensuring that the object represent database records as and relational parts of the system instances of a particular class. are appropriately synchronized. In order to represent 50 records, An excellent and popular example you would need 50 instances of of the Data Mapper paradigm a class, with the state of each can be found in the SQLAlchemy instance reflecting the names, project. SQLAlchemy has been types and values of the columns around for a number of years in that record. already, and makes it possible to There are two basic paradigms for work with relational databases passing data between the object- flexibly from within your Python oriented data structures and the program, without having to write database, both of which were any SQL. described by Martin Fowler. In the In this article, I take a look at first paradigm, known as Active SQLAlchemy, exploring a number Record, each instance is tied directly of its options and features, and to a row in the database, and the considering how it can be used in class itself (as well as each object) Web and other applications.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 37
LJ226-Feb013-bu.indd 37 1/23/13 1:06 PM COLUMNS AT THE FORGE
Connecting to a Database table, People: Installing SQLAlchemy should be straightforward to anyone who has CREATE TABLE People (id SERIAL PRIMARY KEY, installed Python packages before. You can first_name TEXT, get SQLAlchemy from PyPi, the Python last_name TEXT, Package Index, either by downloading email TEXT, it from http://pypi.python.org or by birthday DATE); using the easy_install or pip programs to retrieve and install it. I was able to Let’s also add some initial records: install it with:
INSERT INTO People (first_name, last_name, email, birthday)
pip install sqlalchemy VALUES ('Reuven', 'Lerner', '[email protected]', '1970-jul-14'),
('Foo', 'Bar', '[email protected]', '1970-jan-1'); You might need to install SQLAlchemy as root. Or, you can In order to access this table using install it into a virtual machine, using SQLAlchemy’s ORM, you first need the popular virtualenv package for to create a database session object, Python, which gives you nonroot which itself must be created using control and permissions over a an “engine”. Each database driver Python environment. has its own style of URL. In the You also will need to install a driver case of PostgreSQL accessed via for the database you intend to use. the psycopg2 driver, you would use My favorite relational database is something like this: PostgreSQL, and I use the psycopg
Python driver, also available on PyPi dburl = 'postgresql+psycopg2://reuven:reuven@localhost/atf' and (by extension) via pip. I should note that although I know This URL indicates not only the SQLAlchemy works with Python 3, database and driver type, but also my much of the work I do nowadays user name and password (“reuven” in is still in Python 2, mostly because both cases), the hostname (localhost) that’s what my clients are using. My and the name of the database I’ll examples, thus, also will be in Python be accessing (“atf”). If the database 2, although I believe they will work in is not available at the default Python 3 with little or no changes. PostgreSQL port of 5432, you can Let’s assume you have a database specify that as well in the URL.
38 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 38 1/23/13 1:06 PM COLUMNS AT THE FORGE
You then tell Python to create a new of the SQLAlchemy-provided engine based on this URL: Column class.
from sqlalchemy import create_engine n You connect the class with your engine = create_engine(dburl) database table by defining the _ _tablename_ _ class-level attribute. Now that you have the engine defined, you can create a session For example, the following Python based on this engine. Doing so class provides a mapping to the requires two steps: first you create People database table: a new, custom Session class for
this engine, and then you create an from sqlalchemy import Column, Integer, String, DateTime
instance of the Session class that you from sqlalchemy.ext.declarative import declarative_base
will use to access the database: Base = declarative_base()
class Person(Base):
from sqlalchemy.orm import sessionmaker # import sessionmaker class
Session = sessionmaker(bind=engine) # make custom session type _ _tablename_ _ = 'people'
session = Session() # make instance of session id = Column(Integer, primary_key=True)
first_name = Column(String)
You’re now connected to the last_name = Column(String)
database! But, that’s not quite email = Column(String)
enough. If you want to map your birthday = Column(DateTime)
database table to one or more Python
objects, you need to define a class. def _ _init_ _(self, firstname, lastname, email, birthday):
You do this by defining a normal self.first_name = firstname
Python class, with a few subtle self.last_name = lastname
changes: self.email = email
n The class must inherit from It might not be obvious at first Base, a class returned from the glance, but this class implements the declarative_base function provided Data Mapper design pattern. The by SQLAlchemy. class attributes that you have defined describe the columns in the database n The database columns must be table and can contain a great deal of defined as class attributes, as instances detail, including indexes, uniqueness
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 39
LJ226-Feb013-bu.indd 39 1/23/13 1:06 PM COLUMNS AT THE FORGE
requirements and even custom the records you’ll get, by chaining the integrity constraints, such as those filter_by method to your query: provided by PostgreSQL. The class itself is a standard Python class. for p in session.query(Person).filter_by(id=1): But behind the scenes, because this print p.first_name class inherits from Base, you get a number of other attributes, including That gives the following: _ _mapper_ _, which indicates how your Python class will be mapped to Reuven the database table. You can see this in an interactive Python shell by asking Note that the filter_by method is not to see the printed representation of acting on the results of session.query. Person._ _mapper_ _: Rather, it is changing the SQL that eventually is sent to the database. You Person._ _mapper_ _ can see this by assigning printing to
Reuven You also can see from this query Foo that SQLAlchemy binds parameters to variables inside your query, rather than In other words, session.query directly placing your values. Not only is executing a query against the does this allow you to re-run queries database, without you having to later with different variable values, but specify the SQL. You also can restrict it reduces the possibility that you will
40 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 40 1/23/13 1:06 PM COLUMNS AT THE FORGE
suffer from an SQL injection attack, Relationships which still is surprisingly common. If SQLAlchemy could only do this, it You also can order the results: still would be a nice library, simplifying your queries. But the real power of for person in session.query(Person).order_by('first_name'): SQLAlchemy occurs when you define print person.first_name relationships between tables. For example, let’s assume that I have an Foo Appointments table, indicating when Reuven I’m meeting with various people:
And, you also can do all of the basic CREATE TABLE Appointments ( “CRUD” activities that you would id SERIAL PRIMARY KEY, expect. For example, you can create a person_id INTEGER NOT NULL REFERENCES People, new instance of your Person class and meeting_at TIMESTAMP NOT NULL, then save it to the database: notes TEXT );
p = Person('newfirst', 'newlast', '[email protected]', '1-jan-2012');
session.add(p) Let’s also add some appointments:
session.commit()
INSERT INTO Appointments (person_id, meeting_at, notes)
Notice how I can handle multiple inserts VALUES (2, '1-jan-2013', 'New Year meeting'), (or other actions) inside a single transaction (2, '1-feb-2013', 'Monthly update'); by only issuing session.commit() after adding several objects. Similarly, I can Now I need to create a Python class update the object and the corresponding that represents appointments: row in the database: class Appointment(Base): p.first_name = '!!!' _ _tablename_ _ = 'appointments' session.add(p) id = Column(Integer, primary_key=True) session.commit() person_id = Column(Integer) meeting_at = Column(DateTime) I also can delete the object: notes = Column(String)
session.delete(p) Now, this class will work just fine. session.commit() However, there’s no relationship,
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 41
LJ226-Feb013-bu.indd 41 1/23/13 1:06 PM COLUMNS AT THE FORGE
according to Python, between the per person, representing a one-to- Person class and the Appointment many relationship. class. To make this work, you’ll Let’s say, however, that you want need to change each of these to have a many-to-many relationship table definitions. In the case of between people and appointments, Appointment, you’ll need to indicate such that you can meet with more that the person_id column doesn’t than one person at a time, and you just contain an integer, but that it is can have more than one appointment a foreign key that points to the “id” with a particular person. In order to column on the People table: do that, you need to modify your database table and code somewhat, person_id = Column(Integer, ForeignKey('people.id')) adding a third (association) table. SQLAlchemy makes it easy to do that. On the Person table, you’ll need to Although I don’t have space to show add a line to the class attributes, after it here, the basic idea is that you describing all of the columns: create the third table, and you use the relationship() function to indicate
appointments = relationship("Appointment", backref="person") that there is a secondary relationship between the class and the join table. Thanks to these two lines, you get an “appointments” attribute on Conclusion your Person model. But thanks to SQLAlchemy is packed with features. In the “backref” parameter, you also addition to the introductory examples get a “person” reference on the I showed here, it handles everything appointment. This means you can do from joins to connection pooling, to something like this: dynamically calculated column values, to creating Python classes based on for a in session.query(Appointment): an existing database table. There is print a.person no doubt that it’s a powerful system, one that I expect to use in some of the for p in session.query(Person): Python projects on which I work. print p.appointments That said, I found SQLAlchemy to be a bit overwhelming for the newcomer. Note that the assumption is that Perhaps it’s because I have long used you’ll have multiple appointments the Active Record model in Ruby,
42 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 42 1/23/13 1:06 PM COLUMNS AT THE FORGE
which has minimal configuration and bit formal, trying to tell you too much syntax, but I found the syntax for before moving ahead with actual SQLAlchemy to be a bit overly verbose. code. Hopefully, this article can help Then again, Python has long preferred some more people become interested things be explicit, and there’s no doubt in SQLAlchemy. that SQLAlchemy provides a clear and In conclusion, SQLAlchemy is a great explicit ORM, without much magic and Python module, one that deserves with obvious ramifications for every its sterling reputation and broad function call and parameter. popularity. If you’re interested in The other thing that might throw working with databases from Python off newcomers to SQLAlchemy is programs, you definitely should take a that the documentation is complete, look at SQLAlchemy.■ but not particularly friendly. Once you start to use the system, I expect Reuven M. Lerner is a longtime Web developer, consultant that you (like me) will be able to and trainer. He is also finishing a PhD in learning sciences at understand the documentation and Northwestern University. His latest project, SaveMyWebApp.com, make good use of it. But I found that went live last spring. Reuven lives with his wife and children in even the tutorial documents were a Modi’in, Israel. You can reach him at [email protected].
Resources
The home page for SQLAlchemy is http://sqlalchemy.org, and the documentation is at http://docs.sqlalchemy.org. The Python language is at http://python.org. I suggest that you read through the introductory section and then the ORM documentation, rather than look at the document sequentially.
There are a number of on-line tutorials for SQLAlchemy. Two that I enjoyed, which are freely available to the public, are https://www.youtube.com/watch?v=399c-ycBvo4 and https://www.youtube.com/watch?v=PKAdehPHOMo.
Finally, Rick Copeland’s book, Essential SQLAlchemy, published by O’Reilly in 2008, is a good introduction, particularly if you look at the ORM section. The rest is a bit dry and technical, even if the examples are well written. This book is not completely up to date, and there are several items in it that reflect the fact that it was published several years ago. Nevertheless, having an additional reference can be quite handy and can provide examples for certain features that aren’t otherwise obvious.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 43
LJ226-Feb013-bu.indd 43 1/23/13 1:06 PM COLUMNS WORK THE SHELL
Cribbage: DAVE TAYLOR Sorting Your Hand Continuing our development of a Cribbage game, this month Dave tackles the tricky task of sorting a hand by rank value.
We’ve been working on writing We can prototype this by using the code for the game Cribbage, and command line to see what result we get: last month, I ended this column by creating the code needed to pick a $ sh cribbage.sh | sort -n random subset of six cards out of Card 0: 4S a “deck” and display them in an Card 1: 7C attractive format—like this: Card 2: 9S Card 3: JC $ sh cribbage.sh Card 4: 7H Card 0: 7C Card 5: 8C Card 1: 5H Card 2: 9H What the heck? Oh! You can see Card 3: 10S the problem, right? By telling sort to Card 4: 5D order things numerically, it properly Card 5: AS ignores “Card” but then sees the ordinal value of the card and sorts The primary task on the agenda based on that, rather than on the this month is to sort the cards after actual card value itself. they’ve been dealt. This means we’re Even if we fix this, however, we still going to have to sort the cards by have the problem that face cards will sort rank while ignoring the suit, then before numeric value cards, which isn’t slot them back into the “hand” array. what we want. In fact, we want aces to Is there an easy way to do that? sort as lower than 2s, while jacks, queens Actually, we’ll use the sort function. and kings sort as higher than 10s.
44 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 44 1/23/13 1:06 PM COLUMNS WORK THE SHELL
If you wanted to have aces “high”, we’re using a new variable, orank, the easiest way to do that would to store the corrected value. Why? be to change the display routine, of Because now in the main section of the course: 1 = a deuce, 2 = a three, 12 script we also can access the $rank of = king and 13 = ace. Poof. Everything the card as desired: sorts ace-high. That’s just not how Cribbage scores them. showcard ${hand[$card]} To accomplish Cribbage-rank sorting, echo "$rank ${hand[$card]}" we’ll need to change the output to push out two values: the rank and the For each card chosen, the script has total card value. It’s going to look ugly, an interim output of rank followed by but it’s just an interim result. the numeric value of the card, with Here’s how I tweak the code to no fancy display (even though we’re display these values: still tapping the showcard function for simplicity). The result:
showcard()
{ $ sh cribbage.sh
# given a card value of 0..51 show the suit and rank 13 38
suit=$(( $1 / 13 )) 6 31
rank=$(( ( $1 % 13 ) + 1 )) 8 33
case $rank in 10 35
1) orank="A" ;; 5 30
11) orank="J" ;; 12 24
12) orank="Q" ;;
13) orank="K" ;; Ugly? Definitely. But now we can *) orank=$rank ;; sort it and get useful results, even if esac they might not look like it quite yet:
showcardvalue=$orank${suits[$suit]}
} $ sh cribbage.sh | sort -n 1 26 If you compare it to the version we 2 14 built last month, the main difference is 2 40 that instead of calculating the rank of 3 2 the card and then overwriting it with 7 45 “A”, “J”, “Q” or “K” as appropriate, 10 22
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 45
LJ226-Feb013-bu.indd 45 1/23/13 1:06 PM COLUMNS WORK THE SHELL
It still looks confusing, but you can to display the hand before and after see that it’s in rank order. we’ve sorted by rank. Eventually, the So, how do we get that back into first “showhand” would just be axed, the “hand” array now that we know of course. how to sort it? That’s actually rather Now, let’s get back to the code tricky because of variable scoping needed to sort the cards in our hand issues, as you’ll see. (a feature that a lot of iOS Cribbage Before we go there, however, I’ve games seem to omit, as far as I can tell). written a new “showhand” function My first stab at writing “sorthand” that displays all the cards in the hand on took advantage of a very slick feature a single line, with the help of /bin/echo in the Bourne shell that lets you tie for echoes without a trailing line break: the output of one loop to the input of another with a pipe. For example: showhand() { for card in {0..5} # show our hand neatly do /bin/echo -n "Hand: " showcard ${hand[$card]} for card in {0..4} echo "$rank ${hand[$card]}" do done | sort -n | while read rank value showcard ${hand[$card]} do /bin/echo -n "$showcardvalue, " hand[$index]=$value done index=$(( $index + 1 )) showcard ${hand[5]} done echo "$showcardvalue." } The problem is that the shell’s pipe implementation pushes the second With that available, our main code loop into a subshell without any easy starts to look nice and clean: way to get the changed values back up to the parent shell. The result: by dealhand; the line immediately after the last showhand; # for testing sorthand only done statement, all the new values sorthand; have been lost. showhand; That’s too bad, because it definitely was more elegant. But then again, For debugging purposes, I’m going it’s not about elegant, it’s about
46 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 46 1/23/13 1:06 PM COLUMNS WORK THE SHELL
functional, right? $ sh cribbage.sh Here’s how I actually solved it, Hand: 9H, 6D, KC, AH, 9S, JH. by using a temporary file to store Hand: AH, 6D, 9S, 9H, JH, KC. the intermediate results instead. It’s $ sh cribbage.sh considerably less elegant, for sure: Hand: 4D, QS, AC, 9H, 10C, JS. Hand: AC, 4D, 9H, 10C, JS, QS. sorthand() $ sh cribbage.sh { Hand: 9H, 10C, 7C, 7H, 5H, AS. # hand is dealt, now sort it by card rank... Hand: AS, 5H, 7C, 7H, 9H, 10C. index=0 tempfile="/tmp/.deleteme" It looks like it’s working exactly as for card in {0..5} we’d hope. Yeee-ha! do Yes, there are undoubtedly more showcard ${hand[$card]} efficient ways to write this code, and you echo "$rank ${hand[$card]}" can quite reasonably ask if a shell script done | sort -n > $tempfile is the optimal development environment for this sort of project, but, seriously, while read rank value lighten up. Let’s enjoy this project, not do flagellate ourselves over punctuation! hand[$index]=$value And on that note, let’s wrap up this index=$(( $index + 1 )) month’s column and start thinking done < $tempfile about a considerably harder challenge rm -f $tempfile we’ll face starting next month: how } to evaluate the value of the hand so that we can recommend which four of Note that to get the input of the the six cards dealt should be kept to temporary file as the input for the optimize the Cribbage hand. while loop, I simply redirect stdin You are learning Cribbage as we go, for the loop at the very end of the right? You’ll want it for next month’s loop: done < $tempfile. installment, for sure.■ Let’s test it by dealing a few hands and then showing them Dave Taylor has been hacking shell scripts for more than 30 years. immediately post-deal and then Really. He’s the author of the popular Wicked Cool Shell Scripts after they’ve been rearranged with and can be found on Twitter as @DaveTaylor and more generally the sorthand function: at http://www.DaveTaylorOnline.com.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 47
LJ226-Feb013-bu.indd 47 1/23/13 1:06 PM COLUMNS HACK AND /
Raspberry KYLE RANKIN Strudel: My Raspberry Pi in Austria A $35 Raspberry Pi is now as powerful as my first colocated server. Find out how I tweaked the OS and racked it in a data center.
I remember my first colocated Irssi-in-a-screen session. The server rather fondly. It was a machine served me for more than 1U Supermicro that had been five years until I ultimately replaced decommissioned from my employer it with newer hardware. after a few years’ service. Although Fast-forward to today, and it was too old and slow for my although my primary server has company, the 800MHz CPU, 1GB significantly more resources, I just RAM and 36GB SCSI storage was finished colocating a new server, perfect for my needs back in 2005. again for free, and again with similar A friend was kind enough to allow resources as my old Supermicro: me to colocate the server at his 900MHz CPU, 256MB RAM and facility for free. So, after a lot of 40GB Flash storage. This time planning, I installed and configured though, the server is a Raspberry Pi, Debian, generated SSH keys and set and the facility is located in Austria. IPs so I could manage this machine In this article, I explain how I was remotely. Once it was colocated, able to colocate a Raspberry Pi and it became my primary server for the steps I went through to prepare Web, DNS, SMTP and my perpetual it for remote management.
48 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 48 1/23/13 1:06 PM COLUMNS HACK AND /
Although I certainly prefer servers that provide remote lights-out management, beggars can’t be choosers, so many of my personal servers have had to make do with SSH and the ability to have someone cycle the power.
The Deal for my IP information. When I first saw the deal advertised, I couldn’t believe my eyes. An The Setup Austrian colocation facility, EDIS I’m no stranger to colocating servers GmbH, was offering free Raspberry without remote management. Pi colocation. Although I was a bit Although I certainly prefer servers skeptical, I carefully read through that provide remote lights-out the fine print, but it was pretty clear. management, beggars can’t be If you set up an account, the EDIS choosers, so many of my personal GmbH folks would send you network servers have had to make do with SSH information for your Raspberry Pi. and the ability to have someone cycle Once you configured the network the power. Although I wasn’t sure settings, you would send the how I would use the server, I did know Raspberry Pi along with the SD card, I wanted to keep the OS relatively USB cable and optionally a small USB lightweight. I also didn’t want to take thumbdrive and they would rack it too many chances with a machine I and provide 100Gb/month of traffic would have little access to, so I went on a 100Mb connection. They even with the standard Raspbian “wheezy” offered free remote power cycling of Debian distribution linked to on the the server as long as you were fine Raspberry Pi download page. There with waiting 24–48 hours. I figured already is plenty of documentation the worst that could happen is that on how to set up Raspbian, so I don’t I’m out a $35 Raspberry Pi and some go into that here. Instead, I focus on Flash storage, so I signed up and set the changes I made to the distribution aside a Raspberry Pi, 8GB SD card before I shipped it off. and 32GB thumbdrive while I waited Because Raspbian assumes you will
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 49
LJ226-Feb013-bu.indd 49 1/23/13 1:06 PM COLUMNS HACK AND /
run a desktop, it splits the available so I could ssh in to it. RAM with the GPU. Of course, on a After I confirmed that I could log server, you need the RAM for your in as my user and sudo to root, I services, so the next thing I did was modified /etc/ssh/sshd_config and run sudo raspi-config to launch changed PasswordAuthentication an ncurses interface that let me tweak to no, so I wouldn’t have to worry some of the hardware defaults. I about SSH brute-force attacks. Then, ended up allocating only a little bit once I confirmed I could still ssh in, of RAM to the GPU, and while I was I deleted the pi user and removed its in the interface, I figured it wouldn’t home directory: hurt to expand the root filesystem to fill my SD card, overclock the $ sudo deluser --remove-home --group pi Raspberry Pi to 900MHz, change my locale and time zone, and change the Now that my user was set up, boot behavior so the desktop didn’t the next step was to remove all the start at boot. desktop packages I would no longer The default Raspbian image ships need so I would have extra space for with a standard user and known any services I wanted to install. There password. I didn’t want anyone to wasn’t really a science to this; I just log in to my server except for me, so tried to pick base desktop packages the next thing I did was add my own I thought would have a lot of other user account: desktop dependencies to remove:
$ sudo adduser greenfly $ sudo apt-get remove x11-common openbox-lxde omxplayer
➥openbox libgtk2.0-common lxde-common xarchiver Then, I edited the /etc/group file as root, and anywhere I saw the pi Configure Bulk Storage user, I added my greenfly user to the The base OS for the Raspberry Pi list. In particular, you would want to was on an 8GB SD card. I wanted add your new user to the sudo group, the option to have more storage, because the default sudoers file on and the folks at colo facility stated the distribution gives any members of they would allow external USB that group full sudo privileges. At this drives as long as they were less than point, I also used ssh-copy-id to 4cm long. I had a 32GB USB stick copy my public SSH key to this server that fit that profile and that showed
50 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 50 1/23/13 1:06 PM COLUMNS HACK AND /
up as /dev/sda when plugged in, so to test the colocated network I then proceeded to partition it and settings truly, I didn’t want to go format it: to the trouble, so the very last change I made to the system was the $ sudo fdisk /dev/sda network settings. Before that point, I $ sudo mkfs -t ext4 /dev/sda1 rebooted and updated the Raspberry Pi a few times and made sure I was One thing I didn’t do was add the still able to log in. Once I was ready, disk to my /etc/fstab. I didn’t want I edited /etc/network/interfaces to risk the server stalling in the boot and changed my eth0 network process either because the USB drive configuration from dhcp to static was unplugged or had failed, so I (IPs changed to protect, well, me): decided to add the mount statement to the end of /etc/rc.local. auto eth0 While I’m are talking about iface eth0 inet static /etc/fstab, I also decided I should address 151.236.x.x set up some swap storage for the netmask 255.255.255.0 device. I don’t plan on needing gateway 151.236.x.1 swap, but I didn’t want an out-of- memory issue crashing the server. What I Wish I Would Have Done Unlike with a traditional server, on It turned out it cost me only around Raspbian, the recommendation is $5 to ship the Raspberry Pi from to use dphys-swapfile to create California to Austria with the US a swap file that it takes care of postal service. Of course, the moment mounting for you: I dropped it off, I started thinking about all the things I should have $ sudo dphys-swapfile setup done. In particular, there are two things I wish I would have done. By default, it picks a swap file size First, I wish I would have set up it feels is optimal for your system, but a system so that the Raspberry Pi you always can edit /etc/dphys-swapfile automatically e-mails me whenever and change the size. it boots. That would have gone a long way toward helping with The IP Change my impatience while I waited for While I could set up a local network the server to be racked. Instead,
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 51
LJ226-Feb013-bu.indd 51 1/23/13 1:06 PM COLUMNS HACK AND /
all I could do was ping the IP and any dishonest person along the way hope I set up the network right. To could have pocketed the Raspberry set it up to e-mail me, all I would Pi and said it was lost in shipping. have had to do is install the postfix You may be wondering what I’m package and during install time, going to use this Raspberry Pi for configure it to be a standalone after all. Since there isn’t much Internet mail server. Then I could redundancy, I’m not going to host install the mailutils package and anything critical on it; however, I’m add a mail command near the end considering what kind of redundancy of my /etc/rc.local file: I could get if I partnered up and shared resources with a fellow echo "I'm alive!" | mail -s "I'm alive!" [email protected] Raspberry Pi colo customer. At the moment, I’m just using it to provide Then before I changed the a network sanity check so I can network settings at home, I could perform network troubleshooting have rebooted the system a few from outside the US. Beyond that, times and confirmed that the mail I have set up postfix and nginx was sent. on it and plan to run some sort The second thing I wish I would of rudimentary Web service and have done is pay extra for a tracking possibly backup DNS. Keep an eye number! I was actually pretty angry on this column for updates as I start with myself for forgetting to do to add services to it.■ this. Not only could I have known where the Raspberry Pi was while Kyle Rankin is a Sr. Systems Administrator in the San Francisco it was shipped, I also would have Bay Area and the author of a number of books, including The known when it arrived at the colo. Official Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks. Furthermore, without any tracking, He is currently the president of the North Bay Linux Users’ Group.
Resources
The EDIS GmbH Raspberry Pi Colo Product Page: https://manage.edis.at/whmcs/cart.php?gid=6
Raspberry Pi OS Download Page: http://www.raspberrypi.org/downloads
52 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 52 1/23/13 1:06 PM The Official Conference, Trainings, and Expo of The Apache Software Foundation ...... APACHECON NORTH AMERICA 2013 Open Source Community Leadership Drives Enterprise-Grade Innovation
Join us! February 24th - March 2nd 2013 Portland, Oregon
Conference Website: Sponsorship inquiries send to: na.apachecon.com [email protected]
10% Discount with PROMO CODE LNXJ
Presented for The Apache Software Foundation by: The Open Bastion ......
LJ226-Feb013-bu.indd 53 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
Trying to SHAWN POWERS Tame the Tablet Don’t let your fancy new tablet collect dust!
Like many folks, I received a shiny with a tablet computer, I might be new Nexus 7 tablet for Christmas. This able to do away with most of the brought me great joy and excitement paper in my life. I have cabinets full as I began to plot my future paperless of filed papers that I never use. I life. For most of the evening and an do, however, search my e-mail on a hour or so the next day, I was sure regular basis for communications sent the new Android tablet would change or received years ago. I want that my life forever. Sadly, it wasn’t that same accessibility for items that exist easy. This month, I want to dive head only in paper form now. first into the tablet lifestyle, but I’m not sure if it’s really the lifestyle for Paperless: Evernote or Dropbox me. I’ll try to keep everyone posted I’ve been trying to go paperless since during the next few months (most long before I got a tablet computer. likely in the Upfront section of LJ). There seems to be two schools of And please, please don’t hesitate to thought in the paperless department. send me messages about the ways There are the Evernote people, and you find your Android tablet useful there are the “every-other-kind” of at work/home/play. people. I have Evernote on every electronic device I own (which is a At Work significant number), and I have to The main reason I decided on the admit, for raw information, Evernote Nexus 7 was because with the is amazing. The problem comes with leather case I bought for it (Figure documents. Granted, documents 1), it was small enough to carry to can be added to an Evernote note, meetings easily, yet big enough to but they are like e-mail attachments, view full-size documents. I figured and they can’t be modified once
54 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 54 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
Figure 1. My case doubles as a stand.
attached. This means, at least for me, means I have an inconvenient that the only documents I ever attach combination of Evernote for are “complete” documents that are unformatted information and Dropbox printed as PDF files. for documents. Thankfully, both I don’t have a good solution for applications run very well on Android, how to handle Word/LibreOffice so although I don’t have a central documents in Evernote. So, that repository for all my information, at
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 55
LJ226-Feb013-bu.indd 55 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
After buying a stylus, coming up with a note-taking application proved to be difficult.
least I can access all the information this isn’t a big issue for me. I just from my tablet. whip out my phone if I need to import something optically into Evernote. Getting Data In My biggest hope with the Nexus 7 Evernote includes a really nice was that I could avoid toting around mechanism for using a device’s legal pads and pens to meetings. camera for importing digital snapshots I tend to take “doodle” notes, so of documents, notes, whiteboards a laptop really isn’t ideal for me at and so forth. Unfortunately, the a meeting. (Plus, I tend to become Nexus 7 doesn’t have a rear camera. distracted with a laptop and multitask Thankfully, my cell phone has a really my way into trouble quite often.) I nice camera, and it also has Evernote researched capacitive styli and found installed. Because I never intended the New Trent IMP62B to be just my tablet to replace my cell phone, about the best option (Figure 2). It’s
Figure 2. This stylus is remarkably precise given the size of its tip.
56 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 56 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
less than $10, and it’s remarkably precise for a stylus with a rather bulbous tip. After buying a stylus, coming up with a note-taking application proved to be difficult. I almost can get there with a couple apps, but nothing has been the ideal option for me. The closest I’ve come to perfection is Lecture Notes, which has some critical features:
n Importing PDF files from Dropbox for annotation during a meeting (for example, an agenda).
n Exporting directly to Evernote.
n Very fine lines when writing.
n Simple interface for changing pens, erasing and so on. Figure 3. Lecture Notes is a great I’ll admit, it’s still not as fast as application if you want to take notes writing on paper, but for some quick with a stylus. doodles on a PDF agenda, Lecture Notes does a nice job (Figure 3). point, however, I might as well just My wife actually likes to type on her use a laptop. In a pinch, it’s certainly tablet (an iPad Mini) with the onboard possible to type a few notes with the keyboard. If she’s taking notes, she’ll on-screen keyboard, and if you don’t just open up Google Docs and type have a laptop, a Bluetooth keyboard on the screen. For me, typing on any will help manage some serious screen is awkward and slow. If I have typing. Still, I don’t recommend it. to do any real typing on my tablet, Any Nexus-size keyboards are too I’ll use a Bluetooth keyboard. At that small to type well with, and any
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 57
LJ226-Feb013-bu.indd 57 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
Figure 4. My printer has native Google Print support, but it’s possible to set up a traditional printer.
full-size Bluetooth keyboards are Google Drive. cumbersome to carry around. Speaking of Google Drive, the native Google application does a Printing and Viewing decent job of creating Microsoft- Just a couple years ago, it was compatible Office files. The newest absurd to think about printing from version of Drive even allows editing a phone or tablet. Now, it’s easy to and creating spreadsheet files! When set up network printing for Android combined with Android’s built-in file devices, and Linux users easily can viewer, it’s difficult to find a document share printers with iOS devices Android can’t read. I’ve never been as well. So printing, interestingly stuck in a meeting unable to view an enough, is fairly ubiquitous. Figure 4 e-mail attachment, which would be a shows an example of printing from real showstopper for me at work.
58 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 58 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
Geeking It Up areas of your building, WiFi If you’re stuck wearing a tie and Analyzer is amazing. attending meetings all day, the above information might be all you’re n WiFi Map Maker: I had never heard interested in. For me, although I of this application, but a reader attend more meetings than I care (Roman, I won’t mention his last for, I also have the opportunity to name out of respect for his privacy) be a geek. A tablet computer offers sent me information on it. If you some really great apps for system need to make a quick-and-easy administrators or just geeks on map of Wi-Fi hotspots, this is hard their lunch break. Here are some to beat. It uses the built-in GPS on of my favorites: your tablet to create a thermal map of Wi-Fi coverage in real time. n ConnectBot: this is the de facto standard SSH client for accessing n SplashTop: now that SplashTop remote servers. As with typing long supports controlling Linux documents, the on-screen keyboard workstations along with Windows can be frustrating for more than and OS X, it’s become a whole a few quick server tweaks, but lot more usable for me. Using its the program itself is awesome. If custom application installed on you’ve ever SSH’d into a server your computer, SplashTop allows on a cell-phone screen, the 7" of remote control of workstations with real estate on the tablet will be incredible responsiveness. It’s a bit a godsend. No geek is complete like VNC simplified and on steroids. without a command-line interface, Heck, it’s even possible to play PC and ConnectBot provides remote games over the connection! (Not access to one. that you’d ever do that at work.)
n WiFi Analyzer: I’ve mentioned this At Home: a Boy and His app before in Linux Journal, and Recliner—and Tablet rightly so. It does exactly what’s I don’t think I’ve watched a television on the tin: it analyzes the Wi-Fi show or movie at my house in the past networks in your area. Whether decade without a notebook computer you want to find an open channel sitting on my lap. Whether it’s to look or check signal strength in different up an actor on IMDB or to catch up on
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 59
LJ226-Feb013-bu.indd 59 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
RSS feeds during the boring scenes, worst interface for finding a book in an on-line connection has become a your collection, it has some features requirement for me in my recliner. In that I find indispensable: this case, I’ve found the Nexus 7 to be a decent replacement for a full-blown n With the “Personal Documents” laptop. Not only can I do all the things feature Amazon offers, any I normally do with my laptop, but I also DRM-free ebook can be e-mailed can use an XBMC remote application and stored on Amazon Cloud. They to control the TV. If I happen across a can be retrieved from any Kindle cool on-line video, I can send it to my device or app (excluding the Cloud XBMC unit quickly with iMediaShare, Reader, but I don’t read books on which uses Apple’s AirPlay technology my computer screen anyway). to stream video directly to the TV. It gives me a certain level of satisfaction n WhisperSync used to work only to stream video from an Android device on Amazon-purchased materials, to my Linux nettop running XBMC using but now it works on Personal an Apple protocol, yet having no Apple Documents too. This means I can hardware or software in the mix. Truth pick up my cell phone to read a be told, it works a lot more consistently few pages at the doctor’s office, than the Apple TV and actual AirPlay and then pick up my tablet later does. iMediaShare has both a free and and automatically be right where paid version, which are available on the I left off. Because this works across Google Play store. platforms, it makes the Kindle One thing I never do on my laptop reader my go-to app. is read books. Even though I can read countless Web articles on the n I keep my DRM-free e-book computer, for some reason, I can’t collection at home on Calibre. With bring myself to read actual book- Calibre’s export feature, sending a length material. With the tablet on my book to a specific Kindle device’s lap instead of a laptop, flipping open e-mail address is a single click away. the Kindle app allows me to read a few pages of a book if there’s nothing I really do wish Amazon would interesting on TV. Why the Kindle improve the browsing interface for app? I’m glad you asked. As it turns Android devices. I suspect Amazon out, even though it has the absolute is trying to push people into buying
60 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 60 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
a Kindle Fire, however, since it the Nexus 7 is awesome. I’m not also won’t release the Amazon much of a gamer, but as it happens, Prime streaming app. Oh well, the that’s exactly the type of person tablet WhisperSync feature makes all the games are made for! If I want to play difference for me, and I’m willing a quick game of Solitaire or even to suffer a cruddy interface when shoot a couple zombies, the tablet opening a book. interface is perfect. Entertainment doesn’t stop with Pure, Down-Home Entertainment games, however. I’ve mentioned Plex The tablet size and touchscreen really in recent issues of Linux Journal, but do make it a perfect device for simple it bears mentioning again. If you gaming. Whether you want to sling have a collection of videos on your Angry Birds at a bunch of pigs or use home server, Plex will transcode and the tablet like a steering wheel to stream them to you anywhere. It drive your 4x4 across rough terrain, works at least as well as the AirVideo
LINUX JOURNAL on your Android device Download app now in the Android Marketplace
www.linuxjournal.com/android
For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or [email protected].
LJ226-Feb013-bu.indd 61 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
application on iOS, and the server stream a book through my car’s audio component works excellently on a system. I find traffic jams much more headless Linux server. When you add palatable now that it means more Netflix, Hulu Plus, Smart Audiobook time for “reading”. Player, Pandora, Google Music, I’ve also found Google Map’s Amazon MP3 and the ability to store ability to download maps for off- local media, it’s hard to beat the line use to be awesome. I opted Nexus 7 for media consumption. to get the Wi-Fi-only model of the Nexus 7, so when I’m in the car, I And in between Work and Home don’t have Internet connectivity. One place I never expected to use my My car doesn’t have a navigation tablet was in my car. No, I don’t read system, so the 7" screen and off- books or watch videos during the line maps make for an incredible daily commute, but I certainly enjoy GPS system. Google’s turn-by-turn listening to audiobooks. With its built- navigation is amazing, and the nice in Bluetooth connection, I happily can big screen means it’s more useful
Resources
Dropbox: http://www.dropbox.com
Evernote: http://www.evernote.com
New Trent IMP62B Stylus: http://www.newtrent.com/stylus-pen-imp62b.html
Google Drive: http://drive.google.com
ConnectBot: http://code.google.com/p/connectbot
WiFi Analyzer: https://sites.google.com/site/farproc/wifi-analyzer
Dave’s Apps: http://www.davekb.com/apps
SplashTop: http://www.splashtop.com
iMediaShare: http://www.imediashare.tv
XBMC: http://www.xbmc.org
62 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 62 1/23/13 1:06 PM COLUMNS THE OPEN-SOURCE CLASSROOM
than my phone’s GPS. I don’t have a any time soon, but I do think they great way to mount the tablet in my have a place in our daily lives. car yet, but I suspect with a bit of Hopefully this article gets you started Velcro, it won’t be a big problem. with integrating a tablet computer into your everyday life. I look forward Where to Go from Here? to hearing about and sharing your I’ve given you a glimpse at how I use experiences, so please write me at my tablet on a day-to-day basis. I [email protected].■ hesitated to do this though, because I don’t feel I’m really using the Nexus Shawn Powers is the Associate Editor for Linux Journal. He’s 7 to its fullest potential. Based on a also the Gadget Guy for LinuxJournal.com, and he has an few conversations I’ve had with fellow interesting collection of vintage Garfield coffee mugs. Don’t let readers, however, I don’t think I’m his silly hairdo fool you, he’s a pretty ordinary guy and can be alone. I don’t think tablet computers reached via e-mail at [email protected]. Or, swing by will replace desktops or even laptops the #linuxjournal IRC channel on Freenode.net.
LINUX JOURNAL now available for the iPad and iPhone at the App Store.
linuxjournal.com/ios
For more information about advertising opportunities within Linux Journal iPhone, iPad and Android apps, contact John Grogan at +1-713-344-1956 x2 or [email protected].
LJ226-Feb013-bu.indd 63 1/23/13 1:06 PM NEW PRODUCTS
AdaCore and Altran Praxis’ SPARK Pro SPARK Pro is a product jointly developed by Altran Praxis and AdaCore that provides the language, toolset and design discipline for engineering high-assurance software. The developers say that the new version 11 of SPARK Pro offers many enhancements related to the way that functions and proof functions are handled. These changes are said to improve project efficiency by eliminating the vast majority of rules that previously were encoded manually. The main changes include a more powerful language for specifying proof functions and the ability to use the functions in any proof context. This greatly simplifies the task of writing and maintaining functional contracts for critical software, providing high assurance at lower cost. SPARK Pro combines Altran Praxis’ SPARK language and verification tools with AdaCore’s GNAT Programming Studio and GNATbench Integrated Development Environments. There are SPARK versions based on Ada 83, Ada 95 and Ada 2005, so all standard Ada compilers and tools work out of the box with SPARK, say the companies. http://www.altran-praxis.com and http://adacore.com
Wolfram Research Mathematica Wolfram Research calls its Mathematica application, recently upgraded to version 9, “the broadest, deepest computation system in the world”. Mathematica 9 adds more than 400 functions in new and expanding application areas and also introduces the Wolfram Predictive Interface. The latter, intended to help users fully utilize Mathematica’s vast scope and depth, is a suite of features that intelligently suggests what to try next based on sophisticated heuristics and data from millions of queries from the Wolfram|Alpha site. Other new features include highly integrated units support; major new data science, probability and statistics functionality; full R integration into the Mathematica workflow; 3-D volumetric image processing and others. Supported platforms include Linux x86, Windows and Mac OS X. http://www.wolfram.com
64 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 64 1/23/13 1:07 PM NEW PRODUCTS
Opengear’s ACM5504-5-G-W-I Gateway Improving the management of critical infrastructure in remote locations is what the new Opengear ACM5504-5- G-W-I Remote Infrastructure Management (RIM) Gateway is all about. A new member of Opengear’s ACM5500 product family, this sibling offers IT managers a wide range of connectivity options with its new integrated wireless access point (Wi-Fi 802.11 b/g/n) to complement the cellular, wired and wireless access already present in other gateways. Besides enabling direct management of Wi-Fi-enabled devices, the solution provides the option of convenient wireless access to the management network using mobile devices, such as tablets and smartphones. All RIM gateways in the ACM5500 product family provide serial console-port connectivity, environmental monitoring, power management and monitoring and remote site storage of off-line logs and running configuration files. http://www.opengear.com
Jon William Toigo’s Office Automation 2.0 (Apress)
Those in our midst who look after enterprise-wide IT planning should sneak a peek into Jon William Toigo’s new book Office Automation 2.0. The Apress title is an essential guide to office automation in the post-PC era and helps businesses assess such technologies as virtual desktop infrastructure, mobile clients and cloud services in terms of their practical applications to streamlining workload. Toigo emphasizes that rollouts of the latest enterprise-class technologies cannot produce business value unless management ensures that the front office is trained to use them correctly, and that end-user practices and IT processes are dynamically and efficiently coupled in the organizational culture. Toigo also provides practical guidance for innovative managers who are seeking to make every automation investment dollar count toward the three key metrics of business value: cost-containment, risk reduction and improved productivity. http://www.apress.com
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 65
LJ226-Feb013-bu.indd 65 1/23/13 1:07 PM NEW PRODUCTS
Philatron Wire and Cable’s Flexy CordT
Traditional extension cords can waste you a lot of time, either by having to unwind and rewind them repeatedly, or worse, when you struggle trying to untangle a mess of wire. The answer to this struggle, says Philatron Wire and Cable, is not in the numerous cord-reeling and -wrapping devices on the market, but rather in the company’s new Flexy CordT, which it bills as the “21st Century extension cord”. Flexy CordT line of extension cords are designed with a reduced cord and increased coil diameters, which gives them suppleness properties similar to the Slinky toys. In addition, Flexy Cords are developed with specially engineered materials with “memory” (so they “remember” their original retracted length) and are tangle-proof and kink-proof. Flexy Cords are available in different compact lengths: 4 inches (extending to 8 feet), 5 inches (extending to 10 feet), 10 inches (extending to 20 feet) and 20 inches (extending to 45 feet). http://www.flexycord.com
MetaCase’s MetaEdit+ The new v5.0 release of MetaEdit+ from MetaCase adds a wide range of features to the company’s flagship software development tool. MetaEdit+ is aimed at expert developers who seek to create graphical domain-specific languages and code generators rapidly. MetaCase states that the rich graphical notations go beyond plain icons and links; they can change on the fly depending on model data, be nested to unlimited depth, be retrieved from libraries, and they have a fixed or dynamically varying number of ports to which to connect. These new features allow domain-specific models to mimic closely the problem domains they describe. The new version integrates into programming tools like Visual Studio and Eclipse. Software developers get one-click access from their IDE to MetaEdit+ models, can integrate generated code with hand-written code and libraries, and automate their build process. Versions for Linux, Windows and Mac OS X are available. http://www.metacase.com
66 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 66 1/23/13 1:07 PM NEW PRODUCTS
TeamViewer TeamViewer is a popular remote-control and on-line meetings application for Linux, Windows and Mac OS X. The most notable feature in the new version 8 is the TeamViewer Management Console, a cloud- based administration tool that offers a wide range of capabilities that addresses the needs of corporate environments, with emphasis on accountability, stricter security guidelines and the need for central control of user accounts. Other major new capabilities of TeamViewer 8 include connection reporting of all sessions and browser-based single-click connections. New features that reflect the latest demands of telecommuters include session handover, remote printing, deeper Microsoft Outlook integration, transmission of remote sound and video and enhanced session recordings. http://www.teamviewer.com
Investintech.com’s Able2Extract PDF Converter You might call Investintech.com’s Able2Extract PDF Converter the Swiss army knife of PDF converters. Not only is Able2Extract able to convert PDFs to a wide range of formats accurately, but it also features the unique ability to work across Linux (Ubuntu and Fedora), Mac OS X and Windows platforms. Investintech.com boasts that Able2Extract keeps intact all aspects—images, colors, formatting and fonts—regardless of file format. Supported formats include converting PDF to OpenOffice.org, MS-Office, AutoCAD and commonly used image formats. In addition, users can focus on the content they need by selecting conversion down to a single sentence. http://www.investintech.com
Please send information about releases of Linux-related products to [email protected] or New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 67
LJ226-Feb013-bu.indd 67 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox ConVirt the New Tool in Your Virtual Toolbox Can you manage different hypervisor platforms from a single pane of glass? Yes, you can. JERAMIAH BOWLING
irtualization is now a staple news is that inexpensive open-source of the modern enterprise. As virtualization is on the rise, driven in V more and more shops switch large part due to its low performance to the virtual paradigm, managing overhead. However, one of the primary those new virtual resources is a critical obstacles to large-scale open-source part of any deployment. For admins virtualization adoption has been the using Microsoft- or VMware-based lack of robust management tools. virt- hypervisors, powerful management manager is the most well known and tools are available to keep their virtual used, and although it’s a great tool, it houses in order. Unfortunately, those does not hold a candle to the enterprise products and their accompanying tools tools put out by the big vendors. That’s come with a hefty price tag. The good where ConVirt comes in.
68 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 68 1/23/13 1:07 PM ConVirt is an open-source tool is roughly 2–3GB in size), although capable of managing multiple types the appliance probably will get you of hypervisors including Xen, KVM up and running faster. Make sure that and now VMware from a single pane whichever installation method you of glass. When evaluating ConVirt select, that you open all the necessary for your needs, it is best to think of ports on your CMS and on your it as a front end to the native tools managed servers/hosts that you want of the hypervisors that provides to manage through the console (TCP extended features not available in a 8081, 8006, VNC ports and SSH). standalone hypervisor. Although there The term “managed server” refers is some overlap with virt-manager, to those hosts running a hypervisor ConVirt adds an additional level of that is managed by ConVirt and can enterprise manageability. ConVirt is be used interchangeably with the currently offered in three tiers: Open term “host”. Follow the installation Source, Enterprise and Enterprise procedures available on the Convirture Cloud. This article focuses on the Wiki site to perform the installation open-source version. The open-source of the CMS. Most of the install is version does not include the ability handled by a script that pulls down to manage VMware items, so the the dependencies and installs MySQL. testing environment for this article I won’t go into finer detail on the contains only Xen and KVM servers. server install, as it is well documented Even though I don’t cover it here, the on the site and I would just be ability to manage VMware hosts along repeating the information here. with KVM and Xen hosts from the After the CMS install is complete, same pane of glass should peak the access your management page at interests of many admins. http://youripaddress:8081 (Figure 1). Let’s get started by installing the Use the default login of “admin/admin” ConVirt Management Server or CMS. to bring up the main console. For ConVirt can be installed on most those used to VMware’s vSphere, flavors of Linux or as a pre-configured this interface will feel familiar. The virtual appliance that can be imported layout is broken into three main into a KVM or Xen server. I chose to panels: a navigation panel on the deploy my CMS on a physical server left, a display panel for selected running CentOS 6.2 to allow plenty items in the middle of the page and of storage space (the virtual appliance a panel at the bottom for displaying
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 69
LJ226-Feb013-bu.indd 69 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
Figure 1. The Main Login Screen
task results (Figure 2). Pools that group together like The navigation pane is logically managed servers that share common divided into a tree with your Data items like storage and virtual network Center at the top with Server Pools configurations. Managed servers and Templates listed underneath it. are placed in the server pools along This outline reflects how resources with any guests/VMs that reside on are organized in ConVirt: Data them. Templates fall into their own Center→Server Pool→Managed category, but also are available from Server (host)→Guest. Your Data the navigation pane. Templates are Center is the top-most delineation pre-configured groups of settings of your virtual environment. It could used at provisioning time to carve up/ be a site or an organizational unit. define the virtual resources available Under the Data Center are Server to new guests (processors, memory,
70 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 70 1/23/13 1:07 PM Figure 2. First View of the Data Center
storage and NICS). download the ConVirt tool from the The next step in your deployment site and install any dependencies. is to prepare your hosts to become Then configure SSH on each managed servers. Specific hypervisors managed/server host for root access, have individual requirements before and finally, run the convirt-tool being added to the CMS, but the setup command. Debian/Ubuntu process for preparing each host is users should note that you will roughly the same for each. Create need to set a password on the root a network bridge on each host, account manually in order to manage
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 71
LJ226-Feb013-bu.indd 71 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
any hypervisor from the CMS. I also suggest that you name any bridges you create with identical names (for example, KVM=br0, Xen=Xenbr0), as this helps standardize your guests’ networking options. For this article, I created two KVM servers and one Xen server to manage with ConVirt. With the hosts prepared, you now can add them to the CMS. This starts by adding hosts to a server pool. You can use the pre-configured Server Pools (Desktop, Server, QA Lab) or create your own. I created an additional pool to play with that I named “Production”, and in case Figure 3. Our New Server Group I messed anything up, it wouldn’t affect the default pools. When you Continue adding all of your hosts have your pool selected, right-click as managed servers to the console on it and select Add Server. On the until they have all been added. You resulting screen, select your platform, then can import any pre-existing either Xen or KVM, and fill in the VMs on your hosts by right-clicking hostname or IP address. the managed server and selecting If you have not configured SSH for Import Virtual Machine Config Files. root access on the host, the server will You also might notice from this same fail. If the server is added successfully, menu context that you can move it now should display under the servers between pools. This feature is server pool you chose with a little useful during organizational changes K (KVM) or X (Xen) icon (Figure 3). or when moving test servers into a Click on the newly added server to production environment. Be aware see performance information about that moving a server between pools your host displayed in the center pane also moves any that reside on it, (Figure 4). From this display, you also so be aware of any configuration can view the number, type and status changes that might be applied by of the guest running on the host. moving your server/guests into the
72 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 72 1/23/13 1:07 PM Figure 4. Real-Time Performance Stats on One of Our KVM Servers
new pool. You also are required to Out of the box, ConVirt has two power down any running guests pre-configured templates for use before moving the server. with provisioning. These templates Because I already have covered contain common configuration how to add existing guests to settings for a specific OS installed managed servers, let’s create a new from a CD. Provisioning from the guest from a template (this also is built-in templates is easy. Simply called provisioning). To get a feel for right-click a template, and select all of your options, let’s provision a Provision to create a guest on your guest VM from CD as well as clone selected managed server. a guest from a golden image using a For this example setup, let’s create reference disk. a Linux desktop from the existing
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 73
LJ226-Feb013-bu.indd 73 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
Figure 5. Provisioning a Guest from the Linux Template
Linux CD template. After clicking on Once the guest VM appears under the Provision, you are asked on which host, power it up by right-clicking on server to place the new guest VM, the new guest and clicking Start. and then you’re prompted to provide If you do not want to use CDs, you a name for it (Figure 5). ConVirt also have the option to boot from an then creates a guest based on your ISO file. To do so, change the path name and creates a 10GB virtual of your /dev/cdrom to an accessible hard drive and maps the guest to the ISO file (Figure 6) in the settings of a physical CD/DVD of the host on which template or the guest itself. Once the it’s provisioned. VM has been started, right-click on it Next, insert your physical install and select View Console. If you have a media on the host’s physical drive. Java-enabled browser, you can access
74 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 74 1/23/13 1:07 PM Figure 6. Mounting an ISO to the Guest CD-ROM
the new VM’s desktop via the Web custom machines or one-off builds, console, or if you choose another VNC but if you have to spin up multiple client, ConVirt will display the IP and guests at once, it is a very inefficient port required to access the VM. If you method. It is much more efficient to prefer to administer your host via SSH, create a single VM and clone it over you also can launch a session from the and over again, which is possible in guest’s right-click context menu. ConVirt. To demonstrate this method Provisioning from CD is nice for of provisioning, I created a pristine
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 75
LJ226-Feb013-bu.indd 75 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
Figure 7. Provisioning Settings to Clone the Golden Image
(or “golden”) image of a Windows XP location. In my case, I copied it to an machine. This VM contains all of the NFS share mounted on the ConVirt settings and software needed so that and all of my managed servers. You I don’t need to make changes to each then need to gzip your .xm image new VM that is spun up. After the in its new location to give it a .gz golden image is ready, power it down extension. Next, copy the Windows in the hypervisor or ConVirt, and CD template by right-clicking it in the copy the guest’s .xm file to a separate templates section and clicking on the
76 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 76 1/23/13 1:07 PM Create Like option. best practice to keep your golden You could create a template from image as small as possible to fit as scratch, but copying and modifying a many different size drives (virtual or built-in one is just as quick. If you have physical) that you will deploy it to. very custom settings that differ greatly After your deployment is in place, from those found in the pre-built you may find that you need to move templates, that may be the way to go. guests to another host to balance When prompted, give your template loads between servers, to move a VM a new name. Once the new template from one network site or segment to appears in the list, right-click on it another or to perform maintenance and select the Edit Settings option. on a host with zero downtime to Click on the Storage option and running guests. VMware dominated remove the existing storage defined the market for years with its vMotion for hda. Click on the New button feature that performs this task well. at the top of the window. On the ConVirt provides this same operation. resulting window, set the Option field Note that in order to migrate to Clone Reference Disk. Change the running guests between hosts, both Ref. Disk Type to Disk Image and the hosts must have access to the same Ref. Format to .gzip. In the Ref Loc. shared storage. You may run into field, browse or enter the path to other limitations when migrating your ISO file. Change the VM Device: guests, such as both hosts must have field to “hda”. Your settings should the same processor type and/or must resemble those shown in Figure 7. be on the same hypervisor platform To deploy a new cloned VM from (like KVM or Xen), so plan accordingly. this template, right-click it and select I was unable to determine whether Provision. With the reference disk this was a technical limitation or an method, ConVirt will copy the .gz unlocked feature in the Enterprise file to its destination and expand it version of ConVirt. Either way, to the desired size of the new VM. there are some native tools in the What is really nice is that you can hypervisors that can convert foreign specify a larger disk size than the disk/VM types for importation into one inside your golden image. On their native platform. After you have my XP VMs, the space automatically met all the prerequisites, migrating is was added to the guest partition (not as simple as right-clicking the guest usually an easy task). It is a common and selecting your destination server.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 77
LJ226-Feb013-bu.indd 77 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
storage at the Data Center- level and then attach it to Server Pools, which gives you the ability to mix and match your storage among the pools. Be aware that for all servers in the pool to use the storage, they must connect to the storage using the same logical path (like migration). I found this feature incredibly useful as it really simplifies assignment of any networked Figure 8. Shared Storage Details storage resources you have in You can monitor your migration task your environment (SAN, iSCSI or NFS). in the bottom pane of the console. You also can set certain provisioning One last feature I want to mention settings at the pool level that override is ConVirt’s management of shared those in a template. This means you storage, because I think it is useful can provision the same template with (Figures 8 and 9). With the designer’s multiple storage options. This would be tree-based approach to organizing very handy if you have Server Pools in virtual resources, you set shared different sites or different departments,
78 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 78 1/23/13 1:07 PM each that should use their own storage resources. In this article, I’ve touched on many of the nicer features in ConVirt, but now let me talk about some things that are missing. Before doing so, you should recognize that I am comparing apples and oranges when I talk about ConVirt and vendor-produced management tools. Even comparing the Enterprise version of Figure 9. Server Pools That Can Use This Storage ConVirt is not wholly accurate, as ConVirt is meant servers to communicate with the CMS, to manage a heterogeneous virtual which I am sure most admins won’t environment, whereas Microsoft be crazy about. Snapshot support also and VMware are tuned to their own is noticeably missing from the open- homogeneous platforms. source version. There is an option That being said, I still had a few available for the VMs called Hibernate, gripes with ConVirt. The first is that but that takes a snapshot only of the it requires root access to managed running memory not the underlying
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 79
LJ226-Feb013-bu.indd 79 1/23/13 1:07 PM FEATURE ConVirt: the New Tool in Your Virtual Toolbox
disk. The lack of snapshots bothered I am sure ConVirt comes in under me only for half-a-second when I that. I will say that you really need realized it is available in the Enterprise to know your chops when managing version. The last item missing from different hypervisors at the same ConVirt is administrative roles. You time. I am one of those admins do have the ability to create users and who works with vSphere daily, groups in the console, but as far as I and I have become accustomed to can tell, the only thing that gets you a homogeneous environment, so I is auditing of the tasks that take place really had to get under the hood of on the CMS server. It felt like this was both KVM and Xen to make things added into the product in its most go smoothly. That being said, once basic form, but never fully developed. it is in place, I believe it is easier to In the end, these are minor administer by non-Linux IT pros or complaints, as ConVirt provides far admins who need to perform day-to- more utility than the few features it day tasks in their virtual environment lacks. The software really gives you a than virt-manager or command-line lot of flexibility, especially with KVM, tools. Add in the ability to manage a and you can’t beat the price point. multiplatform hypervisor environment, I’m sure those features unlocked in and the value of ConVirt is apparent.■ the Enterprise version (snapshots, high availability and spanned virtual Jeramiah Bowling has been a systems administrator and networks) are worth the money and network engineer for more than ten years. He works for a bring it more in line with the vendor- regional accounting and auditing firm in Hunt Valley, Maryland, produced management offerings. I and holds numerous industry certifications, including the know how much VMware costs, and CISSP. Your comments are welcome at [email protected].
Resources
Convirture’s Main Site: http://www.convirture.com
Installation Guide/Wiki: http://www.convirture.com/wiki/index.php?title=Convirt2_Installation
KVM: http://www.linux-kvm.org
Xen: http://www.xen.org
80 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 80 1/23/13 1:07 PM February 25 - March 1, 2013 Montreal, Canada
1-up your abilities
PHP - Python - Ruby - Java - .NET HTML5 - Javascript - Mobile
Expand your skills with experts start from across the globe. register on confoo.ca before Jan. 20 Explore for a discount diverse technologies with 160 presentations. Follow @confooca Experience the best of web community and culture.
Sponsored by:
LJ226-Feb013-bu.indd 81 1/23/13 1:07 PM FEATURE Linux on Azure Linux on Azure: a Strange Place to Find a Penguin
Running Linux on Azure might sound like a penguin pitching his tent in the depths of Mordor, but this overview shows that it’s a pleasant computing environment.
Andrew Fabbro
inux enthusiasts might think hobbyist, I suppose for the same the idea of running a Linux reason people climb Mount Everest: L virtual machine on Microsoft’s because it’s there. For the business Azure service is like finding a user, the prospect of spinning up penguin sun tanning in the Sahara. Linux VMs in Microsoft’s fabric offers Linux in the heart of the Microsoft new options for collocating open- cloud? Isn’t that just wrong on so source technologies with existing many levels? Microsoft Azure services. For the cloud Why would anyone want to run market in general, more competition Linux on Microsoft servers? For the is good news for consumers.
82 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 82 1/23/13 2:32 PM The Cloud Marketplace followed suit. Microsoft originally Virtual machines in the form of opted for a different, more complex virtual private servers (VPSes) have cloud strategy. Azure was built as a been offered for nearly a decade “platform as a service” offering (see from a galaxy of providers, using the Cloud Flavors sidebar) in which virtualization technologies, such as developers could write applications Xen, Virtuozzo/OpenVZ and KVM. that ran in various roles and talked These providers subdivide a physical to Azure APIs. In theory, this allowed server into multiple small virtual developers to concentrate on code servers. Users typically subscribe on a and not worry about the abstracted monthly basis, with an allotment of hardware underneath. memory, disk and network bandwidth. In practice, developers were Later vendors, such as Amazon, forced to write Azure-centric Rackspace and now Microsoft, offer applications and adoption was the same service with a finer-grained slow. Many enterprises with mixed commitment. Users can spin up a Windows/Linux environments found VM (or a hundred) by the hour, pay that hosting their own self-managed for bandwidth by the gigabyte and servers on Amazon and other cloud utilize more advanced features, such environments was more attractive as private networks, SAN-like storage than spending time porting and features, offloaded database engines debugging their applications. and so on. In 2012, Microsoft added All of this diversity is good news for “infrastructure as a service” (virtual end users. In 2002, a VPS with 128MB machines) offerings to its lineup, cost nearly $100/month. In 2006, you allowing users to run and administer could get a VPS with 512MB of RAM Windows and Linux virtual machines for $40/month. Today, such VPSes they directly control. can be found for less than $5/month Azure virtual machines are still in the VPS market or for pennies per in “Community Preview”, which is hour from cloud providers. Microsoft lingo for “Beta”. Support is limited to forums, and as you’ll see, Microsoft Enters the Market some sandpapering of the offering Amazon enjoyed early success with still is needed. However, after using its Elastic Compute Cloud, and other the service for a couple months, I find vendors, such as Rackspace, soon Linux on Azure to be stable and easy
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 83
LJ226-Feb013-bu.indd 83 1/23/13 1:07 PM FEATURE Linux on Azure
Cloud Flavors
Cloud-based services come in several different forms, depending on what’s being abstracted and provisioned.
n Software as a Service (SaaS): the provider runs an application and exposes an interface to subscribers. This is simply using a Web-based application. Examples include Salesforce.com, NetSuite and 37 Signals’ Basecamp.
n Platform as a Service (PaaS): the provider offers a platform that users can use to build applications. Subscribers write and provide code, which runs on abstracted hardware and software services. Examples include Microsoft’s Azure original offerings, Oracle’s Fusion cloud or Google’s App Engine.
n Infrastructure as a Service (IaaS): the provider delivers virtual machines and other infrastructure pieces that users can configure as they like. Examples include Amazon’s EC2, Rackspace Cloud, Google Compute and Microsoft’s Azure. Virtual Private Servers also are IaaS offerings with a different financial model.
to use, and it performs well. At the in Community Preview, pricing is time of this writing, Microsoft has not slightly discounted compared to set a date for General Availability. expected General Availability pricing. Actual price comparisons for hourly Comparing Azure to Amazon EC2 VMs depend on how long of a term Azure’s chief competition is Amazon commitment you make. For example, EC2, and it’s not hard to see that Amazon offers both spot instances Microsoft patterned its IaaS offering and prepaid reserved instances, while after its rival’s success. Microsoft also discounts longer-term Like EC2, Azure is priced by the commitments. Storage and bandwidth hour, and the rates are similar. While pricing are very similar. In general,
84 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 84 1/23/13 1:07 PM running a VM on Azure will cost you The Azure value proposition is not about the same as EC2, which is “we are a better cloud” but rather probably just what Microsoft planned. “you can do EC2-like things here EC2 offers features Microsoft has alongside your Azure platforms”. not caught up to yet. For example, For shops that have deep Microsoft the underlying storage virtual deployments, were early adopters machine disk (Elastic Block Storage) of Azure or want to develop can be snapshotted to S3 storage. applications that move into and out However, many Amazon services of Azure, the new IaaS offerings will have parallels in the Azure world. be appealing. For example, Amazon’s SimpleDB is analogous to Azure’s Tables. Both Taking Azure for a Spin vendors offer complex networking Using Microsoft Azure requires features, caching, monitoring and a free Windows Live account, as Content Delivery Network options. well as a credit card to open a In either environment, a VM can be charge account. If your employer sited in the Americas, Europe or Asia, participates in the Microsoft with global CDN nodes. Developer Network (MSDN) program,
Figure 1. The Azure management portal is easy to use and attractively designed.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 85
LJ226-Feb013-bu.indd 85 1/23/13 1:07 PM FEATURE Linux on Azure
Figure 2. The Azure portal displays task message and status.
Figure 3. Five different Linux options are offered when creating a VM.
you already may be entitled to a portal and start adding services. free quantity of Azure services every The Azure control panel is, quite month. Once your account is set up, simply, gorgeous. Perhaps pretty you can head to the management controls are not a big selling point
86 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 86 1/23/13 1:07 PM Figure 4. Setting Up penguin1 in Azure
for a cloud service, but the Azure SUSE Linux Enterprise Server and interface is marvelously interactive. OpenSUSE 12.1. It’s possible to As you set up services, messages on roll your own image and upload the status of operations appear at the it, but this requires working with bottom of the screen asynchronously. Microsoft’s Hyper-V server product, Performance graphs and history are which is something the average integrated into the display, and the Linux user is unlikely to have handy. panel feels much more like a desktop For this article, I create a CentOS app than a “click-submit-and-wait” 6.2 VM called “penguin1”. A DNS Web interface. name is created automatically for Azure offers several flavors of the VM in the cloudapp.net domain, Linux: CentOS 6.2, Ubuntu 12.04, which then can be CNAME’d if you
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 87
LJ226-Feb013-bu.indd 87 1/23/13 1:07 PM FEATURE Linux on Azure
Figure 5. Adding a Network Endpoint to Set Up a Web Server
own your own domain. security feature. By default, only VMs are not directly exposed to port 22 (SSH) is configured. If you the Internet, but rather are given intend to change your default SSH 10.x IP addresses. Inside the Azure port (as often is done to prevent panel, users then can configure script-kiddie scanning), you’ll need endpoints to open firewall ports to change the endpoint in the Azure and map them as they like. For management portal as well. You example, to set up a Web server, also have the option of changing it’s necessary to create a port 80 it in the management portal and (and perhaps 443) endpoint, which mapping it back to 22 on the VM. can be mapped to any port desired Creating “From Gallery” gives you on the VM. the most options for creation. After This network firewall is a nice supplying basic information, such as
88 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 88 1/23/13 1:07 PM name and size, provisioning begins BLOB in Azure storage. So if you immediately and takes about ten delete a VM, it’s possible to retain minutes to complete in my experience. its root volume and later mount it up on another system or use it as What’s the Root Password!?!? the root volume for a new VM. Users of other VPS systems may You also can create as many wonder where they specify the root other volumes as you’d like. These password. The answer is simple: live as BLOBs in Azure storage and you don’t. Whatever account you are persistent. They function much specify will be given sudo authority like SAN volumes, allowing you to to su to root: create and attach them to one VM, then later unmount and attach to [andrew@penguin1 ~]$ sudo su - another. Unfortunately, there is [sudo] password for andrew: no way to resize these volumes, [root@penguin1 ~]# which is a disappointing limitation. While you can create a larger From that point, you can set volume, move data and delete the the root password if you want old volume, this obviously is not a with passwd. scalable approach. When you create a new disk and Storage attach it to the VM, it appears as Linux virtual machines have three a new SCSI device that you can types of storage. mkfs and mount. After creating a First, each is given a 30GB root new 20GB disk in the Azure portal, volume. Storage is locally redundant penguin1’s dmesg shows: and optionally can be made
geographically redundant for about scsi 4:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 4
a 33% cost increase. Unfortunately, sd 4:0:0:0: Attached scsi generic sg3 type 0
short of creating your own sd 4:0:0:0: [sdc] 41943040 512-byte logical blocks: (21.4 GB/20.0 GiB)
template, there is no way to modify sd 4:0:0:0: [sdc] Write Protect is off
this 30GB configuration if you sd 4:0:0:0: [sdc] Mode Sense: 0f 00 10 00
prefer a different filesystem layout sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled,
or want a smaller monthly storage supports DPO and FUA
bill. Root volume data persists sdc: unknown partition table
across reboots and is a permanent sd 4:0:0:0: [sdc] Attached SCSI disk
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 89
LJ226-Feb013-bu.indd 89 1/23/13 1:07 PM FEATURE Linux on Azure
Finally, virtual machines also come with a variable amount of truly local storage. This storage does not live in the Azure cloud but rather is provisioned on the actual physical node.
Now you can fdisk, mkfs and mount: 262059 blocks (5.00%) reserved for the super user
First data block=0
[root@penguin1 ~]# fdisk /dev/sdc Maximum filesystem blocks=4294967296
Command (m for help): n 160 block groups
Command action 32768 blocks per group, 32768 fragments per group
e extended 8192 inodes per group
p primary partition (1-4) Superblock backups stored on blocks:
p 32768, 98304, 163840, 229376, 294912, 819200, 884736,
Partition number (1-4): 1 1605632, 2654208, 4096000
First cylinder (1-2610, default 1):
Using default value 1 Writing inode tables: done
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): Creating journal (32768 blocks): done
Using default value 2610 Writing superblocks and filesystem accounting information: done
Command (m for help): w This filesystem will be automatically checked every 21 mounts or
The partition table has been altered! 180 days, whichever comes first. Use tune2fs -c or -i to override.
Calling ioctl() to re-read partition table. [root@penguin1 ~]# mkdir /data
Syncing disks. [root@penguin1 ~]# mount /dev/sdc1 /data
[root@penguin1 ~]# mkfs.ext4 /dev/sdc1 [root@penguin1 ~]# df -h
mke2fs 1.41.12 (17-May-2010) Filesystem Size Used Avail Use% Mounted on
Filesystem label= /dev/mapper/VolGroup-lv_root
OS type: Linux 28G 2.0G 24G 8% /
Block size=4096 (log=2) tmpfs 872M 0 872M 0% /dev/shm
Fragment size=4096 (log=2) /dev/sda1 485M 86M 374M 19% /boot
Stride=0 blocks, Stripe width=0 blocks /dev/sdb1 69G 180M 66G 1% /mnt/resource
1310720 inodes, 5241198 blocks /dev/sdc1 20G 172M 19G 1% /data
90 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 90 1/23/13 1:07 PM As you might expect, if I were to Rough Edges delete this disk in the Azure portal, The CentOS image could use the operation would fail unless I some improvement. Provided by first unmounted it. OpenLogic, I’ve noted some things Finally, virtual machines also that make me scratch my head: come with a variable amount of truly local storage. This storage n A swap partition is configured, does not live in the Azure cloud but does not appear in /etc/fstab. but rather is provisioned on the Because there isn’t much actual physical node. If there is advantage to creating a swap a hardware or other fault in the partition but not using it, this Azure fabric and your VM migrates is presumably an oversight. to a new host, data stored in this space is lost. This filesystem n Some default services run is meant to be used for state without justification. For information, scratch data and example, why is CUPS needed? other temporary files. On Linux I’m unlikely to print in the cloud. images, it shows up as /mnt/resource. Given that RAID redundancy is Small VMs receive a 70GB provided by the Azure storage filesystem, and the size increases layer and software RAID is not with VM size, up to 800GB for needed, why is mdmonitor set Extra Large VMs. to run at boot? I have experienced a hardware fault on Azure. My VM shut n I periodically receive crash reports down, migrated to a new hardware from fprintd. Removing this host and booted up on the new service is straightforward, but server. Services that I’d defined why the CentOS image is created to start up came up normally, all to support fingerprint biometric network endpoints were moved authentication in a virtual, cloud- correctly, disks I’d mounted were based environment mystifies me. remounted, and the VM worked fine. However, the scratch data n iptables is enabled with a single I had in /mnt/resource was lost, rule to accept bootp, though and the filesystem was empty, the INPUT chain has a default as designed. ACCEPT policy anyway.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 91
LJ226-Feb013-bu.indd 91 1/23/13 1:07 PM FEATURE Linux on Azure
Figure 6. The Integrated Display of Performance Metrics in the Azure Portal
Hopefully, these things will be some circumstances. However, corrected as Linux settles into Azure. these settings can negatively affect performance in other Performance circumstances, depending on In general, I/O performance is the application. Host caching excellent. Measuring with ioping, is OFF by default for both read I found the /data disk I created operations and write operations earlier averaged 6ms latency, while for data disks. Host-caching /mnt/resource showed a zippy is ON by default for read and 0.4ms. The root disk was a slower write operations for operating 18ms, but as Microsoft explains in system disks. As noted, these its documentation, it optimizes the should work best in most cases. I/O performance on volumes tagged However, your mileage may vary. “OS Disk” differently: We recommend you place data intensive operations on a data The operating system disk and disk separate from the OS disk. data disk has a host caching setting (sometimes called Compute performance depends on host-cache mode) that enables the VM size you select. Under the improved performance under covers, Microsoft is using AMD gear,
92 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 92 1/23/13 1:07 PM as evidenced by /proc/cpuinfo: user, performance and pricing are very similar, while the enterprise processor : 1 administrator may find the ability to vendor_id : AuthenticAMD collocate Linux alongside Windows cpu family : 16 and Azure-specific deployments to be model : 8 advantageous. While running Linux model name : AMD Opteron(tm) Processor 4171 HE in the Azure cloud might seem like a stepping : 1 penguin pitching his tent in Mordor, cpu MHz : 2094.702 one more quality player in the cloud cache size : 512 KB space is good news for all.■
Summary Andrew Fabbro is a senior technologist living in the Portland, Some may say the best thing about Oregon, area. He’s used Linux since Slackware came on Linux in Azure is that it gives competition floppies and presently works for Con-way, a Fortune 500 to Amazon and Google. For the end transportation company.
Linux JournaL on your e-Reader
Customized e-Reader editions Kindle and Nook editions FREE for Subscribers now available
LEARN MORE
LJ226-Feb013-bu.indd 93 1/23/13 1:07 PM FEATURE Fabric: a System Administrator’s Best Friend
FABRIC a System Administrator’s Best Friend
Do you routinely make changes to more than a dozen machines at a time? Read this article to find out about a tool to make that task much easier.
ADRIAN HANNAH
94 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 94 1/23/13 1:07 PM ’ll be honest. Even though this most prolific way to install Fabric is library is fully five years old, I using pip (or easy_install). On most I hadn’t heard of Fabric until about systems, you can use your system’s six months ago. Now I can’t imagine package manager (apt-get, install, not having it in my digital tool belt. and so on) to install it (the package Fabric is a Python library/tool that is either will be fabric or python-fabric). designed to use SSH to execute system If you’re feeling froggy, you can check administration and deployment tasks out the Git repository and hack away on one or more remote machines. No at the source code. more running the same task, machine Once installed, you will have by machine, to make one change across access to the fab script from the the board. It is a simple fire-and-forget command line. tool that will make your life so much simpler. Not only can you run simple Operations tasks via SSH on multiple machines, The Fabric library is composed of nine but since you’re using Python code separate operations that can be used
IT IS A SIMPLE FIRE-AND-FORGET TOOL THAT WILL MAKE YOUR LIFE SO MUCH SIMPLER.
to execute items, you can combine it in conjunction to achieve your desired with any arbitrary Python code to make effect. Simply insert these functions robust, complex, elegant applications into your fabfile and off you go: for deployment or administration tasks. n get(remote_path, Installation local_path=None) — get allows Fabric requires Python 2.5 or later, you to pull files from the remote the setuptools packaging/installation machine to your local machine. library, the ssh Python library, and This is like using rsync or scp SSH and its dependencies. For the to copy a file or files from many most part, you won’t have to worry machines. This is super effective for about any of this, because Fabric can systematically collecting log files or be installed easily through various backups in a central location. The package managers. The easiest, and remote path is the path of the file
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 95
LJ226-Feb013-bu.indd 95 1/23/13 1:07 PM FEATURE Fabric: a System Administrator’s Best Friend
on the remote machine that you n prompt(text, key=None, are grabbing, and the local path default='', validate=None) is the path to which you want to — in the case when you need to save the file on the local machine. supply a value, but don’t want to If the local path is omitted, Fabric specify it on the command line assumes you are saving the file to for whatever reason, prompt is the working directory. the ideal way to do this. I have a fabfile I use to add/remove/check n local(command, the status of software on all of the capture=False) — the local servers I maintain, and I use this function allows you to take in the script for when I forget to action on the local host in a specify what software I’m working similar fashion to the Python on. This prompt will appear for subprocess module (in fact, each host you specify, so make sure local is a simplistic wrapper that you account for that! sits on top of the subprocess module). Simply supply the n put(local_path, remote_path, command to run and, if needed, use_sudo=False, whether you want to capture mirror_local_mode=False, the output. If you specify mode=None) — this is the opposite capture=True, the output will command of get, although you are be returned as a string from given more options when putting to a local; otherwise, it will be output remote system than getting. The local to STDOUT. path can be a relative or absolute file path, or it can be an actual file object. n open_shell(command=None) — If either local_path or remote_path this function is mostly for is left blank, the working directory debugging purposes. It opens an will be used. If use_sudo=True is interactive shell on the remote end, specified, Fabric will put the file in allowing you to run any number a temporary location on the remote of commands. This is particularly machine, then use sudo to move it helpful if you are running a series from the temporary location to the of particularly complex commands specified location. This is particularly and it doesn’t seem to be working handy when moving system files on some of your machines. like /etc/resolv.conf or the like that
96 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 96 1/23/13 1:07 PM can’t be moved by a standard user quiet=False, warn_only=False, and you have root login turned stdout=None, stderr=None) — off in SSH. If you want the file this and sudo are the two most mode preserved through the copy, used functions in Fabric, because use mirror_local_mode=True; they actually execute commands otherwise, you can set the mode on the remote host (which is the using mode. whole point of Fabric). With run, you execute the specified command n reboot(wait=120) — reboot as the given user. run returns the does exactly what it says: reboots output from the command as a the remote machine. By default, string that can be checked for a reboot will wait 120 seconds failed, succeeded and return_code before attempting to reconnect to attribute. shell controls whether the machine to continue executing a shell interpreter is created for the
THIS IS PARTICULARLY HANDY WHEN MOVING SYSTEM FILES LIKE /etc/resolv.conf OR THE LIKE THAT CAN’T BE MOVED BY A STANDARD USER AND YOU HAVE ROOT LOGIN TURNED OFF IN SSH.
any following commands. command. If turned off, characters will not be escaped automatically in n require(*keys, **kwargs) — the command. Passing pty=False require forces the specified causes a psuedo-terminal not to keys to be present in the shared be created while executing this environment dict in order to command; this can have some continue execution. If these keys benefit if the command you are are not present, Fabric will abort. running has issues interacting with Optionally, you can specify the psuedo-terminal, but otherwise, used_for to indicate what the key it will be created by default. If you is used for in this particular context. want stderr from the command to be parsable separately from stdout, n run(command, shell=True, use combine_stderr=False to pty=True, combine_stderr=True, indicate that. quiet=True will
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 97
LJ226-Feb013-bu.indd 97 1/23/13 1:07 PM FEATURE Fabric: a System Administrator’s Best Friend
cause the command to run silently, The Basics sending no output to the screen Now that you understand the while executing. When an error groundwork of Fabric, you can occurs in Fabric, typically the script start putting it to use. For this will abort and indicate as such. article, I explain how to make You can indicate that Fabric need a simple fabfile for the purpose not abort if a particular command of installing/removing software errors using the warn_only on your machines. First, you argument. Finally, you can redirect need what is called a fabfile. where the remote stderr and stdout The fabfile contains all of your redirect to on the local side. For Fabric functions. By default, it instance, if you want the stderr needs to be named fabfile.py to pipe to stdout on the local and be in the working directory, end, you could indicate that with but as mentioned previously, you stderr=sys.stdout. can specify the fabfile from the command line if need be. So, n sudo(command, shell=True, open your fabfile and start it pty=True, combine_stderr=True, with from fabric.api import * user=None, quiet=False, to include all the Fabric functionality. warn_only=False, Then define all of your functions. stdout=None, stderr=None, Let’s start with installing some group=None) — sudo works software: precisely like run, except that it will elevate privileges prior to executing def install(pkg=None): the command. It basically works the if pkg is not None: same as if you’d run the command env["pkg"] = pkg using run, but prepended sudo to elif pkg is None and env.get("pkg") is None: the front of command. sudo also env["pkg"] = prompt("Which package? ") takes user and group arguments, sudo('yum install -y %s' % env["pkg"]) allowing you to specify which user or group to run the command as. You then can install a package As long as the original user has via yum on all of your machines the permissions to escalate for by running: that particular user/group and command, you are good to go. $ fab --hosts=host1,host2,host3 install
98 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 98 1/23/13 1:07 PM LinuxJournal_Layout 1 12/12/12 9:47 AM Page 1
Big Data gets real at Big Data TechCon! The HOW-TO conference for Big Data and IT Professionals Discover how to master Big Data from real-world practitioners – instructors who work in the trenches and can teach you from real-world experience! Come to Big Data TechCon to learn the best ways to: • Collect, sort and store massive quantities of structured and unstructured data Over 50 • Process real-time data pouring how-to into your organization practical classes and workshops • Master Big Data tools and to choose technologies like Hadoop, Map/Reduce, from! NoSQL databases, and more
• Learn HOW TO integrate data-collection technologies with analys is and business-analysis tools to produce the kind of workable information and reports your organization needs
• Understand HOW TO leverage Big Data to help your organization today
April 8-10, 2013 Boston, MA www.BigDataTechCon.com Register Early and SAVE! A BZ Media Event
Big Data TechCon™ is a trademark of BZ Media LLC.
LJ226-Feb013-bu.indd 99 1/23/13 1:07 PM FEATURE Fabric: a System Administrator’s Best Friend
A BRIEF WORD ON APPLICATION DEPLOYMENT
Fabric also is used in development teams to deploy new code to production. It is actually used in a fairly similar fashion to how system administrators use it (copy files, run a few commands and so on), just in a very specific manner. Because of how automated Fabric is, it’s easy to incorporate it into a continuous integration cycle and even fully automate your deployment process.
Then, you’ll be prompted for Tips and Tricks the package to install only once. I’ve picked up some neat tricks since Alternatively, since you indicated I’ve started with Fabric. First, you an optional parameter of pkg, you generally never see a Fabric command can indicate that from the command as simple as what is above. When fully line so you won’t be prompted on automated, it looks more like this: execution, like this:
$ fab --skip-bad-hosts -u user -p 12345 -i ~/.ssh/id_dsa --warn-only
$ fab --hosts=host1,host2,host3 install:pkg=wormux ➥--hosts=host1,host2,host3,host4,host5,host6,host7,host8,host9,host10
➥--parallel --pool-size=20 install:pkg=wormux or: Who wants to type that out every $ fab --hosts=host1,host2,host3 install:wormux time they want to run a command? No one! That’s why aliasing almost all Also note that you are prompted of that is so convenient and efficient. for the password for both SSH and Add the following to your .bashrc file: sudo only once. Fabric stores this in
memory and reuses it, if possible, for alias f="fab --skip-bad-hosts -u user -p 12345 -i ~/.ssh/id_dsa
every other machine. Congratulations! ➥--warn-only
You’ve just successfully created your ➥--hosts=host1,host2,host3,host4,host5,host6,host7,host8,host9,host10
first Fabric script. It’s as simple as that! ➥--parallel"
100 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 100 1/23/13 1:07 PM Then, all you have to do each time commands in a with statement, you you want to run Fabric is this: have a solution:
$ f install:pkg=wormux with cd("~/gitrepo"): run('git add --all') Even using this technique, your run('git commit -m "My super awesome automated alias can become cumbersome if ➥commit script for `date`"') you have more than a few machines you commonly administer. A simple More Information solution to that is to add this function There are several ways to get help to your fabfile: with Fabric. The most effective is to use the fab-file mailing list def set_hosts(): (http://lists.nongnu.org/mailman/ env.hosts = open('hosts', 'r').readlines() listinfo/fab-user). The developers are generally very prompt in responding. Then, put all your hostnames in a There is also a Fabric Twitter account file called hosts in the same directory @pyfabric (http://www.twitter.com/ as your fabfile, and modify your alias pyfabric) where Fabric news and to look like this: announcements are released. You can submit and view bugs through the Fabric
alias f="fab --skip-bad-hosts -u user -p 12345 -i ~/.ssh/id_dsa Github page (https://github.com/
➥--warn-only --parallel set_hosts" fabric/fabric/issues). Of course, you also can’t discount the #fabric channel This is particularly convenient if you on Freenode, where you can connect have a variety of fabfiles that you use with the community and get some on different groups of machines, or in quick answers. Finally, you always different contexts. can browse the documentation hosted There are occasions when you need at http://www.fabfile.org.■ to execute certain commands from within a specific directory. Because Adrian Hannah has spent the last 15 years bashing keyboards each command is a discrete and non- to make computers do what he tells them. He currently persistent connection to the machine, works as a Senior System Administrator for a Web startup this is not inherently simple. However, in New York City. He is a jack of all trades and a master of simply by enclosing the necessary none. Find out more at http://about.me/adrianhannah.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 101
LJ226-Feb013-bu.indd 101 1/23/13 1:07 PM FEATURE Fabric: a System Administrator’s Best Friend
COMMAND-LINE ARGUMENTS n -a, --no_agent — sets env.no_agent --shortlist; normal is the same to True, forcing your SSH layer not to talk as simply omitting this option entirely to the SSH agent when trying to unlock (the default), and nested prints out private key files. a nested namespace tree. n -A, --forward-agent — sets n -g HOST, --gateway=HOST — sets env.forward_agent to True, enabling env.gateway to HOST host string. agent forwarding. n -h, --help — displays a standard n --abort-on-prompts — sets help message with all possible options env.abort_on_prompts to True, and a brief overview of what they do, forcing Fabric to abort whenever it then exits. would prompt for input. n --hide=LEVELS — a comma-separated n -c RCFILE, --config=RCFILE — list of output levels to hide by default. sets env.rcfile to the given file path, n -H HOSTS, --hosts=HOSTS — which Fabric will try to load on startup sets env.hosts to the given and use to update environment variables. comma-delimited list of host strings. n -d COMMAND, --display=COMMAND — n -x HOSTS, --exclude-hosts=HOSTS — prints the entire docstring for the given sets env.exclude_hosts to the given task, if there is one. It does not currently comma-delimited list of host strings to print out the task’s function signature, so keep out of the final host list. descriptive docstrings are a good idea. (They’re always a good idea, of course, n -i KEY_FILENAME — when set to just more so here.) a file path, will load the given file as an SSH identity file (usually a private n --connection-attempts=M, key). This option may be repeated -n M — sets the number of times multiple times. Sets (or appends to) to attempt connections. Sets env.key_filename. env.connection_attempts. n -I, --initial-password-prompt — n -D, --disable-known-hosts — forces a password prompt at the start sets env.disable_known_hosts to of the session (after fabfile load and True, preventing Fabric from loading the option parsing, but before executing any user’s SSH known_hosts file. tasks) in order to pre-fill env.password. n -f FABFILE, --fabfile=FABFILE — This is useful for fire-and-forget runs the fabfile name pattern to search (especially parallel sessions, in which for (defaults to fabfile.py), or alternately runtime input is not possible) when setting an explicit file path to load as the fabfile the password via --password or by (for example, /path/to/my/fabfile.py). setting env.password in your fabfile is undesirable. n -F LIST_FORMAT, --list-format=LIST_FORMAT — n -k — sets env.no_keys to True, forcing allows control over the output format the SSH layer not to look for SSH private of --list. short is equivalent to key files in one’s home directory.
102 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 102 1/23/13 1:07 PM n --keepalive=KEEPALIVE — will not override more specific env sets env.keepalive to the given vars that also are specified on the (integer) value, specifying an SSH command line. keepalive interval. n -s SHELL, --shell=SHELL — n --linewise — forces output to be sets env.shell to the given buffered line by line instead of byte string, overriding the default by byte. Often useful or required for shell wrapper used to execute parallel execution. remote commands. n -l, --list — imports a fabfile as n --shortlist — similar to --list, normal, but then prints a list of all but without any embellishment—just discovered tasks and exits. Will also task names separated by newlines print the first line of each task’s with no indentation or docstrings. docstring, if it has one, next to it n --show=LEVELS — a comma-separated (truncating if necessary). list of output levels to be added to those n -p PASSWORD, --password=PASSWORD — that are shown by default. sets env.password to the given n --ssh-config-path — sets string; it then will be used as the env.ssh_config_path. default password when making SSH connections or calling the sudo program. n --skip-bad-hosts — sets env.skip_bad_hosts, causing n -P, --parallel — sets Fabric to skip unavailable hosts. env.parallel to True, causing tasks to run in parallel. n --timeout=N, -t N — sets connection timeout in seconds. n --no-pty — sets Sets env.timeout. env.always_use_pty to False, causing all run/sudo calls to behave as n -u USER, --user=USER — sets if one had specified pty=False. env.user to the given string; it then will be used as the default user name n -r, --reject-unknown-hosts — when making SSH connections. sets env.reject_unknown_hosts to True, causing Fabric to abort when n -V, --version — displays Fabric’s connecting to hosts not found in the version number, then exits. user’s SSH known_hosts file. n -w, --warn-only — sets n -R ROLES, --roles=ROLES — env.warn_only to True, sets env.roles to the given causing Fabric to continue execution comma-separated list of role names. even when commands encounter error conditions. n --set KEY=VALUE,... — allows n -z, --pool-size — sets you to set default values for arbitrary env.pool_size, which specifies how Fabric env vars. Values set this many processes to run concurrently way have a low precedence. They during parallel execution.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 103
LJ226-Feb013-bu.indd 103 1/23/13 1:07 PM INDEPTH Making Linux and Android Get Along (It’s Not as Hard as It Sounds) Android devices don’t come with an “installation CD” for Linux, but by installing a few tools, you won’t need those coasters-to-be anyway! AARON PETERS
Many free software fans, if they Blackjack showed me how nice it is to were like me, breathed a collective carry one device, and since receiving sigh of relief when the Android my first Android device (an original operating system hit the market. Droid I still use to this day), I can’t Before receiving my first smartphone imagine using a device with another (a Samsung Blackjack running mobile OS. Linux kernel, Java-based Windows Mobile 5.5, I believe, that apps—these are all right up my alley. I had to update to through a But, like many great consumer Linux torturous combination of installing products (I’m talking to you, Sharp Windows XP on a partition, installing Zaurus), manufacturers assume in the phone drivers, then running an nearly every case that your “other” update program), I was a steadfast computer will run Windows. Now, it’s “PDA-and-cell” guy who proudly easy enough to install Windows either carried both devices on my belt on a separate partition to dual-boot like a pair of six-shooters. But that or in a VM to run within Linux. But
104 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 104 1/23/13 1:07 PM INDEPTH
this is a bit like killing the proverbial Linux boxen. In this article, I focus fly with a bazooka. Web-based on files and a few approaches for applications and “the cloud” alleviate making sure you always have an some of these difficulties, yet it’s still up-to-date copy of that spreadsheet not an “out-of-the-box-after-a-quick- or source file on your mobile device. install-from-CD” process like it is for Windows users. In the Cloud The good news is, with the The cloud computing movement installation or configuration of a has done a great deal to promote few programs, it’s pretty easy to get platform agnosticism, from your Android device (all the steps consistent (Web-based) UIs to in this article are equally applicable cross-platform APIs that allow to phones and tablets unless stated applications to synchronize otherwise) to play nice with your data. And with most users being
The White Paper Library on LinuxJournal.com
www.linuxjournal.com/whitepapers
LJ226-Feb013-bu.indd 105 1/23/13 1:07 PM INDEPTH
Figure 1. Nautilus Context Menu
constantly connected via 3/4G, long, complicated feature list, just Wi-Fi or wired networks to the Internet, good old-fashioned cloud storage. cloud services have been one of the And with support for both Android most hassle-free ways to make your (via the application in Google data available across devices. Play at https://play.google.com/ store/apps/details?id= Dropbox com.dropbox.android&hl=en) and Of the free file-sharing services out Linux, either for GNOME and other there, Dropbox is arguably the most GTK-centric desktops (using the popular, perhaps because it’s the the Nautilus plugin from Dropbox simplest—no bells and whistles, no shown in Figure 1 and available
106 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 106 1/23/13 1:07 PM INDEPTH
Figure 2. KFileBox Menu and Config Window
at https://www.dropbox.com/ note the “official” Dropbox app install?os=lnx), or KDE (via the above keeps a list of your files, but excellent KFilebox—at the time of it doesn’t actually sync up the files this writing, the project’s home page themselves—that is, if you upload lists 0.4.7 as the most recent version, a revised file to Dropbox from http://kdropbox.deuteros.es, your Linux box, then later go off- but the SourceForge page, line with your mobile device, the http://sourceforge.net/projects/ Android gadget will know that file kdropbox, lists a version 0.4.8 that changed, but you won’t be able to works very well—shown in Figure 2). view or edit it until you go back Pointing each of the above at on-line. However, a free app called the same folder tree will help keep DropSync (https://play.google.com/ all your important folders close at store/apps/details?id= hand. However, it’s important to com.ttxapps.dropsync&hl=en)
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 107
LJ226-Feb013-bu.indd 107 1/23/13 1:07 PM INDEPTH
Figure 3. DropSync Config Screen
will do this for you (Figure 3). In Box addition, Dropbox is supported Box, like Dropbox, offers users free internally by a wide variety of on-line storage space accessible via a individual Android apps, which will Web interface. Box also has an app in let you edit files directly from or the market (https://play.google.com/ save files directly to your Dropbox store/apps/details?id= account. An example of this on com.box.android&hl=en, Figure 4). my Transformer Prime is Epistle One advantage of the Box app over (https://play.google.com/ Dropbox is that it automatically store/apps/details?id= notifies you of updates to files. The com.kooklab.epistle&hl=en), Box service itself also has some nice a very elegant Markdown editor, which features, some of which are available automatically updates the list of files in only with a premium subscription, its folder to a folder on Dropbox. including version management and
108 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 108 1/23/13 1:07 PM INDEPTH
Figure 4. Box Android App
Figure 5. Box “Make Available Offline” Menu
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 109
LJ226-Feb013-bu.indd 109 1/23/13 1:07 PM INDEPTH
Figure 6. Box via WebDAVS
integration with other Web apps, doesn’t have a native client program such as LinkedIn, SalesForce, NetSuite available, it does permit access to and Basecamp. However, it suffers your files via WebDAVS. This means from one of the same weaknesses you can set up a shortcut in Nautilus as the official Dropbox app: when (by connecting via the “Connect to the app is on-line, it updates only Server” option to dav://www.box.net/ information on the files in your Box dav, making sure to select “Secure account, rather than caching a version WebDAV” per these instructions: of the files. Although it does have http://benjaminkerensa.com/ an option to mark files to “Make 2011/10/27/how-to-mount-box- Available Offline” (Figure 5). net-securely-on-ubuntu-11-10) On the Linux side, although Box or Dolphin (for some reason I could
110 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 110 1/23/13 1:07 PM INDEPTH
not get the “Add Network Folder” Google Drive dialog to connect, but simply typing Some heralded the re-branding of webdavs://www.box.net/dav into Google Docs to Google Drive as the Dolphin’s address bar prompted my beginning of the end for Dropbox credentials and worked like a and its brethren (perhaps some still charm, as I proudly display in believe this to be the case). With Figure 6). In some ways, I prefer the built-in editing capabilities of this to being forced into using a Google Docs behind it, Google proprietary client program; on the Drive is certainly a killer tool for other hand, the Dropbox client for collaboration and productivity. Linux does automatically update I’ve used shared text documents local copies of files, while Box’s and spreadsheets with clients and WebDAV access feature will require colleagues, and having an on-line that you’re on-line unless you take place both to stash this important additional steps. information as well as work on it in
Figure 7. Google Drive Android Spreadsheet Editor
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 111
LJ226-Feb013-bu.indd 111 1/23/13 1:07 PM INDEPTH
Figure 8. Polaris Office Displaying Google Drive contents
real time has been a huge time saver app at https://play.google.com/ on more than one occasion. store/apps/details?id=com.google. But placing all your data in Google android.apps.docs (yeah, I included Drive isn’t without its drawbacks. a link, but if you’ve got an Android Google uses its own internal device, you’ve got it already, no?) formats for the text documents, is much improved from the initial spreadsheets, presentations and versions, in which the document drawings in Google Drive. While it’s editor operated through Web- very non-evil about allowing you based text areas. Unfortunately, the to download your files in Linux- spreadsheet editor still requires you friendly formats (even ODF for text to click an Edit link at the beginning and spreadsheets, huzzah!), it still of the row to edit the values in that involves conversion, which carries row (Web-based text fields), shown in with it the risk of misconversion. Figure 7. As for file management, like The recently updated Google Drive Box, Google Drive will save files locally
112 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 112 1/23/13 1:07 PM INDEPTH
for you to edit if you’re off-line, but One’s promise of a “personal cloud” only if you select the Available Offline is certainly enticing. option for each file to which you’ll need access. In addition, Google n Spideroak: if you’re nervous about Drive also is supported by individual entrusting all your sensitive data to apps (like DropBox above). In addition a service provider’s BOFH’s for all to its own app, Google Drive is an you know, Spideroak may be right acceptable storage place for Polaris up your alley. The service’s biggest Office (pre-installed on my Prime, selling point is “zero-knowledge” shown in Figure 8) and Documents encryption on all your data—that to Go. is, even though the company hosts it, even it can’t break into your Others files. It also maintains a version The following items also deserve history on files, a feature typically special mention, and although they’re only for premium customers of not quite as widely known, accepted other services. Finally, in addition and/or supported across the Android to mobile (iOS, Android and community yet, each has some nice Nokia N900), the company has features that are worth a look. comprehensive Linux support, providing clients in DEB (Ubuntu/ n Ubuntu One: Canonical’s entry Debian), RPM (Fedora/OpenSUSE/ into the cloud storage and Web RHEL/CentOS) and even TGZ format services game, it has the benefit of for Slackware users. a commercial supporter of the Linux client. In addition, Ubuntu One goes On the Local Network beyond simple file synchronization For the paranoid among us, there and will have the ability in the are concerns about leaving all your future to keep some of your more sensitive data in the hands of corporate data-centric applications (such as overlords. Fortunately, there are options contacts and notes) up to date as for even the most anti-corporate shell well as stream music. The support jockey to connect Android and Linux is a little patchy (for example, it over a local network. will synchronize contacts, but not calendar or task data, and only on The “Linux” Way: SSHDroid Ubuntu at present), but Ubuntu One option is to synchronize from the
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 113
LJ226-Feb013-bu.indd 113 1/23/13 1:07 PM INDEPTH
Figure 9. SSHDroid Main Screen
Linux side, meaning there needs to files directly over SFTP (one reason be a mechanism for your Linux box why I love kioslaves), I can say this is to see and manipulate the files on probably my favorite way to use my the Android device. SSHDroid Linux and Android machines, for a (https://play.google.com/store/ couple reasons. One, it takes the least apps/details?id=berserker.android. amount of setup: you install SSHDroid, apps.sshdroid&hl=en) provides a full start it up and go to a URL from the SSH server for your device. As shown Linux machine. And, Bob’s your uncle. in Figure 9, starting this app displays a Second, it’s secure. Third, while I screen telling you everything you need generally use it to edit files directly to know, including your current IP over SFTP, once you’re connected, you address, the URL to connect to (it uses can use an application like Unison or the SFTP protocol and defaults to port Krusader to synchronize files. And last, 2222) and the status of the server. the performance for large transfers is Having used this quite a bit to edit not too shabby on my Prime.
114 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 114 1/23/13 1:07 PM INDEPTH
That said, this method is best suited freedom of browsing the Interwebs for those who use the Android device or writing from a hammock in the as a mobile extension of their desktop back yard (which, if you haven’t tried machine—that is, those for whom the it, I highly recommend) but still want Linux box is the boss. For those of you to practice good backup procedures, who do more and more computing on FolderSync (https://play.google.com/ tablets and other mobile devices, it store/apps/details?id=dk.tacit. never hurts to have SSHDroid installed android.foldersync.full&hl=en) is (it’s free and takes up less than 1MB, an excellent solution. It isn’t open rare nowadays). A more Android- source, or even free, but at $2.29 centric solution is described below. for the Pro version, it’s practically a no-brainer once you figure out what The “Android” Way: FolderSync it can do for you. (S/FTP, Samba, WebDAVS) And what is that? It will keep one For those of you who are enjoying the folder on your device synced up with
Figure 10. Configuration Options for Foldersync
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 115
LJ226-Feb013-bu.indd 115 1/23/13 1:07 PM INDEPTH
a folder on your Linux box over SFTP. You can opt to start the sync jobs manually or schedule them, with useful options, such as limiting certain jobs (called Folderpairs, as shown in Figure 10) to certain wireless networks, only synchronizing when the power is plugged in, and you can choose whether files should be updated one-way or bi-directionally. But the great thing about FolderSync is all the different protocols it supports (a part of that selection list is shown in Figure 11). Have a file server at work that exports Figure 11. FolderSync-Supported Protocols a Samba share?
116 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 116 1/23/13 1:07 PM INDEPTH
FolderSync will link up to that, no filesystem. When plugged in via a USB problem. Want to do some updates to a cable, the device appears to be just site on your Web server? Get WebDAV(S) another USB drive. You could move running on Apache, and you’re set. files to and fro, access documents Oh, and remember all those cloud directly on the device, and basically services we talked about? Dropbox, treat the phone or tablet just as you Box and Google Drive? FolderSync does would any other thumbdrive (with that one, that one and that one too. maybe the exception of leaving it in The “Lite” version will allow you your pocket to go through the wash). to sync up with one other folder on Like SSHDroid above, once this USB one device, so if that’s all you need, storage was mounted, you could use you can avoid having to pony up any any Linux tool at your disposal (Unison, cash. But the Pro version will allow Krusader, rsync) to make sure they were you to set up your Android device as up to date. All was well, until Google a central hub for anywhere you stash tried to be too smart for its own good. files. Now if only they started making devices with 1TB Flash drives.... The Honeycomb (3.0 and above) Way Direct Connection From Android v3.0 and up, plugging The last, and slightly old-school way, a device in via USB no longer shows to connect your Android device to up as USB storage (that is, the “easy your Linux box is via a direct USB way”). Rather, you’re required to connection. While this may evoke choose in the device’s settings whether, feelings of nostalgia for longtime on USB connection, you’d like the gadget geeks who remember popping device to use the MTP protocol (that a Palm into a cradle and hitting the is, to appear to the other machine as a “HotSync” button, I find this to be the media player) or the PTP protocol (that worst experience on newer devices, is, to appear as a camera). for reasons I’ll explain next. Now, I’ve read that there’s a technical reason for Google’s decision The Gingerbread (2.3.6 and to do this, mainly that all applications below) Way and data now can reside on a single On Android devices prior to v.3.0, filesystem (as opposed to having to Google did the “right thing” choose, for example, to install apps to enable access to the device’s on the “phone” or on the “SD card”,
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 117
LJ226-Feb013-bu.indd 117 1/23/13 1:07 PM INDEPTH
as I do on my OG Droid). All I would to do so. When I got up the next argue is that, for this user, those benefits morning, it was approximately 5% do not outweigh the terrible experience completed. Before you start asking of trying to use MTP on Linux (PTP for transfer rates and whatnot, I don’t actually works quite well, but only gives have them, but I was able to transfer you access to the “DCIM” folder, so about half that same collection within unless you want to store all your other a couple hours, and over SFTP (so stuff alongside the pictures taken by the with en/decryption overhead) no less. built-in camera, this won’t do). So I’ve pretty much sworn off direct I spent the better part of a connection for the Prime—there are so weekend combing through posts many other ways to shuffle files and on the XDA forums data around, who needs it? (http://forum.xda-developers.com), which is a fantastic resource for all sorts Conclusion of Android hacks, trying to find a nice, One of the great things about Android automated method of mounting the is that the ecosystem is free to come Prime’s SD card. I found a couple resources up with a variety of solutions to a (http://www.omgubuntu.co.uk/ problem and let users sort out which 2011/12/how-to-connect-your- one best fits their needs. It could be android-ice-cream-sandwich- that no one of the above alone will phone-to-ubuntu-for-file-access and suit you—I myself use both SSHDroid http://forum.xda-developers.com/ and FolderSync on almost a daily showthread.php?t=1143044), but basis. But all of the above apps are eventually settled on the script and either free, or have free trial versions, instructions provided via this YouTube so there’s nothing stopping you from video: http://www.youtube.com/ testing them out. Give it a try, and watch?v=3ehnoJn6CEk. After all the robot and penguin will be getting that, I sat down, ready to see the along famously in no time!■ Prime as just another drive in /media, just like the old days. Aaron Peters is a project manager and business analyst at Well, not only is MTP access on Linux a Web/mobile development firm, and he splits his free time inconvenient to use, it’s interminably between learning tech, writing and attacking other people with slow. Once I got connected, I started bamboo sticks. When he and his wife are not trying to corral copying my music collection to the the five animals living with them in Allentown, Pennsylvania, Prime and left it plugged in overnight he sometimes answers e-mail sent to [email protected].
118 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 118 1/23/13 1:07 PM If You Use Linux, You Should Be Reading LINUX JOURNAL™
UU In-depth information providing a full 360- degree look at featured topics relating to Linux UU Tools, tips and tricks you will use today as well as relevant information for the future UU Advice and inspiration for getting the most out of your Linux system UU Instructional how-tos will save you time and money
Subscribe now for instant access! For only $29.50 per year—less than $2.50 per issue—you’ll have access to Linux Journal each month as a PDF, in ePub & Kindle formats, on-line and through our Android & iOS apps. Wherever you go, Linux Journal goes with you.
SUBSCRIBE NOW AT: WWW.LINUXJOURNAL.COM/SUBSCRIBE
LJ226-Feb013-bu.indd 119 1/23/13 1:07 PM EOF
DOC SEARLS Leaving the Land of the Giants The next revolution will be personal. Just like the last three were.
he cover of the December Now here’s the question: Is that 1st–7th 2012 issue of The all that’s going on? Is it not possible T Economist shows four that, in five, ten or twenty years we’ll giant squid battling each other realize that the action that mattered (http://www.economist.com/ in the early twenty-teens was printedition/2012-12-01). The happening in the rest of the ocean, headline reads, “Survival of the and not just among the mollusks with biggest: The internet’s warring the biggest tentacles? giants”. The squid are Amazon, War stories are always interesting, Apple, Facebook and Google. Inside, and very easy to tell because the the story is filed under “Briefing: format is formulaic. Remember Linux Technology giants at war”. The vs. Microsoft, personalized as Linus headline below the title graphic vs. Bill? Never mind that Linux as a reads, “Another game of thrones” server OS worked from the start with (http://www.economist.com/ countless millions (or even billions) of news/21567361-google-apple- Windows clients. Or that both Linus facebook-and-amazon-are-each- and Bill had other fish to fry from the others-throats-all-sorts-ways- start. But personalization is cheap and another-game). The opening slug line easy, and there was enough antipathy reads “Google, Apple, Facebook and on both sides to stoke the story- Amazon are at each other’s throats telling fires, so that’s what we got. in all sorts of ways.” (Raising the Thus, today we might regard Linux as metaphor count to three.) a winner and Microsoft as a loser (or
120 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 120 1/23/13 1:07 PM EOF
at least trending in that direction). tech pubs, because they tended to see The facts behind (or ignored by) the everything as an Intel vs. Motorola stories mostly say that both entities fight. We felt we couldn’t challenge have succeeded or failed largely on either one of those guys head-on, even their own merits. if SPARC was superior on technical Here’s a story that illustrates how grounds (which Sun and its partners stories can both lead and mislead. believed). So we decided the best The time frame was the late 1980s strategy was for SPARC to pick a fight and early 1990s, and the “war” was with another RISC upstart called MIPS. between CISC (Complex Instruction Set This was pure bait for the pubs, Computing, http://en.wikipedia.org/ which came over to this new fight to wiki/Complex_instruction_set_ see what was up. I think we caught computing) and RISC (Reduced MIPS off guard at first, but it defended Instruction Set Computing, itself well and ended up selling years http://en.wikipedia.org/wiki/ later for hundreds of $millions to Reduced_instruction_set_computing). SGI, which eventually went bankrupt. The popular CPUs at the time SPARC is still around, running gear were CISC, and the big two CISC made by Oracle, which acquired Sun. competitors were Intel’s x86 and The big winner in the CPU market Motorola’s 68000. Intel was winning remained Intel and, therefore, CISC. that one, so Motorola and other chip In fact, the x86 architecture still rules, makers pushed RISC as the Next Big at least on PCs and servers, but not in Thing. Motorola had an early RISC mobile devices, where ARM (Advanced lead with the 88000 (before later RISC Machine) now kicks butt. And for pivoting to the PowerPC). what it’s worth, MIPS is now fighting At the time, I was working with ARM in the Android market, and Sun Microsystems and its allies on Motorola’s chip division is the long- SPARC, Sun’s RISC design, which was since-spun-off ON Semiconductor. implemented in various ways by a So, five points here: raft of chip makers, including Texas Instruments, Fujitsu and Cypress 1. Vendors use stories as marketing Semiconductor. In spite of Sun’s heft strategies. in the marketplace, we had trouble getting attention for SPARC with the 2. Vendor war coverage is always to
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 121
LJ226-Feb013-bu.indd 121 1/23/13 1:07 PM EOF
some degree an exercise in Microserfs, is across the Sea of misdirection (http://en.wikipedia.org/ Content in the northwest corner of wiki/Misdirection_(magic)), even the map, bordered by the Cliffs of when journalistic intentions are Surface. In Google Earth are Adsense- worthy ones. land, the Mirkwood of Regulation, the Wastes of Litigation (“Here be 3. The real story is always much more lawyers”), Pagerank Pinnacle (at the complicated than vendor war end of Algorithm Reach), beside which coverage can characterize. lies The Firth of Android. Appleacia has the iPhone Keep. Amazon has 4. “Winners” never win forever, the Cloud Mountains and a volcano especially in tech. named Kindle. Between the latter and Netflix Nation (which lies above the 5. “Losers” don’t always die. Often Satrapy of Spotify) intrudes Pirate Bay. they stay alive by selling out, or Offshore are the eBook Islands. On the they thrive by finding niches and opposite shore are OneClick Castle working them. and Prime Port. Somewhere in the middle, between the Cloud Mountains Now back to our four squid. and Fortress Facebook is the Lost The graphic above The Economist City of MySpace. Out in the Sea of story is an antique-style map Content are small islands called RIM (http://media.economist.com/ Rocks and Nokia. Atop the map is The sites/default/files/cf_images/ Dark Offline. Floating in the Sea of images-magazine/2012/12/01/ Commerce is a Chinese junk flying the FB/20121201_FBD000.png) of the Samsung banner. A peninsula in the fantasy-fiction kind, drawn by David southeast corner features Secondhand Parkins (http://www.davidparkins.com). City, the Bay of E and the Cape of It shows a large mountainous land, Coin. There’s a dragon smiling out with the Sea of Content to the west of the Sea of Commerce, named and the Sea of Commerce to the The Next Big Thing. Finally, in the east. Dividing the land are four center of the map, between the four throne-doms: Applechia, Google thronedoms, is an un-named body of Earth, Amazonia and Fortress water surrounding Identity Island. Facebook. A fifth, Empire of the Parkins’ antique style also depicts
122 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 122 1/23/13 1:07 PM EOF
antique substance in the making— the Maps value-subtract and was because all four of the thrones (or itself an unspectacular successor squid, take your pick) are at least as to the iPhone 4s—which wasn’t affected by their own weaknesses as all that big an improvement on by the strengths of companies they the iPhone 4, which came out way are said to be fighting. And, because back in 2010. Meanwhile, for all so many of us are at their mercy, of Apple’s continued success with their weaknesses are to some degree the iPhone, its entire iOS smart- ours as well. thing hardware market contains just So let’s look at those weaknesses, three devices (iPhone, iPad and iPad and then at where the rest of the Mini) made by only one company. action is, because neither are getting Meanwhile, Android remains an enough attention. open platform with countless First, Apple. hardware implementations from While it’s not wise to bet against a many companies. As I write this, the company as successful as Apple has new Consumer Reports rates various become, it is wise to expect failure Samsung Galaxy devices ahead of the from a company whose success is iPhone, which had formerly topped rightly attributed to a dead and the magazine’s ratings. Countless irreplaceable CEO. Although it was new Android phones also will hit the business as usual for a while after market before the iPhone 6. Steve Jobs perished in September In 2012, Apple also continued 2011, it was clear a year later that the to make fixing or improving its wheels were coming off. First there hardware as hard as possible for was the Maps app debacle, in which anybody not an Apple employee. Apple replaced its Google-based Maps Batteries, RAM and solid-state app on iOS 6 with one based on a storage on new Apple hardware stew of inadequate substitutes—and tends to be hard-wired or -glued. then failed to improve it for months One result is the latest MacBook while Google took its sweet time Pro, with its retina display, which not producing its own Maps app Kyle Wiens in Wired calls “the for the operating system. This not least repairable, least recyclable only hurt Apple and iOS 6, but also computer I have encountered in the new iPhone 5, which featured more than a decade of disassembling
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 123
LJ226-Feb013-bu.indd 123 1/23/13 1:07 PM EOF
electronics” (http://www.wired.com/ become on giant companies and their opinion/2012/10/apple-and- very earth-bound “clouds”. epeat-greenwashing and By coincidence, this month is the http://www.wired.com/gadgetlab/ third anniversary of a column here 2012/06/opinion-apple-retina-displa). titled “The Google Exposure” Credit where due: Apple has been (http://www.linuxjournal.com/ brilliant at retailing and customer magazine/eof-google-exposure). support. On the latter count, nobody In it, I wrote: else is even close. Also, Apple is advantaged by a competitor— I’m just worried about the way Google Microsoft—that seems hell-bent on makes money. Nearly all of it comes sending customers anywhere else. from advertising. That’s what pays for At this point, it’s not clear where all the infrastructure Google is giving Apple is headed. The company’s only to the rest of us. As our dependency “wow” product since Steve died was on Google verges on the absolute, the iPad Mini, which should have come this should be a concern. Think of out years earlier. In the past, it was advertising as oil and Google as one easy to assume that Apple had a “next big emirate. What happens when the big thing” up its sleeve. Now it’s not. oil runs out?...The free rides won’t On to Google. go on forever. There are better ways Last October, Google took the wraps than advertising for demand and off the biggest thing it has in the supply to find each other...and more physical world: giant data centers, which will be found. Google will be in the it immodestly calls “Where the Internet middle of that discovery process, lives” (http://www.google.com/ no doubt. But it’s an open question about/datacenters/gallery/#/). whether Google will make the same The photos doing the bragging are kind of money in a post-advertising as artful as can be, considering that marketplace. I’m betting it won’t. the subjects look like power plants: vast and stark white buildings, with Since then, Google has continued glowing racks inside and huge cooling growing at a 20+% annual rate, and gear outside, veined by an abundance diversifying a bit (for example, by of plumbing. It makes one pause to acquiring Motorola Mobility). But consider how dependent we have the vulnerabilities are still there: for
124 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 124 1/23/13 1:07 PM EOF
Google and therefore also for the rest actually find products and services, of us. Also, the Internet that “lives” rather than merely searching for in Google’s data centers has become them, as they do with Google. an overwhelmingly commercial one, Facebook could host personal especially on the Web. The percentage clouds (http://www.windley.com/ of information on the Web that isn’t archives/2012/11/the_cloud_needs_ about selling something continues an_operating_system.shtml) of downward as more and more eyeball- data kept private for paying routers get into the ad-based game— customers, selectively disclosing and game that game as well. How required data to potential sellers (or far can this go before the whole government agencies, or nonprofits) ad-funded system, with Google at on a secure need-to-know basis— its center, begins to fail in big and treating personal data the way a bank obvious ways? No way to tell, but (as a fourth party) treats customers’ the system we have now can’t go on money. Prototype work on this kind forever. Trees do not grow to the sky. of thing has already taken place at Next, Facebook. Innotribe (http://innotribe.com), An alpha geek told me recently the innovation arm of SWIFT that the most remarkable thing about (http://www.swift.com), the Facebook is the sturdiness of its banking nonprofit that moves infrastructure: it rarely if ever goes $trillions around the world every day. down. Compare that to Twitter, a I know, because I’ve been involved much smaller service notorious for in it. But Facebook won’t go there its familiar “fail whale”. Facebook’s because Facebook, like Google, infrastructure should be good for many sees its main business as advertising things other than housing a locked-in and would rather do business with “social” space where inhabitants get businesses than with individuals. advertised at. What if Facebook started Also, like Google, it would rather offering paid services to its users, sell its users to advertisers than serve turning them into actual customers? For as an intermediary in the far larger example, it could work as a fourth-party retail and services marketplace. agency (http://blogs.law.harvard.edu/ One reason Facebook won’t make vrm/2009/04/12/vrm-and-the-four- that move was suggested to me by party-system), helping customers a top executive at an advertising
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 125
LJ226-Feb013-bu.indd 125 1/23/13 1:07 PM company a couple years ago. He dreading at the same time having an told me the blinders both Facebook “experience” like what it showed. and Google wore were the ones that So far, I’ve met only one Windows keep them focused mostly on each user who likes Windows 8, and other. While this isn’t a verbatim that’s just for some deeply buried quote, it’s close enough: “Google technical stuff. Everybody else either envies Facebook’s ability to get doesn’t like it or hates it outright. personal with users, while Facebook The UI, reportedly nice on phones and envies Google’s ability to put ads tablets, is strange on anything with everywhere on the Web.” Thus, we a keyboard and mouse or trackpad. have locked tentacles rather than The learning curve is more like a wall, evolution by either squid. and—well, nobody asked for all this Next, Microsoft. new stuff. As for the new Surface Today in the mail came our copies tablet, it looks like the second coming of Vanity Fair and the New Yorker, (and going) of the Tablet PC both Condé Nast publications. Both (http://en.wikipedia.org/wiki/ looked different and confusing. Instead Microsoft_Tablet_PC). One version of the usual cover art, there were of the OS doesn’t even run Microsoft’s collections of squares and rectangles Office apps. Some game developers that called to mind the tablet app called the new OS and its Apple-like Flipboard, which organizes “social” “store” for silo-ing apps a “catastrophe” content in the form of picture-tiles (http://www.neowin.net/news/ one can flip through like one would valve-co-founder-windows-8-is-a- a magazine. I have Flipboard, but catastrophe) and a “disaster” its lack of an outline-like organizing (http://www.neowin.net/news/ structure, such as a directory or a table blizzards-rob-pardo-windows-8-is- of contents, annoys me. I thought, not-awesome-for-the-company). This can’t be real. This has to be an ad On the mobile front, Microsoft for something. Then I saw the small teamed up with Nokia to bet the print: “A sample of the new New former mobile-phone giant’s farm Yorker experience on the Windows 8 on Windows-based phones, which desktop.” Oy vay. Microsoft and Condé promptly tanked in the marketplace. Nast hit into a triple-play on that one, Now farmland for both companies is because it made me hate the OS while shrinking like a puddle on a hot day.
126 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 126 1/23/13 1:07 PM EOF
In fact, Microsoft has some legacy to win hearts and minds by standing advantages. It always has been far more behind users—its customers—in the open and supportive toward developers fight against abuses of privacy. than Apple. Unlike Facebook and Instead, Microsoft continues to fight Google, its users are actually paying Google straight-up with an Online customers. And it has always been, Services Division that has lost $billions at its heart, a personal computing over recent years. company. That too should give it a Next, Amazon. kind of advantage over Facebook, Amazon is strongest among The Google, Twitter and everybody betting Economist’s four giant squid, or thrones. on “social” (read: advertising), “the It succeeds, Jeff Bezos says, “by cloud” and “big data”—all of which are starting with the customer and moving corporate/enterprise plays. backwards”. By 2009, Amazon already Over the years, I’ve known and controlled more than a third of all worked with a lot of good people e-commerce (http://www.pcmag.com/ inside Microsoft, all of whom have article2/0,2817,2345381,00.asp). labored to open the company’s Since then, I’ve heard numbers as technology, make it play better with high as 50%. Whatever the number, others in the marketplace, and put you can see the result by looking some truly innovative technologies inside any UPS or Fedex delivery to work. The company’s decision to truck and eyeballing all the boxes default Do Not Track in the “on” labeled Amazon or Zappos (Amazon’s position with the latest rev of Internet shoe store). Explorer was astute, correct and While Apple, Google and Facebook perhaps even brave. It’s the kind all clearly have good engineers and of thing that a clued-in company solid technical infrastructure, Amazon would do. I’ve also seen some tops them all by connecting its excellent Microsoft research on user innards directly both to individuals feelings and preferences in respect and to techies among business to lost on-line privacy. That should customers. It is a rare example of energize Microsoft around some fresh a geek-driven company that also opportunities, but the company seems understands and loves to do business to lack adrenal glands. Opportunities with everybody it can. are lost every day the company fails Amazon’s only shortcoming is one
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 127
LJ226-Feb013-bu.indd 127 1/23/13 1:07 PM EOF
it shares with the rest of retailing, as It is a revolution in which personal well as with its big-squid competitors: empowerment has turned out to be it runs a big data silo where customer good for enterprises because it was data goes in, but not back out to good for individuals. This fact has been individuals. For example, I would like manifest ever since PCs appeared on to have a cooperative data-sharing Earth around the turn of the 1980s. relationship with Amazon, in which To MIS directors in 1983, “personal I tell it everything I own (or feel like computing” was oxymoronic. telling it I own), so it doesn’t bother Computing was a corporate thing trying to sell me what I already have called data processing. It was big but didn’t buy from Amazon. I would and expensive and specialized like my personal API to be one it could and centralized. But those same program against, just as I (or my fourth MIS directors had to start dealing party) can program using its APIs. This with personal computing because requires a respect on Amazon’s part individuals in their organizations for the fact that my life is bigger than and out in the marketplace were the corner that deals with it—and that getting more done with their own I can do more with my own data than word processing, spreadsheets and it can. Also that this will be a Good accounting software than companies Thing for both of us. could get done with their old big-iron But there isn’t any sign that this will data-processing systems. happen, mostly because we don’t yet Likewise, IT directors in 1997 have our own APIs, and managing our had to start dealing with personal own data isn’t something many of us communications (e-mail, instant do yet, least of all so we can deal in one messaging, personal publishing), consistent way with many suppliers. because people in their organizations Mostly, we just fill up hard drives and and out in the marketplace had tools hope whatever we have “in the cloud” of their own that stripped the gears is sort of safe and not going to bite us of what the companies could do with some way in the future. their big old legacy systems. Which brings us to the rest of IT directors in 2009 had to start the world. dealing with iThings and Androids The revolution we’re in is a because that’s what employees and personal one, not a corporate one. users brought to work, and customers
128 / FEBRUARY 2013 / WWW.LINUXJOURNAL.COM
LJ226-Feb013-bu.indd 128 1/23/13 1:07 PM brought to stores, along with zillions of apps that far exceeded what could be Advertiser Index done with company-issued BlackBerries. Thank you as always for supporting our Today’s “big data” bluster—all that advertisers by buying their products! stuff about how marketing can now know more about the customer than
she knows about herself—is mainframe ADVERTISER URL PAGE # talk. Individuals know more about themselves than systems of any kind 1&1 http://www.1and1.com 15
can guess at, no matter how much ApacheCon http://na.apachecon.com 53 data those systems gather. Given the Big Data Tech Con http://www.bigdatatechcon.com/boston2013/ 99 means to control our own lives, with their own personal platforms (not just Confoo http://Confoo.ca 81
ones on their devices, but on their Emac, Inc. http://www.emacinc.com 11 own pile of data), we will be able to do far more with that data than Emperor Linux http://www.emperorlinux.com 13
can any other entity. We also can do Flourish http://www.flourishconf.com/2012/ 33 it cooperatively with other entities, iXsystems http://www.ixsystems.com 7 provided neither of us is busy trying to lock in or control the other. SCALE https://www.socallinuxexpo.org/scale11x/ 2
In the next several years, personal Silicon Mechanics http://www.siliconmechanics.com 3 data and personal operating systems for managing relationships using that data will be as revolutionary as PCs
were in 1983, the Internet was in ATTENTION ADVERTISERS 1996 and mobile was in 2009. We can keep watching giants battle all The Linux Journal brand’s following has they want. But the action that matters grown to a monthly readership nearly one million strong. Encompassing the most won’t be theirs. It will be ours.■ magazine, Web site, newsletters and much more, Linux Journal offers the Doc Searls is Senior Editor of Linux Journal. He is also a fellow ideal content environment to help you reach your marketing objectives. For with the Berkman Center for Internet and Society at Harvard more information, please visit University and the Center for Information Technology and http://www.linuxjournal.com/advertising. Society at UC Santa Barbara.
WWW.LINUXJOURNAL.COM / FEBRUARY 2013 / 129
LJ226-Feb013-bu.indd 129 1/23/13 1:07 PM