A comprehensive masterclass to becoming an instant expert Welcome to Linux & Open Source Despite heavy competition from other operating systems, Linux has not only managed to survive but it has thrived. The recent success of the Raspberry Pi has provided an added impetus for coders, developers and everyday users to take advantage of the many customisable services and utilities that the platform offers. Linux & Open Source Genius Guide Volume 5 is guaranteed to keep any ardent Linux user busy for weeks with the number of useful tutorials, masterclasses and guides on offer. We’ve covered a wide range of fun and useful tips & tricks from creaing your own Windows 8.1 VM to taking pictures and video with the Raspberry Pi Camera. You’ll find tons of useful guides that include dual-booting Linux on any device and an ultimate Qt masterclass. We’ve even featured the essential Linux software for your everyday needs. So, fire up your machine, turn the page, and follow our experts as they guide you through all areas of using and developing Linux.
Linux & Open Source
Imagine Publishing Ltd Richmond House 33 Richmond Hill Bournemouth Dorset BH2 6EZ +44 (0) 1202 586200 Website: www.imagine-publishing.co.uk Twitter: @Books_Imagine Facebook: www.facebook.com/ImagineBookazines
Publishing Director Aaron Asadi
Head of Design Ross Andrews
Production Editor Hannah Kelly
Senior Art Editor Greg Whitaker
Designer Abbi Denney
Photographer James Sheppard Printed by William Gibbons, 26 Planetary Road, Willenhall, West Midlands, WV13 3XT
Distributed in the UK, Eire & the Rest of the World by: Marketforce, Blue Fin Building, 110 Southwark Street, London, SE1 0SU Tel 0203 148 3300 www.marketforce.co.uk
Distributed in Australia by: Network Services (a division of Bauer Media Group), Level 21 Civic Tower, 66-68 Goulburn Street, Sydney, New South Wales 2000, Australia Tel +61 2 8667 5288 Disclaimer The publisher cannot accept responsibility for any unsolicited material lost or damaged in the post. All text and layout is the copyright of Imagine Publishing Ltd. Nothing in this bookazine may be reproduced in whole or part without the written permission of the publisher. All copyrights are recognised and used specifically for the purpose of criticism and review. Although the bookazine has endeavoured to ensure all information is correct at time of print, prices and availability may change. This bookazine is fully independent and not affiliated in any way with the companies mentioned herein.
Linux & Open Source Genius Guide Volume 5 © 2014 Imagine Publishing Ltd
Part of the
bookazine series Contents Your guide to what’s inside
Tips & Tricks 18 Host your own webmail server 60 SSH from a web browser 22 Network scanning using Nmap 64 Master version control with Git 26 Create the perfect Windows 8.1 68 Take pictures and video with the virtual machine Raspberry Pi Camera 30 Understand regular expressions 70 Record slow-motion videos with the Raspberry Pi 34 Use TeX and LaTex to create beautiful documents 72 Browse privately with Onion Pi 38 Set up a wireless access point with a Raspberry Pi 40 Configure your PC for remote maintenance Masterclass 78 Dual-boot Linux on any device 44 Build your own private cloud with ownCloud 88 Get started with Android Studio 48 Monitor network traffic 96 Get to grips with Go Program! 52 Access your network anywhere with a VPN 100 Qt masterclass
56 Use gnuplot for drawing 112 Program with Erlang
Page 8 Build your own distro
6 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Developer guide Essential software 120 Write once, use anywhere 160 Presentation software with Qt 164 Music players 124 Answer real-world questions with Gt 168 Cloud storage services 128 Handle Twitter’s three-legged 172 Integrated development oAuth process 176 Ultimate IRC client 132 Create dynamic templates with Flask, Jinja2 and Twitter 180 System recovery 136 Build a blog with Django 184 Video editors 142 Create a real-time CPU 188 Photo managers meter with Node.js and Duino 146 Build tic-tac-toe with Kivy 150 Get started with Gulp.js 154 Master Vim in easy steps
Page 78 Dual-boot Linux on any device
Linux & Open Source Genius Guide 7 Build your own distro
Discover the different methods available for creating your own customised distro and have everything working your way in no time
here a few reasons why you There are other, niche reasons for wanting to might want to build your own build your own distribution too. You might need to distribution. You might want to put something small and lightweight together for an build a custom install CD to match older computer. You might need to build a live media the policy of your organisation. For ISO that you are able to carry around with you and example, a GNOME desktop with to bring your favourite set of tools to bear when you TChrome as the web browser might be the standard need them. You have the fl exibility to create your desktop where you work. That touches on another own distro whatever your needs. motivation for wanting to create a customised The methods of creating a custom installer: if the creator of the distribution makes a distribution are varied, but they can be divided decision that you don't like. Canonical's decision to into two main categories: you can modify a switch to its own UI, Unity, ranks amongst its most running distribution and then distribute it, or controversial decisions. However, by using some of you can modify the installation ISO (called the methods that we explore here, you could 'remixing') so that it installs your modifi ed create a distribution that is standard distribution in the way you have specifi ed. Ubuntu, but with a traditional desktop We're going to take a look at four different that you are more comfortable with. approaches.
8 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Remastersys Remastersys is a tool that extracts the confi guration from a running Ubuntu Pros or Debian installation and then turns You can use the this into an installable ISO image. This standard tools to confi gure a means that you carry out the customisation using distribution the standard tools that you normally use, such as the package management system and GUI Cons Needs the confi guration tools. When you have everything set expertise to up the way you want it, you can clone the system and carry out the customisations, deploy it. Additionally, you can use Remastersys to doesn't work on make a clone of a working system. all distros, has an uncertain future Fetch Remastersys The development status of Remastersys is currently in transition. At time of writing, the best policy is to visit the Remastersys website and to cut and paste the repository details from there. For example, if you are using Ubuntu 13.10, download the GPG key and add it from the command line with: Tip wget -O - http://www.remastersys.com/ubuntu/ The 'Start the installer remastersys.gpg.key directly' option on the sudo apt-key add remastersys.gpg.key GRUB menu is more then add… dependable than deb http://www.remastersys.com/ubuntu installing via the live precise main CD option …to the end of /etc/apt/sources.lst by invoking a text Q Booting from the installation ISO editor as root. Following that, type sudo apt-get install remastersys remastersys-gui in order to install Remastersys and its GUI. “The future of the GUI portion of the Using Remastersys project seems less certain” When you have the installation set up the way you want it, launch Remastersys by typing sudo you skip this stage, new users will simply have the remastersys-gui. The fi rst option we need to visit is default settings for the distribution. the customisation page which is reached by clicking Finally, build the installation ISO simply by clicking The future of on the Customise button. From here, you can change on the Distribution button on the main menu page. branding options such as the various splash images. The ISO is deposited into the /home/remastersys/ Remastersys From within this page, click on Copy Settings. This folder. Use networking to transfer the ISO fi le to the The long-time developer of Remastersys takes you to a further page on which you can select outside of the VM. We usually install Filezilla and recently decided to give up development. the user whose settings will be copied to /etc/skel/. In transfer to a local FTP server. You can now boot the Fortunately, he has chosen to release the other words, these are the settings that will become ISO on the target machine and carry out a regular source code so that other developers can the defaults for all new users on the new system. If Ubuntu installation. take up the mantle. The future of the GUI portion of the project seems less certain, but Remastersys is also fully functional from the command line. For the moment, the binaries of Remastersys are still freely available from the developer's web site (www.remastersys.com). The situation is constantly in fl ux, so search around for the latest forks. The System Imager project, which uses Remastersys, is a good source of up-to- date information, and can be found here: http://system-imaging.blogspot.co.uk. Q Copying the skeleton information for new users Q Building the ISO
Linux & Open Source Genius Guide 9 Build your own distro
SUSE Studio SUSE Studio allows you to build a customised SUSE Linux installation Pros using a web interface. Although it's Could hardly be easier to easy to use, that doesn't mean it has use, sharing of compromised on options. appliances is built into the site Initially, you choose a base template such as KDE Desktop or Server. From this point, you begin Cons the confi guration properly. The fi rst tab is labelled Build speed varies, Software, which is where you choose software you might hit a wall with really complex packages with an interface that is categorised and customisations searchable. Tip You can upload RPMs that aren't in the standard repos using the software page
Q Waiting for the ISO to build Q Adding in some custom branding Example deployment: Start customising Business desktop As this is a business desktop, let's add LibreOffi ce Here we're going to put together an example to it. To do this, select the Software tab and type appliance. In this case, the appliance will be a the word 'libre' into the search box. The search business desktop that based around GNOME. We'll is real-time, so you should soon be presented add a few customisations as we go along, and we with a list of matches. Note that they are sorted want to fi nish up with an installable ISO that we can by popularity and the package called LibreOffi ce use for deployment. should be at the top of the list. Click the '+add' Begin by setting up an account on the SUSE Studio button to add this package. For a big software suite website (http://susestudio.com). You do this by such as LibreOffi ce, it may take a few moments following the 'Sign In Or Create An Account' link on the for the interface to register all of the needed front page, and it is possible to use one of your existing dependencies. Add Firefox too. Staying in the social networking accounts such as Facebook or Software tab for moment, it's worth noting that you OpenID if you prefer. are able to add extra repositories and even custom Q Selecting software packages to begin Once you have an account, click on 'Create New RPM packages. customising the desktop Appliance...'. On the next screen, choose the GNOME We'll select the localisation options next. Desktop base template, making sure that you are Proceed to the Confi guration tab and select the selecting from the templates that relate to the latest General sub-tab. In here, select the English (UK) version of SUSE Linux. Scroll down to the bottom of option as the language and keyboard layout and the window to choose your architecture and then Europe and United Kingdom as the region and time give your appliance a meaningful name. Click OK, and zone respectively. after a short delay, we can start honing the appliance Note that you could also have selected Ask on to match our own requirements. fi rst boot for any of these options as well. “It's worth noting that you are able to add extra repositories and even
custom RPM packages” Q Booting from the installation ISO
10 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Tip The build process and download speed vary according to server load so get things ready before you need them
Q This screen shows us confi guring details such as the users and network settings in the Confi guration>General page
We'll leave the network options as they are, but this automatically log the user in. Add the command is where you would disable DHCP and specify a static firefox to the Autostart desktop user log-in section to IP address for the workstation, or disable the firewall if automatically start Firefox. Opinions vary, but these you needed to. At the bottom of the page, we can see a options allow the user to get straight to work. list of users and groups. It's a good idea to change the root password from the default. Now click on 'Add new Build the ISO user...' and create a standard user who is a member of The options within the Build tab are particularly the Users group. interesting because they allow you to specify the On the Personalize sub-tab of the Confi guration output format of your custom build. This means page, we can add some custom branding. This might that you don't necessarily have to carry out a full fi t in well with the policies of your organisation, and it installation in order to use your custom build. is also extremely handy for at-a-glance identifi cation For example, you can create a virtual machine that of a desktop within a busy IT environment. will directly boot within a visualiser. If you want to The Files tab is worth a visit if you need to work like this, you will probably need to skip back to Q The SUSE Studio login page add custom files to the distribution. You can the Appliance sub-tab within the Confi guration tab add single files or archives. For example, if you to defi ne the parameters of the VM. Here, you can minutes to complete, depending on how large and wanted to add a file to the desktop of every choose options such as allocated memory and set complicated your custom image is. Although it may new user, you should upload it and specify up the LVM partition arrangement. Apart from the take several minutes for your image to build, once that it should be placed in '/etc/skeleton/ various VM environments you can directly create, you built, your appliances remain on the site and can be Desktop'. If you wanted to place a file within the can also create a traditional ISO installer, a hard disk downloaded without delay. The fi nal tab, Share, is an home directory of the user that you have created called image or a live CD/USB image. intriguing function that allows you share your fi nished John, add it to '/home/john/'. In order to create a traditional installation ISO, appliances with other people. As a finishing touch, pop into the select 'Preload ISO (.iso)' in the Default format Boot the fi nished ISO. Confi rm that you wish to Configuration>Desktop page. Tick the box to and click on the 'Build' icon. This can take a few erase all data on the hard disk when prompted.
Linux & Open Source Genius Guide 11 Build your own distro
Ubuntu minimal installation
Canonical provides a minimal Ubuntu install CD. It's smaller than the regular Pros installation ISO and it installs a minimal Excellent way of keeping the distro version of the distribution. At its most standard yet basic, it gives the user a command line, network minimal too connectivity and not much else. From this bare- bones beginning, it's possible to selectively add Cons components while leaving out most of the cruft that Time consuming to carry out from tends to come with a standard distribution. start to fi nish We're going to work from within a virtual machine for safety and convenience. In our case, we're going to use Oracle VirtualBox but any of the major Q Creating a blank hard disk image in VirtualBox virtualisers will work. Once we have it set up the way we want it, we can use Remastersys to turn it into an ISO that can be distributed. We can then transfer this ISO from within the VM to an FTP server. Example deployment: Minimal Openbox Desktop Fetch the installation media from http://tinyurl.com/ ygawub and create a new virtual machine. 512MB is a sensible minimum when allocating memory, but more Tip memory can also help greatly with speeding things Add the kernel up. An 8GB hard disk file should be adequate for most extensions for your people's requirements. It's usually worth allocating as virtualiser to enable many CPU cores to the VM as you can. things like cut and Begin the customisation paste between host Once you have booted the ISO from within the VM, and guest begin by fi lling in the localisation details using the text mode interface. Next, the installer will attempt to fi nd your network using DHCP. Following the network detection phase, fi ll in a hostname that will be used to identify this computer on the network. Once you've done this, select a mirror that is geographically close to your location. The installer should now begin to download Q Starting a minimal installation packages. Once the packages have come through the network, set up the username for the standard user. You should be able to use common sense to ask the question that comes Clonezilla next, regarding your time zone and default user Clonezilla (http://clonezilla.org) is a live and password. Q Fetching the initial set of base packages CD that can be used to make complete system backups. It uses an algorithm that avoids copying the empty space on “You'll usually find it's worth a hard disk and produces fi les that are as small as possible. You might want to use Clonezilla as an alternative method to allocating as many CPU cores to the distribute a customised distribution as a VM as you can” hard disk image.
12 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Q Fetching the Ubuntu MinimalCD ISO fi le
Tip Using a minimal install CD like this will create quite a lot of network traffic while it is pulling packages through
Q Use automatic partitioning
When prompted, allow the installer to allocate the disk partitioning by selecting the 'Guided - use entire disk' option. Confi rm that you want to write Tasksel to the disk when prompted. The actual layout Tasksel is a system that installs and that you use now isn't important as we will be confi gures a series of packages related producing an ISO that will carry out the installation to a particular 'role' such as that of a web of our custom distribution from scratch. Once the server, a full KDE desktop or a media partitioning has completed, the installer will fetch the packages need for the base installation and workstation. You might be prompted for begin installing them. task selection during an install, but you can access this feature on a running installation at any time. Customisation decisions Q Tasksel helps assign a specifi c role to Add the tasksel package with sudo apt- When the base installation is complete, you will your distro be presented with the Software selection menu. get install tasksel, and then type sudo At this point you have to make a decision. If you First reboot tasksel to run it. You'll be presented with want to, you can select one or more of the provided Upon booting the minimal installation for the fi rst the familiar text-mode interface, space to templates. For example, you could select Kubuntu time, you should be prompted for your username and select an option and tab to switch fi elds. desktop option and have a fairly complete desktop password. We can now start to customise the system. Naturally, you can add multiple tasks. system from the beginning. Install X.org server and Openbox window manager There are other options available to establish (feel free to substitute another WM/DE if you prefer) package manager), LXTerminal (more comprehensive a LAMP web server or a Mythbuntu media by typing sudo apt-get install xorg openbox. terminal application), Mousepad (a GUI text editor) system, and many others too. Most of the rest When this has completed, type startx to test the GUI. and LXDM (graphical login manager). That little lot will of this tutorial assumes that you don't select Click on the backdrop to bring up a menu that will add about 30MB to the installation ISO that you will any of these options so that we can customise allow you to launch a terminal window. create, and about 100MB on the hard disk. completelyfrom scratch. Now you can begin customising the system. Make What you actually add is up to you. Apart More downloading and installation follows. things as comfortable as you like, but remember that from adding packages, you can add desktop Confi rm that you want GRUB installed to the MBR anything that is installed on this system will end up on customisations such as changing the backdrop. when asked. This brings us to the end of the initial the target system. sudo apt-get install firefox When you've got things just how you want them, installation phase. Next, eject the ISO and reboot synaptic lxterminal mousepad lxdm will install create a distribution medium using Remastersys or a the VM when prompted. and set up the Firefox web browser, Synaptic (GUI disk cloning tool such as Clonezilla.
Linux & Open Source Genius Guide 13 Build your own distro
Tip Ubuntu Builder Ubuntu builder creates a /home/ubuntu- builder/ work directory Ubuntu Builder is a GUI application that into which it deposits its allows you to take the contents of a Pros This is a nice, output ISO standard Ubuntu installation ISO and simple tool - modify it to create a new, customised there's little you can do to make this ISO for redistribution. It's a fairly simple application, all go wrong however, and not designed for deep modifi cations of the type that some of the other methods allow. Cons Ubuntu Builder is a standalone application Not a huge amount of customisation that runs on your desktop, and it even runs on depth. Lacked distributions other than Ubuntu. It works by polish and felt a bit buggy in use. ISO modifying a standard Ubuntu installation ISO, build process is downloading and inserting or removing packages also a bit slow for you.
Q The end result: Ubuntu 13.10 with MATE, a more traditional desktop
The installation 01 Start the process by adding the Ubuntu Desktop PPA by typing sudo add-apt-repository ppa:f-muriana/ubuntu-builder into a terminal. Now run sudo apt-get update followed by sudo apt-get Fetch ISO upgrade to update the package lists on the system. 03 You can fetch the current ISO by clicking on the 'Get Ubuntu' button in the main interface. However, Use sudo apt-get install ubuntu-builder to it's worth mentioning that we actually found manually fetching the latest standard install ISO from the Ubuntu carry out the installation. website to be more reliable.
Launch Ubuntu Builder Select and unpack the ISO Add the MATE repository 02 You can now launch Ubuntu Builder by 04 Point Ubuntu Builder to the standard 05 Next, click on the 'Edit sources.list' button. typing sudo ubuntu-builder in the Terminal or by installation ISO by clicking on the 'Select ISO' button. This will open a text editor. Cut and paste the clicking on its launcher icon in the launcher menu. This should invoke the unpack procedure in a Terminal appropriate repository line (beginning with deb) At this point in the process you should be able to see window, enabling us to modify the contents of the ISO. from the MATE installation guide (http://wiki.mate- the basic root interface. Wait for this process to fi nish. desktop.org/download).
14 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Tip Use the Synaptic button to reinstall LightDM, the graphical login manager, when changing desktop environment
Select MATE as the Create the remixed ISO Install the ISO 06 desktop environment 07 Click on the 'Build' button at the top of the 08 Use the installation disk in the same Click on the Select DE/WM button. In the menu, main window. This will open a Terminal window that way that you would normally install Ubuntu... select MATE as the desktop. This should invoke a displays the progress of the ISO build process. This and that's really all it takes! You are now ready Terminal screen while the packages are replaced. might take a long time (an hour or more) depending on to start using your new, customised distribution. Allow this process to fi nish. the speed of your machine. Enjoy!
Linux & Open Source Genius Guide 15 Tips & Tricks Improve the way you work
18 Host your own webmail server 40 Configure your PC for remote 64 Master version control with Git Cut out the middleman by managing your own maintenance Git is quite easy to learn, but you go much further webmail for personal accounts and avoid any Learn how to add the ability to log back in and with an understanding beyond a simple cheat unnecessary downtime maintain a Linux system remotely sheet, so we’ve combined this ‘how-to’ with a little of ‘why’ 22 Network scanning using Nmap 44 Build your own private cloud Discover Nmap and learn how to use tshark to with ownCloud 68 Take pictures and video with the examine Nmap traffic A fast-track guide to setting up your own Raspberry Pi Camera file management system in the cloud using Follow our short tutorial on how to get your new 26 Create the perfect Windows 8.1 ownCloud Pi Camera set up on your Raspberry Pi, and how virtual machine to use it Work in and develop for Windows without ever 48 Monitor network traffic having to properly dual-boot it, by creating a Use tshark to examine network traffic, solve 70 Record slow-motion videos with virtual machine network difficulties and add network data to the Raspberry Pi a MongoDB database Use the updated Raspberry Pi firmware to record 30 Understand regular expressions slow-motion videos with the camera module Leap beyond wildcard expansion, to detailed 52 Access your network searches of text files and accurate validation of anywhere with a VPN 72 Browse privately with Onion Pi program output Work around insecure and restrictive network Turn your Raspberry Pi into a highly secure and access with your own virtual private network very portable router to keep your system safe 34 Use TeX and LaTex to create and your browsing anonymous, wherever you are beautiful documents 56 Use gnuplot for drawing TeX and LaTex: typesetting for perfectionists! Gnuplot is a powerful 2D and 3D program that “We’ll show you TeX isn’t just for numbers, it’s perfect for creating can be used to plot mathematical functions invoices, books and PDFs too and also show scientific data how to get your new 38 Set up a wireless access point 60 SSH from a web browser Pi Camera set up on with a Raspberry Pi There are times when you are stuck using a Discover how to wirelessly connect to your locked-down machine. As long as you have a your Raspberry Pi Raspberry Pi, or any existing network connected browser, though, you can still connect to your to it remote machines and how to use it”
48 60
16 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Linux & Open Source Genius Guide 17 Tips & Tricks
A traditional mail client layout exists in Open and favourite emails are remembered Rainloopthat connects with the folders and between sessions so your unread accounts are emails of your server accurate wherever you log in
Customise your experience with different Thepreview can be made fullscreen in the same folders, extra accounts and even social window, instead of opening a different page or network login support tab – this reduces server load Host your own webmail server Cut out the middleman by managing your own Advisor Rob Zwetsloot models webmail for personal accounts and avoid any complex systems and is a web developer profi cient in Python, unnecessary downtime Django and PHP. He loves to experiment with computing While using webmail may be incredibly webmail application based on PHP, you can quickly convenient, you’re also at the mercy of and easily set up your own webmail server with your another company’s server and privacy own custom settings and email addresses. It has a Resources policies. With the way that people are modern UI and supports all the latest mail server Rainloop: connected online today it’s almost impossible to go protocols, such as SMTP and IMAP. http://rainloop.net/downloads back to the mail client system of old, even if security You will need a server or always-on PC in A server and privacy are far superior to Gmail. order to host your webmail, otherwise it will But there is another solution that satisfi es both only work when your computer is actually on. requirements; by hosting your very own webmail Be aware that it may also increase your bandwidth server you can have both the convenience of usage on a monthly basis, so don’t send huge fi les worldwide access while having the privacy of a over it unless you need to. desktop mail client. By using Rainloop, a free Interested? Let’s get going.
18 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Server know it. If you’re accessing it on a local machine, use 01 ready 127.0.0.1. Enter the following URL: http://[IP address or domain]/webmail/?admin
Log in 07 The username and password for this default Server ready of Rainloop from the website and unzip it. Otherwise, version of Rainloop is ‘admin’ and 12345 respectively. 01 Before we even begin to look at Rainloop in use wget to download it: Log in to access the interface; our fi rst task is to depth, we need to make sure the system hosting it is $ wget http://repository.rainloop.net/ then change the default password. Create a secure server ready. Install the following packages from the v1/rainloop-[current version].zip password using standard password etiquette, as the terminal: And then unzip it to proceed. URL is quite common. $ sudo apt-get install apache2 php5 php5-mysql libmysqlclient15-dev mysql- Elevate permissions server poppassd 04 Open the terminal if it’s not already and use Some or most of these packages may already be cd to move to /var/www/webmail. Run the following installed, but it’s worth checking. two commands to elevate the permissions of the necessary fi les: sudo find . -type d -exec chmod 755 {} \; sudo find . -type f -exec chmod 644 {} \;
Add domains 08 Head over to the Admin Panel where you’ll fi nd domains and then add a domain. Here you can add your own personal email or work domains and Set up directories Final permissions use the IP or server address in either or both of the 02 We’ll need specifi c directories created to get 05 Finish off making sure all the permissions are IMAP and SMTP fi elds. You don’t need to add Gmail Rainloop to work. You can do this in a fi le manager, but set by running: or a handful of other web services, as they are already it works in the command line like so: sudo chown -R www-data:www-data /var/ listed there. $ sudo mkdir /var/www/webmail www/webmail Move to the directory for the next step, as this is Rainloop is now just about ready to go, and most of the where all our Rainloop fi les will live. rest of the setup will occur within the interface itself.
Domain ports 09 Make sure the ports are correct on the server addresses: for a local server, the default ports should be fi ne. Also, make sure that ‘Use short login form’ is Download Rainloop Access Rainloop checked, name the server as the @ address of your 03 Now it’s time to get Rainloop. If you have 06 Now log into Rainloop from a browser; either email (eg example.com) and then click Add to save it a graphical interface, download the latest version through the IP of the server or its web domain if you to the list.
Linux & Open Source Genius Guide 19 Tips & Tricks
Contacts database Log in to 10 To support contacts, we need to add a your email MySQL database. Open up the terminal again 15 and type in: $ sudo mysql -u root -p Enter your password and you’ll be dropped into the MySQL shell to create the database.
Create the database 11 To create our contacts database, enter the following command: create database rainloop; This, as you might have guessed, simply creates the database named rainloop. To confi rm the operation and quit out of the MySQL shell use: “You can have both the convenience exit; of worldwide access with the privacy of a desktop client” you’ve added to the server. It will log you in and work just like any email client, with a column of emails on the left and a preview pane located on the right. You can then expand the emails to take up fullscreen. Rainloop will remember which emails you have read and connect to drafts and sent folders whenever you log in.
Enable Contacts 12 Back in the admin panel on Rainloop, go to the Contacts tab and check the Enable Contacts Activate password plug-in box. Below that, change the type of the database to 14 Now you need to add the plug-in on the admin MySQL. You’ll then also need to add the username panel of Rainloop. Go to the packages tab and fi nd the and password for the MySQL server. plug-in on the list; activate it by clicking the arrow next to release date. Password changing plug-in Add a signature 13 You’ll need to add a specifi c password change Log in to your email 16 Click on the gear symbol at the bottom to plug-in to be able to change any passwords on a Linux 15 To log in to your webmail, go to the webmail access the Settings. From here you can create an mail-server. To do this, open up the terminal and fi rst address for your server: identity, each with an individual signature. This is install the plug-in to your system with: http://[IP or domain]/webmail useful for if you’re using multiple accounts, or if $ sudo apt-get install poppassd And use your normal login details for the mail address multiple people using the same email.
20 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Add a 16 signature
Add extra folders admin page. This way you can consolidate all of your 17 Located next to the settings cog on the emails into one single login. This can be done from the main screen is the new folder button. You can main page, or more preferably from the Accounts tab add folders and then move emails to them in the Settings menu. for better fi ling, and these will sync with the server. You can also have parents and children More on social of folders in your hierarchy. 22 Having your social media work as a log on means that you can only use them on the one account that you've set them up for. It also means that people can more easily log in to your accounts if they have access to your regular system and you don’t log out.
Choose your account 20 Once you’re logged in, go to the Settings menu and locate the Social tab. In here you can choose between the three social media types – you basically need to log in under one or all Manager folders of them here fi rst before adding the feature to 18 In settings you can create folders, but you can the main interface. also delete, hide and edit the folders too. There’s also an option to change the system folders, allowing you to fi lter spam, sent or other types of messages to pre- determined folders. This is currently the only way to fi lter email.
Connect via social media More to come 19 Rainloop has an interesting feature of 23 There are some features missing at the allowing you to log in to your webmail for specifi c moment that you might miss from some regular accounts via Twitter, Facebook or Google sign Add a new account clients – Out Of Offi ce messages and automatic in. Before you have a go at this, make sure that 21 You can add another account to your fi lters, for example. These features will eventually you’re signed in with the webmail account you want to login for that email address, as long as it’s come, whether offi cially or via plug-ins, so keep an eye activate this on. included in the accounts you have created on the out for updates.
Linux & Open Source Genius Guide 21 Tips & Tricks
Scanning an Scanning a iPad Linux machine
About Scanning an Nmap HP printer
Network scanning using Nmap Advisor Discover Nmap and learn how to use tshark to Mihalis Tsoukalos is a UNIX system administrator also profi cient examine Nmap traffi c in programming, databases and mathematics. He has been using Nmap is an open source tool created by When scanning hosts that are not on your Linux since 1993 Gordon Fyodor Lyon that supports port local network, keep in mind that sophisticated scanning, operating system detection intermediate devices such as routers, fi rewalls Resources and version detection and is very famous and proxy servers can mislead Nmap and provide – it has even been seen in movies, including The incorrect information on purpose. Nmap: www.nmap.org Matrix Reloaded and Elysium. Although regular users can perform various Nmap WireShark site: www.wireshark.org Nmap stands for Network Mapper and supports scans, particular command line options demand more than 15 scanning techniques. Nmap can root privileges to run. RFCs: www.ietf.org/rfc.html be useful to network administrators as well as Make sure you experiment with Nmap in order Internetworking with TCP/IP, Volume advanced users and hackers. It can be used for to fi nd out exactly which options and scanning I, Douglas E. Comer, Prentice Hall securing your own network, but it can be also used techniques work best for you, but we’ll walk you for hacking (or cracking) purposes. through some of the options.
22 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
A simple is a good starting point when trying to evaluate 03 scan the security status of a computer. Remember that although TCP SYN scans do not leave any log info, modern fi rewalls and capture programs can detect and report TCP SYN scans.
Get Nmap ports of localhost (127.0.0.1). As you can see from Ping scan (nmap -sP) 01 Installing Nmap on a Debian 7 Linux system the output, the 3306 TCP port is only available 06 This is a very fast scan as it only sends is as easy as running the following command with root through the 127.0.0.1 IP address, which means ICMP Echo Requests and accepts ICMP Echo privileges: that the MySQL server process accepts local Replies. It is not easily noticeable, especially on # apt-get install nmap connections only. busy networks. It does not need special privileges When running Nmap, the –v parameter provides in order to run but it does not return much additional output to the user about the scanning TCP connect() scan (nmap -sT) information and it cannot be used in combination progress and can be used in combination with every 04 The TCP connect() scan is useful when you with other types of scans. It’s used for fi nding the other command line parameter. While an Nmap scan do not have root access to a computer. Its weakness active machines on a network. is performed, you can press the ‘v’ key to increase the is that it leaves traces in the log fi les of the remote verbosity of the output and the ‘V’ key to decrease the computer because it opens TCP session to the verbosity of the output. remote machine.
Find Nmap version 02 You can fi nd the version of Nmap you are using by executing the following command: $ nmap V Nmap version 6.00 ( http://nmap.org ) Platform: x86_64-unknown-linux-gnu Compiled with: liblua-5.1.5 openssl- 1.0.1e libpcre-8.30 libpcap-1.3.0 nmap- libdnet-1.12 ipv6 Compiled without: $ UDP scan (nmap -sU) TCP SYN scan (nmap -sS) 07 The UDP protocol is used by many services A simple scan 05 The SYN scan never appears to log fi les and this type of scan is the only way to detect open 03 The simplest scan that you can execute on a because the TCP connection is never initialised. UDP ports using Nmap. The UDP scan does not Linux system is the following: Its disadvantage is that it requires root access create too much network traffi c and works well for $ nmap localhost to run. What it gives you is information about the examining machines that are running Microsoft The output gives you information about the open open, closed or fi ltered ports of a machine, which operating systems, but it needs root privileges to run.
Linux & Open Source Genius Guide 23 Tips & Tricks
Nmap output 12 options
Scan multiple hosts at once 08 The following command scans 256 IP addresses, from 192.168.2.0 to 192.168.2.255: $ nmap 192.168.2.0/24 “Nmap can be useful to network administrators as well as advanced users and hackers”
saves the output in Normal format, which is suitable for easy reading and printing. Use a text fi le as input (nmap -iL) The Grepable format (nmap -oG
24 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Capture Examine Nmap traffi c using tshark 22 the traffi c 21 You will now learn how to study the traffi c that is created by an Nmap UDP scan using tshark, the command line version of WireShark. First, the network data will be captured. Then, a tshark display fi lter will be applied in order to display the desired traffi c. Finally, specifi c traffi c will be examined and reviewed.
Capture the traffi c 22 The Nmap command that is going to be used is the following: $ sudo nmap sU 192.168.2.10 Just before running Nmap, the following tshark command will be executed for capturing network data: $ sudo tshark w UDPscan.tcpdump
Watch the Nmap traffi c 23 If you want to display the network packets Scan a machine running Mac OS X that go to or come from the host with the 192.168.2.10 17 As far as Nmap is concerned, a machine IP address and also use UDP port 400, run command: that is running Mac OS X does not differ that $ sudo tshark -R "(ip.addr == much from a typical UNIX machine, and its 192.168.2.10) and (udp.port == 400)" -r output demonstrates this. UDPscan.tcpdata x You can examine the network traffi c of other UDP port Scan a Cisco 877W ADSL router numbers using analogous commands. 18 As you can see, it took Nmap 16.07 seconds to scan a Cisco 877W ADSL router – and its guess was pretty accurate! Nevertheless, you have to keep in mind that Nmap operating system detection is not always accurate and therefore cannot be trusted all the time.
Scan a Linux machine (part 2) 15 Next, you should try something smarter, like “nmap –A –T4”. The –A option enables operating system and version detection (know as TCP/IP fi ngerprinting), script scanning, and traceroute while the -T4 option is used for faster execution. Please note that if you run Scan a Windows machine Discuss Nmap network traffi c. the same command without root privileges, you will 19 Windows machines have the habit of having 24 Nmap UDP port scanning starts by get a less detailed output. more open ports than needed, so it is always a good sending empty UDP packets – packets that The –T option is useful for slowing down the scans idea to examine them using Nmap. Here Nmap was have no additional data – to various UDP ports in order to avoid the creation of too much traffi c unable to correctly detect the operating system of the waiting for an answer. that can slow down or fl ood a network or a host. The machine, which dictates the presence of a fi rewall You can see that for UDP port 137 (netbios-ns) allowed values of the option are 0-5. A smaller number somewhere in between. there is a reply that states that the host cannot dictates a slower scan. understand the actual packet, so Nmap knows that Scan an HP printer the port is defi nitely in use! Scan an iPhone 5 20 Network printers also have an IP address, so On the other hand, although a packet 16 Nowadays, a mobile phone can have an IP Nmap can be used to scan them as well. was sent to port 80, the answer was a ‘Port address and therefore can be scanned using Nmap! Nmap found out that it is an HP printer that runs unreachable’ packet, so Nmap implies that It is interesting to scan new devices to fi nd out their HP embedded and has a web server running (ports UDP port 80 is not in use. open ports and their behaviour during a scan. The 80 and 443 are open). Following the Nmap scan, the If there is no reply at all from the host for a output shows that an iPhone does not have very many printer needed a reboot because all of its lights were given UDP port, then the port is considered open ports. blinking! open but fi ltered (port 138).
Linux & Open Source Genius Guide 25 Tips & Tricks
Use VirtualBox to emulate any other Install Windows under Linux with no x86-based operating system or Linux problems thanks to the way VirtualBox distro on a host computer handles virtual machines
Use the tiled interface or the Desktop Share fi les from the host computer and interface in Windows 8.1 – whatever you attached USB storage to the virtual choose to develop for machine to speed up fi le transfer Create the perfect Windows 8.1 VM Work in and develop for Windows without Advisor ever having to properly dual-boot it, by Rob Zwetsloot models complex systems and is a web developer profi cient in Python, creating a virtual machine Django and PHP. He loves to experiment with computing Sometimes using Windows is The virtualisation software, Oracle VM VirtualBox, a necessary evil. A lot of offi ce makes it easy to create virtual machines, and it’s Resources environments still use Windows, and what we’ll be using in this tutorial. It’s a bit of a VirtualBox: sometimes you’ll need Windows-specifi c different process to install Windows, though, and applications and software if you want to work from we’ll be covering that in detail. We can also have it www.virtualbox.org home. There’s also the problem of developing cross- interact with the folders and storage from the host VirtualBox Guest Additions platform apps – constantly switching between machine, allowing you to set up a shared folder to ISO of Windows operating systems is time-consuming and can, access fi les and dev builds. frankly, get exhausting. The solution to a lot of these You’ll need a decently powered, modern PC to be issues is to never dual-boot your machinein the fi rst able to run the VM smoothly, and make sure you have place, but instead to virtualise Windows, in particular plenty of RAM. Windows 8.1, on your computer. Here’s how to create your own virtual machine…
26 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Install 01 VirtualBox
System settings 06 Select the newly created VM and then hit Install VirtualBox should defi nitely try to fulfi l that requirement, Settings on the top bar. Go to the System tab fi rst 01 The necessary fi rst step, VirtualBox is we suggest using half of your system’s and enable EFI if you’re using Windows 8 or later. available in most repositories as the package available RAM. Next, go to Display; make sure Video Memory is set to virtualbox. Some distros – such as Fedora – don’t maximum and tick Enable 3D Acceleration. include it in the repositories; however, you can get binaries along with the source code from the VirtualBox website listed in the resources section.
New virtual machine Storage space Windows disc 02 Start up VirtualBox and click on New on the 04 Windows requires a little more hard drive 07 Click on the Storage tab to add the Windows top menu to create a new virtual machine. It will fi rst space than a Linux distro. VirtualBox will suggest installation disc. Click on the empty disc icon ask you to name it, and will try to fi t a category to the 25GB; however, if you can spare the space, you should underneath the IDE controller, and then the disc icon name. The category is mostly optional, although make up it to 50 or 100 if you plan to use it for a bit more than that shows up on the right. If you have the disc, insert sure it’s set to 32-bit or 64-bit for the version you plan basic development or offi ce work. it into your PC and choose ‘host drive’ from the menu. to use. Otherwise, fi nd and select the ISO.
Storage type First boot 05 Choose VDI as the type of hard drive 08 Once fi nished, click Start while the Windows you want, and then make sure it’s dynamically VM is selected. It will open a window and boot into Volatile memory allocated. This means the space for the storage Windows installer. Go to View on the top settings and 03 You’ll need to set the RAM that the virtual won’t be taken up all at once, so if you’ve created a click on Fullscreen to set the VM to always launch machine will take from the host system. 2GB is the 100GB virtual hard drive, it will only use the space it and stay in full-screen mode. If it’s staying at a recommended amount for Windows 8; while you needs at any one time. Name it and click Create. prompt screen, you may have to disable EFI.
Linux & Open Source Genius Guide 27 Tips & Tricks
VirtualBox 15 additions
Install preparation 09 The fi rst step in installing is to set the language, as you normally would. Click Install and then agree to the licence. After this, you’ll be asked how you want to install; click on the Custom install option to install from scratch.
“You can also access USB storage devices via the host computer”
Storage 10 Unless you plan to do any crazy dual- booting, you will only need to select the empty space on your hard drive. Otherwise, Windows gives you tools that lets you create partitions so that you can install a Linux distro or other Windows version elsewhere.
Personalise Final setup 12 On Windows 8 and 8.1, you’ll now need to 14 Wait a while and it will grab your account personalise your install with a name, colour scheme settings and any other data you may have associated and update settings. By default, Windows will with a Windows 8 install under your Microsoft download and install updates when it sees fi t. You account. After this, it will bring up the main tiled can change the type of updates it will automatically interface for you to start using. install, or turn it to manual. VirtualBox additions Microsoft account 15 You may have noticed that the screen may Wait for it 13 You’ll need to log into, or set up, a Microsoft- be the wrong resolution. To fi x this, you’ll need to 11 Windows can take a while to install, and based account to use the latest versions of Windows. install the guest additions via VirtualBox. Bring up will go through several phases, including rebooting If you already have one, you can enter it here and log the VirtualBox menu pop-up at the bottom of the once or twice during the process. Leave it alone and in. Otherwise, you’ll need to link an email address to a screen and click Devices and Insert Guest Additions it will do its thing without any interruptions. new account. CD image. It will automatically download it if you don’t
28 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Share 17 settings
already have it. Right-click and select All Apps, then Host USB should be prominently displayed – if not, make sure fi nd the Computer. Click it and fi nd the CD. 19 You can also access USB storage devices, you’ve updated Windows. Otherwise, click the app and Open it and click on the relevant Windows .exe for like an external hard drive or USB stick, via the host download it. During the upgrade process, you’ll have to your system and follow the instructions. computer. You’ll have to do it manually for each USB restart and reconfirm some settings. device, though. Plug in the fi rst USB device that you want to connect to the virtual machine and make sure Shared folders it’s mounted. 16 One of the better things we can take advantage of with a virtual version of Windows is a shared folder that easily allows transfer of fi les between the host system and Windows. Create a specifi c folder on the host to use a shared folder.
Share settings 17 Go back into the VM and click Devices followed by ‘Shared folder settings’. Click the folder symbol with a plug sign. Click the down arrow and Other to then browse your fi le system and select the shared folder we set up. Name it if you wish. Boot to desktop 22 Windows 8.1 now has the ability to boot into the desktop. Once you’ve completed the update, enter the VirtualBox setup Desktop. Right-click on the taskbar and open Properties, 20 Go into the settings for the Windows VM and go to the Navigation tab and then select ‘Go to the select the USB tab. Make sure the USB controllers desktop instead of Start when I sign in’. That’s actually are enabled and then click the USB plug with a plus the name of the option. symbol. Select your USB device and it will add it to Access the share the VM; whenever it’s plugged in, it will automatically Finish up 18 Make sure you’ve selected automount and mount on Windows. 23 After a reboot, that setting will come into then press OK. Windows will now easily be able to see effect. You’ll now be able to properly start using your it in the fi le system; you’ll need to go to Network then Update to Windows 8.1 virtual Windows machine for anything you’d use a VBOXSVR to fi nd the folder. You can then drag it to the 21 If you’ve installed Windows 8, updating to 8.1 normal Windows machine for, without having to dual- desktop to create a shortcut. is free. Go to the Store and the Windows 8.1 update boot and constantly switch back and forth.
Linux & Open Source Genius Guide 29 Tips & Tricks
Even the simplest regexp can Online regexp testers like 2problems.com let you make you more productive at refi ne your regexps – though ’perl pi.bak’ the command line also gives you a backup of your original test fi le
We’re going to show you use of the Regexps are great, but it’s important to know their metacharacters one-by-one, until ‘. * [ ] limitations – they’re ideal with text fi les, such as ^ $ \ ? | { } ( )’ is more than just line noise logs, but steer clear of complex HTML searches Understand regular expressions Advisor Leap beyond wildcard expansion, to Richard Smedley started using computers long before detailed searches of text fi les and WYSIWYG, and still maintains that the command-line, and Emacs, accurate validation of program input is the most productive working environment We’re always searching for something (BRE); grep -E (or egrep) gives Extended Regular – the fi le where we wrote that recipe Expressions (ERE). For other languages, most (Python or baking); the comment in adopt PCRE (Perl Compatible Regular Expressions), 100,000 lines of code that points to an developed in 1997, by Philip Hazel, and understood Resources unfi nished module; the log entry about an iffy by many languages, though not always implemented Server like LAMP/WAMP/MAMP connection. Regular expressions (abbreviated as in exactly the same way. We’ll use grep -P when we regexps hereafter, but you’ll also see regex and re) need to access these. Emacs has its own regexp ownCloud plug-ins: are a codifi ed method of searching which, to the style but, like grep, has a -P option to use Perl- apps.owncloud.com unenlightened, suggests line noise. Yet, despite a compatible regexps. history that stretches back to Ken Thompson’s 1968 This introduction is mostly aimed at searching QED editor, they’re still a powerful tool today, thanks from the shell, but you should easily be able to adapt to grep – ‘global regular expression print’. Using grep it to standalone Perl scripts, and other languages exposes only the limited Basic Regular Expressions which use PCRE.
30 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
POSIX 09 classes
Word up! 01 You’re probably used to searching a text fi le for occurrences of a word with grep – in that case, the word is the regular expression. More complicated regexps are simply concise ways for searching for parts of words, or character strings, in particular positions.
Reserved character 02 Some characters mean special things in regexp pattern matching: . * [ ] ^ $ \ in Basic Regular A to Z Guide Expressions. The ‘.’ matches any character, so using 08 The range in [] can be anything from the it above doesn’t just fi nd the full stop unless grep’s -F ASCII character set, so [ \t\r\n\v\f] indicates the option is used to make the string entirely literal. whitespace characters (tab, newline et al). [^bd] oom$ matches all words ending in ‘oom’, occurring Mmmmm, cooooool at the end of the line, except boom and doom. 05 The other quantifi ers are + for at least one of the preceding regexps (‘_+’ fi nds lines with POSIX classes at least one underscore) and * for zero or more 09 The POSIX classes for character ranges (coo*l matches col, cool, coooooooool, but not cl, save a lot of the [A-Za-z0-9], but perhaps most useful for different spellings of mmmmmmmmm or useful is the non-POSIX addition of [:word:] which zzzzzzzzzz). matches [A-Za-z0-9_], the addition of underscore helping to match identifi ers in many programming Atlantic crossing languages. 03 Extended Regular Expressions add ? | { } ( ) to the metacharacters. grep -E or egrep lets you use them, as above, where ‘standardise|standardize’ can match British or American (and ‘Oxford’) spellings of ‘standardise’.
No number 06 Feeling confi dent? Good, time for more goodies. [0-9] is short for [0123456789] and matches any element in the square brackets. The ^ inside the brackets is a negation, here matching any non-number but the other ^? … Colourful? 04 ‘|’ gives a choice between the two Start to fi nish ASCII style characters in the parentheses – standardi(s|z)e 07 The ^ matches the expression at the 10 Where character classes aren’t implemented, – saving unnecessary typing. Another way to fi nd beginning of the line; a $ matches the end. Now knowledge of ASCII’s underpinnings can save you time: both British and American spellings is ‘?’ to indicate you can sort your document.text from text.doc and so [ -~] is all printable ASCII characters (character one or zero of the preceding element, such as the u fi nd lines beginning with # or ending in a punctuation codes 32-127) and its inverse [^ -~] is all non-printable in colour. mark other than a period. ASCII characters.
Linux & Open Source Genius Guide 31 Tips & Tricks
Bye bye, 14 IPv4
Beyond grep 11 Find and Locate both work well with regexps. In The Linux Command Line (reviewed in LUD 111), William Shotts gave the great example of find . -regex ‘.*[^-_./0-9a-zA-Z].*’ to find filenames with embedded spaces and other nasties.
Nice one Cyril “Regular expressions are like a 12 Speaking of non-standard characters, while [:alpha:] depends on your locale settings, and may only find ASCII text, you can still search for characters of particularly spicy hot sauce – to other alphabets – from accented French and Welsh letters to the Greek or Russian alphabet. be used in moderation and with restraint, only when appropriate”
Bye bye, IPv4 14 FOSDEM was all IPv6 this year, so let’s not waste any more time on IPv4 validation, as the future may actually be here. As can be seen in this glimpse of IPv6 validators, despite some Perl ‘line noise’, it boils down to checking appropriate amounts of hex.
Validation 15 By now regexps should be looking a lot less like line noise, so it’s time to put together a longer Ranging repeat one, just building from some of the simpler parts. A 13 While {4} would match the preceding element common programming task, particularly with web if it occurred four times, putting in two numbers gives forms, is validating input is in the correct format – a range. So, [0-9]{1,3} in the above screenshot fi nds such as dates. be fi ltered using current date). Note that one-, two- or three-digit numbers – a quick fi nd for In this case we’re looking at validating dates, (0[1-9]|[12][0-9]|3[01]) checks numbers 01-31, but dotted quads, although it won’t fi lter out 256-999. eg for date-of-birth (future dates could then won’t prevent 31st February.
32 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
A regexp 23 too far
Perl -pie 21 Aside from grep, Perl remains the most comfortable fi t with regexps, as is far more powerful than the former. With perl -pie on the command line, you can perform anything from simple substitutions on one or more fi les, to…
Perl one-liner 22 …counting the empty lines in a text file (this from Krumin’s Perl One-Liners, see next month’s book reviews). /^$/ matches an empty line; note Perl’s use of // to delimit a regexp; ,, could also be used if / is one of the literals used. Boundary guard 18 As well as ^ and $ for line ends, word A regexp too far boundaries can be matched in regexps with 23 Now you know the basics, you can \b – enabling matches on, say, ‘hat’ without build slightly more complicated regexps – but, matching ‘chatter’. The escape character, \, is as Jeff Atwood said: “Regular expressions used to add a number of extra elements, such are like a particularly spicy hot sauce – to be as \d for numerical digit. used in moderation and with restraint, only when appropriate.”
Literally meta 19 Speaking of boundaries, placing \Q \E around a regexp will treat everything within as literals rather than metacharacters – meaning you can just quote a part of the regexp, unlike grep -F where everything becomes a literal. Back to basics 16 Now we have the basics, and can string them together, don’t neglect the grep basics – here we’re looking at how many attempts at unauthorised access were made by SSH in a given period. An unnecessary pipe replaced with grep -c.
Why vi? Lazy = good Tagged offender 17 Whatever your position in the venerable and 20 Time to think about good practice. * is a 24 Finally, know the limitations of regexps. Don’t affectionate vi/Emacs war, there will be times and greedy operator, expanding something like <.*> by use on HTML, as they don’t parse complex languages servers where vi is your only tool, so grab yourself a grabbing the last closing tag and anything between, well. Here the legendary StackOverflow reply by Bob cheat-sheet. Vi and vim mostly follow BRE. Here we including further tags. <.*?> is non-greedy (lazy), Ince to a query on their use with HTML expresses the see one of the \< \> word boundaries. taking the fi rst closing tag. passion this question engenders.
Linux & Open Source Genius Guide 33 Tips & Tricks
\usepackage imports a macro package – in this case maths – Run latex on the .tex text fi le and it gets to and that’s where the power lies. There are macros for everything. work doing all the typesetting, so you only Note too what \LaTeX produces in the right-hand panel need to worry about structure and content
Your text editor will provide highlighting of LaTeX You can use TeX’s xdvi viewer to look at the document, or comments and commands, making it easy to just convert it to PostScript or PDF for printing – and thanks to read back either text or structural information tablets, PDFs are undergoing a small-screen renaissance Use TeX and LaTeX to create beautiful documents Advisor Learn how TeX isn’t just for numbers; it Richard Smedley started using computers long before also makes invoices, books and PDFs WYSIWYG, and still maintains that the command-line, and Emacs, You use LibreOffi ce for your documents, numbers of pixels per inch, beautiful PDFs become is the most productive working environment perhaps Scribus for making a company important, so feel the irony as you produce them brochure, and edit most documents from the *NIX command line. online. You’ve heard of TeX – in the TeX itself is simply a formatting engine, stable and Resources context of mathematical equations, mostly – but virtually bug free. LaTeX is a complete document TexLive: why would you use it to lay out everything from preparation system built as a macro package on TeX, www.tug.org/texlive/ invoices to 100-page PDFs? Quite simply, because and containing templates and add-ons to create Comprehensive TeX Archive it produces beautiful-looking documents that are just about any document. We’ll use LaTeX2e, known Network: easy to maintain, and using a simple markup that as Standard LaTeX, which has barely changed in 20 www.ctan.org/ makes keeping track of document evolution and years – it hasn’t needed to. changes far simpler than it is with word processors The commands are straightforward, but there and desktop publishing packages. Format once for are many options. Follow our quick intro, modifying each type of content you write, then concentrate just the documents, then move online and build bigger, on the content. Thanks to tablets with insanely large better docs as your knowledge increases.
34 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
06 Right-to-left
Bugs will be features! 01 TeX is extremely stable, with years passing between bug reports! Since release 3, the version number of TeX is converging to π, and is now at 3.1415926 (released March 2008). The version number will be changed to π upon creator Donald Knuth’s death, and all future bugs found are to be regraded as features!
TeX Live WYSIWYM formatting 02 TeX Live is the recommended way to install 08 These strengths are a product of the What the LaTeX packages you’ll need. It’s available for You See is What You Mean (WYSIWYM) paradigm. almost all distros. To try our invoice example, also You write your .tex document in any text editor and install the latex-invoice RPM, or apt-get install let LaTeX take care of the typesetting: running latex texlive-latex-extra. on the fi le produces a .dvi, which you can convert to PostScript or PDF. TeX in your browser 03 You needn’t wait until you can download It’s just text and install TeX Live. JaxEdit is an online LaTeX 09 Remember it’s just text. With structure. editor with live preview – for maths but not much And a few commands that all begin with \ and may else yet – which you can try at jaxedit.com. You can Just a fraction have braces {} and possibly square brackets []. The also download the JS and run it in your browser. 05 The step 4 example code produces this problem for the beginner is knowing where to start output when converted to a PDF. We’ll look at some with so many options, so let’s start simply. of the commands in a minute, so you can get started trying LaTeX out, but fi rst we want to make sure that you realise it’s not just for maths.
Right-to-left 06 LaTeX spread through the academic world and quickly gained popularity with linguists and language researchers, leading to excellent layout of non-Latin alphabets. Pictured at the top of the page is an example of Arabic in ArabTeX (which also handles Hebrew). e=mc2 Macro power 04 Most people have heard good things Booked up 10 Remember those imported macros on the about TeX and LaTeX in the context of laying out 07 LaTeX continues to be virtually the default tool title-page screenshot? There are thousands of mathematical equations. In fact, it was because of for producing postgraduate theses and other academic user-contributed packages at CTAN. Here, your how badly typesetters were laying out mathematical works – this is because of its structural strengths, and advisor is looking at a Dutch .sty for producing equations that Donald Knuth felt compelled to write consequent abilities at producing bibliographies and invoices. Useful, but not as powerful as the offi cial TeX originally… nested contents pages. invoice package…
Linux & Open Source Genius Guide 35 Tips & Tricks
Draught preview and place them on top of the following letter. Find 15 In the pic at the bottom of the page, we’re oodles more in The Comprehensive LaTeX Symbol previewing an earlier draught of the invoice. Note List, by Scott Pakin. we’ve used separate programs – GEdit, xterm and xdvi – in this case, but done it all within Emacs on the screenshot of the opening page. The point is, we’re dealing with text, so it’s easy to fi t in with your workfl ow, and the tools with which you are familiar. It might also be a time to try something new [will you stop pushing people onto Emacs, already – Ed], or even use LaTeX to generate your slides, or your website’s PDF downloads.
Special character 16 Note the \& in LinuxUser \& Developer – &, like #, $, %, ^, _, {, }, ~ and \, is a special character, with special meaning to LaTeX, so must be called like: \# \$ \% \^{} \& \_ \{ \} \~{} Error messages \textbackslash. 18 For more complex documents, with More macro power references and tables of contents, you may have 11 …which can do the adding up for you. Having Like the hat to rinse and repeat the latex command until tried in the past to produce invoices both from 17 Note the {}s after the escaped ^ and ~ in everything is fully processed. If there’s a problem, spreadsheet templates and word-processors, it is the previous step. This tells LaTeX that they are LaTeX will tell you about it. Here, we’ve left in special with great relief that your advisor turns to something standalone, otherwise it will treat them as accents characters by mistake. that just needs text to be fi lled in, yet produces readable output. “The best way to learn is, as with Boilerplate 12 Everything before \begin{} is what’s called any coding language, by reading .tex boilerplate, analogous to
in an HTML document. \documentclass must be specifi ed in the {} braces: {letter}, {article} or {book} are documents out there and making common types. Options such as twoside for double- sided printing, paper size, and font size go in the [] your own” brackets. 12-16Make a date 13 \usepackage{}, \title{}, and \date{} are common boilerplate. \include{fi lename.tex} will import other fi les, so big projects can be split, such as a short story collection of chapters from different authors. Text commands can be embedded in fi elds, as here with date.
Desktop calculator 14 Using the \Fee command followed by {activity}, {rate} and {units done} for each invoice item lets invoice.sty work out totals for you and produces clear, readable invoices (see main picture). VAT options are also available.
36 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Pretty pictures 19 LaTeX seems to work best with graphics from EPS fi les and PDFs, or tools generating vector graphics from data on-the-fl y. However, your advisor doesn’t excel at all things visual, and has seen others do more than dump a pic in a doc, as here.
txt2tex Know the score 20 If you’re converting a lot of documents to 21 The WYSIWYM choice for music is not so TeX, txt2tex will take some of the tedium out of the clear, with Philip’s Music Writer (PMW) arguably job – download it and run the demo. But you’re by no easier to learn than TeX’s MusicTeX markup. means limited to text-based documents in TeX. See Additionally, Lilypond, a standalone MusicTeX fork, what LaTeX can do with music and games… produces beautiful scores. Choices, choices.
TeXnicians 23 and TeXperts En passant 22 Install one of the chess packages (skak or TeXmate) and you can draw a board from Forsyth– Edwards Notation: \fenboard{r5k1/1b1p1ppp/p7/1p1Q4/2p1r3/ PP4Pq/BBP2b1P/R4R1K} With TeXmate you don’t need the numbers to end lines.
TeXnicians and TeXperts 23 There are some great guides to using LaTeX – The Not So Short Introduction to LaTeX2¡ (Or LaTeX2¡ in 157 minutes) by Tobias Oetiker, Hubert Partl, Irene Hyna and Elisabeth Schlegl is a good concise start, but don’t just read a book...
Try it out 24 The best way to learn is, as with any coding language, by reading .tex documents out there and making your own. If you’ve got a document you’re about to write, give yourself an extra hour and fi nd a .tex with similar structure to build it upon.
Linux & Open Source Genius Guide 37 Tips & Tricks
Create an access Our expert’s Raspberry Create an access point for a point for an existing Pi with a suitable new independent network wired network wireless N dongle that can be used anywhere
Set up a wireless access point with a Raspberry Pi Advisor How to wirelessly connect to your Liam Fraser Liam is the creator of the Raspberry Pi, or any existing network RaspberryPiTutorials YouTube series and volunteers as a Linux server administrator for the Raspberry connected to it Pi Foundation A Raspberry Pi with a wireless dongle fi rewalling rules to make sure they can’t connect to Resources that supports Access Point mode is a very anything on your LAN. Another is simply connecting versatile tool. n Latest Raspbian Image As well as connecting to to it over wireless rather than Ethernet, which can existing wireless networks, the Raspberry be useful if the Ethernet port is already in use. For raspberrypi.org/downloads Pi can become an access point and have multiple example, a Raspberry Pi acting as a portable PXE A router or switch devices connected to it. It can either have its own server (used to boot a computer over the network) A USB wireless dongle that network or bridge onto an existing one and make it would have an Ethernet cable connected to the supports Access Point mode wireless. One potential application of this is to create computer it’s booting, but you might need to log in and A Wi-Fi device to test with a wireless access point for guests, with appropriate see what’s going on at the same time.
38 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Install the required software network, but we will set up our own later on. Reboot Create an independent network 01 Log into the Raspbian system with the using sudo reboot so that the changes take effect. 06 We’re now going to change the network username pi and the password raspberry. Get the confi guration to be independent from your latest package lists using the command: Confi gure hostapd existing network. The fi rst thing to do is to pick a 04 We fi rst need to edit /etc/default/hostapd private address range that you’d like to use. For sudo apt-get update to start the daemon on boot. Uncomment the this example, we’ll use 192.168.0.0/24. However, ‘DAEMON_CONF’ option and replace it with: we recommend using a random address range in We’ll be using hostapd, which is a daemon case you ever want to route to other networks. The that handles access point management and DAEMON_CONF="/etc/hostapd/hostapd. fi rst thing we’ll do is assign a static address to the authentication. We’ll also need bridge utils, which conf" bridge interface. Change the bridge section of /etc/ will be used to bridge the Ethernet and wireless network/interfaces to look as follows: interfaces together. iw is used to get information We then need to create the hostapd confi g fi le at the about the wireless interface, and dnsmasq is used path specifi ed above. It should have the following iface br0 inet static as a DHCP server. Install these with: contents: bridge_ports eth0 wlan0 bridge_waitport 0 sudo apt-get install hostapd bridge- interface=wlan0 address 192.168.0.1 utils iw dnsmasq bridge=br0 network 192.168.0.0 driver=nl80211 netmask 255.255.255.0 Check for AP mode country_code=UK 02 It’s crucial that you have a wireless dongle ssid=pinet Confi gure a DHCP server that supports Access Point (AP) mode. To verify that, hw_mode=g 07 We installed dnsmasq in the fi rst step. connect the wireless dongle and type iw list. There channel=1 Confi guring it is really simple. Start by taking a copy will be a section titled ‘Supported interface modes’. wpa=2 of the default confi g: Ensure that AP mode is in that list. Our expert’s wpa_passphrase=super_secret output looked like this: wpa_key_mgmt=WPA-PSK sudo cp /etc/dnsmasq.conf /etc/ Supported interface modes: ieee80211n=1 dnsmasq.conf.orig * IBSS wmm_enabled=1 * managed Then edit /etc/dnsmasq.conf to contain the * AP Note that if you don’t have a wireless N capable following lines: * AP/VLAN device, or are having stability issues, you’ll want * WDS to leave out the last two lines. You may also need interface=br0 * monitor to change the wireless channel if you are having dhcp-range=192.168.0.2,192.168.0.254, * mesh point stability issues. Once you’ve done this, you can 255.255.255.0,12h start the hostapd daemon with sudo /etc/init.d/ hostapd start. Reboot for the changes to take effect. You’ll want to Bridge interfaces disconnect the Raspberry Pi from your existing network 03 We’ll bridge the Ethernet interface and the Connect to the network at this point, otherwise there will be two DHCP servers wireless interface so they can share the same IP 05 The SSID in the confi g fi le is the network on the network. address and traffi c can pass between them. To do name, and the WPA Passphrase is the password. this, edit /etc/network/interfaces (using sudo) and When you connect to the network, the connection Try it out change it to look as follows: will be bridged to the existing wired network and 08 Devices on the wireless interface will be you should receive an IP address from your existing able to talk to devices on the wired interface, and auto lo br0 router doing DHCP. Congratulations on creating an any device plugged into the Pi will get an IP address access point! so that it can talk to the other devices. iface lo inet loopback
allow-hotplug eth0 iface eth0 inet manual “It’s crucial that you have a wireless
allow-hotplug wlan0 dongle that supports Access Point iface wlan0 inet manual (AP) mode. To verify that, connect iface br0 inet dhcp bridge_ports eth0 wlan0 bridge_waitport 0 the wireless dongle and type iw list.
At the moment, we’ll be bridging onto an existing Ensure that AP mode is in the list”
Linux & Open Source Genius Guide 39 Tips & Tricks
Install Webmin, the Open a terminal on the Use VNC over SSH to gain Via SSH tunnelling, you can run extremely compressive web- remote computer and remote desktop access to full X Window applications on based system confi guration because it’s SSH, it’s put yourself in full control of your desktop while they execute and monitoring GUI completely secure the other computer on the remote computer
Configure your PC for remote maintenance Advisor How you can add the ability to log back in Michael Reed is a technology journalist and he’s been hacking and maintain a Linux system remotely away at Linux for over 15 years If you’ve set up a PC using a Linux distro confi guration. Note that once you have SSH terminal for a non-expert user or family member access up and running, you can carry out the rest of to use, you’ve made the fi rst step in the tutorial remotely over the LAN. In fact, if you then Resources providing them with a secure and well- skip ahead to the port forwarding section, you can do Client computer running Linux featured computing platform. However, if the user the other stages of the tutorial over the internet. Remote computer running Linux in question needs a bit of hand-holding, it’s only a Initially, this tutorial will assume that you have LAN matter of time before you will need to carry out some both the client computer (your computer) and the maintenance that can’t be explained over the phone. remote computer (their computer) attached to This project involves installing SSH for command- your LAN. Ultimately, we will set up the router in the line access and for secure tunnelling, VNC for remote owner’s home and a dynamic DNS server so that we desktop access, and Webmin for overall system can fi nd the remote machine on the internet.
40 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Set up the network 01 This tutorial presumes that you have the remote computer and the client computer on your LAN. Make a note of the login username and password of the remote computer and its IP address (type ifconfig to fi nd it).
a password and a username. Type ssh-keygen -t dsa to begin. You can hit Enter to accept the defaults when prompted, but type in a password when asked.
Install SSH 02 To enable remote terminal logins, install the SSH server by typing sudo apt-get install openssh-server (or yum -y install openssh- server on Red Hat). Open /etc/ssh/sshd_confi g in a File copying (SSH) text editor (as root) to begin confi guring SSH. 07 You can place fi les on the remote computer or fetch fi les from it with the scp command. scp [re m ote user]@[remote IP]:[local file] Copy key [destination directory] to fetch the fi le(s) onto 05 Copy the pubic key over to the machine that the remote machine and scp [local file] [re m ote will be used for remote maintenance. You carry this user]@[remote IP]:[destination directory]. out by typing ssh-copy-id [re m ote login name]@[IP address of remote computer] on the client computer.
Confi gure SSH server 03 Alter the line PubkeyAuthentication so that it reads PubkeyAuthentication yes. Make sure that PasswordAuthentication is set to no. This is because we want to use encrypted keys rather than usernames and passwords to remotely log into the machine. Save Test SSH the file and type sudo service ssh restart to 06 SSH into the remote box from the client restart the server. by typing ssh [remote username]@[remote IP address]. This should give you command line on the Generate keys other machine without prompting you for a password. Web proxy (SSH) 04 We need to generate both a public and Try out a few commands to test it, and type Ctrl+D to 08 Let’s say that the user complains that private key so that we can log into SSH without using end the session. they can’t access a specifi c website or the web
Linux & Open Source Genius Guide 41 Tips & Tricks
Connect to VNC 12 There are a few VNC clients, but this example presumes that you have installed Vinagre. Use it to connect to the VNC server at localhost. Remember, we connect to localhost because we set up the SSH tunnel for all traffi c to the remote computer on port 5900 in the previous step. You should now have remote desktop access to the remote computer. If there are any problems with the connection, examine the text output in the terminal for errors.
in general. We can test the connection by using apt-get install x11vnc (or yum install vnc- SSH tunnelling. Type ssh -D 8888 [username]@ server on Red Hat). [IP address] to begin. This will establish a local proxy server that is actually accessing the web via the remote machine. You can then confi gure Firefox to access the web through this proxy by specifying Socks 5 proxy on localhost. If you can SSH to the other machine and access the problem site, it must be a software problem on the other machine, rather than a networking problem.
Run X applications (SSH) Install Webmin 09 You can use tunnelling to remotely run 13 Proceed to the downloads section of X applications. Just the thing to, for example, the Webmin website (www.webmin.com) and edit the confi guration in a GUI application Set up VNC tunnelling download the package for your Linux architecture. without using full desktop sharing. Type ssh -X 11 and start server In the case of Debian, install the .deb fi le by [userna m e]@[rem ote IP] to open the connection. Set up an SSH tunnel for VNC by typing ssh -L typing sudo dpkg -i [name of .deb you You then start the application in the normal 5900:localhost:5900 [remote user]@[remote downloaded]. Follow this up by typing sudo apt- way from the command line. Type firefox & IP]. As well as setting up the tunnel, this launches get -f install to satisfy the dependencies. In to start Firefox on the remote machine and a terminal connection. In the terminal, type x11vnc the case of Red Hat, type yum install [name of interact with it on your local desktop. You can -safer -localhost -nopw -once -display :0. RPM]. reduce things to a single command by typing ssh -f -T -X [username]@[ip address] [name of application]. “In this case, we’re going to ignore Set up the VNC server 10 We’re going to install VNC for remote the VNC server’s own security desktop access. In this case, we’re going to ignore the VNC server’s own security features and simply features and simply tunnel it over tunnel it over SSH instead. Begin by installing the VNC server on the remote machine by typing sudo SSH instead”
42 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Test Webmin 14 Webmin runs in SSL mode by default, so access it by browsing to https://[IP address of remote computer]:10000. Accept the browser certifi cate that it offers. Use either the root username and password or regular username and password that can run sudo, in the case of Ubuntu. In the future, tunnel to Webmin for maximum security without having to open up an extra port on the router. We use the same technique as with VNC. Type ssh -L 10000:localhost:10000 [username]@[IP address] in order to set it up and then browse to https://localhost:10000.
Set up static address Locate remote IP address 16 How you set up a static IP address varies from 18 on the internet distribution to distribution. On an Ubuntu machine, You can ask the person who runs the remote system click on the Network icon in the Control Panel and to simply visit whatismyip.com and email their select Edit… and then select the IPv4 Settings tab. current IP address to you, but setting up dynamic Select Manual from the Method: drop-down list. DNS is a more robust solution. Go to the No-IP Now add an IP address that is part of the current website (www.noip.com) and set up a free account. subnet. For example, if typing ifconfig reveals Now install the ddclient package. that the current IP address is 192.168.0.5, choose something higher such as 192.168.0.100. You may have to manually add DNS server addresses for your ISP. Google for them. Accept the changes and reset the machine. Set up the remote network 15 It’s time to return the computer that we have set up for remote maintenance to its home base. Once the computer is reconnected to the network, check that it can access the internet as usual. Set up ddclient 19 Add/alter the following lines in /etc/ddclient.conf: protocol=dyndns2 use=web, web=checkip.dyndns.com/, web- skip='IP Address' server=dynupdate.no-ip.com login=user name password='password' Port forwarding [hostname].no-ip.biz 17 Set up port forwarding by making use of run_daemon=true the documentation for your router (or portforward. com). You need to forward port 22 to the static IP Now restart it by typing sudo service ddclient address that we set up. restart.
Linux & Open Source Genius Guide 43 Tips & Tricks
ownCloud automatically categorises This is the point from The expanded menu shows the uploaded fi les. Different types can where you upload your fi les the different admin actions be accessed via the links here to ownCloud available
Handily, there is a built-in The built-in image viewer can Hover the cursor over a fi le and a menu PDF viewer available show the pictures uploaded allows you to rename, download, share or in ownCloud to ownCloud check other versions of that fi le Build your own private cloud with ownCloud Advisor A fast-track guide to setting up your own Nitish Tiwari is a software developer by profession and an open source enthusiast by heart. As well fi le management system in the cloud as writing for leading open source magazines, he helps fi rms set up and using ownCloud use open source software for their business needs ownCloud lets you create your own fi le because you don’t need to install the server. We will management/sharing/backup system then explore some other use cases where it can be Resources without having to rely on a third-party deployed. Then we will go through some of the third- Server like LAMP/WAMP/MAMP cloud service. It also provides other party apps/plug-ins available, which can be hooked functionality like contacts management, calendar onto ownCloud to provide further functionality. ownCloud plug-ins: management, plug-in support, users, groups etc. We’re using ownCloud version 5.0.13 and the apps.owncloud.com All these features make ownCloud a fully fl edged dependent PHP versions for this tutorial, although enterprise-level fi le management tool. some of the plug-ins discussed here may require In this tutorial we will provide a step-by-step older versions of ownCloud. Please check the installation guide to set up ownCloud on your system, corresponding plug-in documentation link before although setting it up on a third-party server (for trying to install the plug-in. There has also recently example a web server provider) will probably be easy been an upgrade to ownCloud version 6.
44 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Introduction 01 ownCloud is an open source fi le sync and sharing application, available in a free community edition as well as an enterprise edition. It allows you to back up, share and manage fi les uploaded to the server. With multiple interfaces – like the web UI and Android/iPhone apps – it allows you to be in touch with your data at almost any point of time. The ownCloud desktop client for Windows, Mac OS X and Linux lets you sync your fi les seamlessly with the ownCloud server, akin to Google Drive or Dropbox Now, access the folder via your web browser. The clients. With this full ecosystem support for fi le welcome page opens up and prompts you to enter syncing, ownCloud truly is all about ‘Your Cloud, Your the admin ID and password. Below the prompt Data, Your Way’, as they say in the documentation. you’ll also see an ‘Advanced’ link, where you can WAMP or MAMP server is probably the best way change settings related to the database etc. Once Installing the server forward, since all the required tools come bundled done, just click on ‘Finish Setup’ and that’s all! You 02 There are two possible setups here. You with these servers. Your advisor uses the MAMP have your ownCloud. may want to install ownCloud on your system or in a server in his system to host ownCloud. small home/offi ce setup, where it is accessible in a Get going with ownCloud LAN. Otherwise you may want it to be available on the Installing ownCloud 04 Now that ownCloud is installed, and the internet. In the latter case, you can skip this step, but 03 With the server active, we can try to host admin user created, you may want to add more if you are planning the former, you will have to identify the ownCloud application. Before that, download users and then each of those users may want to a machine as the server and install a server such as the application code from ownCloud’s offi cial upload their fi les. In this step we will see how to Apache to that system. But, as discussed earlier, a website: just click on the ‘Tar or Zip fi le’ link on do this. First, the user management – click on server is not the only requirement for ownCloud. You the ownCloud install page. Unzip the archive and the ‘admin’ button in the top-right corner of the also need a database and PHP support. So, a LAMP/ place the extracted folder in the root of the server. homepage. In the drop-down list that appears,
Linux & Open Source Genius Guide 45 Tips & Tricks
left side of the homepage lets you create events and share them with other users or groups. It also supports multiple calendars and syncing with iCal. The contacts option supports adding and managing the contact details. Contacts can also be uploaded as .vcf fi les – and, like fi les and events, you can even share the contacts among users or groups.
Roundcube mail plug-in 07 With contacts and calendar support, click ‘users’. This takes you to the user management you would probably start to think, why isn’t email page where you can add/remove users and also supported? Thankfully, with support for external plug- segregate them in groups. You also have the option credentials. The client then connects to the server ins, ownCloud lets you extend the functionality in any to assign the admin for a group. To upload fi les, you seamlessly and starts syncing. The app is highly way you wish. Open source webmail client provider can simply click on the ‘new’ button in the ownCloud confi gurable and lets you change many settings, Roundcube offers an external plug-in which brings homepage and then select the fi le to upload in the including bandwidth usage and multiple sync folder your mailbox to ownCloud. To install the plug-in, go upload window. support. Also, it is available for all the major platforms: to the ownCloud offi cial plug-in portal, download it Windows, Linux and Mac OS X. and paste the downloaded fi les to the htdocs/apps ownCloud sync client folder. Now, click on the ‘apps’ link in the drop-down 05 Along with the server application, ownCloud Other applications of ownCloud that appears after clicking on admin (at the top-right also provides the desktop sync client, which can be 06 It may appear straightforward, but there corner of the page). Here you’ll see all the apps; go to installed on the user’s system. The sync client makes is a lot more to ownCloud than just fi le syncing. the Roundcube app and enable it. This enables the sure any fi les present in its folder get uploaded ownCloud not only helps sync fi les between multiple app, but you still need to create the database and automatically to the server. Just download the client, devices and platforms, it also lets you manage your confi gure the web server. For more details, you can point it to your ownCloud server URL and enter your calendar and contacts. The calendar link on the check the installation guide.
46 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
once encryption is enabled because if you forget “ownCloud not only helps sync fi les your password, there is no way to retrieve the data by default. To protect yourself against such loss, you can between devices, it also lets you manage enable ‘recovery key’ in the ownCloud admin settings for encryption. your calendar and contacts” rescue. This plug-in lets you create and store short links from the web. It comes as a preloaded plug-in with ownCloud. To start using it, you just need to enable it from the apps page. Once enabled, you can just drag and drop the button from the Shorty interface to add a site to your list. You can also shorten the URLs with a configurable back-end service like goo.gl, ti.ny etc.
File encryption plug-in 09 Security and privacy have become major Journal plug-in Shorty plug-in for areas of concern in recent years. With ownCloud you 10 The last plug-in we will look at in this article 08 weblinks management can be sure of using your own server; still, encrypting is the journal entry one. This plug-in lets you create With the internet having grown so vast, it’s become the data makes the whole setup even more secure. journal entries in your ownCloud calendar. The plug-in difficult to track webpages and the content you like. ownCloud ships with an encryption plug-in and once ships with ownCloud, but before you enable it, enable On top of that, there are now so many devices: phone, you enable it, all your files get automatically encrypted. the TAL Page Templates plug-in (also preloaded with office PC, home PC etc. So, the link you saw at home The encryption is done server-side and only the ownCloud). The entries are saved as VJOURNAL yesterday is difficult to find when you want to show it ownCloud portal can decrypt the data, using a key records in the calendar and can be sorted and filtered to your colleagues in the office. Shorty comes to your that is generated with your password. So be careful by date/time.
Linux & Open Source Genius Guide 47 Tips & Tricks
Capturing network data Applying a fi lter during and displaying it on network data capturing screen using tshark
Applying a display fi lter Displaying statistics on using tshark the captured data
Python script avaliable online: http://bit. Monitor network traffic ly/1gH6W0S Advisor Use tshark to examine network traffi c, Mihalis Tsoukalos is a UNIX system administrator solve network diffi culties and add with expertise in programming, databases and maths. He has been using Linux since 1993 network data to a MongoDB database
Gerald Combs created Ethereal, the If you try to run tshark as a normal user, you Resources ancestor of Wireshark, back in 2006. may not be able to use any network interfaces for tshark: When he went to work in a new job, he could capturing network traffi c due to UNIX permissions. www.wireshark.org/docs/man-pages/tshark.html not use the name Ethereal any more so he Your advisor fi nds it more convenient to run tshark renamed his tool Wireshark. The rest is history! as root (sudo tshark) when capturing data and as a Wireshark: www.wireshark.org This tutorial will present you tshark, the command- normal user when analysing network data. DHCP, RFC 2131: line version of Wireshark, which is a very popular Before you start capturing, it is better to have www.ietf.org/rfc/rfc2131.txt and capable network protocol analyser. The main a given issue that you want to solve or examine in Display Filter Reference: advantage of tshark is that it can be used in scripts. Its mind. This is the fi rst step for a successful network www.wireshark.org/docs/dfref/ main disadvantage is that it does not have a GUI. traffi c analysis. You can get tshark either from its website –by If you are already familiar with Wireshark, learning compiling its source code – or directly from your Linux how to use tshark will be easy for you. Having a good distribution. The second way is quicker and simpler. knowledge of TCP/IP comes in handy too.
48 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Installing and running tshark 01 In order to install tshark on a Debian 7 system, you just have to run the following command as root:
# apt-get install tshark
To fi nd out if tshark is properly installed, as well as its version, you can execute this command:
$ tshark -v
Capturing network 02 data using tshark Tshark can be used as a replacement for tcpdump, which is the industry standard for network data capturing. Apart from the capturing part where both tools are equivalent, tshark is more powerful than tcpdump and therefore if you want to learn just one tool, tshark should be your choice. The fi rst command you should run is tshark D to list the available network interfaces. The simplest way of capturing data is by running the tshark command without any parameters. You will get the output on screen – which, as you can easily understand, is not helpful at all!
creation of huge capture fi les. This can be done using The http.response.code != 404 display fi lter the -f command-line parameter followed by a fi lter in searches for HTTP traffi c with a response code double quotes. not equal to 404. The tcp.port == 80 && ip.src == The most important TCP-related fi eld names are 192.168.2.2 display fi lter searches for TCP traffi c tcp.port, for fi ltering the source or the destination TCP that both uses port number 80 and comes from the Two command-line parameters port; tcp.srcport, for checking the TCP source port; 192.168.2.2 IP address. If you have an error on your 03 The single most useful command-line and tcp.dstport, for checking the destination port. Display Filter, tshark will let you know by displaying parameter is -w, followed by a fi lename. This Generally speaking, applying a fi lter after data an error message. parameter allows you to save network data to a fi le for capturing is considered more practical and versatile As you can easily understand, the possibilities are later processing. than fi ltering during the capture stage because most endless and only depend on your imagination and The following tshark command captures 500 of the time you do not know in advance what you want the problem you are trying to solve. network packets (-c 500) and saves them into a fi le to inspect. Nevertheless, using fi lters during network If you deeply understand Display Filters and have called test.pcap (-w test.pcap): capturing can save you time and disk space and that a good knowledge of TCP/IP and networks then is the main reason for using them. network problems will not be a problem! $ tshark -c 500 -w test.pcap Remember that the fi lter strings should always be written in lower case. Exporting captured data Another useful parameter is -r, followed by a 06 into a readable format fi lename, which allows you to read and analyse a Applying fi lters Imagine that you want to extract the frame number, previously captured fi le. 05 after network capturing the relative time of the frame, the source IP address, Filters that are applied after data capturing are the destination IP address, the protocol of the packet Applying fi lters during capturing called Display Filters by tshark and Wireshark. and the length of the network packet from previously 04 Tshark allows you to fi lter network data You should use the -R command-line parameter captured network traffi c. The following tshark by capturing specifi c types of traffi c, avoiding the followed by the Display Filter in double quotes. command will do the trick for you:
Linux & Open Source Genius Guide 49 Tips & Tricks
$ tshark -r login.tcpdump -T fields -e frame.number -e frame.time_relative -e ip.src -e ip.dst -e frame.protocols -e frame.len -E header=y -E quote=n -E occurrence=f
The –E header=y option tells tshark to fi rst print a header line, the –E quote=n dictates tshark not to include the data in quotes and the –E occurrence=f tells tshark to only use the fi rst occurrence for fi elds that have multiple occurrences.
Solving a DHCP problem 07 The problem: some computers on a network could not connect to the network although other computers were okay. All computers were using the DHCP protocol to get their network settings. The IP of the offi cial DHCP server was 10.0.10.10. DHCP is short for Dynamic Host Configuration Protocol and is a protocol that provides configuration information to hosts on TCP/IP networks. DHCP is based on BOOTP (the Bootstrap Protocol) and extends it by adding more capabilities. DHCP and BOOTP protocols are both using the UDP protocol with UDP ports 67 and 68.
More about the DHCP protocol 08 The fi rst packet of a usual DHCP transaction between a DHCP client and a DHCP server (IP 192.168.1.1) has a DHCPDISCOVER message from the machine searching for a DHCP server. Since the machine does not have an IP address yet, the source IP of the packet is 0.0.0.0 and the destination IP is the broadcast IP address (255.255.255.255). What distinguishes the network card of a machine from another network device found in the same LAN is its MAC address, which is unique. Therefore the DHCPDISCOVER message should include the MAC address of the device requesting a DHCP server. The next message is the DHCPOFFER from the DHCP server (IP 192.168.1.1) and is a broadcast message since the client machine still has no IP address. Then the client machine requests from the DHCP server confi guration parameters with the DHCPREQUEST message. Next, the DHCP server sends a DHCPACK message back to the client machine that includes all the confi guration parameters. From now on, the DHCP client can use the offered “The Python script inserts network confi guration information and any parameter that is unique to that particular machine, like the IP address, data into a MongoDB database for is reserved by the DHCP server and is not offered to further processing and querying. The any other networked device. name of the script is insertMongo.py”
50 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Solving the problem choosing the 192.168.1.254 DHCP server was that it using the appropriate command-line arguments 09 Tshark output shows that there were two responded fi rst! Pretty naive reason, yet it caused using the following Perl commands: DHCPOFFER messages on the network from two many problems. different IP addresses (192.168.1.254 and 10.0.10.10) After fi nding out that there was a second DHCP my $command = "$TSHARK_BINARY -r $filename -T instead of only one DHCPOFFER message from the server that triggered the problem, it was easy to fields -e frame.number -e ip.src -e ip.dst legitimate 10.0.10.10 DHCP server! This was the fi rst fi nd out the computer that caused the problem. -E header=y -E quote=n -E occurrence=f"; truly useful hint for solving the actual problem. This particular computer was running a Linux my @netDATA = `$command`; As the DHCP server did not get any answer from the virtual machine (VM). The OS on the VM had its client, it re-sent the DHCPOFFER message (packet DHCP server running and that was the cause of the number 6), but as you can see, it was already too late problem! Pretty tricky, don’t you think? (packet number 4)! The IP address of the ‘extra’ DHCP server was Creating a Perl script that uses tshark 192.168.1.254. The 192.168.1.254 DHCP server 10 The purpose of the checkIP.pl Perl script is offered the 192.168.1.60 IP address to the machine. to fi nd invalid IP addresses. The checkIP.pl script As you can guess, all computers that could not assumes that the network data is already captured properly connect to the network were getting IPs in with tshark. the 192.168.1.1-253 range. Several steps are needed in order to solve the The client machine preferred the wrong DHCP problem. The fi rst step is reading the fi le with the More about the Perl script server to get its information. The reason for network data. Next, it is running the tshark binary 11 The next step is reading the output of the tshark command that was saved in the @netDATA variable line by line. After cleaning up input lines from unnecessary space characters and parsing it, the script uses the Data::Validate::IP Perl module for catching erroneous IP addresses and then prints the IP on screen:
if ( ! is_ipv4($sourceIP) ) { print "Packet number $frameNumber contains a bogus source IP!\n"; }
You can alter the script in order to catch the type of errors you want, such as traffi c from unwanted hosts or traffi c to specifi c TCP or UDP ports.
Inserting network data into a MongoDB 12 database The Python script supplied online inserts network data into a MongoDB database for further processing and querying. The name of the script is insertMongo. py and it assumes that the network data is already captured with tshark or tcpdump. The next shell command runs the Python script with input from tshark:
$ tshark -r ~/1000.pcap -T fields -e frame. number -e ip.src -e ip.dst -e frame.len -E header=n -E quote=n -E occurrence=f | python insertMongo.py Total number of documents written: 1000
The 1000.pcap fi le contains 1,000 network packets and the script informs you that there were 1,000 documents written in MongoDB, so you know that everything is okay. You can now start querying the MongoDB database!
Linux & Open Source Genius Guide 51 Tips & Tricks
Other architectures such You can add the VPN server to The Linux clients run the Android-powered devices as Microsoft Windows can your existing Linux setup. We’ll same software but with a such as phones and tablets connect to our server, using run it manually at first, so that different confi g fi le can also connect the offi cial client we can monitor its output
Access your network anywhere with a VPN Advisor Work around insecure and restrictive internet Michael Reed is a technology journalist access with your own virtual private network and he’s been hacking away at Linux for over This tutorial will show you how to add a In addition, you can use a VPN (virtual private 15 years VPN server to your existing Linux setup. network) connection to access any resources on This means that you can connect, securely, your network, because effectively you are on that Resources to a computer that you trust by using a network. So you need never curse yourself for A Linux box technique called ‘tunnelling’. Using this connection, forgetting to bring those important fi les with you you can securely access the internet from your ‘remote’ ever again. A second Linux box on a LAN (optional) computer, bypassing any filtering that is part of the Once the server is set up, you can connect to it An Android phone (optional) policy of the Wi-Fi provider. You can access anything using a variety of clients. On a Linux machine, you that your properly set up computer in your home or will typically use the same service as the server (optional) A Microsoft Windows box your office can access. The connection is a secure, with a slightly different confi guration fi le. Microsoft encrypted one. This is significant, because in theory it is Windows computers and Android devices such as possible to eavesdrop on the communications that are smartphones and tablets have offi cial and third- carried out using a public Wi-Fi hotspot. party OpenVPN clients available for them.
52 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
chmod 400 /etc/openvpn/{server.key,server. crt,ca.crt,ta.key}.
Generate three client keys 06 Use the command ./build-key client1 to generate the fi rst of the client keys. When prompted, accept client1 as the common name. Each key must have a unique common name. Now repeat the procedure for two more client keys. As Install OpenVPN on the server before, agree to sign the certifi cate and commit. 01 Install OpenVPN using the package manager. For Red Hat, type yum install openvpn or apt-get install openvpn on Debian (both root). In addition, install easy-rsa, a collection of scripts that aid in the creation of the certifi cates and keys that we need.
Build server certifi cates and keys Copy easy-rsa scripts 04 Type source ./vars followed by 02 We can’t run easy-rsa from where it ./clean-all and then ./build-ca. Fill in the is installed because we need to alter it. Find questions when prompted – you should be able to hit Begin server confi guration the installation directory with find /usr -name Return to accept the defaults in most cases. Build the 07 The server and each client use a configuration easy-rsa. Copy the directory to your current directory certifi cates and keys for the server with the command file located in the /etc/openvpn folder. On the server, with (for example) cp -r /usr/share/easy-rsa/. and ./build-key-server server. Answer questions as when the OpenVPN service starts, it scans that move into it with the cd command. before and accept server as the Common Name. directory and launches a service instance for every Don’t specify the challenge password or optional valid configuration file it finds. Start by shutting down company name when asked. Answer y when asked the OpenVPN service by typing service openvpn about signing the certifi cate and when asked to stop (as root). commit. Diffi e-Hellman parameters are an element in the encryption scheme that OpenVPN employs. Type ./build-dh to create the parameter fi le.
Locate the example configuration Install OpenVPN on the server 08 Use find /usr -name server.conf* to 03 Open the fi le vars in a text editor. The Copy the server keys locate the example server confi guration fi le. Use offi cial OpenVPN documentation recommends 05 Enter the keys directory with the cut and paste within the terminal to copy it to the that you edit at least the KEY_COUNTRY, cd command, and type ls keys to examine current directory with the cp [na m e of file] . KEY_PROVINCE, KEY_CITY, KEY_ORG and the key fi les that we have created. Use command sequence. If it has a .gz extension, it’s KEY_EMAIL fi elds. Set KEY_CN to ‘server’. Close and cp * /etc/openvpn/ (as root) to copy all of the compressed. If so, decompress with gunzip [name save the fi le. keys to where OpenVPN can see them. Then do of file].
Linux & Open Source Genius Guide 53 Tips & Tricks
client1.crt to /etc/openvn/ on the client machine. How you do this is up to you, but bear security in mind as anyone who has these fi les can, potentially, use them to access your server. A memory stick (delete them afterwards) or a trusted network such as your LAN should be fi ne.
Edit the client confi guration 12 Open client.conf in a text editor. Change remote to the IP address of the server. For now, use the IP address of the server on your LAN (run Edit server confi guration Type openvpn server.conf (as root). If everything is ifconfig on it). Later, we’ll change this to the 09 Open server.conf in a text editor and give it working okay, the final message should be ‘Initialization external IP address so that it can be accessed a quick examination to get an idea of what you can Sequence Completed’. from the internet. Change the CA certifi cate (ca) to change, but we’ll leave most of it as it is. For now, just ca.crt, the user certifi cate (cert) to client1.crt and add /etc/openvpn/ to the start of the key filenames. the private key (key) to client1.key, each with /etc/ openvpn/ at the beginning of the name. Each client needs a copy of the CA and unique versions of the other two fi les.
Confi gure the client 11 Move to another Linux machine on your LAN (the client). Install OpenVPN using the package management system. Now run find Start the server /usr -name client.conf to fi nd the example 10 For now, we’ll start the server manually so client confi guration fi le and copy it to the current that we can see the text output and check for errors. directory. Copy the fi les client1.key, ca.crt and Test the client 13 Type openvpn client.conf (as root) “We can connect to the server, to start the client. Type ifconfig to confi rm the creation of the tun0 device. If you can via the tunnel, but we can’t yet ping the server from the client by typing ping 10.8.0.1, you have now established a access the internet” tunnelled, encrypted connection between the client and the server.
54 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Enable packet forwarding 14 As it stands, we can connect to the server, via the tunnel, but we can’t yet access the internet. Start by editing /etc/sysctl.con and uncomment the line net/ipv4/ ip_forward=1 to enable packet forwarding. Now restart the server machine.
resolvconf/resolv.conf.d/head and type resolvconf -u (all as root). You should now be able to ping and browse to sites on the internet from the client.
Forward traffi c with iptables 17 On the server, copy server.conf to /etc/ openvpn/. We now control the OpenVPN server with service openvpn [co m m and]. ‘Command’ can be Forward traffi c with iptables start, stop or restart and the check command makes 15 On the server, type iptables -t nat the service start on boot. Add the iptables we used to -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j the end of /etc/rc.local. MASQUERADE (as root) to enable forwarding of traffi c through the tunnel to the internet-connected Connecting from an Android device interface (eth0). Install the dnsmasq package to 19 Most Android phones and tablets don’t include your system. Add the line interface=tun0 to /etc/ built-in OpenVPN support. Search for an OpenVPN dnsmasq.conf. Restart dnsmasq by typing service client on Google Play. Your choice of client may dnsmasq restart. depend on compatibility with your device, and some applications require a rooted device. Copy the key files to your SD card and use the setup page of the client.
Connecting from the outside 18 Set up your router to forward UDP port 1194 to the IP address of your server. How you do this depends on the model of your router (see Set up routing for the client portforward.com for a comprehensive database Connecting from a Windows client 16 Add the lines push "redirect-gateway def1" of router models). Update the remote fi eld in your 20 The OpenVPN website (openvpn.net) has and push "dhcp-option DNS 10.8.0.1" to server.conf. client.conf fi les to point to the external IP address of a client that runs on Windows. Copy the three client This causes the clients to execute the appropriate your network (use www.whatismyip.com to fi nd it). fi les to the machine, then install the client and follow route commands to direct all network traffi c via the Consider using a dynamic IP service such as No-IP the on-screen prompts. The Windows client can make VPN. You may have to manually add the DNS server (www.noip.com) so that you don’t have to update the use of the push directives that we added to server. to the client: add the line nameserver 10.8.0.1 to /etc/ clients every time the address on your router changes. conf, so we don’t need to set up DNS manually.
Linux & Open Source Genius Guide 55 Tips & Tricks
Gnuplot can fi nd Gnuplot can draw Gnuplot can plot Gnuplot can do the intersection of diffi cult plots many lines on the 3D too two parabolas same output
Use gnuplot for drawing Advisor Gnuplot is a powerful 2D and 3D program Mihalis Tsoukalos is a UNIX system administrator that can be used to plot mathematical with expertise in programming, databases and maths. He has been using Linux since 1993 functions and also show scientifi c data
This tutorial will introduce you to a very using lines, points, vector fi elds, surfaces, boxes and Resources useful scientific tool called gnuplot. contours. It also supports output in many different fi le Gnuplot homepage: Gnuplot is copyrighted but freely formats, including EPS, FIG, JPEG, LaTeX, METAFONT www.gnuplot.info distributed. There are versions of gnuplot (invented by Donald Knuth the creator of TeX), PDF, for UNIX, IBM OS/2, MS Windows, DOS, Mac OS X, PNG and PostScript. Gnuplot Python package: VMS, Atari and many other platforms! Although gnuplot can show its graphical output gnuplot-py.sourceforge.net Gnuplot is a command-line tool that was initially on screen with the help of a windows manager, the 3D plots using gnuplot: developed in order to allow scientists and students gnuplot commands that are going to be used here www.packtpub.com/article/3d-plot-using-gnuplot to visualise mathematical functions and data. It now will capture the output directly to a fi le. This implies supports web scripting and integration as a plotting that gnuplot is suitable for scripting. There is also engine for third-party applications like Octave. a Python module for using gnuplot while writing Gnuplot can draw many types of plots in both two Python code, called py-gnuplot, which will be (2D) and three (3D) dimensions. It supports drawing explored in this tutorial.
56 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Install gnuplot 01 You can install the command-line version of gnuplot on a Debian system by using the following command: # apt-get install gnuplot-nox Alternatively, you can install the gnuplot-qt package that is the Qt version, or the gnuplot-x11 package that is the X-package version. Colin D Kelley and Thomas Williams fi rst developed gnuplot back in 1987. Note: When executing gnuplot commands, please remember that the gnuplot language is case sensitive.
First run 02 You can fi nd the version of gnuplot you are using by executing this command from the shell: $ gnuplot -V gnuplot 4.6 patchlevel 0 If you run the gnuplot command without any arguments you will enter the gnuplot shell, as shown in the screen above. Type quit at the gnuplot prompt to leave the gnuplot The default size of the output is 640x480 pixels. shell and return to the UNIX shell. If you want to generate a higher analysis output (2048x2048 pixels), you can give the following A simple gnuplot example command instead: 03 The fi rst example will show how to draw the gnuplot> set term png size 2048, 2048 2*x linear function. You can defi ne the size of the output according to your gnuplot> set term png needs. Terminal type set to 'png' Options are 'nocrop font "/usr/share/fonts/ Plot mathematical functions with truetype/liberation/LiberationSans-Regular. 04 gnuplot – part 1 ttf,12" fontscale 1.0 size 640,480 ' The previous output was very simplistic. In this part, gnuplot> set output "plot2x.png" you are going to learn some commands that will help gnuplot> plot 2*x you beautify the output, add some text to it and make gnuplot> quit it look more professional. mtsouk@mail:~/docs/article/working/GNUplot. gnuplot> set term png size 2048, 2048 LUD/code$ ls -l Terminal type set to 'png' total 4 Options are 'nocrop font "/usr/share/fonts/ gnuplot> set ylabel "Setting the y-axis label" -rw-r--r-- 1 mtsouk mtsouk 3958 Oct 26 19:17 truetype/liberation/LiberationSans-Regular.ttf, gnuplot> plot sin(x) * cos(x) plot2x.png 12" fontscale 1.0 size 2048,2048 ' gnuplot> quit The first command tells gnuplot that the output will use gnuplot> set output "plotSinCos.png" As you can comprehend, the set xrange command the PNG file format. It then defines the filename that gnuplot> set xrange [-2*pi:2*pi] defi nes the graph boundaries for the x-axis. For will be used for saving the output image. After that, it gnuplot> set title "Linux User and Developer" a trigonometric function such as sin(x) * cos(x), it defines the function to plot. After quitting gnuplot, you gnuplot> set xlabel "x-axis: from -2*pi to makes more sense to defi ne boundaries relative to can find the output file in you current directory. +2*pi" the number pi.
Linux & Open Source Genius Guide 57 Tips & Tricks
Using gnuplot and Python Running the Python script produces the following 07 On a Debian 7 system, you can install the output because of the debug=1 option: gnuplot Python package by executing the following $ ./LUD.py command: gnuplot> set term png size 2048, 2048 # apt-get install python-gnuplot gnuplot> set output "LUD.png" In order to make sure that python-gnuplot was gnuplot> plot cos(x)/x title "Using the installed correctly, you can run the following one-liner Gnuplot Python module" Python script and check its output for errors: $ python -c 'import Gnuplot' If everything is okay, you will immediately see the shell prompt. If the module is not installed, the error message will look like the following: $ python -c 'import Gnuplot' Traceback (most recent call last): File "
A simple Python and gnuplot script dummy variable is t for curves, 08 The next Python script uses the gnuplot u/v for surfaces Python module: gnuplot> set isosamples 50 #!/usr/bin/python gnuplot> set ztics .5 Plotting two functions at once import Gnuplot gnuplot> set xtics .4 06 Let’s say that you want to see two functions gp = Gnuplot.Gnuplot(debug=1) gnuplot> set ytics .4 on the same graph in order to compare them. Gnuplot gp('set term png size 2048, 2048') gnuplot> set urange [-pi:pi] can assist you with this. The following command gp('set output "LUD.png"') gnuplot> set vrange [-pi:pi] demonstrates how: g5 = Gnuplot.Func(cos(x)/x', title= 'Plotting a gnuplot> set pm3d depthorder gnuplot> plot x*x, x*x*x function with py-gnuplot') gnuplot> splot cos(u)*cos(v), sin(u)*cos(v), As you may understand, all you have to do is separate gp.plot(g5) sin(u) with pm3d the two functions with a comma! If you want to make The import Gnuplot command loads the correct smooth palette in png: using 160 of 160 the output look even better, you can give the following Python module, whereas the gp.plot(g5) command available color positions command: plots a graph that is called g5. The gp(‘set term png size gnuplot> gnuplot> plot x*x, x*x*x with filledcurves fs 2048, 2048’) command defines the output file format The splot command can display a surface as a As you may recognise, the ‘with fi lledcurves fs’ does and its dimensions, whereas the gp('set output "LUD. collection of points, or by connecting those points. the job for you – it fi lls the second function with green, png"') command dictates the filename that will be used The set isosamples 50 command sets the number which is the default colour for the current curve. for saving the output. of points at which a function of two variables will be evaluated to generate a surface plot. The special pm3d option was set as ‘pm3d “Gnuplot gives you the choice of depthorder’. This tells gnuplot to draw the surface starting with the elements farthest away from our plotting multiple graphs side-by- viewpoint, producing a more pleasant output. The with pm3d (palette-mapped three-dimensional) parameter side on the same output screen” of the splot command is for drawing solid surfaces.
58 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Using multiplot mode 10 Sometimes it is preferable to show multiple plots at once. Gnuplot gives you the choice of plotting multiple graphs side-by-side on the same output screen. The key point for doing this is the set multiplot command. If you give this command, the prompt will change from ‘gnuplot’ to ‘multiplot’. The following commands generate a very pretty output: gnuplot> set term png size 2048, 2048 Terminal type set to 'png' Options are 'nocrop font "/usr/share/fonts/ truetype/liberation/LiberationSans-Regular.ttf,12" fontscale 1.0 size 2048,2048 ' gnuplot> set output "MultiPlot.png" gnuplot> set xrange [-1:1] Using gnuplot scripts gnuplot> set size 1,1 12 As you can imagine, it would prove very tedious gnuplot> set origin 0,0 to type every single gnuplot command each time you gnuplot> set multiplot wanted to plot something. Gnuplot allows you to write multiplot> set size 0.5,0.5 the commands you want to execute in a plain text file multiplot> set origin 0,0.5 and run it as a script. To run a gnuplot script named multiplot> plot x test.script, you just have to execute the following multiplot> set size 0.5,0.5 command from the shell: multiplot> set origin 0.5,0.5 $ gnuplot test.script multiplot> plot x*x If you’re inside gnuplot, run the following command: multiplot> set size 0.5,0.5 gnuplot> load “test.script” multiplot> set origin 0,0 Note that it is generally considered good practice to end multiplot> plot x*x*x a gnuplot script with the reset command in order to reset multiplot> set size 0.5,0.5 the gnuplot environment to its default status. multiplot> set origin 0.5,0 The following is an example of a gnuplot script: multiplot> plot 1/x can help you and plot it for you! The following set term png size 2048,2048 crop multiplot> unset multiplot command illustrates what gnuplot shows when you set output "3Dwireframe.png" gnuplot> reset try to plot an erroneous data fi le: set param The size and origin commands position each graph gnuplot> plot 'badDataFile' set isosamples 20 in its place and make it take up just a quarter of the ^ set ztics .3 screen. The reset command clears all labels on the Bad data on line 325 set xtics .2 axes as well as all other settings. It is very useful if Gnuplot prints an error message on screen telling set ytics .2 you want to start a new graph from scratch. you the line number of the fi rst error – this does not set urange [-pi:pi] indicate that the rest of the fi le is fi ne, just that there set vrange [-pi:pi] Plotting data fi les is at least one error! A correct data fi le gets plotted set hidden 11 Imagine that you have a fi le that contains without problems by using the following command: splot sin(u)*cos(v), cos(u)*cos(v), sin(u) groups of data points that you want to plot. Gnuplot gnuplot> plot 'LUDdata' reset
Linux & Open Source Genius Guide 59 Tips & Tricks
There is a URL always For a given connection, you Here you can fi nd a list of all available pointing you to a can set the hostname, the of the previous connections goo.gl URL containing an FAQ username and any other SSH you made through Secure for Secure Shell options you need Shell in Chrome
SSH from a web browser Advisor There are times when you are stuck using Joey Bernard As a true renaissance man, he splits his a locked-down machine. As long as you time between building furniture, helping researchers with scientifi c computing problems have a browser, though, you can still and writing Android apps connect to your remote machines Resources SSH is the de facto way of securely the server side. Naturally, the available options are Secure Shell: connecting to remote machines where limited, but it is one of the leanest options. The issue goo.gl/RYHiK you need to get work done. Normally, is that you need to use a supported browser. The FireSSH: this is achieved through an SSH client second option is a Java-based one. A Java applet is fi ressh.net application installed on your desktop. Unfortunately, loaded into your browser to handle the actual SSH Shellinabox: there are situations where this is just not feasible, connection management. Unfortunately, this is only https://code.google.com/p/shellinabox for any number of reasons. In this tutorial we will an option if you have Java installed and are allowed MindTerm: look at a few different options for how to regain a to run Java applets in the browser. The third option www.cryptzone.com/products/mindterm/ command-line connection to your remote machines. is even leaner on the client side than the fi rst option, No matter how locked down a machine may and has the added advantage of running in almost DropPages: be, you will almost always have a web browser any browser. The downside is that it requires you to droppages.com available. We can leverage this and get an SSH install a piece of server-side code to facilitate the Pancake: connection established through this browser. There actual SSH connection management. pancake.io are several different technologies that can be used Hopefully, by the end of this tutorial, you will have to give us this connection. The fi rst option we will found an option that fi ts your own situation and look at is a purely browser-based application that helps you manage your remote machines no matter requires nothing extra on either the client side or where you are.
60 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Closing connections 06 Close your connection the same way as any SSH client, by typing in exit. When the connection closes, ‘Secure Shell’ offers you the option to reconnect (R), choose another connection (C), or Finding an SSH client plug-in where you can enter the host, username and simply fi nish and exit (x). If you choose ‘x’, the current 01 Both Chrome and Firefox have SSH clients password. browser window will stay open but will be inactive. in their respective app stores. In this tutorial, we will be looking at Secure Shell from the Chrome store and FireSSH from the Firefox store.
Terminal options 04 ‘Secure Shell’ in Chrome does not have a terminal preferences window yet, so you need to Installation open a JavaScript console (by clicking the menu item Saving connections 02 In the case of both browsers, installation View>Developer>JavaScript Console) and entering 07 All of your previous connections get stored should be straightforward. All you need to do is fi nd the changes you want to make. For example, you can as a list available at the top of the connection screen. the relevant app in the browser store and click on the set the background colour with the following: Click on a connection to edit the SSH options before Install button. Most browsers also require a restart term_.prefs_.set(’background-color’, ’wheat’) fi ring off and connecting to the remote machine. before the SSH client is ready to use.
Working in SSH 05 You can do almost everything with ‘Secure Finding a Java plug-in client Open a new connection Shell’ that you would normally do with a regular 08 There is a Java applet that you can 03 For the rest of this tutorial, we will use the client. You can do port forwarding by including use called MindTerm. In this case, you need to Chrome version. To open a new connection, simply the relevant options when you make the original wrap MindTerm in a simple webpage in order to get click on the ‘Secure Shell’ icon on the browser connection. You place these types of options in the the browser to load it for you and host it somewhere homepage. This will open up a connection window SSH Arguments box. visible. You can also run it directly as a Java app.
Linux & Open Source Genius Guide 61 Tips & Tricks
“You can do almost everything with ‘Secure Shell’ that you would normally do with a regular client”
Installation 09 If you need to host MindTerm somewhere non-local, you can place it on a hosting service if you Closing connections have one. If not, you can get a Dropbox account and 13 You close your session with the exit host it there as a static webpage. There are services command, just like with a regular SSH client. Once like droppages.com or pancake.io that will help the connection is shut down, MindTerm resets itself you here. and is ready for a new connection to a new host.
type, font type and size, and colours, among many Saving connections other items. 14 Whenever you connect to a new host, MindTerm asks you whether you want to save it in the list of hosts under an optional alias. To get access to these saved connections, you will need to click Open a new connection on the menu item File>Connect…. This will pop up a 10 The screenshot above is made connection window where you can select the server using the MindTerm jar fi le standalone. The from a drop-down box. behaviour is the same in the browser. When it starts up, it asks you to enter either a server Client/server browser-based SSH alias or a server hostname. If this is a new 15 The previous two methods have an machine, it will ask you whether you want to advantage where all of the SSH connections are save it as an alias. Working in SSH essentially only through the client and the server. 12 With MindTerm, you also have easy access to This also means that the machine you are working Connection options all of the SSH connection options. Clicking on the menu on also needs to allow network connections on the 11 The advantage of a Java applet is that you item Settings>Connection… will pop up a new window ports that you need, most often port 22. But what have more tools available to you. Clicking on the where you can set port forwarding, as well as more can you do if your desktop is locked down to only menu item Settings>Terminal… will pop up a full esoteric items such as the type of cipher or the type of allowing HTTP traffi c? In this case, you need to move preferences window where you can set the terminal compression to use. the workhorse part of your SSH connection off to
62 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
clicking in the terminal window will bring up a set of options that get saved in a web cookie that survives over sessions.
Working with SSH 21 Unlike the other methods, you do not have the option to set up more complicated options, like another machine, and connect to it over HTTP with remote host. There are tons of options available as SSH tunnelling. All you get on the client side is pure your browser. The most common choice for this is command-line parameters. The most common ones HTML, with nothing else. If you need more, you will shellinabox. are options like --port=x, and --verbose. need to use one of the other methods.
Starting the client 18 To start the client, you simply need to open your browser and enter the URL Installation to the remote machine. It will look like Closing connections 16 Once you download the source, you http://mymachine.com:4200, where you might have 22 Like every other SSH client, you shut down need to install it with the usual ./configure; changed the port being used. This will open up a your connection with the exit command. This leaves ./make; ./make install step that should be in terminal where you can enter your username and your browser window open with a button popping most Linux users’ repertoire of skills. You do this on password. up in the centre labelled Connect. Clicking on this the remote host that you want to connect to. button will refresh the screen and reopen the connection to your remote host.
Saving connections 23 Unfortunately, there is no real way to ‘store’ Connecting to a different machine a set of connection strings to different machines 19 Once you log in, you can always within shellinabox. The best option open to you is just SSH to another machine. But if this is to confi gure a series of daemons on different ports something that you always do, you can get tunnelling to different machines, and then you can shellinabox to do this for you by using the option save the URLs to these servers as bookmarks in your -s /:SSH:mynewhost.com. This means you could browser. have connection tunnels to multiple different machines, each with its own port. Where to now? 24 Hopefully this tutorial will have shown you Connection options some options that are available when you get stuck Starting the server 20 Because this is the leanest of the web- with an overly locked down machine. If these don’t fi t 17 Once shellinabox is installed, starting it based SSH clients available, you simply don’t have your exact situation, don’t be afraid to look for some is done by simply starting up shellinaboxd on the the same level of confi guration options. Right- of the other options available out in the wild.
Linux & Open Source Genius Guide 63 Tips & Tricks
Join in with any FOSS project Which fi les did I change Fork a repository – grab Share your code without that has a Git repository. since the last commit? The some open source code sharing your passwords, GitHub and other sites open ‘git status’ command keeps to start working on your logs, autosaves and other easy code sharing to all track of where you’ve been own version irrelevancies, using .gitignore
Master version control with Git Advisor Git is quite easy to learn, but you go much Richard Smedley As a system administrator, further with an understanding beyond a writer, consultant and occasional coder, Richard has a simple cheat sheet, so we’ve combined lot of stuff to keep track of, and now leans heavily on Git this ‘how-to’ with a little of ‘why’ Resources Git has become so popular so quickly, choice for distributed version control and source Git: it’s easy to have missed out on why you management of new projects. git-scm.com/ should integrate it into your coding (and On personal projects that don’t need a central A GitHub account: writing) life. We’re going to get you quickly repository, Git is great for keeping track of changes https://github.com up to speed about why Git is special, and how to get and experimenting with different approaches in Basic command line skills to grips with using it. your project using branching, giving you the choice to Git is a distributed version control system, merge changes and/or roll them back. famously written by Linus Torvalds after the To really benefi t from Git you need not just a few BitKeeper debacle left the Linux kernel in need of a commands on a cheat sheet, but an understanding capable home. Now, through GitHub and a dozen of how it works – so we’ll touch on a few Git concepts other popular repositories that make it social to as we go along, making it easier for you to go and fi nd store and share code, Git is becoming the default out more later.
64 Linux & Open Source Genius Guide Linux & Open Source Genius Guide ickr.com/photos/phil_g
A project, any project 01 Git manages projects in the form of fi les – so not just code, but anything from a set of recipe cards to the text of that historical romance you’re git init writing under a pen name.
CREDIT: Phil Gould cc-by-sa 2.0 http://www.fl A .git directory is created: you now have a working tree, and you’re now ready to commit changes to your repository. Initialisation is only done once per repository.
Where was I? Installation 07 If you haven’t committed changes for a Naturally, your distro includes an easily while, git status is useful for showing just what 02 Staging post installable binary. Most distros separate out changes you’ve made as you tunnel through Now that you have a working repository, packages for Git-to-Subversion functionality, a 05 Rails, for example, making changes across views, let’s use: GUI interface, revision tree visualiser, et al. Install controllers and helper fi les. What’s changed in the git-doc package for extremely comprehensive git add those fi les is shown by git diff (see the large HTML-formatted documentation. screenshot on the next page). …to stage the code. Passing ‘.’ as an argument adds Undone by stages everything in the current directory, recursively (ie The two-step commitment process allows including subdirectories), to the index. Now we use 08 one to change one’s mind at the staging step – so if commit and at last our code is in the repository, and you have added a fi le you’ve edited, but don’t after tracked by Git. all want to commit it, then…
git reset HEAD filename
…will remove that fi le from staging.
Who are you? 03 Initial confi g simply consists of a few git config --global commands, which write your details into ~/.gitconfi g. Without the --global switch, git config writes to ./.gitconfi g so you can Status tailor individual repositories. 06 If we beaver away at writing some more code, then ask Git for a status report, it will tell us A tree, initially that we have made some changes that have yet to 04 Those repositories are created where your be committed. It’s good practice to commit often – code is; cd into a project folder and initialise your Git is your source code management tool after all, fi rst Git repository with: so give it the changes to manage.
Linux & Open Source Genius Guide 65 Tips & Tricks
Not that one 09 If you’ve staged a totally new fi le, then git rm --config filename
…is the better option – rm to remove a fi le, but --cached to just take it from staging. Here git status shows the change from stage to ‘untracked’ for the fi le we removed. Branch line 12 git checkout gets you using another branch. Commit objects It’s quite important to commit your changes before 10 Time to improve our understanding of you use checkout to switch branches – otherwise what’s going on behind the scenes in Git. Each some strangeness will result, such as losing all the commit object contains a snapshot of the fi les, changes you’ve made to the branch, so show some references to parent commit objects, and a SHA-1 commitment and commit… name (40-character hash of parts of the commit) that uniquely identifi es the commit.
Releasable code 14 You can have as many branches as you like, depending on your team and strategy. One simple approach is to keep the master branch in a releasable state and commit your incremental changes to a development branch until you’re ready to merge your changes back for a new release.
HEAD master Committed to change 11 While the latest commit object is HEAD – 13 Git will prompt you if you have forgotten to and its parent is HEAD^ – that refers to the latest save before changing branch. You can use commit in the current branch. By default we just git checkout -- nameoffile have a master to work on, but we can create new branches from any commit object – the above …to deliberately remove the uncommitted changes to work on an earlier version of the project’s fi le you have made. ‘Commits are cheap’ so keep format. commiting in your own branch. Track what’s changed 15 Sometimes, simply keeping track of what “Git manages projects in the form of files you’ve just done defeats the fallible human brain. What was that small change which seems to have – so not just code” broken something?
66 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
merge master into your branch to keep your tree up to date with changes there.
Pushing code git diff 21 git push is used to push changes to the is your friend, showing you what has been changed remote repository. Using push.default in your but not yet staged with git add. Not the fi les confi g, or arguments (eg simple) to the command, effected (git status does that), but each line added, you can get push to refuse to push if the upstream removed or edited. branch’s name is different from the local one.
git diff --cached will look at the changes that have been staged but not yet committed.
Other useful diff commands to Git include
git diff HEAD^ Ignorance is bliss 18 Now, before you work with others on …which shows changes since the commit previous a project, use the .gitignore fi le to keep your to the latest commit. password and local datase details out of the public repository, along with auto-saves from your text editor, notes to yourself, and anything else unnecessary for the project but in the directory. Time-saving aliases 22 Aliases such as gc for git commit,
ga for git add --all
and gst for git status
save valuable key strokes for frequent users. You might wish to add your own preferred aliases to your .bashrc, or download one of the macro collections to be found on GitHub.
Try out commands Branch log Push me pull you 23 Grab some code and start trying out the 16 Keeping track of commits for your current 19 git pull origin master will fetch the latest commands. git clone can be used to make a local branch is a simple matter of running git log forthat copy of the remote master and merge it into your copy of a complete Git repository from elsewhere, branch. This will show the chain of commit objects local copy, thereby allowing your local copy to including branches (git branch -r will show them). since initialisation. The fi rst few digits of the SHA-1 combine other people’s updates with your own Those ‘Fork me on GitHub’ buttons will all be hash shown in log is a good reference to HEAD on changes. Whether you pull frequently or just after competing for your attention. the branch you’re working on, if you want to check if specifi c changes is up to you. a colleague is using the same. Go use Git Fetch 24 Now, armed with some understanding of Confl icting code 20 git fetch just gets the remote changes what Git’s about, go forth and spread thy code! 17 Git makes merging branches easy, but will without forcing a merge. It creates a local copy for Online there are plenty of tutorials on advanced Git also tell you where there is a confl ict if a merge you to work from, so you can examine with git diff usage and we hope to return to the topic soon in cannot take place – usually where both branches and make changes before merge to handle any LUD, showing you many versatile uses for Torvalds’ have made changes in the same place. You can also problem confl icts you can foresee. other creation!
Linux & Open Source Genius Guide 67 Tips & Tricks
Create automated video- and The Pi Camera doesn't Use the camera for time- photo-related tasks easily take up any USB slots, lapse photography, a robot's instead plugging directly eyes, or a webcam in any into the Raspberry Pi, number of suitable projects which also powers it
Take pictures and video with the Raspberry Pi Camera Advisor Follow our short tutorial on how to get your Rob Zwetsloot models complex systems and is a web new Pi Camera set up on your Raspberry developer profi cient in Python, Django and PHP. He loves to Pi, and how to use it experiment with computing One of the most recent Raspberry Pi as an optical sensor for a Pi-powered robot, like the accessories is the tiny Pi Camera board one we’re making this issue. Because it doesn’t take Resources – a small PCB with a camera sensor up any USB slots, and draws very low power, it can Raspberry Pi mounted to it that connects via a ribbon be a lot more versatile than a standard webcam. Raspbian: to the Raspberry Pi..Because of this, it’s not exactly The Pi Camera itself is not a low-quality piece www.raspberrypi.org/downloads plug-and-play, so you’ll need to do some extra setup of kit either – with a 5MP sensor, it’s also able to Pi Camera on your Raspberry Pi to get it to work. create up to 1080p quality video, the same as the Ashton’s picam module: The Pi Camera has multiple functions, such as for Raspberry Pi’s HDMI output. So grab your Raspbian https://github.com/ashtons/picam time-lapse photography, using as a webcam, or even SD card and make the most out of your Pi Camera.
68 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Attach Camera 01 To attach the Camera to the Raspberry Pi, locate the slot between the Ethernet and HDMI port and gently lift up the fastener. Insert the ribbon of the Camera board, making sure to align the ribbon’s connectors with those on the Pi.
This will show a fi ve-second preview of the input of pip install https://github.com/ashtons/picam/ the camera and then capture the last frame of the zipball/master#egg=picam video. Picam photos Record video Pi preparation 07 We can now use Python to construct a script To record a video, we use a similar command, Before we try to enable the Raspberry Pi 05 to take photos with the picam module. Very simply, all 02 raspivid, like so: Camera, we need to make sure our fi rmware and you need to do is enter: software are all up to date with a quick software upgrade. In Raspbian, we do this by opening the $ raspivid -o video.h264 import picam terminal and using: i = picam.takePhoto() $ sudo apt-get update It will also take fi ve seconds of video by default. i.save(‘/home/pi/test.jpg’)
…followed by: And running it will take a photo called test.jpg. $ sudo apt-get upgrade Advanced photos 08 You can have it take photos of specifi c size Pi confi g and quality with a time-based name by editing the 03 Once that’s fi nished, run in the terminal or code to look like this: command line: import picam $ sudo raspi-config import time ii = picam.takePhotoWithDetails(640,480, 85) …to start the standard confi guration screen. Navigate filename = “/tmp/picam-%s.jpg” % time. strftime(“%Y%m%d-%H%M%S”) down to Enable Camera, press Enter and then simply Picam key over to enable and confi rm with another press of ii.save(filename) If you want to do a little more with the Pi Enter. Select Finish and then reboot. 06 Camera, there’s a simple Python wrapper currently available called picam. You’ll need to install it fi rst, Picam video and more Take pictures though, and we’ll use pip for that. Install pip with: 09 Picam also allows you to take video in a 04 To take pictures with the Raspberry Pi similar way to the above, with the main difference Camera, you’ll simply need to enter: being that you’ll use the recordVideo command. You $ sudo apt-get install python-pip can use the code to take photos or video at regular intervals for time-lapse, or have it trigger during a $ raspistill -o image.png …and then enter: specifi ed event.
Linux & Open Source Genius Guide 69 Tips & Tricks
Learn how to record Allow your Raspberry Pi to Play it at normal speed for multiple types of slow- play back the video to get high frame rate video for motion videos for yourself the full effect specifi c video projects
Record slow-motion videos with the Raspberry Pi Use the updated Raspberry Pi fi rmware to record slow-motion videos with the camera module
The Raspberry Pi camera has been an The fi rmware updater Resources interesting little add-on for the Raspberry 01 The fi rst thing you need to do is make sure Latest Raspbian Image: Pi, allowing people to have slightly more you update the fi rmware on your Raspberry Pi. Boot raspberrypi.org/downloads control when it comes to projects that into Raspbian and open the LXTerminal. To install Raspberry Pi camera board require pictures and video. The functionality of the the fi rmware updater, simply type in the following Internet connection camera is always being slightly enhanced and now a new function has made its way to the camera: slow- motion video. With this you can create short clips as slow as 33 per cent normal speed, with all the same control as you have with the normal camera functions. At the time of writing, our favourite PiCamera Python module doesn't include the ability to access the slow- motion functions.
70 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
command. For newer versions of Raspbian, it will Camera setup likely be already installed. 04 Just in case you're using a new install of Raspbian, you'll need to $ sudo apt-get install rpi-update make sure that the camera module is enabled. Make sure that you've plugged in the camera while it's off, and then in the LXTerminal use:
$ sudo raspi-config
Go to the Enable Camera option and select to enable it. You may need to restart in order for it to Playing it back actually take effect. 07 Although your Pi has the means to record the video, it can't play it back. First tests You can either grab the video from the 05 We can now do a quick test of the slow- SD card, or install VLC using the following motion capabilities by entering the following: command, which should have no trouble viewing Update the fi rmware the video fi le. 02 Updating the fi rmware is very $ raspivid -w 640 -h 480 -fps 90 -t straightforward; back in the terminal just type in the 5000 -o slowvideo.h264 $ sudo apt-get install vlc following:
$ sudo rpi-update Other uses 08 If you do plan to use the slow-motion And it should automatically download and install all function, obviously you can use it as intended. the necessary fi les. Once this is complete, reboot However, you can also use it to create high frame your Raspberry Pi to activate the new fi rmware rate videos. Video editors such as Kdenlive will let you increase the playback speed, and you can also Alternate updating view them in VLC at normal speed. 03 Problems have been known to occur with the fi rmware update and there Final warnings are two main ways you can try and fi x 09 Doing these slow-motion videos – even them. First of all, try updating Raspbian at the low resolution we're using – is taxing on the with apt-get update followed by apt-get processor. Try to only capture short clips while upgrade. If that method doesn't work, We've told it to make a video at 640 x 480 resolution, using it and defi nitely do not take a lot of them in try using the fi rmware updater like so: to fi lm it at 90 frames per second and to do so for rapid succession. fi ve seconds. This is the maximum resolution for this $ sudo UPDATE_SELF=0 rpi-update speed of video.
Other recording modes 06 As well as being able to record video at 90 frames per second, you can also go down to 60. You cannot increase the resolution though, so it's stuck to 640 x 480 for both 90 fps and 60 fps video for the time being.
“You can also use slow-mo to create high frame rate videos. Video editors such as Kdenlive will let you increase the playback speed” QBe careful not to overwork the processor
Linux & Open Source Genius Guide 71 Tips & Tricks
This isn’t a Using not much more than a hardware hack – a Raspberry Pi, you can route spare SD card can one or more systems through be used for the Tor Hook into the internet just a Tor-enabled access point, about anywhere there’s router, and other guaranteeing anonymity SD cards can be an internet connection – a used for different relative’s house, hotel functions without rooms and more any problems
Connect everything over a wireless network – no need to directly connect to the Pi with a cable Browse privately with Onion Pi Advisor Turn your Raspberry Pi into a highly Rob Zwetsloot models secure and very portable router to keep complex systems and is a web developer proficient in Python, Django and PHP. He loves to your system safe and your browsing experiment with computing anonymous, wherever you are Resources You can easily turn your Raspberry Pi access point using just a Raspberry Pi. This project into a portable wireless router, requiring is fairly straightforward: after setting up the wireless A Raspberry Pi very little power and giving you a wireless access point, we install Tor and do some basic setup Raspbian: network wherever there’s the most basic tasks so that it routes traffi c properly, and securely. www.raspberrypi.org/downloads of internet connections. What if it’s not enough to This will keep you anonymous online – a handy Compatible Wi-Fi adaptor: know you can search the web, though? What if you feature in a time fi lled with privacy concerns all www.adafruit.com/products/814 want to be wholly secure as you do it? Then it’s time to around the web. upgrade the router with Tor to protect your privacy on When the Pi is not connected to the internet, it the internet. should still function as a wireless router, allowing at This ‘Onion Pi’, as dubbed by Adafruit, combines the very least a wireless LAN in your location. Raspbian and Tor to create and secure a wireless Here, we run you through the simple process…
72 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Set up DHCP 06 Now we need to confi gure the DHCP server. Edit the confi guration fi le with:
$ sudo nano /etc/dhcp/dhcpd.conf
And start by putting a # in front of the two option domain-name entries, then remove the # in front of authoritative, seven lines down. Server address 07 At the end of the confi guration fi le, add the following:
subnet 192.168.42.0 netmask 255.255.255.0 { range 192.168.42.10 192.168.42.50; option broadcast-address 192.168.42.255; option routers 192.168.42.1; default-lease-time 600; max-lease-time 7200; Install Raspbian SSH connection option domain-name "local"; Raspbian is the Raspberry Pi distro we’ll be Plug your USB wireless adapter into the Pi 01 04 option domain-name-servers 8.8.8.8, using for the Onion Pi. Download the zip fi le, extract and turn it back on. On another computer connected 8.8.4.4; the image and then apply it to an SD card using: to the same network, open a terminal or type into the } command line: $ dd bs=4M if=[version number]-wheezy- raspbian.img of=/dev/[SD card location] $ ssh [user]@[IP address] Save and exit.
You can also use NOOBS to install Raspbian if Then enter the password for your Raspbian if it asks you wish. for it.
Set up Raspbian 02 Go through the initial Raspbian setup and make sure to turn on the SSH server, and to disable autoboot to desktop – this is unnecessary and will only use extra power. You can also tell it to fi ll up the rest of the card if there’s room for it.
Pi IP Install DHCP 03 We’ll be accessing your Raspberry Pi via SSH 05 To make life easier for any system connecting to set it up. To do this we need to know its IP address – to the Pi access point, we need to install a DHCP you can fi nd it by typing ifconfig into the command server to it. We do this with: line. Make a note of it and turn off your Pi. $ sudo apt-get install hostapd isc- dhcp-server
DHCP will automatically assign IP addresses to network-attached devices, meaning you won’t need static IPs. DHCP server 08 Edit the server confi guration fi les so that it’s set to work in conjunction with the wireless adaptor:
$ sudo nano /etc/default/isc-dhcp- server
Scroll to INTERFACES and change it to:
INTERFACES="wlan0"
Linux & Open Source Genius Guide 73 Tips & Tricks
Incoming Wi-Fi Network addressing 09 We need to set up the Wi-Fi adaptor to be 14 Setting up a NAT will allow multiple clients to both static and accept incoming signals. First: connect. To do this, run:
$ sudo nano /etc/network/interfaces $ sudo nano /etc/sysctl.conf
Put a # in front of iface wlan0 and following lines And add to the bottom of the fi le: with wpa roam, iface default and any other affecting wlan0. net.ipv4.ip_forward=1
Save this, and then fi nish by running:
$ sudo sh -c "echo 1 > /proc/sys/net/ ipv4/ip_forward"
WLAN confi guration 12 interface=wlan0 driver=rtl871xdrv Static IP ssid=[access point name] 10 Now give the wireless interface a static IP – hw_mode=g after the line allow-hotplug wlan0, enter the following: channel=1 IP tables macaddr_acl=0 Run the following three commands to make iface wlan0 inet static auth_algs=1 15 sure the internet connection is forwarded correctly: ignore_broadcast_ssid=0 address 192.168.42.1 wpa=2 sudo iptables -t nat -A POSTROUTING -o netmask 255.255.255.0 wpa_passphrase=[password] eth0 -j MASQUERADE wpa_key_mgmt=WPA-PSK sudo iptables -A FORWARD -i Save and exit, and then set wlan0’s address with: wpa_pairwise=TKIP eth0 -o wlan0 -m state --state rsn_pairwise=CCMP RELATED,ESTABLISHED -j ACCEPT $ sudo ifconfig wlan0 192.168.42.1 sudo iptables -A FORWARD -i wlan0 -o Hostapd eth0 -j ACCEPT 13 After saving and exiting, we need to edit hostapd to point it to this new fi le. Open it with: Apply confi guration So that this still works after a reboot, type: $ sudo nano /etc/default/hostapd 16 $ sudo sh -c "iptables-save > /etc/ And then fi nd the line #DAEMON_CONF="". Remove iptables.ipv4.nat" the #, and change it to: Then add to the end of /etc/network/interfaces: DAEMON_CONF="/etc/hostapd/hostapd.conf" up iptables-restore < /etc/iptables. ipv4.nat
WLAN creation 11 We need to create a new fi le that holds all the information for our wireless network. We are going to make it password protected so that only the people we want to can access it. To create the fi le, start with:
$ sudo nano /etc/hostapd/hostapd.conf
And then enter the text from the next step.
74 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Wi-Fi fi nal 17 Finally, set it up as a daemon so it runs at boot with the following commands: sudo service hostapd start sudo service isc-dhcp-server start sudo update-rc.d hostapd enable sudo update-rc.d isc-dhcp-server enable
And the wireless access point part will be fi nished.
Install Tor 18 After a reboot, we now need to install Tor. Do this simply with:
$ sudo apt-get install tor
Once it’s installed, you’ll need to edit the Tor confi g fi le with:
$ sudo nano /etc/tor/torrc
Follow the next step to add all the necessary information to it.
Tor confi gure If you want to keep SSH open to connect remotely, $ sudo iptables -t nat -L 19 Put this below the FAQ comment: you’ll need to make an exception for that with: If you’re happy with it, save it to the NAT fi le like Log notice file /var/log/tor/notices. $ sudo iptables -t nat -A PREROUTING before with: log -i wlan0 -p tcp --dport 22 -j REDIRECT $ sudo sh -c "iptables-save > /etc/ VirtualAddrNetwork 10.192.0.0/10 --to-ports 22 iptables.ipv4.nat" AutomapHostsSuffixes .onion,.exit AutomapHostsOnResolve 1 Reroute Logging TransPort 9040 21 Route all DNS traffi c fi rst, using: 23 We should create a log fi le in case TransListenAddress 192.168.42.1 you need to debug later. To do this, use these DNSPort 53 $ sudo iptables -t nat -A PREROUTING three commands: DNSListenAddress 192.168.42.1 -i wlan0 -p udp --dport 53 -j REDIRECT --to-ports 53 $ sudo touch /var/log/tor/notices.log $ sudo chown debian-tor /var/log/tor/ And then route any TCP traffi c with: notices.log $ sudo chmod 644 /var/log/tor/notices. $ sudo iptables -t nat -A PREROUTING -i log wlan0 -p tcp --syn -j REDIRECT --to- ports 9040 You can also check it with: $ ls -l /var/log/tor
Secure the router 24 Finally, we can activate the Tor service so that we can start using the access point securely with:
$ sudo service tor start
Table fl ush You can check this if you wish with: 20 We now need to fl ush the current IP tables $ sudo service tor status so that we can get the routing to go through Tor. First of all, do: To make it turn on at boot, you simple add it to rc.d with: $ sudo iptables -F Check and save $ sudo iptables -t nat -F 22 You can check the table setup with: $ sudo update-rc.d tor enable
Linux & Open Source Genius Guide 75 Masterclass Become a Linux power user
78 Dual-boot Linux on any device Give yourself more choice with your computers as we attempt to dual-boot anything and everything from Windows to Macs 78 88 Get started with Android Studio Get your head around the early access preview of the next stage in Android development 96 Get to grips with Go Program! An introduction to one of the hottest programming languages of the moment. Find out if Go is right for you 100 Qt masterclass 88 Get your head around event-driven programming, responsive development, and reading and writing files, all with Qt 112 Program with Erlang Write, document and debug Erlang programs and use OTP to create a fault-tolerant server process
100
“Dual-booting a PC is fairly simple, but what about other devices that we use or carry around with us?”
76 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Linux & Open Source Genius Guide 77 Masterclass
“Dual-booting a PC is fairly simple and inifinitely useful but what about computer-like devices that we use?”
78 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Dual-boot Linux on any device Give yourself more choice with your computers as we attempt to dual-boot anything and everything
Dual booting is not quite one of the dark ourselves the greatest amount of choice possible with computing arts – it’s not like you’re our computers. compiling Gentoo or anything – but there’s Looking around the home and the offi ce, we still an element of risk and reward in noticed a few extra devices we wouldn’t normally the process. Dual-booting a PC is fairly simple, and associate with dual booting: live USB keys, infi nitely useful, but what about computer-like devices Raspberry Pis, Android smartphones/tablets and that we use or carry around with us? We decided to see even a Mac. Adding what we already know about if we could bring the convenience and power of dual dual-booting normal PCs, we set about creating booting to a range of other everyday devices and give the ultimate guide to dual-booting anything.
Warning! It should go without saying that everything we’re attempting in this feature carries some risk of losing all your data. While you should always have a backup solution in place, we know it can sometimes slip your mind. This is your reminder to make sure everything you need is backed up, saved ! elsewhere or ghosted in case of any problems.
Linux & Open Source Genius Guide 79 Masterclass
Dual-boot your PC Get more out of your PC by adding a second Linux distro Linux distros are basically designed The fi rst step in the process is choosing your these days to work well with each other in second distro – what exactly are you trying to multi-boot situations. Modern GRUB will achieve? Are you developing and testing across automatically detect other bootable distros multiple platforms, or do you need a work and leisure during an install and add them to the list. Even if it distro? Will your system have two inherently different doesn’t, it’s fairly simple to add them. Going deeper, functions while booted into each operating system? you can mount partitions across multiple distros Once you’ve made your decision on what exactly you for sharing fi les, profi les and space. To get started, want and need, you’ll then need to create a standard though, you’ll have to create some space on your hard installation medium – either a burnt CD or a live- drive. If you used the standard installation tools with booting USB stick will suffi ce. your main distro, you’re likely using up all the space. You can install in two different ways from here: either setting up the new boot partitions manually, or QAn fstab file, mounting partitions on boot using the Install Alongside option that a lot of modern distros include. You’ll need to create a root partition During the installation, GRUB will usually be for your new distro – we detail how to best go about reinstalled by the new distro so it can boot into both this at the bottom of the page. It’s best to have a wired the new and the old distro, with the new distro set internet connection even if your system has wireless, as a default. We cover the best way to edit or change just in case it doesn’t have the right drivers out of it back depending on your preferences on the next the box for it. Some distros like to download updates page; however, this doesn’t cover shared space or during the install, or even download the entire distro auto-mounting between the two. This can be done by QChoose between installing alongside, or having during installation, so having a more guaranteed way of adding entries to /etc/fstab, as pictured in the fstab full control of the partitions accessing the internet is useful. screenshot above.
Making space The most effi cient way to set up a hard drive and share multiple distros Reclaim space New partitions At the very least, your system will already have a root partition for We recommend having at least 20GB of space for your second your distro, plus a swap partition. This swap can be shared by both distro, or at least the root partition. Make sure not to reclaim all the distros, so all you need to do is trim the storage partition for the available space from the hard drive, as while the ext3 and ext4 fi le main distro. Most live discs and installers will include partitioning systems don’t fragment easily, you may still catch an important tools. However, dedicated rescue and admin distros such as or essential fi le if you shrink too much. You can always try having SystemRescueCD also include them. dedicated space for your home folder on the hard drive as well.
80 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Understanding the boot menu Get GRUB under your control so you can make startup effi cient and easy
GRUB is currently the standard Linux bootloader easily change the order of menu items by renaming that you’ll use. It’s the first thing you see after system the fi les to go in a different numeric order. POST and is designed to let you easily and quickly choose what you plan to do on your computer. It’s Recover highly editable – from small things such as length until If you do decide to reorder the entries, you’ll then need timeout and default selection, to colour schemes and to run: customised boot entries. $ sudo update-grub Proper order QWhile the config file warns against it, there’s no One of the first customisations you’ll usually want to This command can also be used as part of the harm in changing the default selection make to a new boot menu is the default selection and recovery of GRUB from the original distro as well perhaps the timeout. To do this, enter the terminal and – if that’s your primary distro on this system. Then open the grub.cfg file with: instead of reordering the items or changing the defaults, it will be better to start by using it to manage $ sudo nano /boot/grub/grub.cfg GRUB. Boot into it, open the terminal and reinstall GRUB with: Find the line set default=”0” and change the number to the entry in the bootloader that needs to $ sudo grub-install /dev/sdaX be default. Timeout can be found in the folder /etc/ grub.d in the 00_header fi le. The fi les here determine …where X is the location of your hard drive. This is the order in which entries are added to the bootloader, more than likely 1 on a single hard drive system. so as well as changing the default selection, we can After, use update-grub to update the boot menu.
QGRUB’s boot menu also includes recovery and Build your own boot scripts memory testing options Create custom boot entries in GRUB 2 and even GRUB to make the bootloader your own Which GRUB? There are two versions of GRUB: the older GRUB2 chainloader (hdX,Y)+1 GRUB 1 (or GRUB Legacy) and the newer For booting Linux, we point it towards the root of the } GRUB 2. Some prefer GRUB Legacy as distro to boot and the information it needs: EOF parts of it are easier to use; however, #!/bin/sh -e GRUB 2 can be customised a lot more with cat << EOF GRUB themes and images and smarter boot menuentry "Linux distro" { GRUB 1, while simpler looking, needs more ordering. Generally though, if your main set root=(hdX,Y) information to boot into Linux properly: or primary distro comes with one version linux /boot/vmlinuz title Linux distro of GRUB, you should continue using that initrd /boot/initrd.img root (hdX,Y) rather than installing a different version } kernel [/path/to/kernel] root=/dev/sdaX ro over it. EOF initrd [/path/to/initrd] hdX,Y and /dev/sdaX Windows is a little bit different – after telling it where The Windows boot code is again fairly simple, using How exactly do we fi nd out the numbers Windows lives, we then chainload into the Windows the chainloader command to use the Windows to replace these with? In the terminal, fi rst bootloader: bootloader: use fdisk -l to list the drives and partitions #!/bin/sh -e title Windows currently in your system as sda1, sdb3 etc. cat << EOF rootnoverify (hd0,0) For hdX,Y, start counting the drives from 0, menuentry "Windows" { makeactive so sda1 would be hd0,1 and so on. set root=(hdX,Y) chainloader +1
Linux & Open Source Genius Guide 81 Masterclass
Add Linux to Windows Turn a new laptop or PC into a dual-booting machine with dual operating systems for maximum coverage
As always, we start with partitioning can then use an install-alongside option if your distro the hard drive. However, unlike with dual has one and it will automatically set up a good swap booting with another Linux distro, there partition at the same time. won’t be a swap partition available for the GRUB will automatically add a chainloading new Linux distro to use. The swap partition needs to option to boot Windows during startup using its own be the same size as your system memory, so take that bootloader. Windows will be none the wiser and boot into account when creating space for the Linux distro. up as normal, albeit now with a little less space. You Also consider that Linux can easily see and mount can use the fstab tips from the previous pages in your NTFS partitions, while Windows doesn’t like extended Linux distro to automatically mount the Windows fi le systems as much, so keeping the majority of partition and have full access to your fi les and folders; QUbuntu and other distros will recognise and space to be shared on the NTFS partition may be wise. however, you’ll need to look for specifi c ext drivers for know how to install alongside Windows The swap partition can be placed at the end of the Windows to access your Linux fi les. hard drive; this way it’s easy to work around if you have to recreate or resize partitions at a later date. This Secure boot partitioning can be done on the Windows side, but Secure Boot and how it affects it’s preferable to use a Linux tool such as GParted to UEFI and dual booting is still an perform the task. This can be found on some rescue issue – not all distros work with distros, and a few live-booting distros will have that or it. However, Ubuntu, Fedora and their own partitioning tool. You can either just make the their derivatives all work just fi ne on QChainload straight into Windows with no space before installation, or set up the partitions while systems with Secure Boot without much ore than extra work thanks to GRUB you’re at it. One benefit of only making space is that you disabling it in the UEFI menu at startup.
Fragmentation
Windows Vista and later has automatic defragmentation tools built in to cut down on the issues previous Windows systems used to encounter. It’s not perfect, though, and NTFS is still prone to fragmenting more than Linux journal fi le systems as fi les are updated or changed over time. We suggest invoking the Defrag tool from within Windows by going to Start>All Programs>Accessories>System Tools>Disk Defragmenter in Windows 7 and later.
QThere are also some open source tools you can use to defrag Windows
82 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Add Windows to Linux Add or just restore Windows to a system that didn’t always need it
In some ways, it’s currently a little easier to add Windows to a system already running Linux than it is the other way around. Linux systems don’t invoke any Secure Boot privileges, giving you free rein over the hardware, which in this case means installing Windows. The main issue arising from this is that Windows will overwrite the Master Boot Record, meaning you won’t be able to boot into Linux right after the installation. We’ll cover a quick way to recover your GRUB bootloader after installation, but for a more in-depth guide, look towards the end of this tutorial.
Make space 01 We advise allowing Windows to have as Choose wisely much space as possible, as it can fi ll up a lot quicker 03 The Linux partitions will be labelled as than Linux. Microsoft recommends a minimum of unknown by the Windows installer; however, it will 20GB; however, that’s far too low if you plan to use recognise the free space on the hard drive. Select the Windows fairly regularly. 100GB, if you can spare it, free space and click Next to continue with the lengthy should be more than enough. Windows install process.
Restoring GRUB 05 Once you’ve checked if GRUB is installed, we can now begin to restore it. First, mount the root Linux Live booting partition and then type in the terminal: Install Windows 04 We’ll need to recover GRUB now so that $ grub-install root-directory=/[mount point] 02 Insert the Windows installation disc and we can boot into both Linux and Windows. Use an /dev/sdX reboot your system. Select ‘Boot from disc’ from Ubuntu live disc – or your original distro’s live disc if it …where X is the boot hard drive, more than likely sda the boot menu and then go through the basic has one – and boot it up. Check to see if you need to in a one hard drive system. Reboot and log into your language setup and licence agreement. Select install GRUB to the live environment, specifi cally the Linux distro and fi nish off with: Advanced when it asks you how you want to install. grub-pc package. $ update-grub
Linux & Open Source Genius Guide 83 Masterclass
Dual-booting USB storage Create a portable, dual-booting stick or hard drive for recovery, sysadmin or anything you want
Carrying around a live USB or custom live-booting USB storage device can be advantageous if you regularly need to do system maintenance or don’t always know what computer you’ll be working at next. Just one distro may suffi ce for some but, as we’ve mentioned previously, there are many reasons you might be in need of two at a time. Having two fully functional distros working on one USB drive or large USB stick can offer a lot more than an Ubuntu live USB, and it’s actually not too different from setting up a dual-booting system from scratch.
First roots GRUB reinstall Create swap 03 Now create the 15 to 20GB space you’ll 05 Boot into a recovery distro (or Ubuntu’s) 01 Make sure you’ve saved any important data want for the fi rst distro in its installer and set it to be live environment and make a note of what on the external drive before we go any further. The root by selecting ‘/’ in the menu. It will automatically fdisk -l labels the main boot partition on the fi rst thing you’ll need to do is format it and then create mount the swap the partition we created when external drive as. Mount it with: the swap partition – make it 4GB in size so that it can booting into itself and our other distro. Carry on with $ sudo mount /dev/sdXY /mnt make the most out of more powerful systems. Put it the installation instructions as before. And then reinstall GRUB 2 from the terminal with: towards the end of the hard drive like before. $ sudo grub-install --root-directory= /mnt/ /dev/sda
GRUB update Room to share Second distro 06 After rebooting, you’ll be able to get back 02 You can create some shared space for 04 Put in the installation medium for the into the main distro on the external drive. Once there, both distros to use, either independently or as a second distro with the USB storage still plugged mount the other distro’s root partition however you shared home directory if you’re feeling a bit braver. in and boot into its live environment or installer. wish and run: Again we’re saving about 15 to 20GB for the root Choose the external drive again when selecting $ sudo update-grub directories, so keep that in mind while creating it. the location, create a custom partition in the space It will automatically detect the other install and Use ext3 or ext4 as the fi lesystem, or NTFS if you remaining as ext3 or ext4 and set the mount point to update the boot menu next time you boot from the want it to be cross-platform. ‘/’. Install as normal. hard drive.
84 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Dual-boot Android ROMs If ditching stock Android is not enough for you, why not try using multiple ROMs on your device?
Android being an open source operating anything else it suggests installing. It will likely ask system has resulted in a number of you to reboot to recovery to fi nish installation, and custom ROMS that extend or update then show up a boot selection on next boot. Compatible the capabilities of your phone or tablet. Using only one at a time, though, means that if you Adding ROMs devices want to change, you need to spend an afternoon Find the .zip fi le of your preferred alternate ROM Q fl ashing and restoring another custom or stock and download it to the storage of your Nexus Nexus 4 Last year’s Android ROM, leaving you without the device for that time. device. Do not extract it. Reboot the device flagship phone is able For users of Google Nexus products, the to recovery with adb reboot bootloader (on a to run the Ubuntu Touch MultiROM Manager is a fairly straightforward app connected PC), or turn it on by holding down image as well that takes the pain out of the process, requiring you volume-up and power and select recovery from to only have your phone rooted for it to work. This the fastboot menu. You’ll be sent to the TWRP can be downloaded straight from the Google Play recovery, which now has a new option under Store and will then ask to be granted root access. Advanced called MultiROM. Select this, then The next step is installing MultiROM Manager to Add ROM and hit Next to then locate the zip fi le the bootloader, updating the kernel modules and we just put on the device. Swipe to install.
QNexus 7 (2012) The 7-inch tablet that revolutionised the market also runs the Ubuntu Touch image
QNexus 7 (2013) Google’s latest tablet offering is much more powerful, but cannot QMultiROM Manager handles all the flashing and QChoose your distro from the simple advanced installing for you bootloader run Ubuntu Touch
Linux & Open Source Genius Guide 85 Masterclass
Dual-boot a Mac Apple products can still get a bit of open source love – you just need the right tools
While the quality of Apple software Room for more Live boot is debatable on a number of levels, 02 Open Disk Utility from the OS X Utilities; 04 Boot into your Linux distro from the disc the hardware inside the fi rm’s Mac click on the main hard drive and then the Partition and follow the normal installation process we’ve computers is often extremely good. tab. You should see the main startup partition, detailed for other dual boot scenarios – either Since the switch to Intel-based architecture which, unless you’ve changed it, will take up all the use install alongside or create your own custom a few years ago, more Linux distros have hard drive. Resize it, as we’ve advised before, and partition structure. It may take a little longer than technically been able to work on Macs, albeit click Apply. usual, so you’ll need to be a little patient. When it with some restricted drivers installed to make asks to reboot, stay at the rEFInd screen. the most of them. It’s not as straightforward as the other setups we’ve tried in this guide, though, Restricted drivers and involves installing a custom bootloader to 05 The wireless drivers won’t be included allow us to chainload into live discs and GRUB. with most distros for a lot of Macs. Boot into your Once this is done, however, you can enjoy a lot distro and update all the packages over a wired more freedom from your Mac and make full use connection, then go to Settings>Driver Manager and of its powerful, high-quality hardware. install the suggested Broadcom wireless driver.
Some rEFInement 01 Head on over to the rEFInd website (www.rodsbooks.com/refi nd) and download the latest binary zip of the tool. Open Terminal in OS X and cd to the download directory. It’s much like a Linux terminal, so ls will also list the contents. Install it with ./install.sh
Boot from disc 03 Reboot your Mac and the rEFInd boot manager should show up at boot. You may need Boot order to reboot a couple of times for it to work. It will 06 You can change the default selection in automatically boot into OS X unless you press a rEFInd by editing its conf fi le. Boot into OS X and directional key. If the option to boot from CD isn’t locate the refi nd folder from the EFI directory. there, press Esc to refresh the boot selections until Open refi nd.conf in a text editor and locate the line it appears. default_selection. Uncomment it and change it to “Since the switch 2 for your Linux installation, or keep it at 1 for OS X. to Intel-based architecture, more Linux distros have been able to work on Macs”
86 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Dual-booting a Raspberry Pi Short of SD cards for Pi projects? Put two Pi images on one card
The Raspberry Pi, with access to only a limited SD card, may not be the most obvious dual-bootable device. However, due to the recent release of the New Out Of Box Software (NOOBS), multiple Pi distros can be installed onto one card. The catch is that you need to have used NOOBS in the fi rst place to get the dual-booting setup to work. If you haven’t already done this, installing NOOBS onto an SD card is easier than copying an image directly to it. Using the same kind of partitioning tools we’ve used to shrink space and create new structures, wipe an SD card you wish to use on the Pi and create a FAT32 partition that fi lls up the card. Preferably, as we’re dual-booting, you should use as large a card as possible, with at least 2GB for each distro. Once the formatting is complete, download NOOBS from the Raspberry Pi website and extract the fi les onto the SD card, then plug it into the Raspberry Pi and turn it on. You can then select which distros you want to install and it will automatically install them all to the disc, and create a bootloader to use at system startup. Raspberry distros Home & Projects Raspbian is the preferred distro of the Raspberry Pi, with tons of learning software and easier access to some of the more interesting Pi projects and hardware. It’s easy to set up and can be used as a standard desktop OS.
HTPC It’s a tough call between OpenELEC and Raspbmc – both run on the excellent XBMC. However, OpenELEC has been around for longer and has some very special speed optimisations that put it just ahead of Raspbmc.
Customisability Arch Linux on Raspberry Pi allows you to build it from the ground up, only coming with a command-line interface and a basic Linux setup to start with. This allows you to totally customise it and only use the QIf you've previously used NOOBs to install a distro, you can get back to the OS selector by holding down packages and software you wish. shift as you turn it on
Linux & Open Source Genius Guide 87 Tips & Tricks
88 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Get started with Android Studio Get your head around the early access preview of the next stage in Android development
Announced mid-last year at Google I/O, compiling or installation on Linux, Android Studio is Android Studio will be the replacement an incredible easy and quick way to get straight into for the current Android development coding your apps – whether you’re an experienced solution in Eclipse. This Eclipse setup Android/Java dev working directly in code, or a is part of the Android Developer Tools, and takes novice looking to use the graphical, drag-and-drop advantage of the IDE’s Java-based development approach to app creation. environment and plug-in support to currently Android Studio is still currently in beta as a free create the development portion of Android apps. early preview – we’ll cover some of the current With the rest of the SDK, virtual devices can be quirks with the software. However, it’s defi nitely a created and used to test code before deployment great way to get used to using this software before onto real devices. the fi nal switchover occurs. We’ll also cover how The Android Studio aims to be an all-in-one you can export from Eclipse if you’ve already been solution for development and testing. Coming with using that, and how to then distribute your app once the entire SDK, and not even requiring traditional it’s fi nally complete. “Coming with the entire SDK, and not even requiring traditional compiling or installation on Linux, Android Studio is an incredible easy and quick way to get straight into coding your apps”
Linux & Open Source Genius Guide 89 Masterclass
Introducing Android Studio
The new Android Studio is based on IntelliJ IDEA, a much smaller and streamlined Installation is very simple as well – the fi les contain both the IDE and the IDE than Eclipse. Android Studio utilises its features to create an all-in-one Android SDK you’ll need to create your apps, so you won’t need to install the SDK via ADT development environment with a smart visual view that is great for people just separately like you will have done with Eclipse if you’ve developed for Android in getting into app development, and the standard text editor for those who know the past. The Studio allows you to edit how the SDKs are utilised, how imports and their way around Java and the Android API. exports work, and even has a plug-in manager.
Recent Projects The recent projects view New Project allows you to go Start a new project straight into any by going through project you wish the handy Android to work on without new project wizard, having to look in selecting API levels the Open Project and setting the icon prompt
Docs and How-Tos Some tips and how-tos for IntelliJ, the base of Android Studio, are available Check for updates straight from the Android Studio welcome screen has its own update manager, allowing you to keep it up to date. It will notify you when a newer version is available
Installation
Runtime Download 02 Use cd in the terminal to navigate to 01 Head over to the Android Developers the folder you just created, specifi cally the First run website and grab yourself a copy of the early android-studio/bin/ directory. To launch Android 03 If this is your fi rst time using Android access preview to Android Studio from here: Studio, enter into the terminal: Studio, you can safely ignore the prompt to bit.ly/1bWrFXz $ ./studio.sh import any settings. Otherwise, fi nd the fi le you You’ll need to then extract the contents – If it asks about your Java settings, for now created from before, or your old installation put it in a folder you have easy access to, as press Enter, as we can fi x that later if it becomes folder, to get previous settings. We can import you need to run the Studio from the folder. a problem. from Eclipse later on.
90 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Key features Development views Helpful shortcuts As the Java for the Android Android Studio has handy, configurable apps is split into visual and keyboard shortcuts. Here’s a list of the function elements, you can most common to get you started… use a visual WYSIWIG editor for the XML code, allowing Command look-up you to place buttons and (autocomplete command name) text and other UI elements Ctrl Shift A wherever you want on the + + screen. The text part is a full Project quick fi x IDE, with code navigation, debugging, syntax Alt + Enter highlighting and a smart code analyser that warns you Reformat code of any obvious issues with what you’ve written. Ctrl + Alt + L
Show docs for selected API Virtual devices Ctrl + Q Test your apps on a virtual device that you defi ne, either Show parameters for selected method by using settings for the main Nexus devices, or by using Ctrl + P generic settings for different- sized Android phones or Generate method tablets. You can edit the system settings to limit or Alt + Insert increase RAM, and even allow it to use the system GPU if you Navigate open tabs need it to. It’s not perfect, but Alt it’s a great way to quickly test + functionality. Alt +
Jump to source F4
Delete line Ctrl + Y
Search by symbol name Ctrl + Alt + Shift + N
Build Ctrl F9 Debugging + Eclipse migration A specifi c debugger can analyse the code as Build and run For existing Android developers, it’s easy to migrate it runs and give you a full rundown of what’s from Eclipse and the ADT to Android Studio. The happening when, allowing you to go through the Shift + F10 tools are already in place to export the necessary logs and fi gure out any possible issues, or work fi les from Eclipse, and Android Studio includes a out where problems are already occurring. You can Toggle project visibility handy feature that allows you to then import them, pause the program at any point to isolate issues, and any other Studio fi le. This is helpful now and instead of having to search through code as it Alt + 1 will be essential when it replaces Eclipse. continues to run.
Linux & Open Source Genius Guide 91 Masterclass
Create a new project
FAQ Android Studio offers a quick way to start your app: specifi c versions of Android in mind, making newer name, package name, theme, advanced icon options API objects available for some. Since Android Studio is still under some and the ability to select between different Android We’ll look at a basic setup fi rst so we can make our development, there are some bugs that still APIs. The latter allows you to put together an app with Hello World app later. need to be ironed out. Here are a couple of known issues with Android Studio that you can work around.
1) Error: Gradle project refresh failed Android Studio has a newer version of Gradle that has some backward compatibility issues. These can be fi xed by fi rst clicking the link to search build.gradle from the error dialog. From there, double-click the line under the build. gradle usage, opening the project build.gradle fi le. Edit the classpath to change the gradle Create New Project Set image version to “0.5.+”. Finally, save the fi le and From the Welcome screen, click on Create Here you can set your image, giving you a rebuild your project. 01 02 New Project to open up the dialog window. Name nice preview of how it will look on different types of the project HelloWorld – we’ll be creating the code devices. You can set padding, background colour 2) Error: Failed to import Gradle project for that shortly. Leave all the other default settings and more to your app icon. For now though, click This can be a problem after upgrading Android – they relate to what versions of Android will be the Next icon, as we don’t need an icon for this Studio, as the project fi les may point towards able to use the app. sample project. an SDK element that no longer exists. To fi x this, you’ll need to quickly install the Android Support Repository. First, open Android SDK Manager, fi nd and expand Extras, and install Android Support Repository.
Migration It’s quite easy to export existing projects from Activity selector Activity name Eclipse and then import them into Android 03 For now, we’ll just use the blank activity. 04 Here you can name the activity and Studio. First of all, make sure your ADT plug-in When making other apps, this activity selector can customise the way it works, with tabs, drop-down is up to date, then select File and Export. Open help you choose how the app with behave – such menus and other actions. For now, we’ll leave this Android from there and select Generate Gradle as being full-screen or having a multi-column fl ow as is. Hit Finish and the project will be built. It may fi les, then your projects to export before clicking to view several items. take a little while to do this. Finish. In the basic Android Studio window, click on ‘Import project’, fi nd the build.gradle fi le and then press OK at the pop-up. You should then be able to start development on the project within Android Studio.
Project structure Views 05 Android Studio will create a lot of fi les 06 The default view is the visual development for what will be a simple app – luckily we just interface – it includes elements down the side that need to pay attention to MainActivity.java and allow you to drag and drop functionality. You can Main_Activity.xml for the moment. The interface also click the Text tab at the bottom to get straight is also initially set up to allow for WYSIWYG visual into the Java code, with a live preview updating as development. you type.
92 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
A view to develop
The two views for Android Studio serve different purposes – and people. The The text view works like any good IDE, with a hierarchical view of code, code default visual view allows you to drag and drop functions onto the interface, with navigation tools, autocomplete and, of course, proper syntax highlighting. guidelines to make sure you keep them aligned with other UI elements. This then Whichever view you use is up to you – however, you will ultimately have more adds the elements to the code, so you can edit them further in there. control with the text view.
Layouts & widgets Live preview Toolbar Component tree Drag and drop See a live Quick access to Even in the visual different layouts and preview of the preview and debugging view, you can break functions to create way your app tools, along with the down the code your app visually – a will look, even ability to change the in the individual great way for novices while using the theme, activity type components to see to get started text mode and the device preview how they work
Useful resources There are plenty of online resources that you can projects, updates to the SDK, and some Support for IntelliJ IDEA, the base for Android use to help you with your development – whether troubleshooting tips. Studio. Some of the support questions may be you need to fi gure out a more elegant method to Stack Overfl ow – stackoverfl ow.com more suited to the IntelliJ side of things. create a specifi c function, or need to wrestle with One of the best places on the internet for both XDA Developers – some bugs. Here’s a list of some of the places you enthusiast and professional developers to get www.xda-developers.com/ should visit: advice on an issue – either by searching for a Android Developer Site – A community built around smartphone related question or submitting your own. developer.android.com development, the XDA forum is a good way to fi nd IntelliJ IDEA Support – out about Android-specifi c tips and tools, and even As well as the place to get Android Studio, the intellij-support.jetbrains.com/home to promote your application. Developer site has regular posts on interesting
Linux & Open Source Genius Guide 93 Masterclass
Make your fi rst application
Creating your fi rst application in Android Studio is really very easy, already displayed within them – however what we’re going to do in and we’ll start by showing you the simplest one of all – the classic this tutorial is learn how to simply make Hello World in Android, and Hello World app. All basic apps, when created, have Hello World then we’ll go through what other simple tasks we can do using it.
Visually android:layout_height=”wrap_content” import android.widget.Button; 01 To do it visually, simply select Plain TextView, android:text=”Hello World!” This allows us to use some extra Android functions to and place it on the display. Double-click it and type android:id=”@+id/textView”/> press and use the Quit button. in ‘Hello World!’. All static text can be placed this way The layout variables can also be set as fi xed pixel onto the interface, and it will create the necessary width/height. Coded actions code in the text view as well. 06 Add the following code below the line Quit button “setContentView(R.layout.activity_main);”: Relative text 04 The fi le we’re editing now is just the display. Button button = (Button) 02 The relative layout should be included in the To do something, we need to start editing the .java fi le. findViewById(R.id.button) code by default. Create a text parameter by opening While we’re still in the .xml, place a button and give it button.setOnClickListener(new up triangle brackets with: the caption ‘Quit’. Open up MainActivity.java from the OnClickListener() { Package name The name of your package package com.linuxuser.helloworld; according to the Android import android.os.Bundle; system import android.app.Activity; import android.view.Menu; import functions import android.view.View; Import the various functions import android.view.View.OnClickListener; import android.widget.Button; we need to get the code to public class MainActivity extends Activity { work on an Android device @Override public void onCreate(Bundle savedInstanceState) { MainActivity class super.onCreate(savedInstanceState); The MainActivity class includes setContentView(R.layout.activity_main); all the functions in our main Button button = (Button) findViewById(R.id.button); activity that we created button.setOnClickListener(new OnClickListener() { @Override Main layout public void onClick(View v) { This sets what the main finish(); layout, and displays it System.exit(0); } Quit button }); The section containing } the code to make sure our @Override button (with id button) public boolean onCreateOptionsMenu(Menu menu) { // Inflate the menu; this adds items to the action bar properly quits the app if it is present. with System.exit getMenuInflater().inflate(R.menu.main, menu); return true; Settings menu } This creates the Android settings } menu that appears at the top of the app 94 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Working with virtual devices Create a virtual Android device to test and debug your Android apps, choosing from a selection of generic phones and tablets Home screen Physical buttons Emulate a list of There’s a range of Android devices, physical buttons you from Nexus can use that work on phones and certain devices, with tablets to generic power and volume phones defi ned by working on all screen resolution Hardware specs Edit specifi c hardware variables Android buttons to better emulate Software buttons certain aspects of are included on a device, or give it the virtual device, more power via your as they are part of host system modern Android Virtual device manager New device Virtual reality 01 From the Project View, fi nd the 02 In the pop-up dialog, name the virtual 03 Click OK and Android Studio will confi rm Android Virtual Device (AVD) manager icon device, select the device you wish to emulate and the options you’re using. Press Start while the along the top toolbar and click it. From here then the target API level. Some devices will have a device is selected and choose whether you want to you can create and edit a range of devices minimum Android version target. Your can edit this get a pixel-accurate screen, or a scaled version. It for testing your apps on. Click on New in the later, but for our app we won’t need to change any will then boot up a virtual phone or tablet you can right-hand panel so that we can get started. other settings. test on. Debugging Android Studio includes tools to debug the apps you create. To use them, fi rst make sure a device is running to emulate the app. Then, click the Debug button – QUse in the shape of a green bug – and the fi le Android will rebuild and launch on the device. This Studio’s time, though, it will bring up a full logging debugging window, a console and a debugger with tools to full tools to fi gure out where problems analyse your may be occurring in your apps. code Linux & Open Source Genius Guide 95 Masterclass Get to grips with Go program! An introduction to one of the hottest programming languages of the moment. Find out if Go is right for you Advisor Sean M Tracey is a 1 //Example 1: Example variables & creative technologist at a Assigning variables 2 a = 1 functions code listing leading digital agency. He 3 b = 2 spends a lot of his time living in Go is very inside of Node.js, Python straightforward 4 and Arduino and fl exible 5 //Example 2: 6 a, b := 1, 2 7 In 2007 Robert Griesemer, Rob Pike and 8 //or Ken Thompson started writing the 9 language we’re going to be talking about 10 var a, b int = 1, 2 here. Ken and the two Robs felt that, over 11 the years, languages such as C++ had become too 12 //Example 3: cumbersome to quickly develop with and, instead of 13 var a, b, c = 1, true, "Hello" following the path of developers before that tried to fi x Setting constants 14 these problems, they decided to start again from is much stricter, for 15 //Constants can only be set like this scratch. That’s how we have Go. Go’s been out in the obvious reasons 16 const x = 12 real world for a little while now. It was designed to be 17 quick to learn, quick to write and quick to run and it is 18 //##### Functions certainly that. At a glance, Go (or Golang) will look very 19 familiar to those of us who spend our days writing 20 //Example 4: programs in a C-like language, but Go has a few 21 func example(x int, y int){ gotchas and nuances that we’re going to look at in this 22 fmt.Println(x + y) article. We’re going to quickly walk through setting up Function defi nitions 23 } Go on a Linux system (we’ve used Ubuntu 12.04 but it’ll are very user-friendly 24 run on OS X and Windows), the Go I/O standard library, in design and 25 //Shorthand variable declaration, loops, structs, slices and various implementation 26 func example(x, y int){ other little bits that developers will fi nd useful in their 27 fmt.Println(x + y) coding adventures. 28 } 29 Installing Go 30 //With return If you’re quite lucky, you might be able to install Go 31 func timesTwo(x, y int)(int, int){ with one swift command of the terminal: 32 return x * 2, y * 2 33 } sudo apt-get install golang 34 35 func main(){ If all has gone well, you will now have Go on your 36 h, i := timesTwo(2, 6) system. Hurrah! But, for the sake of variety, let’s 37 fmt.Prtinln(h, i) // h == 4, i == 12 assume that’s not happened. Let’s go and install from 38 } source instead so we can all share in the Go goodness. 39 Head on over to code.google.com/p/go/downloads/list 40 //Alternative way with namesd variable return and grab go1.1.2.src.tar.gz (at the time of writing, this Notice how Go allows you 41 func timesTwo(a, b int) (x, y int) { was the latest version of Go). Extract the tar.gz to to assign two variables by 42 x = 1 * 2 wherever you like and head into the src folder that returning two values 43 y = b * 2 you’ll fi nd in there. 44 return Assuming you have GCC or another C compiler 45 } installed on the system, we’re ready to get this thing 96 Linux & Open Source Genius Guide Linux & Open Source Genius Guide going. As it often is, Google has been very kind and has included a handy Bash script to install and test Where’s it going? Go for us. Fire it off with: New languages pop up all the time. So what makes sudo ./all.bash Go different? And then go and make yourself a cup of tea. Like most Go is a really exciting language: it’s quick to remember how seriously people took Node.js source builds, this can take a little while. Just before learn, fun to write and really fast at running. when it was fi rst released? JavaScript… On the Bash script exits, you should see something along It’s true, new languages appear all the time. the server? Wha? Now look where it is. the lines of: Anybody and their mother can learn how to Go already has a massive package base to write a compiler, but Go has some of the best save developers time, so there’s nothing getting Installed Go for [YOUR ARCHITECTURE] in software engineers in the world tinkering in the way between a developer and a pet /home/[USERNAME]/go. away with it. It’s got a long way to go before it project. As for current real-world applications, Installed commands in /home/[USERNAME]/ can stand to challenge the likes of Python or Google recently rewrote its content delivery go/bin. Java and it’s going to take time for people to system with Go. Now that’s what we call *** You need to add /home/[USERNAME]/ acknowledge it as a genuine competitor, but commitment. go/bin to your $PATH. *** If you can see that, everything has gone well. Now, library for Go. It can be written so that one statement want to write a function that accepts variables, we just like the script says, we need to add Go to our imports many libraries or just one at a time like so. can write it like so: system PATH. Everybody has their own preference when it comes to their system but you can put it Single statement: func example(x int, y int){ into your .profi le fi le in your home directory so it’s import("fmt", "math", "net") fmt.Println(x + y) accessible across the system like so: } Multiple statement: PATH="$HOME/gosrc/go/bin" import "fmt" Notice how we declare the types after we name the import "math" variable we’re expecting. If the types are the same we Now, enter go in your terminal and you should see the import "path" could write instead: help dialog listing all the commands we can invoke when executing a Go script. When executed, Go will look to run the main() function func example(x, y int){ fi rst. fmt.Println(x + y) Hello world } Let’s write a quick ‘Hello world’. Create a fi le called Variables lets.go. If you execute: Variable declaration is a little different in Go than If we wanted to return values we would write it is in other languages. There are multiple ways of something like this instead: go run lets.go achieving the same effect, If you look at Example 1 on the preceding page, it can also be written like func timesTwo(x, y int)(int, int){ You should see ‘Hello, world’ pop up in your terminal. Example 2. return x * 2, y * 2 Let’s step through what we’ve written here. package In each of these instances, a == 1 and b == 2. } main is basically Go’s way of creating a namespace You’ll notice that when we used the var keyword, we for the code contained in the fi le. If we were to write, declared the variable type just before we assigned a func main(){ say, a module for use in other programs then we could value. When we declare the variable type at the end h, i := timesTwo(2, 6) use most any name we like. The general convention of this declaration, all of the variables will be that fmt.Prtinln(h, i) // h == 4, i == is that the package name is the same as the import type. If we are assigning values in this way and the 12 path. import(“fmt”) simply imports the stand I/O types vary then Go will assign them based on the } value passed. In Example 3, a will become an integer, b will be a 1 package main helloWorld.go Boolean and c will be a string. Go has constants too. “Google has 2 code listing They can be set like so: 3 import("fmt") 4 const x = 12 included a handy 5 func main(){ But you can’t declare the type like you can with a 6 fmt.Println("Hello, world") variable. That’s inferred from the value. Bash script to 7 } Functions install and test Q Now that we have a working environment, we Functions in Go can do some cool things. In our ‘Hello can run a nice little ‘Hello world’ world’ we used the most basic implementation. If we Go for us” Linux & Open Source Genius Guide 97 Masterclass Notice how we assigned two variables by returning create a subset using the following code: var places = [3]Coordinates{} two values. If we were so inclined, we could name numbers = []int{1, 2 ,4, 8, 16, 32, 64, Then we could pass through a struct to the slice like these variables and return them like so: 128} this: lessNumbers = numbers[1:3] places[0] = Coordinates{1.1, 2.2} func timesTwo(a, b int) (x, y int) { x = a * 2 lessNumbers will be a slice of the values from 1 to 3 Packages y = b * 2 of numbers. You may have noticed so far that we always include return package main whenever we write a Go script. Go } Structs comes with a large catalogue of packages that Go has structs which are kind of like classes in C or can be used in any Go program. The main package Functions in Go can also return closures. But we objects in JavaScript. We can create instances of tells our program where to execute from. If we were must be careful to realise that we aren’t returning structs to store variables. Let’s say we wanted a writing a package to be used in another script, we a function – we’re returning the result of a function way of storing map co-ordinates. Let’s look at the would give it another name. Let’s say we made a that was executed when the function that was called example code below. Bournemouth package: was returned. It’s a little confusing: When we set the variable, the order in which we pass the variables into the struct is the order that package Bournemouth func aFunction(x, y int) int{ the properties within will be assigned. Reassigning randomNumber := 5 a value after its creation is simple: func isAwesome() bool{ return func(x, y int) int { return true return x + y + randomNumber london.longitude = 0.1075 } } } If we wanted to have a slice of structs then we would func isSunny() bool{ declare the type that the slice is to contain the struct return true Notice that in aFunction() we’re specifying that like so: } an integer is being returned, not a function. If this was JavaScript, you’d be returning the function as an object that could be executed in the scope from 1 numbers = []int{1, 2 ,4, 8, 16, 32, 64, 128} which it was called with reference to the scope 2 //numbers[3] == 8 that it had. In Go, we’re only ever executing the 3 function in the context with which it was called; 4 numbers := make([]int, 5) unlike JS, the function itself is not returned – the 5 result of that function is. 6 numbers = []int{1, 2 ,4, 8, 16, 32, 64, 128} 7 lessNumbers = numbers[1:3] Slices 8 Arrays in Go are interesting things. For a start, they’re 9 known as slices. You can create a slice that has a set 10 //#### Structs capacity but no length. If we know what we’re going 11 to put into a slice right away then we can create one 12 package main like this: 13 14 import "fmt" numbers = []int{1, 2 ,4, 8, 16, 32, 64, 15 128} 16 type Coordinates struct { //numbers[3] == 8 17 latitude float32 18 longitude float32 Like in C, If we don’t immediately have the 19 } information that we’re going to store in the slice, we 20 have to tell it how many items it can contain. We do 21 func main(){ this by constructing a slice with the make keyword. 22 london := Coordinates{51.5072, 0.1275} This slice will have a length of 5: 23 bournemouth := Coordinates{50.7200, 1.8800} 24 fmt.Println(london.latitude, london.longitude) //51.5072, 0.1275 numbers := make([]int, 5) 25 fmt.Println(bournemouth.latitude, bournemouth.longitude) //50.7200, 1.8800 26 } If we assign numbers[0] = 2 it would behave as we’d expect, but if we try to assign numbers greater than the length of the slice we’ll get an out of range error. Example slices & structs.go code listing Note that slices can be sliced into smaller slices. For instance, if we have a slice of numbers, we can Q A simple implementation of structs to store individual geo co-ordinates 98 Linux & Open Source Genius Guide Linux & Open Source Genius Guide script, it’ll output how far it is between points A and B. Using packages together And there we have it, our fi rst Go program that uses all the parts of this tutorial. Go has many well-built packages. This example shows how they can work in tandem Finishing up It’s always hard covering the ins and outs of a new In this small example we’re using quite a ADDRESS] it will parse the URL fl ag, make an coding language, since every individual aspect of few of Go’s built-in packages. We have the HTTP request to the page specifi ed, get the the latter could easily take up this space on its own. HTTP library, the I/O library for writing fi les, response body of that request and then store Hopefully, this tutorial will have given you enough of a and the fl ag library for parsing arguments it in a fi le. If you combine the elements of this reason to get excited about using Go in some of your made to the script. When you execute go run code with the distance calculator, you should day-to-day projects. requestStore.go -URL=[ANY VALID WEB be able to pass in new points. You’ll notice that we’re not calling the main function returned value of the distance function by 1.6.) “Arrays in Go are here. This is because this script isn’t going to be In the main() function we can assign pointA and executing any code on its own – it exists to be used by pointB to any item in the places slice. When we run the interesting” a program. Run: go build 1 package main distance.go 2 code listing …and if you don’t see any errors, then we’ve 3 import( successfully made a package. Run: 4 "fmt" 5 "math" go install Bournemouth.go 6 ) 7 …and Go will move this package to the pkg folder 8 in our Go build. Now if we were to import the 9 var places = [3]Coordinates{ Coordinates{51.5072, 0.1275, "London"}, Bournemouth package into one of our programs, Coordinates{50.7200, 1.8800, "Bournemouth"}, Coordinates{55.8580, 4.2590, the methods isAwesome() and isSunny() would be "Glasgow"} } available to the main package, just like Sqrt() or Pi() is 10 when we import math. 11 type Coordinates struct{ 12 latitude float64 Putting it all together 13 longitude float64 So we’ve covered packages, functions, variable 14 name string declarations, structs and slices. That should be 15 } enough to write a simple but useful program. So let’s 16 do that. Create a fi le called distance.go. 17 func distance(a,b Coordinates) float64{ The fi rst thing we’re going to import (after the 18 usual fmt) is the math package. We’re going to be 19 return math.Sqrt( ( a.longitude - b.longitude ) * ( a.longitude - doing some calculations, so we’re going to need the b.longitude ) + ( a.latitude - b.latitude ) * ( a.latitude - b.latitude ) ) square root function that this package exposes. * 69.174; //69.174 miles = 1 degree of latitude on Earth's surface Next, we create the Coordinates struct, Just like 20 we did before but this time we’re including a name 21 } property too. 22 Now we want to create a places slice. For this 23 func main(){ example we’re going to have three locations (London, 24 Bournemouth and Glasgow) so we’ll create a slice 25 pointA := places[0] with a length of [3]. We’re going to be containing 26 pointB := places[2] Coordinates structs in this slice so we need to declare 27 that after the [3]. After that, we enter the geolocation 28 fmt.Println("It is ", distance(pointA, pointB), " miles from ", co-ordinates for each location – latitude then pointA.name, " to ", pointB.name) longitude – then the name. 29 The distance function is quite simple. The 30 } calculation will work out the difference between the two points on a map and then convert that to miles. (If you’d like kilometres, you can multiply the Q The code for distance.go. Using co-ordinates, we can calculate the distance between two points Linux & Open Source Genius Guide 99 Masterclass MASTERCLASS Event-driven programming with Qt Advisor Tam Hanna has been in Keeping your program components separated the IT business since the days of the Palm IIIc. Serving as journalist, tutor, speaker and simplifi es testing. Enjoy the benefi ts of event- author of scientifi c books, he has seen every aspect of the mobile driven coding, with minimal extra effort market more than once Did you ever feel like reusing a part of a program, series of coloured rings). It will use a computational Resources only to find yourself restricted due to various engine which communicates by receiving and dependencies on existing code? Did you ever want emitting signals. Documentation: to handle user input in an engine, but didn’t want to Traditionally, creating an event framework was qt-project.org/doc/qt-5.0/qtcore/ signalsandslots.html make the algorithm dependent on the GUI toolkit? a significant undertaking. Many a developer chose The signal-slot system permits you to connect to deploy Qt because of the signal-slot system. This QtGui index: application components to one another using an honour is well deserved: it is difficult to find a use case qt-project.org/doc/qt-5.0/qtgui/qtgui-index. html event-like paradigm. Instead of having objects call up which can not be handled by the framework. one another, signals are emitted to the framework. In addition to reducing complexity, signals and slots QML intro: These are then routed to so-called slots, which are are needed for interacting with controls provided by qt-project.org/doc/qt-5.1/qtdoc/ invoked in order to handle the processing. Qt’s GUI stacks. Finally, most modules and libraries qmlfi rststeps.html Most computer science problems can best be integrate themselves into the signal-slot system. understood via an example. In the following tutorial, So, if you want to use Qt, you will need to know Further reading... we will create a little application which is to analyse your way around signals and slots. In real life, an event planet.qt-project.org ohmic resistors (whose value is determined by a is any occurrence of note: the opening of a club, the qt-project.org/doc qt-project.org/forums blog.qt.digia.com QAfter startup, Qt Creator shows a ‘start pane’ QFour lists and a button are needed with some instructions for getting started 100 Linux & Open Source Genius Guide Linux & Open Source Genius Guide QAdd some colours to the first three list boxes launch of an aircraft and the birthday of a friend can all Declaring signals and slots be considered events. In computer science, a similar Our QtGui-based example will constrain itself void recalculateValue(); definition can be construed. to dealing with resistors with four bands. This int myVal1; When dealing with Qt, an event is any happening provides us with a total of four incoming and int myVal2; which the class decides to share with the outside one outgoing events. QObject-derived classes int myVal3; world. Most QtGui widgets emit a wide variety of are automatically integrated into the signal-slot int myVal4; events. They can be handled in order to create the framework – the header of our ResistorCalcClass }; actual application logic. Your own classes can emit looks as following: as many or as few events as you see fit. As we will see Being based on a QObject-derived class requires the later, the framework imposes very few limitations in class ResistorCalculator: public QObject presence of the Q_OBJECT macro. It is needed for this regard. { the moc compiler,. Due to the widespread usage of the term event, Q_OBJECT Slots can be declared publicly or privately. In compatibility issues could arise if Qt used a keyword practice, private slots are rarely used: some versions called ‘event’. This is averted by the use of the signal public: of Qt even permit you to substitute the public slots and slot monikers. In Qt, a signal is an event. The ResistorCalculator(); declaration with a short form. The actual defi nition framework routes signals to one or more slots, which of the slot takes the form of a void member function, then process it further. public slots: which receives the parameters that are specifi ed in Like in the real world, event routing takes place on void ring1Changed (int _toWhat); the signal. an interest basis. If a partygoer alerts his mate to bring void ring2Changed (int _toWhat); Declaring signals can prove a bit more diffi cult. a second crate of beer, it is likely that other partygoers void ring3Changed (int _toWhat); Signals get emitted by the object, and therefore also hear this call. However, they simply ignore the void ring4Changed (int _toWhat); have no ‘local’ representation. Due to that, their statement as it is of no interest to them. declaration consists of a void function with a Qt mirrors this behaviour in the registration process. signals: parameter for each bit of data that is to be passed to When a signal is emitted, the framework checks if any void newValue (QString _newVal); the recipient. slots have registered interest in it. Notifications get Finally, the class needs a few member variables dispatched only to those objects. private: to track the state of the incoming data. Linux & Open Source Genius Guide 101 Masterclass Implementing signals and slots Slots receive incoming events and are implemented as normal member functions. It is permitted to invoke a slot like any other function from your code. In our case, the ring-changed functions update the data stored in the model. They also invoke the common recalculation function. It has the job to compute the current value, which is then emitted via the aforementioned signal: void ResistorCalculator::ring1Changed (int _toWhat) { myVal1=_toWhat; recalculateValue(); } Now let’s look at the function which determines the actual return value. Our example uses an abridged computational algorithm which does not provide valid values for some input combinations: void ResistorCalculator::recalculateValue() { QConnect the signals from the list with your newly created slot float retval; retval=myVal1*10+myVal2; the borders of the form to achieve a decent size. The corresponding parameters. Click OK to confi rm and switch(myVal3) toolbar on the left side contains a group of controls: close the editor. Select the currentRowChanged and { drag four Item-Based List Widgets and a Button onto the newly created properties in order to link them up. case 0: the form. Feel free to arrange the controls in any way Repeat this for the four other lists. Finally, link the retval=retval; you fi nd useful. clicked() signal of the button with the closed() slot break; Adjust the properties displayed in the bottom-right of the form – it is shown only if the Show inherited case 1: dialog to create a user interface similar to the one checkbox is enabled. retval=retval*10; shown on page 61. Due to the relative complexity of the break; editing involved, you can also take the mainform.ui file Connecting stuff . . . provided on the cover disc – the editing of the object With that, it is time to connect the QtGui with the default: names and the button caption is of no concern to the engine. The four slots you created in the editor must be emit newValue("No Multiplyer"); signal-slot system. integrated by hand into the mainform class along with return; Qt Creator provides a special tool for editing the list corresponding signals – the editing of the header is } contents. It can be accessed by right-clicking the list. straightforward; the method body looks like this: emit newValue(QString::number(retval)); Then, click Edit items in order to bring up the editor. void MainWindow::ite m1C h a n g e d(int _newVal) } Just add a few colours of your choice. { In the next step, it is time to create the signal and In this part of the code, the emit call is the only one of slot connections. This can be done in a special editor, relevance. It is called to fire off the signal in question. accessed by clicking the Edit Signals/Slots icon in Please note that emit does not act as a return the toolbar. Once it is activated, click one of the lists statement. Method execution continues normally after to mark it as the signal source. As long as you hold the emitting of the signal. the mouse button, the cursor will change into an inverse tree signal. Drop it on the form’s background Creating a QtGui user interface to set the form as the recipient. Create a new slot Now let’s create the user interface. Double-click on by clicking Edit in the popped-up form. Click the ‘+’ mainwindow.ui to open it in the resource editor. Drag button under the Slots list to create a new slot with “The four slots must be integrated by hand into the mainform class QCreate a slot in the main form by clicking the ‘+’ along with corresponding signals” button. Here, three newly created slots are shown 102 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Creating attractive user interfaces with Qt QtGui framework and QML: two solutions for a common problem When Qt was fi rst envisioned, JavaScript and HTML were in their infancy – using them to create complex user interfaces was infeasible. Furthermore, the then-current computer systems were unable to cope with the computational demands raised by animated GUIs. Instead, developers worried about the ever-rising screen sizes seen on desktop workstations. The QtGui framework (which we’ll use to create our example application) addresses these issues. It lets you use native controls QThe signal-and-slot mode provides an overview of the connections in your form and provides a variety of helpers which handle be passed in about the signals and slots and their resizing and positioning. However, it is not well emit changed1(_newVal); parameters. In principle, the parameters of the signal suited to animation. } and the slot must match 1:1 – special conditions are In Version 4.7, Nokia introduced a new GUI Here, we forward the signal via a slot. Theoretically, we explained in the URLs given in Resources (page 60). stack called Qt Modeling Language (QML). This JavaScript- and JSON-based system could also connect signals to one another – this special By the way, signals and slots created in the editor are offered impressive animation facilities and case is described in the documentation. The signal from connected automatically. initially was intended solely for mobile phones. the engine must also be processed. For this, create In theory, QML elements consist solely of another slot with the following code. Also, include the Analysing the benefits rectangles, bitmaps and texts which are QMessageBox via an #include: After all this coding, it is time to recapitulate what rendered using OpenGL. void MainWindow::datareceived(QString _val) we have done. In addition to learning about the This approach greatly simplifi ed porting the { manipulation of QtGui-based user interfaces, a framework to new operating systems. Due to QMessageBox aMessageBox; component was created which computes resistance that, Digia decided to make QML available on aMessageBox.setText(_val); values. ResistorCalculator is completely independent all platforms. A set of predefi ned controls will aMessageBox.show(); of the user interface: it just needs four incoming signals eventually simplify the creation of commonly } and a recipient for its slot. used widgets. For a class as trivial as this one, the benefits might Using QtGui usually leads to quicker results, Finally, it is time to connect the signals and slots. This is not be visible immediately. However, let’s assume as controls do not need to be assembled by ideally done in the constructor of the form, which is to that ResistorCalculator would implement a complex hand. Furthermore, the native integration be modified as shown in Fig 1. computation determining trajectories or amortisation. ensures that the results look exactly like other The connect macro takes pointers to sender In this case, reuse would be a highly attractive desktop apps. and receiver. In addition to that, information must proposition. Exchanging the QtGui front-end for a QML front-end or even a headless server becomes a question of adapting a few connect() calls in the MainWindow::MainWindow(QWidget *parent) : Fig 1 constructor of the host class. QMainWindow(parent), The power of this, alone, justifies the extra effort ui(new Ui::MainWindow) needed for implementing signals and slots. However, { the isolation has further benefits. Imagine the creation ui->setupUi(this); of an automatic test script. Thanks to the signal-slot myEngine= new ResistorCalculator(); system, all it has to do is create an instance and fire off connect(this,SIGNAL(changed1(int)),myEngine,SLOT(ring1Changed(int))); a few signals – mock objects are not needed. connect(this,SIGNAL(changed2(int)),myEngine,SLOT(ring2Changed(int))); In short, using the signal-and-slot system tends connect(this,SIGNAL(changed3(int)),myEngine,SLOT(ring3Changed(int))); to pay off in most cases. With that, it’s time to end our connect(myEngine, SIGNAL(newValue(QString)),this,SLOT(datareceived(QString))); travel through the fascinating world of Qt for this issue. } Next time, we’ll look at layouts and size policies. Until then, keep coding… Linux & Open Source Genius Guide 103 Masterclass Responsive development with Qt Monitors and devices come in all shapes and sizes. Qt’s layout system ensures that your app looks good, no matter how big the display Since Trolltech set out to develop Qt, screen If the user increases the size of the window, the QtGui-based user interfaces achieve this via sizes have increased rapidly. Ten years ago, a button remains ‘stuck’ at its original position. the layout system. Controls are to be placed in 15-inch CRT monitor was considered a decent Additional screen real estate would be occupied containers which contain information about screen – today, cheap notebooks come with a by white space. the intended arrangement. In addition to that, 17-inch Full HD display. Developers were left An ideal computer analyses the user interface. invisible meta controls (spacers) expand at standing in the rain – an unadjusted application Elements would be resized and repositioned runtime, pushing widgets to the border of their looks ‘lost’ when running on a large screen. as to their ‘usefulness’: content controls would relative container. A well-made Qt application looks great on gain size, whereas modifi ers and buttons would The knowledge gained in the following article is not a large variety of screen sizes. Your advisor is remain clickable. limited to the QtGui stack. Cascades, XAML and most aware of some products which use the same .ui This can be achieved by liberating the layout JavaScript frameworks take a similar approach to file on smartphones and workstations – on both engine from the constraints of developer- dynamic resizing of elements. Web developers refer to systems, users are satisfied with the results. provided co-ordinates. A ‘fl exible’ user interface it as ‘responsive web design’, but tend to use the same This is achieved by a ‘reversal of control’. In is specifi ed in terms of relations and growth processes and idioms outlined here. traditional user interface systems, controls are behaviours, dependent on the screen size of the positioned where the developer drops them. Let’s device being used. At runtime, a layout engine A first layout assume a window with a size of 200x200 pixels. performs the actual positioning and determines It is now time to start applying all of this knowledge The size of the controls would be specifi ed inside the actual co-ordinates using the ‘description’ in practice. Our first example will consist of a group the window – a button is located at 60|60//120|70. provided by the developer. of buttons which are arranged above one another – “A well-made Qt application looks great on a large variety of screen sizes” QApplying a layout to the form immediately arranges the controls QOur empty layout shows up as a red line 104 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Missing form layout! A tiny issue causing strife and troubles since day one When a form is fi rst created, Qt Creator does not provide it with a layout. This does not hinder you from placing widgets and sublayouts onto it – you can even set size policies and other layout-related attributes. These will, however, be ignored completely at runtime. This strange behaviour is caused by a rarely documented property of the Qt layout engine: forms which do not contain a form layout are not managed. Therefore, attributes such as size policy will not be adhered to. Fortunately, identifying this problem is relatively easy. Resize your form in the designer and look at the behaviour of the controls: if they do not adjust their sizes accordingly, your form is likely missing a layout… “Changing the size of the form will not affect QThe three buttons have become part of the layout we will then expand it to include a group of other Your buttons will immediately be arranged above this placement – widgets which will be arranged in a more or less one another. Changing the size of the form will not complex fashion. affect this placement – the three buttons will stay in the three buttons Start by launching Qt Creator. The following tutorial the middle of the window as it keeps changing its size. uses Qt Creator 2.8.1, the examples are based on Qt This behaviour is shown in the editor and at runtime. will stay in the 5.1.1. Older editions of the integrated development environment will behave similarly – the layout system More layouts middle” itself has not been changed much since Qt 4.0. Forgetting to assign a form layout disables the layout Layouts tend to be based in forms. Double- system in its entirety. Sadly, most cases require should not affect the size of the elements of a layout- click the fi le mainwindow.ui in order to open it in more than one layout per form – arranging a group of managed form – this is achieved by more subtle the WYSIWYG editor. The toolbox on the left side buttons horizontally under a list is not possible with means discussed later in the article. of the screen contains a group of controls. Drag just a horizontal or a vertical layout. For now, it is enough to place the text box above a few buttons into the form and drop them off at Qt permits nested layouts. A horizontal layout can the buttons. The toolbar on the left side of the a convenient position. Then run the program and contain a vertical layout which then, in turn, contains screen contains a group called Layouts. Select a resize the MainWindow by dragging its borders. The another horizontal layout. The high performance horizontal layout and drag it onto the form in order to positions of the buttons will remain unchanged. of contemporary computer systems ensures that add it. Initially, the layout will show up only as a slim Close the application after you have convinced complex forms will still be rendered in real-time. In red line. yourself that static layouts cannot adjust to changes the last ten years, your advisor has not experienced a Due to an internal limitation of the editor, layouts in the form size. Return to the WYSIWYG editor and single case of lag caused by overcomplex layouts. can not be modifi ed while a form layout is in place. take a look at the icons next to the filename chooser Creating nested layouts is relatively easy. In our This can be fi xed by clicking the form and selecting displayed in the top-right corner of the screen. example, add a text widget to the main form by the option ‘Break layout’ in order to strip the form of The sixth button from the left has a symbol made dragging it in from the toolbar. Qt Creator will display a its layout intelligence. When this is done, the layout up of three horizontal lines shown above one another. blue line in the form. This informs you that the control will show up as a small red square – drag the three Select the form in the tree and click the button in will be placed in a layout and provides indication of its buttons into the rectangle in order to add them to the order to add a horizontal layout to the form. location in the layout hierarchy. Keep in mind that you internal HorizontalLayout. Linux & Open Source Genius Guide 105 Masterclass When finished, the three controls will be displayed inside the red rectangle. The layout can be moved en bloc: selecting the layout as a whole is easiest if you drag a ‘selection rectangle’ around part of the controls. Actual movement is then accomplished via the arrow keys. In the fi nal step, don’t forget to re-enable the form layout. If this is done, the resulting form is made up of a large and expanding text box and three slim buttons glued to the bottom of the screen. A question of policy Looking at the text box and the buttons in the example leads to a puzzling conclusion. The Qt framework somehow knows that some controls can profit from extra height, whereas others do not. This information is used at runtime in order to give additional height to the textbox, while the buttons remain small. Each widget has a sizePolicy attribute for the vertical and the horizontal direction. These properties can be modified by selecting one of the widgets and setting up its properties accordingly. The seven attributes that are supported, as of Qt 5.0, are outlined in Fig 1. QInflating the size of the form does not affect the arrangement of the widgets This, however, is not the only possible way the layout system can be affected. If controls are intended to display real estate is available, it is shared according to grow at different rates, this can be configured using the the stretch values set in the corresponding direction. Attention, Horizontal and Vertical Stretch properties. By default, For example, if you had three buttons with their they are set to zero – so the controls all ‘grow’ and horizontal stretch factors set to one, two and three excessive width ‘shrink’ together. they would correspondingly take up one, two and three Manufacturers of monitors Assigning a value to horizontal stretch and/or times as much space at the bottom of the created form. have their own agenda vertical stretch permits you to modify the size-gaining In the last years, the diagonals of screens have behaviour. When controls are laid out, each widget is Minimum and maximum sizes grown considerably. At the same time, the assigned the minimum required space. If additional In an ideal world, the computer would be able to aspect ratio has become fl atter and fl atter: automatically detect the minimum and maximum today, screens tend to be much wider than sizes needed by widgets. Sadly, practical attempts at they used to be. This new situation provides Fig 1 using artificial intelligence for determining minimal considerable challenges to user interface QSizePolicy::Fixed This widget can neither and maximal heights has, by and large, proven to be developers. Very wide texts are diffi cult to read shrink nor grow unworkable. Instead of that, Qt provides developers – it is for good reason that most books and QSizePolicy::Minimum This widget can newspapers are printed in portrait mode. When theoretically grow, but with two properties called minimumSize and the minimumSize is maximumSize. creating an application optimised for very wide more than enough Their behaviour is largely logical. A control will displays, try to think about how the additional QSizePolicy::Maximum This widget can be horizontal screen real estate can be used. reduced in size if other never become smaller than the values specifi ed Android tablet apps tend to provide useful widgets can use the in its minimumSize, and never exceed the size of space better – it can inspiration thanks to their fragment layouts, NOT grow maximumSize. A newly set up control usually has a which are used to give additional context. QSizePolicy::Preferred This widget can be minimumSize of 0 and a maximumSize of 16777215 shrunk if needed – For example, an email client would display – this means that the framework can compute the growing is possible a hierarchy of folders next to the actual but does not lead to minimal size from the content and can make the advantages message. QSizePolicy::Expanding This widget can profi t widget as large as possible. from extra space, but can be shrunk if the need arises QSizePolicy::MinimumExpanding This widget can profi t from extra space, but “Analysing and comparing the sizes can NOT be shrunk if the need arises QSizePolicy::Ignored This widget tries to of the widget to the ‘base size’ allows grab as much space as possible you to compute the ideal font size” 106 Linux & Open Source Genius Guide Linux & Open Source Genius Guide class Ui_MainWindow Fig 2 { Where is the public: QWidget *centralWidget; intelligence? QVBoxLayout *verticalLayout; QTextEdit *textEdit; Analysing the process behind QHBoxLayout *horizontalLayout; the WYSIWYG editor QPushButton *pushButton_3; When a .ui fi le is saved, the Qt toolchain QPushButton *pushButton_2; QPushButton *pushButton; transparently creates a ui_*.h fi le. This header QMenuBar *menuBar; usually contains a setupUi method which QToolBar *mainToolBar; assembles the windows described in the ui fi le QStatusBar *statusBar; out of the classes provided in the QtGui stack. As an example, take a look at the snippet in void setupUi(QMainWindow *MainWindow) Fig 2, which was taken from one of the forms { created as this article was written. if (MainWindow->objectName().isEmpty()) Please keep in mind that these fi les are MainWindow->setObjectName(QStringLiteral("MainWindow")); automatically regenerated whenever the form MainWindow->resize(395, 228); is saved. Due to this, manual changes in the ui centralWidget = new QWidget(MainWindow); fi le are not recommended – developers using centralWidget->setObjectName(QStringLiteral("centralWidget")); source control systems are advised to keep verticalLayout = new QVBoxLayout(centralWidget); these fi les out of the CVS at all expense. verticalLayout->setSpacing(6); verticalLayout->setContentsMargins(11, 11, 11, 11); verticalLayout->setObjectName(QStringLiteral("verticalLayout")); Conclusion textEdit = new QTextEdit(centralWidget); Due to a long tradition of Visual Basic, Windows users textEdit->setObjectName(QStringLiteral("textEdit")); do not expect their applications to show an overly large verticalLayout->addWidget(textEdit); amount of flexibility as forms are resized. Due to the widespread adoption of Qt, Linux and UNIX users are far less tolerant of applications which do not resize “Qt provides developers with two themselves intelligently. Creative use of the layout system provides you with the aforementioned logic with minimal extra code. additional layout components” Arranging your controls in layouts can be done in mere Advanced layouts seconds, but leads to significantly better applications. Most computer problems can be solved by using the not be achieved without the use of spacers. Spacers Practical experience shows that the layout vertical and horizontal layouts. Sadly, some more are a design-time-only component which is not visible system can be daunting due to the large amount of complex layouts can not be addressed in this fashion. to the user. functionality offered. Most developers tend to get Qt provides developers with two additional layout At layout time, spacers have the habit to try to acquainted with the handling within few hours – let this components which help to simplify the creation of extend to the maximum size possible. Placing a article be your guide to more attractive and responsive advanced forms. spacer on the left side of a control ensures that it user interfaces. Grid layouts permit you to arrange widgets in a will be pushed to the right side of the screen as far shape similar to the ones used in Excel or LibreOffi ce as possible. Aligning a control to the middle of the spreadsheets. After dragging one of the widgets onto screen is possible by using two spacers at the sides the screen, the layout will pop up on the screen as an of the control. empty rectangle. Then, controls can be dragged in by one another. When dropping the widget, a blue line Font sizes? will be shown on the right or the left side of the layout As screen densities increase, font sizes eventually – dropping it there adds a new row or column to the become problematic. Keeping them inline is grid layout. currently not possible in the framework. Fortunately, Form layouts are a specific from of the addressing the issue is easily done in code. Most aforementioned grid. A form is made up of two controls contain an event handler which will be columns: the one on the left contains the labels, invoked whenever the layout engine changes the size whereas the control on the right side is intended for of the elements of the screen. collecting the actual input. Analysing and comparing the sizes of the widget to the ‘base size’ allows you to compute the ideal font size. Spacers for efficiency The actual ‘correlation’ can be done in a linear or in a Creative use of layouts permits you to create a large stepped fashion – advanced information on this can be QThe three buttons have become part of the variety of attractive forms. Sadly, some effects can found in classic typography literature. layout Linux & Open Source Genius Guide 107 Masterclass Reading and writing files with Qt Reading and writing fi les stored on disk is essential for more complex applications. Qt provides a common interface which eliminates wrangling with the native APIs Even though the C standard library provided some cause problems for beginners. Future parts of this variety of commonly used storage locations. As some sort of common API for disk access, creating a truly series will introduce you to databases, XML and operating systems store data in multiple folders, a cross-platform application remained a daunting other forms of ‘advanced storage’. StringList is returned. We simply grab the fi rst entry in task. This is caused by differences in the underlying Our program will start out by displaying a list of the the list and set it as the working directory. myCWD is a operating system. ‘C:\’ is a valid path under Windows, files and folders found in the home directory of the user. declared member of the MainWindow class. but does not work for most UNIX systems. Sadly, finding its location is not as straightforward as QStandardPaths was introduced with Qt 5.0. When The C/C++ fi le system API was specifi ed in one might think at first glance – keep in mind that our dealing with legacy versions of Qt, you will sometimes the 1970s and ’80s. Back then, databases and application might run under Windows, Linux and Mac see code using static methods of the QDir object. By XML fi les were unknown – data was stored in fl at OS X at some point in the not-so-distant future. and large, they behave similarly to the module used in fi les containing a group of entries ‘back-to-back’. Qt provides a class which lets us determine the this feature here. Accessing XML or SQL records in vanilla C/C++ location of the user’s desktop path. So, the first ‘act’ of requires an awful amount of glue code. our program consists of determining where to start our Enumerating the directory Fortunately, Qt provides a variety of classes which journey through the file system: Once the CWD has been determined, we must proceed help you develop applications swiftly and effectively. to showing the user an overview of its contents. This is If you design your programs correctly, they can move MainWindow::MainWindow(QWidget *parent) : done via a process called enumerating the directory. from Linux to Windows and/or Mac OS X without .... Performing an enumeration is quite simple: on query, rewrites – Qt handles the idiosyncrasies of the { the file system will return a list containing all of the different fi le systems. Furthermore, a group of utility .... entries in the folder. We can process these for display or functions is made available for accessing complex myDesktopPaths=QStandardPaths::st a n d a r d Lo c at further analysis. In our case, we’ll use them to populate data formats such as XML and SQL databases. ions(QStandardPaths::DesktopLocation); a QListWidget displayed in the main window. The code This time, we will walk through the creation of myCWD=myDesktopPaths[0]; needed is as in Fig 1. a fi le-fi nding utility. The intent is to teach you a } variety of procedures needed when accessing the fi le and directory system. In addition to that, we Calling the static standardLocations method of introduce and analyse common ‘pain points’ which QStandardPaths lets you query the system for a QHead to the Qt project website (http://qt-project.org/) to download the latest version of Qt. On the QThe file sleuth displays the contents of the website you can also access helpfuflu l forums, development topics and tools author’s desktop 108 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Fig 1 void MainWindow::enumerateCWD() Recursion { myDir.setPath(myCWD); What goes around tends to enumContents=myDir.entryInfoList(QDir::NoFilter, QDir::DirsFirst); come around for(int i=0;i Linux & Open Source Genius Guide 109 Masterclass come home to roost. Under UNIX, the root of the system is ‘/’. Windows instead provides the user with a group of drives which are not unifi ed under a common root. Fortunately, handling this is relatively easy. The cdUp function returns false whenever the file system does not provide a ‘parent folder’. Even though this can also be caused by lack of writing privileges, we will – for now – assume it to be an indication for the user hitting the root folder. In that case, we will provide the user with a list of ‘drives’: void MainWindow::m yIte m Clic ke d (QListWidgetItem* _inItem) { if(_inItem->text().compare("[..]")==0) {//G o u p if(myDir.cdUp()==true) { myCWD=myDir.absolutePath(); ui->listWidget->clear(); enumerateCWD(); } else QSelect what kind of widgets Qt Creator should use to pre-populate your layout file {//Enumerate drives enumContents=myDir.drives(); ui->listWidget->clear(); for(int i=0;i 110 Linux & Open Source Genius Guide Linux & Open Source Genius Guide absoluteFilePath(), this); If we are looking at a directory, the parsing logic A common example would be the analysis of log myFileWindow->show(); is a bit more complex. First of all, we must ensure fi les created by servers. Slovakian fi rm Tamoggemon that the fi le system will not send us to a path already Holding uses a Qt program to determine the ‘pain Traversing the file system for real seen: the ‘virtual directories’ . and .. are sought out points’ faced by users of TouchCalc – manually With that out of the way, it is now time to go ‘searching’ and eliminated beforehand. Then, the method is analysing the data would be a prohibitively for files and folders on the computer. Our example will invoked again to analyse the contents of the actual expensive process due to the large number of limit itself to comparing the names of files and folders folder in question. transactions involved. to a pattern entered by the user. For this, the routine Debugging recursive routines is not easy. A great With that, this part of our journey through the world in Fig 2 is responsible. It can be triggered via a button way to simplify the process is outlined in the example of Qt ends. added to the bottom of the ‘file navigator’. above: we use qDebug to emit info about every Understanding this routine requires you to have invocation of the recursive function. This way, a sort grasped the concept of recursion – outlined on the of ‘call stack’ is printed into the Application Output “Our example second page of this tutorial. MySeekClicked starts window as the program is run. out by allocating a QList which is to contain all files Due to space constraints, our program will not emit program shows returned from the seeking algorithm. the list of found files. Instead, set a breakpoint at the In the next step, the actual seeking function myResultList=myResultList line in order to see the seekThisFolder is invoked. It requires a pointer to contents. all important the common storage and a QDir instance pointing at the folder which must be scanned. The calling Conclusion obstacles faced of entryInfoList then returns a list of the contents Even though our example program is not particularly found. complex, it shows all important obstacles faced by by developers We proceed to inspecting each element returned. developers creating applications which access the file If it is a file, its name is compared to the search string system. The procedures outlined here can be used to creating apps entered; matching files are then dumped into the QList. realise a wide variety of utilities. which access the void MainWindow::m y S e e k Clic ke d() Fig 2 { QList Linux & Open Source Genius Guide 111 Masterclass The fi nal version of the Running Erlang from Documentation of an Erlang The Erlang sqFile Erlang module the UNIX shell module in HTML format graphical monitor Program with Erlang Advisor Write, document and debug Erlang Mihalis Tsoukalos is a UNIX system administrator programs and use OTP to create a also profi cient in programming, databases and mathematics. He fault-tolerant server process has been using Linux since 1993 This feature will introduce you to the equipment. Although it looks like a special-purpose Resources Erlang programming language, which is programming language, it is not. If you are using a Erlang: used to build massively scalable NoSQL data store such as CouchDB or Riak, you are www.erlang.org software real-time systems with already using Erlang without even realising it. Try Erlang: requirements for high availability. Some of its uses The fi rst thing that you need to do if you are used www.tryerlang.org are in telecoms, banking, computer telephony, to programming with computer languages such as eCommerce and instant messaging. The runtime C, Perl or Python is forget everything you know about Mnesia DBMS: system of Erlang has built-in support for classes, variables and assignment of variables. You www.erlang.org/doc/apps/mnesia/ concurrency, distribution and fault tolerance. will need to think more in terms of pattern matching, MochiWeb: Erlang was developed at Ericsson Labs pathways and message passing. It may sound weird https://github.com/mochi/MochiWeb in order to serve the switching of telephone but it is also powerful and fun! 112 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Introducing Erlang 01 Your Linux distribution probably includes a ready-to-install Erlang package that you will have to install. Alternatively, you can compile Erlang and install it manually. On a Debian 7 Linux system, about a hundred The Erlang new packages will be installed after executing the shell following command (as root): 02 # apt-get install erlang The Erlang programming language makes hard things easy and sometimes makes easy things hard. It was not designed for big data, but it was designed for massive concurrency and is a great language for fault tolerance and distributed applications. The syntax of Erlang does not look like a C-based language. The Erlang way of programming concentrates more on data fl ow and you should learn to design your programs based on recursion and message passing. Erlang can also program web services with the help of the MochiWeb library. Variables start with capital letters in Erlang and ‘words’ starting with lower-case letters are atoms. Atoms are constants with their own name as their value. The Erlang shell Erlang numbers and variables The Hello World 02 The shortest interaction you can have with 03 Erlang uses two kinds of numbers: integers 04 program in Erlang the Erlang shell is the following: and fl oats. ‘10’ is an integer whereas ‘21.0’ is a Writing the ‘Hello World’ program in the Erlang fl oating-point number. Integer numbers are always programming language can prove to be a little $ erl exact, but fl oating-point numbers may get truncated tricky because you will have to create an Erlang Erlang R15B01 (erts-5.9.1) [so urce] [64-bit] a little bit depending on their value. When you are module in a separate fi le, along with a function that [smp:8:8] [async-threads:0] [kernel-poll:false] using fl oats, you should always have a number to the prints your message. left of the decimal point. In other words, you cannot Then, you will have to compile the fi le and run Eshell V5.9.1 (abort with ^G) write .6; you should write 0.6. the function to fi nally get the ‘Hello World’ message 1> q(). When defi ning a new variable inside the Erlang displayed on your screen. ok shell, you cannot change its value afterwards, as the The name of the fi le must match the name of 2> mtsouk@mail:~$ following example demonstrates: the module; therefore, if the module will be named ‘hello’, the fi le should be called hello.erl’. You can As you can see, if you want to exit the Erlang shell, 1> MyVar = 12. call the function any name you want; ‘helloWorld’ you will have to type ‘q().’ and press the Return/Enter 12 is a representative name. key. Make sure that you do not forget to put a dot at 2> MyVar = MyVar + 5. Then, you will have to type the following inside the end of the command because Erlang commands ** exception error: no match of right the Erlang shell: always end with the dot character. hand side value 17 The q() function is the easiest way to exit the 3> MyVar = 5. 1> c(hello). Erlang shell, but keep in mind that the q() function ** exception error: no match of right {ok,hello} quits everything Erlang is doing. If you are working hand side value 5 2> hello:helloWorld(). locally then there is no problem, but if you are hello, world working on a remote system it’s better to quit by Erlang interprets ‘MyVar = MyVar + 5’ as ‘12 = 12 + ok typing Ctrl+G and then q. The reason is that you 5’, which is obviously wrong. Erlang uses the single 3> may shut down the Erlang runtime on the remote assignment model, which allows each variable to machine when quitting using q()! be assigned a value only once in a given session or The fi rst command compiles the hello.erl source The Erlang shell is perfect to test most of your context. Your fi rst exposure to Erlang may confuse fi le and the second command calls the helloWorld() stuff before putting your code into a module. and frustrate you, but you will soon feel captivated function of the hello Erlang module. The output is The built-in line editor, erl, is a subset of Emacs. and excited. shown on screen. Linux & Open Source Genius Guide 113 Masterclass More information about Erlang you will get the following warning message when you -import(square, [area/1]). 05 A key part of the Erlang Run-Time System try to compile the square module: (ERTS) is the BEAM processor. BEAM stands for There is no command to import all functions at once ‘Bodgan's Erlang Abstract Machine’, a virtual machine 1> c(square). from an existing Erlang module. If a function of a that interprets optimised BEAM code. When you square.erl:8: Warning: this expression will module is not used inside the module, you will get a compile an Erlang program, the compiler converts the fail with a 'badarith' exception descriptive warning message. code into a BEAM fi le. {ok,square} The Erlang syntax may look strange the fi rst time Reading fi les in Erlang you see it, but if you start writing Erlang programs This happens because variables must start with 09 This part will make the previously developed more often, you will stop thinking about its syntax in a capital letter in Erlang. The last statement of a square module able to read the contents of a text fi le. less than a week! function is also the return value of the function. So, the Each line of the text fi le should be a number that will The Erlang shell is the perfect place to test most area() function returns a number. represent the length of the square’s side. The name of your code before creating a module, but you can of the new module will be sqFile. Fig 01 shows the also compile an Erlang program using the Erlang Explaining the code added code. compiler (erlc) without entering the Erlang shell: 07 At the top of the module, the -module The fi le:open built-in function (BIF) inside the directive dictates the name of the module, which readlines() function opens the fi le for reading. Each $ erlc hello.erl is square. The -export directive defi nes the list line is processed by the process_line function. Finally, of functions that are visible to the world – public the save_line function prints some output on screen Similarly, you can run an Erlang function without functions in object-oriented terminology. The using the io:fwrite() BIF to make sure that the fi le was entering the Erlang shell: square module exports two functions, called length read correctly. and area. Note that both the name and the total The io:fwrite() function is analogous to the printf(…) $ erl -noshell -s hello helloWorld -s init number of parameters of a function are mentioned. function found in C. The trim_whitespace function stop Erlang considers functions with the same name uses regular expressions to remove empty spaces hello, world but a different number of parameters as totally from its input. different functions. All module code must be Please do not forget that Erlang cannot solve every included in functions. Writing fi les in Erlang programming problem effi ciently – you may need to Listing the available functions as well as other 10 This part will make changes to the previously use another programming language for a specifi c type useful information about a module can be done using developed sqFile module to support the writing of of problem. Always use the best tool for the job. the following command: data to a new file. Each line will contain three numbers separated by a space character: the side of the square, An Erlang module 1> square:module_info(). the perimeter and the area of the square. The input 06 When creating programs, Erlang file – a file called data will be used – contains float programmers put their code in modules. Modules An alternative way of getting information about a numbers. The filename used for saving the output will help you organise, store and share code easily and module is the following: be output – its name is hard-coded. Opening a file for effi ciently and are the preferred way of writing Erlang writing is performed using the following line of code: programs. 2> m(square). The source code for the ‘square’ module, which {ok, FILE} = file:open("output", write) calculates the perimeter and the area of a square Using the square module given its side, is the following: 08 If you look inside the directory where square. You should not forget to close the fi le after all write erl resides, you will also see a binary fi le called operations are done, by using the following line of -module(square). square.beam, which is the compiled module code in code: -export([length/1, area/1]). BEAM format. You should compile your Erlang code length(Side) -> 4 * Side. every time you make changes to the source code. file:close(FILE) If you want to use the area function from the area(Side) -> Side * Side. square module in another module that you create, The actual writing of the output to the ‘output’ fi le is you should include the following command inside done using the following Erlang code: If you use side instead of Side as the variable name, the new module: io:format(File, "~.3f ~.3f ~.3f~n" , [Number, L, A]). “Erlang programmers put their The Erlang Debugger 11 Erlang comes with a Debugger! You have to follow a number of simple steps in the Erlang shell code in modules. Modules help you in order to start the Debugger. For debugging the sqFile module, you will have to execute the following organise, store and share code easily commands: and efficiently” 1> c(sqFile, [debug_info]). 114 Linux & Open Source Genius Guide Linux & Open Source Genius Guide {ok,sqFile} readlines(FileName) -> Fig 01 2> im(). {ok, Device} = file:open(FileName, [read]), <0.42.0> try process_line(Device) 3> ii(sqFile). after file:close(Device) {module,sqFile} end. 4> iaa([init]). true process_line(Device) -> 5> sqFile:readlines("data"). case io:get_line(Device, "") of eof -> []; The fi rst command compiles the Erlang module and adds some extra debugging information, which is a Line -> save_line(Line), common practice in every programming language. Line ++ process_line(Device) The second command starts a new graphical end. monitor that is the main Debugger window. The third command interprets the code of the Erlang module save_line(Line) -> io:fwrite("**** ~s~n", [trim_whitespace(Line)]). that is going to be examined. The fourth command tells Erlang shell to automatically attach the trim_whitespace(Input) -> re:replace(Input, " \\s+", "", [global]). Debugger to any process that is executing interpreted code when that process is started. The last command executes a module function and makes the Debugger start running. The Erlang Debugger Documenting Erlang code 11 12 The extra lines you are going to put in the sqFile module for documenting your code do not affect the functionality of the module. Erlang needs them for generating the HTML output. Documenting your code is an excellent practice that you should always follow even if you think you do not have enough time to write the few extra lines. In order to document the mylength function, you can put the lines from Fig 02 before its implementation. And the lines in Fig 03 add general documentation for the entire module. In order to generate the HTML files and put them in a directory called documentation, you should execute the following command in the Erlang shell: edoc:files(["sqFile.erl"], [{dir, "documentation"}]). Remember that documentation is produced for the functions that are in the exported list even if Fig 02 you write documentation code (@doc) for other %% @doc This function calculates the length of the perimeter of a square. functions. %% For a square with a side length of A, the length of its perimeter is 4 * A. mylength(Side) -> 4 * Side. Reading fi les in Erlang 13 The Erlang Process Manager is a useful Fig 03 tool for keeping track of your processes and %% @author Mihalis Tsoukalos Linux & Open Source Genius Guide 115 Masterclass information about a process, you can double-click on it. If you want a shorter list of processes, you Reading fi les should check the Hide System Processes option. 13 in Erlang The Process Manager is a tool that you will defi nitely need when you get more comfortable with Erlang and OTP. During the learning process, it will mostly help you discover what is going on behind the curtains. Erlang OTP 14 OTP stands for Open Telecom Platform and comprises Erlang’s open source libraries and tools. Despite its name, OTP is about much more than just telephones! It is the most important and vital part of Erlang. If half of Erlang's greatness comes from its concurrency, distribution and error-handling capabilities, then the OTP framework is the other half of it. OTP was designed for building projects with big teams. It has been rewritten twice before, so it is now in its third version. OTP is about taking all the generic components, putting them into libraries, making sure they work fi ne and reliably, and then reusing that code when possible. So, what is left for the programmer? The programmer has to deal with things that will change from application to the mechanisms you will use and interact with tcp:send() function to actually send some data to the application. If you are going to write real-world processes. They are a way of formalising common TCP server. After finishing with the gen_tcp:send() Erlang software, you will eventually have to learn patterns. Line 6 defi nes a macro that tells the function, you should return to the TCP Server and run the OTP framework. Erlang compiler that every time it sees the word the flush() command to display the received data. ?SERVER, it should replace it with ?MODULE. So, To close the TCP server and client connections, The serverLUD module macros are just simple text replacements, as in C. you’ll have to run the gen_tcp:close() function. The 15 and its supervisor In line 7, the count of the number of calls that were f() function executed on the Erlang shell of the TCP The name of our module that uses OTP is serverLUD. made to the server is kept. In line 22, a function that server tells the shell to delete and forget any bindings The commands in Fig 04 show you how to interact with prints a message on screen is defi ned. In line 14, it has. the serverLUD server process. the defi ned function calls the start_link function of Connecting using the UDP protocol is almost Now, it is time to write an Erlang program (module) gen_server in order to start up the process. identical – you just have to use the gen_udp Erlang that will supervise the serverLUD server. Its name module instead of the gen_tcp module. will be simpleSUP. The important thing is that by Using TCP in Erlang writing Erlang code to supervise the server, if the 17 This part will show you how to create a simple The Mnesia database serverLUD server goes down for some reason, the server that listens to a TCP port and interacts with a 18 Mnesia is an extremely fast, distributed supervisor will automatically start a new serverLUD TCP client using the Erlang shell. You will need two DBMS written in Erlang that is part of the standard process, offering high availability to your Erlang Erlang shells: one shell for the TCP server and one for Erlang distribution. server – which is the main reason that Erlang was the TCP client. In order to create a new Mnesia database, you will used at the Ericsson labs. The TCP server is created using the have to run the mnesia:create_schema() function. What you should keep in mind is that the gen_tcp:listen() Erlang function. Next, you have to The mnesia:create_schema([node()]) command supervisor is responsible for the serverLUD server set the socket to an accepting state by calling the creates a Mnesia database in the current node without serverLUD knowing about it and without the gen_tcp:accept() function. After executing the gen_ only because of the [node()] parameter. After the programmer having to change a single line of code tcp:accept() function, the Erlang shell waits for a TCP mnesia:create_schema() function is executed, inside the serverLUD.erl fi le! connection without allowing you to type any more Mnesia creates a new directory structure to host its Erlang commands. database. Explaining the serverLUD The TCP client tries to connect to the TCP server Mnesia queries are similar to SQL and list 16 OTP module by calling the gen_tcp:connect() function. When comprehensions. ‘List Comprehensions’ is an Erlang If you are not familiar with Erlang code, you are likely specifying an IP address in Erlang, you should use feature that allows you to create lists without having to find the code on both modules difficult to read and commas instead of dots to separate the four parts of to use some other language features (funs, maps or understand. the IP address. After executing the gen_tcp:connect() fi lters). Line 5 tells Erlang that the module will implement function, the gen_tcp:accept() function of the TCP Populating and querying an Mnesia database is the gen_server behaviour. A behaviour pre-defi nes server returns. Next, you should execute the gen_ beyond the scope of this feature. 116 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Multiple CPU cores and Erlang 19 As Steve Jobs once said, “The way the Using TCP processor industry is going is to add more and more 17 in Erlang cores, but nobody knows how to program those things. I mean, two, yeah; four, not really; eight, forget it.” The good news is that Erlang supports multiple CPU cores natively! An Erlang program is implemented as a large number of small processes that do simple tasks and communicate with each other. Functional Programming languages forbid code that has side effects. Erlang processes interact with one method by exchanging messages without sharing any data with other processes, therefore allowing Erlang code 1> serverLUD:start_link(). %% start server Fig 04 to utilise multiple CPU cores without problems. {ok,<0.34.0>} You only have to make sure during the designing 2> serverLUD:say_hi(). %% run a server function phase of your Erlang application that it has lots of Hi Linux User and Developer! processes that do not interfere with each other. ok As a result of all the above, an Erlang program can 3> serverLUD:get_count(). %% get the number of connections so far run four times faster on a quad-core CPU without 1 changing a single line of code! 4> serverLUD:say_hi(). %% make another connection Hi Linux User and Developer! What to do next? ok 20 One article cannot cover every aspect of a 5> serverLUD:get_count(). programming language such as Erlang. You should 3 experiment and write many lines of Erlang code to learn 6> whereis(serverLUD). %% find the process ID of the process the Erlang way of thinking and programming. First, <0.34.0> learn how to use the Erlang shell well and then try to 7> serverLUD:get_count(). %% get the number of connections so far 4 write your own programs using your favourite editor and 8> serverLUD:stop(). %% stop the server compile them using erlc. ok If you are really into Erlang and want to master 9> it, it would be wise to buy Armstrong’s book – Joe =INFO REPORT==== 25-Jan-2014::17:28:10 === Armstrong is the creator of Erlang. terminating server Erlang will certainly change you as a programmer and definitely make you look at programs in a totally 9> whereis(serverLUD). %% the server process is not running any more! different way. undefined 10> What to do 20 next? QWritten by Erlang creator Joe Armstrong, this is a highly recommended read Linux & Open Source Genius Guide 117 Developer guide Essential tools for coders 120 Write once, use anywhere with 142 Create a real-time CPU meter 150 Get started with Gulp.js Qt with Node.js and Duino Explore the many benefits of using the Qt is the most flexible cross-platform Get a Node.js-powered Arduino to display Gulp.js task runner in your web projects framework for C++ and can even create cross- the current processor usage of a computer in to help streamline your workflow and platform GUIs. Learn how real-time optimise your code 124 Answer real-world questions 146 Build tic-tac-toe with Kivy 154 Master Vim in easy steps with Qt Ease into the workings of Kivy by creating this Take a step ahead, learn some advanced Computer simulation can solve a variety of real- pen-and-paper classic in just over 100 lines features and get more productive with the world problems, ranging from retail purchases of Python Vim editor to trading card game modelling 128 Handle Twitter’s three-legged oAuth process Building and signing requests according to OAuth can be tricky. We explain the authentication process for Python and Flask 132 Create dynamic templates with Flask, Jinja2 and Twitter We create a dynamic webpage with Twitter and Flask’s rendering engine, Jinja2 136 Build a blog with Django Learn how to program and publish a Django photo blog site in easy steps 132 146 118 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Linux & Open Source Genius Guide 119 Developer guide Write once, use anywhere with Qt Advisor Qt is the most fl exible cross-platform Tam Hanna has been in the IT business since the days framework for C++ and can even create of the Palm IIIc. Serving as journalist, tutor, speaker and author of scientifi c books, he has cross-platform GUIs. Here’s how… seen every aspect of the mobile market more than once During the initial development of C, programmers, who started to develop a product creating GUI applications was considered called Quasar toolkit (later known as Qt). Work on the Further reading... a non-use case: it was developed on framework started in 1991 and the product managed planet.qt-project.org a computer system attached to non- to achieve a cult following. The tremendous success qt-project.org/doc graphical dumb terminals. The average developer of the Qt-based KDE framework led to concerns spent his time churning away on engine code – which, about the closed nature of the framework. qt-project.org/forums if designed correctly, would result in an easily portable Qt’s creators Trolltech addressed this issue with application. the introduction of a dual-licensing scheme. From As time went by, operating system vendors that date onward, the framework was provided in two extended C/C++ with a variety of custom libraries. forms. Commercial users could buy the code, which This led to an ever-decreasing level of portability: permitted them to keep changes to themselves, while apps for Windows, Linux and Symbian are built with a GPL’d version also became available. C++, but the non-domain-level code cannot be In addition to that, a legal trapdoor was shared between platforms. This ired two Norwegian established to protect the availability. It is QAfter startup, Qt Creator shows a ‘start pane’ with some instructions for getting started 120 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Installing Qt Getting Qt up and running is really easy Due to the ubiquity of the Qt framework, sudo chmod u+x qt-linux-opensource- most Linux distributions contain a package 1.4.0-2-x86-online.run containing the aforementioned IDE. However, this version tends to be well out of date – the sudo ./qt-linux-opensource-1.4.0-2- safest and sanest approach is to obtain the x86-online.run latest version directly from the Digia download servers. As of this writing, the current version This will then start the automatic installer. QAfter starting a project and choosing where to is 5.1.1 – the following steps are based on it. It will provide sensible default values for store it, select Desktop on this screen The easiest way to install Qt can be had by all settings – for the average developer, responsible for the fl ood of updates which have visiting the URL qt-project.org/downloads. just clicking Next repeatedly will provide a shaped the framework’s history: if the current Then, click the option for the ‘Qt Online workable installation. commercial maintainer does not update the free Installer’ which best fi ts your system. After The download itself can take up to 30 version at least once a year, it is automatically the download is complete, open a terminal minutes, during which you can continue to use placed under a very liberal open source licence. and execute the following commands: your system normally. Nokia purchased the framework with the intent to unify its various developer offerings. This project sudo qtcreator First steps came to nothing. Qt is now owned by an IT consulting SOFT ASSERT: "i != -1" in It is now time for us to start working on a little company named Digia. Even though the free licence file ../../../../src/plugins/ ‘Hello world’ application. Click File>New File or has long been replaced by the LGPL, a commercial projectexplorer/taskmodel.cpp, line 172 Project in order to open the Project Creation version remains available. Ports for iOS and wizard. Then, select Applications>Qt Gui Android are in development – Qt is available almost Qt Creator has a slightly unusual layout, which can Application and click Choose. In the next step, everywhere. best be understood by looking at the GUI in its default the product will ask for a name and a storage state. The IDE is subdivided into multiple submodules, location. IDE à la Qt which can be activated via the ribbon on the left-hand After that, a relatively important dialog shows Digia provides users of the framework with a custom side of the screen. Click any of the icons to activate the up. Qt Creator contains a relatively advanced cross- IDE. Qt Creator works best when run in superuser corresponding view – it will show up in the remainder compilation framework, which permits you to target mode. Start it by entering the following commands of the GUI. The bar at the bottom allows you to fi nd multiple different versions and platforms with one into a terminal hosted in /opt/Qt/ – the soft asserts out further information about compile processes and click. For our basic project, just select Desktop – we emitted are normal and do not point to mistakes in application status – by default, it is hidden to save will use this dialog more in later parts of the tutorial. confi guration: screen real estate. The ‘Class Information’ dialog is not signifi cant and can be closed by clicking Next. Then, click Finish to start the generation of the project skeleton. Then, click Play to start the program – it will display the project skeleton in Edit mode. A new build process When clicking the Play button at the bottom-left corner, the framework compiles and runs the application. In the background, a quite complex process takes place – understanding it is paramount to using Qt successfully. A Qt-based application is compiled by the native toolchain intended for the operating system. The process is controlled by makefi les which, however, are generated automatically by a tool called qmake. It analyses the .pro-fi le of a solution and uses that information to set up the toolchain. This cumbersome-sounding process offers a signifi cant benefi t to the developer: as the .cpp fi les are processed by the native compiler, they can contain any kind of code which the native operating system understands. It is fully possible to embed a native API call into an event handler or a similar QQt Creator supports a large variety of project types portion of code – if you do not need Linux & Open Source Genius Guide 121 Developer guide Going cross-platform Taking our application to another operating system is really easy Currently, we have no ‘non-framework’ code, so there are no obstacles from a code point of view. Even though Qt Creator contains decent cross-platform capability, it tends to be limited to compiling within the same platform. This means that a UNIX-like system could easily create a binary for a different UNIX-like system running on a different processor type (producing an executable fi le for Windows requires considerable effort). When targeting a different desktop operating system, bringing up an instance and installing Qt Creator, there is the fastest way. Digia provides complete installers for Windows, Mac OS and various fl avours of UNIX. Just copy the project fi les over, open the .pro fi le and start a compile process. The situation in the mobile space is a bit QQtGui-based forms can be edited using a WYSIWYG editor different. Here, cross-compilation is a matter of necessity. The choice of desktop operating system is sometimes made by the manufacturer cross-platform compatibility, there is no need GUI from file of the device in question. BlackBerry 10 and to provide ‘protection’ around the native code. Qt currently contains two different GUI stacks, which Android can be programmed with any desktop Should you ever want to port your app to another can be used co-operatively. QtGui (Qt 5 renamed this operating system, whereas Windows Phone platform, just use the C preprocessor to blot out the module to ‘widgets’) uses a group of controls mapped and iOS tend to require a device running the problematic code section. to the native operating system. Qt 4.7 introduced ‘corresponding’ desktop operating system. The .pro fi le of our example can be opened by a second system called QML, which uses a JSON- clicking it in the fi le list on the left side of the screen. It derived language to describe the modules making up will look like this: the user interface. The toolbar on the left of the screen contains a Even though rumours pertaining to the death of variety of controls, which can be dragged onto the QT += core gui QtGui tend to pop up every now and then, Digia has form in the middle. The list at the bottom right shows greaterThan(QT_MAJOR_VERSION, 4): QT += repeatedly confi rmed that it is strongly committed to the properties of the currently selected control. widgets keeping the system around for the foreseeable future. The tree view above it provides an overview of the TARGET = untitled Using QtGui tends to bring results quicker. As the structure of the user interface. TEMPLATE = app base project we created was based on QtGui, we will During compilation, a program called uic (User SOURCES += main.cpp\ continue to use it during the rest of the tutorial. Interface Compiler) will transform the .ui fi les into mainwindow.cpp QtGui-based forms are created entirely in source header fi les containing the code needed to generate HEADERS += mainwindow.h code. However, Qt Creator provides a WYSIWYG the actual form. In the case of our skeleton program, FORMS += mainwindow.ui front-end which simplifi es the management of the the fi le will be called ui_mainwindow.h (it can be user interface. Double-click any .ui fi le in the project found by opening the fi le mainwindow.h, right- In principle, a .pro fi le contains a list of fi les and overview in order to activate the design view, as clicking the relevant inclusion and then instructing Qt settings which are needed to build the product. pictured above. Creator to follow the declaration of the symbol). Qt is made up of multiple modules – including the ones needed via the QT-Statement. The target- and template-Statements determine the type of binary “Qt Creator provides a WYSIWYG front- which is to be generated. The HEADERS-, SOURCES- and FORMS-Blocks list the fi les of each type. When a end which simplifi es the management of new fi le is added, Qt Creator updates these sections automatically. the user interface” 122 Linux & Open Source Genius Guide Linux & Open Source Genius Guide In addition to that, this listing also demonstrates the use of the Q_OBJECT macro. It is required in all classes which are derived from a QObject-derived class – its use will be discussed in the next part of the series. For now, it suffi ces to say that it must be there to avoid compiler errors. #ifndef MAINWINDOW_H #define MAINWINDOW_H #include public: explicit MainWindow(QWidget *parent = 0); QThe source code of main.cpp is shown with syntax highlighting ~MainWindow(); Its structure will be similar to this code snippet: pushed onto the screen. Finally, exec() is invoked: it private: will return only when the program is to be terminated: Ui::MainWindow *ui; class Ui_MainWindow }; { #include Linux & Open Source Genius Guide 123 Developer guide Answer real-world questions with Qt Advisor Computer simulation can solve a variety Tam Hanna has followed the mobile phone market since the year of real-world problems, ranging from retail 2004. His fi rm Tamoggemon Holding k.s. uses Qt to maximise code reuse purchases to trading card game modelling across different mobile and desktop operating systems. Did you ever wonder how many booster Let us start our observations by looking at the packs must be bought to have one copy situation of a well-off Slovak who enters his Tesco Further reading... of each card in a set? Did you baulk at store to pick up his fi rst stone. Each one of the 36 planet.qt-project.org purchasing expensive stockings due to available pictures is ‘new’, as our friend’s collection qt-project.org/doc qt-project.org/forums the high total cost of ownership? Did you ever feel is completely devoid of smurf dominoes at this point blog.qt.digia.com overwhelmed by analysing the impact of a fi nancial of time. decision you have to make? The situation looks far less satisfactory on the Even though most problems can be solved by second run to the supermarket. As the collection trying to fi gure out the mathematical formulas, now contains one stone, the chance for fi nding a brute-force simulations can bring useful results far new one is now reduced to 35/36ths. As more and more quickly. The method outlined in this article can more stones get collected, returns diminish: more be used for all problems, which can be broken down stones reduces the odds for fi nding a new one. into a group of easy-to-analyse ‘passes’. Running enough of them leads to a result fi eld, which can then Well-off computers be analysed further. Modelling this situation on a computer is This story was inspired by a promotional comparatively easy. The collection of each individual campaign run by Slovakia’s Tesco retailer. The can be represented by an array of 36 booleans: if one company gave its customers a ‘domino gem’ for of them is false, the element in question is not yet each purchase. Each of the stones was imprinted found in the user’s collection. with a picture of a different smurf character. The actual simulation is performed by using the Customers were encouraged to complete an random-number generator to pick a domino stone. album by fi nding one copy of each of the stones – a Its ID is compared against the collection in order task made more complex as the packages did not to determine if the stone is already there – if it is reveal their contents until after they were opened. not there yet, the fl ag’s state is changed and the But how many packages would need to be number of missing stones is reduced by one. purchased in order to have a realistic shot at A loop is used to keep adding stones until having a full, completed album? all stones are found in the collection. Counting RNG-Output 1 2 3 4 5 6 7 5 6 7 Assigned elements 1 2 3 4 QIn this case, the first three elements get more Slot for elements 1 2 3 4 random ones than the fourth one. This causes an imbalance in the result 124 Linux & Open Source Genius Guide Linux & Open Source Genius Guide the number of iterations will inform you about the amount of packages that are needed by the Multithreading and race conditions individual in question. We will start our work by implementing a single Speeding up simulations by executing multiple runs pass. It shall be encapsulated in a method that in parallel returns the number of attempts needed to its caller, as follows: In the last few years, thermal and physical condition, which happens whenever two limits have set constraints on the maximal threads access the same location in int MainWindow::runAttempt() performance achievable from a single memory at the same time. { core. Chip vendors have approached this In this case, the local value stored in one //Res et problem by creating multi core processors thread will overwrite the results produced int missing=36; containing multiple cores on a single die. by another one. The collection process int attempts=0; Developers can use these systems to would then end up with wrong data, which, int whatDoIHave[36]; run a group of threads in parallel. If they in the worst case, could cause a crash. for(int i=0;i<36;i++)whatDoIHave[i]=0; are designed correctly, tremendous speed This situation can be remedied by increases can result – some algorithms using Mutexes and Semaphores. They Processing begins with the declaration of the scale in an almost ideal fashion. ensure that the threads do not access the variables needed for storage. Using method-local As long as the individual tasks are same sequence at the same time. Sadly, storages is helpful when it comes to creating multi- independent from one another, no weird creating reliable multithreaded code threaded simulations, as there is less potential for behaviour will occur. Sadly, practical tends to be complex – please consult the race conditions. programs tend to suffer from race corresponding literature. In the next step, we commence with the actual population process. It is implemented in the form of a while loop, which runs until the amount of missing stoned reaches zero: //Start to populate while(missing!=0) { attempts++; The individual attempts are modelled by invoking the qrand function. It returns a random integer ranging from zero to the highest value of the random number generator. Its value can be found by looking at the RAND_MAX constant, which is declared in stdlib.h. For our example, we are satisfi ed by knowing that the highest number returned is much larger than the number range used for modelling. If the ranges were to end up being ‘close’ to one another, you might fi nd that mapping diffi culties could occur. We process the returned number via the modulo operator. The resulting value is then used as ‘index’ to the array – if the element is not yet set, the missing QGauss Curves describe randomised processesimbalance in the result counter is decreased accordingly: Feel free to run the code from a command line insanely lucky Slovak might fi nd a new gem on each int myResult= qrand() % ((35 + 1)); or GUI. Each invocation of the method will return a purchase, whereas a very, very unlucky one might if(whatDoIHave[myResult]==0) different number, determined by the sequence of the purchase a million packages and still end up with { random numbers returned by the RNG. missing ones. whatDoIHave[myResult]=1; Processes that are – by and large – dependent on missing--; Increase accuracy by random subhappenings tend to align themselves in a } collating results pattern described as Gaussian standard deviation, } Colleges and schools tend to provide mathematical as can be seen displayed in the above graph. return attempts; problems which have one – or, in the worst case – a Our program will return the data needed to } small amount of permissible and correct results. As determine the various supporting points of the our simulation is based on a random process, it can diagram. This can be achieved by running the same With that, the code is done. return a large variety of different and valid results. An process over and over again until a large enough Linux & Open Source Genius Guide 125 Developer guide sample of data has been collected. Most software- myBinningArray[i]=0; that customers must expect to buy a lot of packages based random number generators need a ‘starting for(int i=0;i<50000;i++) before their collection is complete. value’, which initialises the numerical processes { contained. It is customary to pass in the current time int idx=runAttempt()/10; How many iterations? in a production environment. if (idx>=27)idx=27; Our example analyses a total of 50,000 individual myBinningArray[idx]++; fl ows. Determining the ideal amount of iterations is QTime time = QTime::c u rre ntTim e(); } a science of its own – there is no universal principle qsrand((uint)time.msec()); governing the number of runs needed to have a Finally, data must be ‘normalised’ in order to simplify satisfactory result. In the next step, a binning array is created. Their the processing. This is achieved by dividing the Due to the tremendous power of today’s role can be diffi cult to grasp on the fi rst glance. number of iterations seen by each slot of the bin computer systems, most real-life problems can be The table below contains a ‘schematic’ description, with the total. The result is then multiplied with 100 overanalysed in a few seconds worth of processing which should be consulted as you work through the to achieve the percentage of iterations that fell in the time. Letting the example run for a minute or so will following example. bucket in question: usually lead to an amount of data so large that the Each attempt is classifi ed by the amount of results can be considered statistically signifi cant. purchases needed to complete the collection. We for(int i=0;i<27;i++) Looking carefully at the graph below reveals that ‘band’ these up so we end up with a group of slots qDebug() << (float)myBinningArray the diagram contains three different coloured lines. of ten each. The binning array is then made up of a [i]/(float)50000*100; The bright blue one was achieved by analysing 500 group of ints that count the amount of occurrences iterations, whereas the yellow one shows the result of each band. For example, the slot zero would Divisions tend to require quite a bit of computing of 50,000 runs. contain a count of the runs where zero to nine power. Due to that, C treats all division operations In principle, you can be satisfi ed with the results purchases were needed. involving two integers as int – the normalisation of the analysis if the values start to ‘converge’ and Our routine uses a total of 27 different slots, process above would not work out, as the result of change less as the number of runs increases. handling a range of ten each. So, the zeroth element the division would be rounded to either zero or one. of the array would be responsible for all iterations Casting the two parameters to fl oat ensures that the Stockings galore ranging from zero to nine, and onward. Here, the arithmetical operation will be carried out with fl oat The methods outlined here can be applied to a upper level was determined by ‘good luck’ – higher data types, thereby preserving accuracy. wide range of real-life problems. One amusing results are collated into the highest element via an if The example that we have used dumps the problem solved recently involved stockings. clause as follows: computed results into the command line. You can Purchasing a single pair of expensive stockings copy them into a spreadsheet such as the one is perceived as a waste of money, as a ladder in int myBinningArray[27]; included in LibreOffi ce. This method was used to one of the stockings would render the other one for(int i=0;i<27;i++) generate the chart shown in fi gure fi ve – it is clear completely useless. Purchasing multiple packs of ... 20-29 10-19 0-9 QA binning array is made up of multiple slots for data Q Customers must expect to purchase a large number of gems to get a complete set 126 Linux & Open Source Genius Guide Linux & Open Source Genius Guide hosiery will reduce the impact of a single ladder. QTime time = QTime::c u rre ntTim e(); Fig 01 However, it also increases the amount of money qsrand((uint)time.msec()); bound in underwear. for(int numberOfPacks = 1; numberOfPacks < 11; numberOfPacks++) Before getting into the actual model, we must { clarify one thing in advance. The results of this int stockingsInStore=numberOfPacks*2; program will not return an ‘ideal’ value deciding on int counter=0; the amount of stockings to buy. Instead, the data while(stockingsInStore>=2) used will serve as input into the decision process { made by the owner. counter++; Let’s set out on modelling the actual problem at stockingsInStore -= modelStockingWearing(); hand. For our program here, a ‘transaction’ can be } started as long as two intact stockings can be put qDebug() << numberOfPacks << " " << counter; to use. They will be assumed to break down with a } random probability of three per cent when worn – aging will not be a factor for brevity reasons: int MainWindow::modelStockingWearing() { int lostStockings=0; for(int numberOfPacks = 1; numberOfPacks < 11; numberOfPacks++) Fig 02 //Left sto c k in g { int myResult= qrand() % ((100 + 1)); int counter=0; if(myResult<=3)lostStockings++; for(int i=0;i<1000;i++) //Right stocking { myResult= qrand() % ((100+1)); int stockingsInStore=numberOfPacks*2; if(myResult<=3)lostStockings++; while(stockingsInStore>=2) return lostStockings; { } counter++; stockingsInStore -= modelStockingWearing(); modelStockingWearing is a relatively simple } function. It invokes the random number generator } twice in order to fi gure out whether the left and/or qDebug() << numberOfPacks << " " << counter/1000; the right stocking was damaged during the outing. } This result can then be processed further in a mother routine. See Fig 01. In principle, this method is easy. It runs the modelStockingWearing-Function until the number of remaining stockings falls below two. The amount of possible iterations is then dumped into the command line for further processing. See Fig 02. The improved version of the harness routine runs the internal test a thousand times. It then divides the counter by thousand in order to receive the average value per iteration. A chart with the results of our computation (right) shows that the price per ‘wearing session’ does not decrease signifi cantly when more than four packages are purchased in one sitting. Conclusion While using computers to solve non-virtual problems by brute force is a process that is not currently perceived as acceptable – and this story glosses over some mathematical details – the results returned by a modelling system based on the methods described tend to be accurate enough for everyday use. QThe price of each ‘wearing’ stops to reduce after the fourth package was bought Linux & Open Source Genius Guide 127 Developer guide Handle Twitter’s three-legged OAuth process Building and signing requests according to OAuth can be tricky. We explain the authentication process for Python and Flask Advisor Sean M Tracey is a creative technologist at a leading digital agency. He spends a lot of his time living inside of Node.js, Python and Arduino Resources Python 2.7+ Flask 0.10.0: fl ask.pocoo.org Flask GitHub: github.com/mitsuhiko/fl ask A Twitter account Your favourite text editor QThe OAuth Settings page for your new Twitter app –you’ll fill in various bits as you go Once upon a time, Twitter was a favourite great, lightweight Python server framework to defi ne need to create an app for their service. Go to https:// API for developers looking to write a routes, create sessions and handle parameters dev.twitter.com and sign in. If you look in the top- quick weekend hack. Setup was quick, passed to and from Twitter. Flask is perfect for this right corner, you’ll see your Twitter profi le picture; documentation was plentiful and the project: as we’ll see shortly, defi ning routes and hover over it and a drop-down will appear. Select ‘My data available was vast. Fun times! But now things handling parameters is a piece of cake and saves us Apps’ and then click the ‘Create new app‘ button that are a little different. With the Twitter API v1.1, having to deal with a lot of the overheads that come appears on the next screen. Follow the instructions authentication is not quite as easy as it once was with frameworks such as Django. to set up the app. The information you enter here (though the documentation is still plentiful and isn’t important right now – at the end of the process the data vast). Twitter now requires adherence Install Flask you’ll see a fi eld titled ‘OAuth Settings’ with a table of to OAuth protocol v1 for all requests made to its 01 So, before we start writing code, let’s deal settings. We’ll need those in a little bit. API. It’s tempting to install a library and ignore all with the prerequisites. If you have pip on your system the complicated stuff – and entirely appropriate then installing is as simple as entering sudo pip Importing Python modules if you’re putting together a quick hack project. install flask into your terminal. If you’re into 03 These are the modules we’ll be using for However, understanding what’s going on in the building from source, you can head over to Flask’s this project. With the exception of Flask, all of the authentication process will help you along when it GitHub and grab the latest version there. modules should be included with all versions of comes to debugging problems you may encounter Python 2.7+. These modules are all used in the later if you have to start handling multiple users or Create a Twitter app process of generating, signing and encoding each requests. For this tutorial, we’ll be using Flask – a 02 Before we can do anything with Twitter, we request we send to and from Twitter in the auth 128 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Full code process. We need to be specifi c when importing avaliable online: sessions, requests etc from Flask, otherwise http://bit. flask.py (cont. on page 131) ly/1gH6W0S those aspects of the library won’t be included in the scope of the endpoints. import urllib, urllib2, collections, hmac, binascii, time, random Create global variables from hashlib import sha1 04 for Twitter keys from flask import Flask, redirect, request, session Back in step 2 we created an app with Twitter. #Twitter OAuth Variables Now we need those variables that we saw consumer_key = “Srq7dzdIDule3RxSz9Ew” in the OAuth Settings area of the ‘Create consumer_secret = “Bsurvs8Bk41VHjhLWzZCUHvZw9btpdxNZVgxjG8b0” my app’ page. Fill in the consumer_key and app = Flask(__name__) consumer_secret variables. For now, leave the oauth_secret and oauth_token variables as blank @app.route(‘/authenticate’) def authenticate(): strings – we’ll assign those later. session.clear() session[‘oauth_secret’] = ‘’ “With the requestParams = { “oauth_callback” : “http://192.168.1.5:5000/authorised”, “oauth_consumer_key” : consumer_key, Twitter API v1.1, “oauth_nonce” : str(random.randint(1, 999999999)), “oauth_signature_method” : “HMAC-SHA1”, “oauth_timestamp” : int(time.time()), authentication is “oauth_version” : “1.0” } not quite as easy theSig = sign_request(requestParams, “POST”, “https://api.twitter.com/oauth/ request_token”) as it once was” requestParams[“oauth_signature”] = theSig request = urllib2.Request(“https://api.twitter.com/oauth/request_token”, “”) Assigning a namespace request.add_header(“Authorization”, create_oauth_headers(requestParams)) 05 to the Flask app Now we create an instance of the Flask class. This try: httpResponse = urllib2.urlopen(request) variable will be how we access all of the classes and except urllib2.HTTPError, e: properties associated with the Flask package. The return e.read() __name__ property in this instance gives our Flask responseData = getParameters(httpResponse.read()) class an idea of the scope it’s operating in. session[‘oauth_token’] = responseData[‘oauth_token’] session[‘oauth_secret’] = responseData[‘oauth_token_secret’] Creating our fi rst route 06 In order to authorise our user, we need return redirect(“https://api.twitter.com/oauth/authorize?oauth_token=” + to create an endpoint that they can visit to trigger session[‘oauth_token’]) the process. This is where Flask comes in handy. @app.route(‘/authorised’) Using the routes decorator (@app.route()) we can def authorised(): determine paths that trigger functions or actions. if request.args.get(‘oauth_token’, ‘’) == session[‘oauth_token’]: The fi rst endpoint we want to create is @app. route(‘authenticate’). The function we declare verifyRequestParams = { immediately after the decorator will be executed “oauth_consumer_key” : consumer_key, “oauth_nonce” : str(random.randint(1, 999999999)), when that endpoint is called. Note that although “oauth_signature_method” : “HMAC-SHA1”, we’ve done so here, you do not need to name the “oauth_timestamp” : int(time.time()), “oauth_version” : “1.0”, function the same as the endpoint. Flask can “oauth_token” : session[‘oauth_token’] deliver static fi les as well as use template rendering } engines, but for the sake of brevity we’re going to use signVerification = sign_request(verifyRequestParams, “POST”, “https://api. return at the end of the function to deliver a simple twitter.com/oauth/access_token”) text response showing our progress. verifyRequestParams[“oauth_signature”] = signVerification Creating a session with Flask verifyRequest = urllib2.Request(“https://api.twitter.com/oauth/access_token”, 07 In order to maintain various properties “oauth_verifier=” + request.args.get(‘oauth_verifier’)) and values specifi c to multiple users, we’re going verifyRequest.add_header(“Authorization”, create_oauth_ to be storing our values in Flask sessions. A Flask session is almost exactly the same as using a Linux & Open Source Genius Guide 129 Developer guide cookie to store values, but with sessions we get a layer of cryptography to help keep user credentials “Flask is perfect for this project: secure. The values in our sessions behave just like a dictionary, so we can call all of the methods that defining routes and handling dictionaries have. For example, we can call session. clear() if we want to log a user out of a service or restart the authentication process. We need to make parameters is a piece of cake” sure that before we make any calls to our sessions, the fi rst part of our request body. This is made up prepare the values we’ll need to sign against and we’ve set our session key, otherwise we’ll get an of the method we’ll be using for the request and the then submit these to Twitter to receive a request error. In this example we’ve chosen to insert the key URL we’ll be submitting to. We then iterate through token. The oauth_callback fi eld won’t do anything just before app.run() so it will be set just before our the keys in the OrderedDict, concatenating and URL- right now, but it’s where our app will redirect after server is executed. Like most keys, you must keep encoding the key value pairs of the dictionary so we we obtain our fi rst access token. We want to point this key secret. have something like key=value. Finally, we check it to our route /authorised. On line 27 we pass whether or not we’re at the last key in the dictionary; the requestParams through to the sign_request Preparing to authorise our app if we aren’t then we’ll append a ‘&’ to the parameter method. We then add the signature returned to the 08 Now we need to start thinking about how string in preperation for the next one to follow. requestParameters that passed through to the we are going to submit data to Twitter. The OAuth Ultimately, we’ll have a string that will appear as signing function, as we’ll be using these to build our protocol expects every request to be signed with a GET&http%3A%2F%2Fapi.twitter.com%2Fetc%2F authorisation headers next. SHA1 hash so Twitter can check the integrity of the ?key1=value1&key2=value2&key3=value3. data it has received before allowing access to any Building the OAuth personal information. The signing key 13 authorisation headers In order to sign the request properly, we need to 10 Now we need to create a signing key Much like when we sign our requests, we need order the parameters of the request alphabetically, for our signature. As the code shows, it’s just to properly encode and format our authorisation URL-encode them, concatenate them and then sign our consumer_secret + oauth_secret with an headers, too. The main difference between against them each time. This is where the headache ampersand in the middle. However, you may sign_request() and create_oauth_headers() is of OAuth begins. Because we’ll be doing this for most notice that we don’t have an OAuth secret yet. that the signature base string (the string we build to every request, we’re going to write a convenience That’s okay: for this fi rst signing request we generate our SHA1 signature) is all of the variables function that does all of this for us: sign_request(). don’t need one. You may have noticed in our joined with ‘&’, whereas our auth headers are joined /authenticate endpoint that after we popped the with ‘,’ and prepended with the word ‘Auth’. Other Creating our signature session values, we reassigned session[‘oauth_ than that, the two serve the same purpose. Here 09 So, what’s going on in our sign_request() secret’] to “”. This way we won’t get a KeyError we’ve separated the two out to emphasise the function? First, we need to URL-encode the message when we try to sign our request. When importance of the difference – if either of these two base URL of the request, which we can do with we get our callback in our /authorised endpoint, functions returns a malformed string, the request urllib.quote(). Next, we create an ordered dictionary we’ll assign our session[‘oauth_secret’] for will fail. Using the urllib2 module, we can prepare the (OrderedDict) which sorts the parameters we’ve future requests. request we’ll submit. Notice the second parameter passed through to the function alphabetically of the request function on line 31. If we omit this according to the key value. This is crucial as each Hashing the key parameter, the request method will be GET; if it is request must be signed with all of the parameters 11 Finally, we’re ready to hash the key. Using included – regardless of the content – it will be a concatenated alphabetically in order to be valid. Python’s HMAC library, we pass our signing key, POST request. The parameter’s main function is to Next, we create the variable requestString to build our result string (the concatenation of our escaped assign the body property of a request; it’s a little base URL and method with our parameters string) clumsy, but it’s something we’ll need in a bit. and from Python’s hashlib library, the SHA1 hash function. This results in a binary buffer with our Making the fi rst request A little about signature – that’s great, but we can’t send a buffer to 14 Once we’ve gotten the prepared auth Twitter’s server. On the next line, we pass our hashed headers back, we add them to the HTTP request and oauth_nonce variable through to the binascii library to convert it then call the open() method and wait for a response. Most of the values included in the OAuth to a Base64 value. Now we have our signature which If all has gone well, we should receive a string in the headers and signing process are self- we’ll return to the function that called it. request response with our fi rst auth token and auth explanatory. However, one isn’t so obvious secret. To save us having to individually parse the in its purpose. OAuth_Nonce is a randomly Obtaining a request token string for properties, the function getParameters() generated value – often alphanumeric 12 With Twitter API v1.1, we can no longer just will split the string up and return a dictionary we can but as there is no defi ned convention, an request an access token to access user data. access. We’ll then store these values in our session integer between 1 and something large First, we need to obtain a request token and then for later. will serve its purpose. It exists so Twitter exchange that for our fi rst access token. We don’t can check that it’s not been sent the same need the user to interact with anything at this point, Redirecting the user request twice. so we can trigger this on the back-end and handle 15 Now we have our fi rst set of credentials, we it ourselves. In the requestParams dictionary, we can redirect our user to the Twitter authorisations 130 Linux & Open Source Genius Guide Linux & Open Source Genius Guide Full code avaliable online: http://bit. screen to let them grant access to their data. ly/1gH6W0S Because we haven’t sent anything to our client yet, headers(verifyRequestParams)) we can use Flask’s redirect() method to send our try: user to the Twitter authorisation view. This request httpResponse = urllib2.urlopen(verifyRequest) is one of the few that we can use URL parameters to except urllib2.HTTPError, e: return e.read() send tokens to Twitter. responseData = getParameters(httpResponse.read()) Handling access session[‘oauth_token’] = responseData[“oauth_token”] 16 If the user grants our application access, they will be redirected to the callback URL we return “Authorised “ + session[‘oauth_token’] defi ned in step 12. The redirection will include URL def getParameters(paramString): parameters that are our access token and a token verifi er string. The Twitter documents state that paramString = paramString.split(“&”) we should check the access token we received pDict = {} in the URL parameters is the same as the one we for parameter in paramString: obtained moments ago. If all has gone well, Our / parameter = parameter.split(“=”) authenticate endpoint will have stored our token in the session dictionary. If our stored token and pDict[parameter[0]] = parameter[1] the one Twitter returned matches, we can prepare return pDict to exchange our access token (which currently only has restricted privileges) for a fully fl edged access def sign_request(parameters, method, baseURL): token with read/write permissions (depending baseURL = urllib.quote(baseURL, ‘’) on the permissions you requested when you fi rst p = collections.OrderedDict(sorted(parameters.items(), key=lambda t: t[0])) created the app). requestString = method + “&” + baseURL + “&” Third (and fi nal) parameterString = “” 17 token exchange for idx, key in enumerate(p.keys()): So, same as before: we prepare our parameters paramString = key + “=” + urllib.quote(str(p[key]), ‘’) for submission with sign_request(), build our if idx < len(p.keys()) - 1: paramString += “&” auth headers and then fi re off our request. Just like when we fi rst retrieved our request token, this parameterString += paramString is something that we can do in the background result = requestString + urllib.quote(parameterString, ‘’) without user interaction. Providing everything has gone swimmingly, you will then be returned a fully signingKey = consumer_secret + “&” + session[‘oauth_secret’] authorised access token that you can start building print signingKey your own requests with. hashed = hmac.new(signingKey, result, sha1) signature = binascii.b2a_base64(hashed.digest())[:-1] Fire up your server 18 As you can see, we have ended our fl ask.py return signature fi le with: def create_oauth_headers(oauthParams): if __name__ == ‘__ main__’: oauthp = collections.OrderedDict(sorted(oauthParams.items(), key=lambda t: t[0])) app.secret_key = headerString = “OAuth “ ‘R4nDOMCRypt0gr4ph1cK3yf0R5355i0N’ app.run(host=’0.0.0.0’, debug=True) for idx, key in enumerate(oauthp): hString = key + “=\”” + urllib.quote(str(oauthp[key]), ‘’) + “\”” if idx < len(oauthp.keys()) - 1: If we didn’t include any parameters in the run hString += “,” function then our server would run and be accessible headerString += hString at 127.0.0.1: 5000, but it wouldn’t be accessible to external machines – that kind of defeats the point of return headerString having a server. By passing host=’0,0,0,0’, Flask will if __name__ == ‘__ main__’: listen on any IP address it can for potential requests with a default port of 5000. debug=True is an error app.secret_key = ‘R4nDOMCRypt0gr4ph1cK3yf0R5355i0N’ app.run(host=’0.0.0.0’, debug=True) message beautifi er that Flask includes. Rather than crashing our server, an error page will be delivered detailing what went wrong in place of our expected return. Linux & Open Source Genius Guide 131 Developer guide Create dynamic templates with Flask, Jinja2 and Twitter We create a dynamic webpage with Twitter and Flask’s rendering engine, Jinja2 Advisor Sean M Tracey is a creative technologist at a leading digital agency. He spends a lot of his time living inside of Node.js, Python and Arduino Resources Python 2.7+ Flask 0.10.0: lask.pocoo.org Flask GitHub: github.com/mitsuhiko/fl ask A Twitter account Your favourite text editor QThe template uses a loop to generate a list of Twitter tweets In the previous tutorial, we looked display tweets. Flask comes with the super-nifty Cut and paste them into a new fi le called helpers.py at using Python and Flask to handle Jinja2 templating engine, If you’re familiar with in the root of your project folder. At the top of this fi le the Twitter OAuth process and build Node.js or front-end JavaScript, the syntax will look we want to import some libraries. requests to obtain tokens. We’ve used very similar to the Handlebars rendering engine. Twitter in these tutorials because of the large But, before we dive into that, we need to rearrange import urllib, collections, hmac, binascii, amount of easily digestible data available. However, some of the code we wrote last time. time, random, string since Twitter adheres to the standards set out by OAuth 1.0, the code we’ve used to sign and build Rearranging our code from hashlib import sha1 requests can be modifi ed to work with any third- 01 Server code can get messy and party API using the same standard without a great unmaintainable quickly, so the fi rst thing we’re going Now we can head back over to server.py and import deal of work. to do is move our helper functions to another fi le and the helper functions back into our project. We do For years PHP has been a mainstay of template import them into our project, much like you would this by simply calling import helpers. Because generation, but now with well-documented a module. This way, it will be clear which functions Python is smart, It will look in the current directory frameworks such as Flask, Sinatra and Handlebars, are our server logic and endpoints and which are for a helpers.py fi le before it looks for a system the ability to use powerful scripting languages generic Python functions. Open your server.py fi le module. Now every function included in helpers.py greatly improves our ability to make great web (or whatever you decided to call it) that we wrote is accessible to our project. All we need to do to call services. To continue on from last time, we’re going in our last tutorial and locate the getParameters, them is prepend our the methods we called before to use Python, Flask and its templating engine to sign_request and create_oauth_headers functions. with helper.function_name and it will execute. 132 Linux & Open Source Genius Guide Linux & Open Source Genius Guide For sign_request, we’ll need to pass our {% if session[‘oauth_token’] != “” %} Fig 01 oauth_secret and consumer_secret for each request Hello, You’ve authenticated! We need to authenticateAlready Authorised
rather than accessing it from the session. Adjust the
Let’s get some tweetsAuthorisation required
Our fi rst template 03 Now that we have a rendering engine, we need to create templates for it to use. In the root of our project’s folder, create a new folder called templates. Whenever we try to render a template, Flask will look in this folder for the template specifi ed. To get to grips with templating, we’ll rewrite some of our authentication logic to use a template, rather than manually requesting endpoints. In templates, create an index.html fi le. You can treat this HTML fi le like any other – included in the resources for this tutorial is an index.html that includes all of the necessary head tags and declarations for this fi le.
Rendering our template 04 In server.py, let’s create a route for ‘/’ to handle the authorisation process.
@app.route(‘/’) def home():
if not ‘oauth_token’ in session: session.clear() session[‘oauth_secret’] = ‘’ session[‘oauth_token’] = ‘’
return render_template(‘index.html’)
It’s a simple function: all we want to do is check whether or not we have an oauth_token already
Linux & Open Source Genius Guide 133 Developer guide
and create those properties in the Flask session so we don’t throw an error if we try to access it “Now we know how to build erroneously. In order to send our generated template in response to the request, we return render_ templates, let’s grab some tweets template(‘index.html’). to display” Template variables 05 We can choose to send variables to our In server.py defi ne a new route, get-tweets: If we add the following function to our helpers. template with render_template(‘index.htm’, py fi le, we can quickly build one for each request. variableOne=value, variableTwo=Value) but @app.route(‘/get-tweets’) in this instance we don’t need to as each template @app.route(‘/get-tweets/
134 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
something to render, we’re ready to work through tweetRequestParams = { Fig 02 it, otherwise we’ll just print that we didn’t get “oauth_consumer_key” : consumer_key, anything back. “oauth_nonce” : helpers.nonce(32), Once again, any template logic that we want to “oauth_signature_method” : “HMAC-SHA1”, use to generate our page is included between {% “oauth_timestamp” : int(time.time()), %}. This time we’re creating a loop; inside the loop “oauth_version” : “1.0”, we’ll be able to access any property we have of “oauth_token” : session[‚Äòoauth_token’], that object and print it out. In this template we’re “user_id” : session[‘user_id’], going to create an
- options. {% for tweet in data %}
- Wrapping up 14 So, that is pretty much it for templating easy to build and deploy dynamic sites. Flask is agreat tool for any Python developer looking to runa web service. {{ tweet[‘text’]|forceescape }} Although we’ve used Twitter to demonstrateFlask’s power, all of the techniques described can be used with any third-party service or database {% endfor %} resource. Flask can work with other rendering
We didn’t get any tweets :(
With such great integration between Flask and {% endif %} Jinja2, it makes little sense to use another engine outside of very specifi c circumstances.Linux & Open Source Genius Guide 135 Developer guide
Django test web Using Django Executing the ‘python Displaying the SQL server running Admin to create a manage.py syncdb’ command commands that Django uses new blog post for the fi rst time for creating a database table
Build a blog with django Advisor Mihalis Tsoukalos is a UNIX system administrator Learn how to program and publish a Django also profi cient in programming, databases and mathematics. He photo blog site in easy steps has been using Linux since 1993
Django is a high-level Python web Django follows a slightly changed version of the Resources framework that encourages rapid MVC (model-view-controller) design pattern called Django: www.djangoproject.com development and clean, pragmatic model-template-view (MTV). The MTV handles the Django built-in template tags and design and allows you to build controller work by the core and all the other work web applications quickly. Django focuses on is done in models, templates and views. According fi lters: https://docs.djangoproject.com/en/dev/ref/ templates/builtins automating as much as possible and adhering to Django philosophy, what is truly important is to the DRY principle. Since Django is a collection not terminology but getting things done quickly. Django tutorials: www.mikesdjangotutorials. co.uk/ of libraries written in Python, in order to create a Join the Django community at https://www. site using it you basically write Python code that djangoproject.com/. SQLite: www.sqlite.org uses the Django libraries. If you already have a This tutorial will go through all the required steps Apache: www.webforefront.com/django/ good knowledge of Python, then you only have to needed for creating a simple photography blog, so setupapachewebserverwsgi.html understand how the Django methodology works. make yourself comfortable and enjoy the ride…
136 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Installing Django 01 On a Debian 7 system, you can install Django using the following command: # apt-get install python-django To make sure that everything works as expected, type the following Django command, which prints the version of your Django installation: django-admin version If your output prints something like 1.4.5, then everything is okay. Otherwise check your PATH settings or look for error messages. django-admin is the main Python script for Django administrative tasks. Run django-admin without any command-line parameters to display usage information and a list of all the available commands. Although at the time of writing this article the latest Django version is 1.6.1, Debian uses an older version.
The Django web server 02 Django has its own ‘lite’ web server that is used during development for testing the application so that you do not need to set up Apache just to test your Django site. The test web server allows one connection at a time and has no security. By default, it listens to localhost (its IP is 127.0.0.1) only. The default port number that the test server listens to is 8000. If you want to change the server port number, pass it as a command-line argument (python manage.py runserver 8580). If you want to change the IP address of the server, pass it along with the port (python manage.py runserver 192.168.1.100:8080). If you make changes to the project, you do not need to restart it; just hit reload on your web browser and it will restart itself. Instead of keeping a log fi le, it prints descriptive error messages and other information on screen that are very useful during development.
Creating your project and 03 your application the django-admin Python script. You will see many as a Python package; a fi le named settings.py, Execute the django-admin command with specifi c fi les ending with .pyc. These are byte code fi les which is the project’s confi guration fi le (a very parameters to start a new project. Each project can automatically created by the Python interpreter important fi le that you will need to edit multiple contain many applications, so the project itself is like a and are executed by the Python virtual machine. times); a fi le called urls.py that is the Master Table big container of different applications. The fi rst thing to do is add the application of Contents fi le for the project; and a fi le named You can create a new project, called LUD, by name to the list of the INSTALLED_APPS in the wsgi.py that contains a simple SGI application executing the following command: LUD/settings.py fi le. If you do not do this, you will used for deploying Django with WSGI app servers. $ django-admin startproject LUD not be able to use the application. You are going to need the wsgi.py fi le during If you want to create a new application called deployment phase. photoBlog, you have to go to the LUD project’s Directory structure and Inside the photoBlog directory there exists a directory that is automatically created and has 04 fi les of your project fi le called models.py where you create the data the same name as the project and execute the Django automatically generates certain fi les models of the application, a fi le named test.py following command: and directories when you create a new project that has the unit tests, and a fi le called views.py $ python manage.py startapp photoBlog or a new application. As far as the project that includes the functions that handle data and The manage.py script is automatically created goes, the LUD directory contains a fi le called various other things. There is also an __init__.py for every Django project and is a wrapper around __init__.py that makes Python treat the directory fi le that has the same purpose as before.
Linux & Open Source Genius Guide 137 Developer guide
Making changes to the 05 LUD/settings.py fi le As you already know, the fi rst thing to do after creating a new application is adding it to the list of INSTALLED_APPS. For the photoBlog app, you just have to add the following line: ‘photoBlog’, Next, add valid values to the ADMINS fi eld. Then, change the TIME_ZONE value to match your location. Later on in this article, you will learn about the other changes you need to make to it in order to support the Django administration interface. The next step will show you how to use a database with your Django project.
Connecting to your database 06 Django can communicate natively with PostgreSQL, MySQL, SQLite and Oracle, but you can also fi nd third-party drivers for other DBMSs. When a request for a URL page comes, Django uses a series of regular expressions to match the URL to one of the defi ned views. Django has also its own authentication system. “Django uses a series of regular For reasons of simplicity, your advisor is using an SQLite database, which is more than enough expressions to match the URL to one for medium-size projects. The main advantages of SQLite are that it does not need a server process to run and that it is a single fi le database, meaning of the defined views” that one single fi le has all necessary database data. application and try to make it as generic and self- system comes with built-in template tags and fi lters, If the database fi le (photoBlog.sqlite) does not suffi cient as possible. but you can also create your own. exist, it will be automatically generated the fi rst The templates used in this project will be located time you synchronise the database (using the Using models inside the templates directory and you can reference syncdb command). If you want to use PostgreSQL, 08 Using Django's ORM (object-relational them in Django code. You should add the following MySQL or Oracle, you should create a new database mapper) is one of the project goals. ORM allows all line – using the full path – inside the LUD/settings.py and a user with a password at this point. Python classes that were defi ned inside models.py to fi le to make Django recognise the directory with the Make the required changes and run the access the selected database without having to deal template fi les: python manage.py syncdb command. The with the database directly. /home/mtsouk/docs/article/working/djangoBlog. great thing is that you don’t need to write any SQL You will need to edit photoBlog/models.py LUD/code/LUD/templates commands to communicate with the database. using your favourite text editor to create a new When your server goes live and is moved to a class called photoBlogPost. Each photoBlogPost different hard disk location, you will need to make About the project class will contain a myID fi eld, a Title, a Body, the more changes, but this is suffi cient for development 07 You will be able to see our example Django Publication Date, the Number of Likes and a Photo purposes. site live at blog.highiso.net – your advisor’s that can either be a binary object or a link to an image You can have multiple template paths if needed. To photography blog. Having a live site to test a web fi le, depending on its type. Django prefers having make your app to use templates, you should add the framework such as Django gives you a great static resources served by your web server, so a fi le following two lines inside your photoBlog/views.py file: opportunity to learn new software better, faster and link will be used. Django automatically adds a fi eld from django.template.loader import get_ deeper. Your only two concerns should be security called id that will be the primary key of the table. template and having an online Linux server to host your site. After creating your models, you should run python from django.template import Context You should also have some system administration manage.py syncdb to tell Django to create the tables knowledge. inside the SQLite3 database fi le. About Django views Django sites are fast, easy to maintain and It is possible to change your models afterwards, 10 Django views are functions that describe contain only the necessary components – as but it is good to be as accurate as possible during how to display information on screen when you visit opposed to content management systems, which design time to avoid potential problems in the future. a URL. You defi ne different functions that describe are huge pieces of software. how different pages will be displayed. A view takes Each Django app you create is autonomous Dynamic templates an HTTP request and returns an HTTP response. and therefore can be easily used in other projects. 09 Django supports dynamic templates, which The fi rst argument of every view should always be an Keep that in mind when you are designing a new are a way of presenting data dynamically. Its template HTTP request.
138 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
You will have to edit fi le photoBlog/views.py MEDIA_ROOT = '/home/mtsouk/docs/article/ The admin interface to add views to your Django website. Every working/djangoBlog.LUD/code/LUD/photos/' 14 Django’s admin interface allows you to work view should correspond to a URL defi ned in the The MEDIA_URL variable is set to ‘uploaded_photos’ on your model data using a webpage instead of the LUD/urls.py fi le. so all images are stored inside directory …/code/ command line. LUD/photos/uploaded_photos. The fi rst thing to do is to alter the Project URLs Timestamps will be used for making the fi lename LUD/settings.py fi le. You should uncomment the line 11 When you use Django for the fi rst time, you unique (it is crucial that fi lenames are unique) using that says ‘django.contrib.admin’, which is the Django will have to defi ne many URL patterns. The more the following Python module: module that allows us to use the admin interface. you use Django, the more you are going to reuse from time import time Next, you should tell Django about the URL of the previously defi ned patterns. A new method – called get_fi le_name(…) – admin interface, which will happen inside the LUD/ URL patterns are processed in order; as soon as a generates a new unique fi lename that is used by urls.py fi le. By just confi guring a URL, you can use match is found, it stops any additional matching. The the Django system. The unique fi lename is then the admin interface application! A good technique idea of this is to put specifi c expressions before any stored inside the database as a string. Django’s is making the URL to the admin interface a kind generic expressions. ImageField does the rest of the work for you! of ‘second-level security’ by naming it something URLs are defi ned inside the fi le LUD/urls.py. All fi les are stored locally. Later on, when your diffi cult to guess. You can defi ne additional URLs elsewhere, but production web server is serving the images, you The last task to do is creating a fi le named admin. everything starts at LUD/urls.py. will have to create an Apache alias that matches py inside the directory of your app (photoBlog). This URLs usually use regular expressions to match the string stored in the database. fi le will have these contents: a URL given by the user on a web browser. The from django.contrib import admin following line will match a URL that has the word Creating the views for the project from photoBlog.models import photoBlogPost ‘welcome’ after the address and execute the 13 The welcome function now uses dynamic admin.site.register(photoBlogPost) function welcome, located inside Python module templates, so it is renamed to welcome_template. photoBlog, and its Python code which is in the Renaming the function to include the word fi le views.py: ‘template’ is not necessary, but it helps us to “A veiw takes an url(r'^welcome/$', 'photoBlog.views.welcome'), remember how it is implemented. A function named about_template will be created, Uploading and loading photos which will contain static HTML code and will also use HTTP request and 12 The fi rst thing to do is declare the full dynamic templates. Another function named list_ path of the directory that will hold user-uploaded template is also implemented. This function reads returns an HTTP fi les using MEDIA_ROOT. This will happen inside all the available blog posts from the database and LUD/settings.py. displays them on screen as a list. response”
Linux & Open Source Genius Guide 139 Developer guide
The beta version of the website! screen, write it to a fi le or to a network socket. count = a.nLikes 15 What is currently missing and is going to By default, the Django logging configuration emails count += 1 be added is a navigation menu for the website. all ERROR or CRITICAL messages as long as the DEBUG a.nLikes = count First, you need to add the following in the value is set to False, which is not happening during a.save() LUD/settings.py fi le: development. When DEBUG is True, all messages # For Supporting the Navigation Menu caught by the logger are displayed on screen. return HttpResponseRedirect ('/n o d e / %s/' % TEMPLATE_CONTEXT_PROCESSORS = ( myID) 'django.contrib.auth.context_processors.auth', Implementing the Number The like_blogPost method updates an SQLite3 'django.core.context_processors.debug', 17 of Likes feature database record simply and elegantly. Last, you 'django.core.context_processors.i18n', First , add the following line to LUD/urls: should add the following to the templates/display_ 'django.core.context_processors.request', url(r'^node/like/(?P
140 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Setting up Apache 18 The primary deployment platform for Django is WSGI, which is the Python standard for web servers and applications; therefore you are going to need to have mod_wsgi enabled on your web server. Run apache2ctl -t -D DUMP_MODULES as root to see if it is already installed and enabled. You will need to set up the DNS record for the website domain. You adcvisor did this using the web interface of Linode. The name of our example site is http://blog.highiso.net/. The confi guration for creating the Apache virtual host for the website is as in Fig 02. After successfully setting up Apache and WSGI, the Django site will no longer be in development mode, so do not forget to change the ‘DEBUG = True’ setting in LUD/settings.py to ‘DEBUG = False’. You will also need to revise the directory paths defi ned inside LUD/settings.py if you are going to move the development Django site to a new hard disk location. It is very important to change the ALLOWED_HOSTS value to the following for supporting connections from every internet host: ALLOWED_HOSTS = ['*'] For the admin site to work properly on the production server, you will need to run the following command after setting STATIC_ROOT and STATIC_URL: # python manage.py collectstatic Last, you need to use the full path of the SQLite3 database inside fi le LUD/settings.py. If everything is working with Apache and WSGI, you will see log messages similar to Fig 03 inside /var/log/apache2/error.log. This part of the tutorial cannot cover every possible diffi culty that may come up, because even small differences in program versions may create small problems – you will have to observe the log fi les and “The primary deployment platform for correct the errors that appear. Django is WSGI – the Python standard Examining database 19 tables and data for web servers and applications” It is very interesting to look at the tables of the SQLite3 database. The site is based on 11 tables. The presented Django application allows you to If you want to enhance the site, you can The most important table is called photoBlog_ quickly create and publish a photo blog from scratch, implement keywords, user comments and search photoblogpost. You can easily understand that in less than two hours, provided that you are familiar functionality. You should also try to implement the fi rst part of its name is the name of the app with Django and Python. It needs some CSS code, but Django forms. (photoBlog) and the second part is the name of it is fully functional. When trying new things, always run the Django the model in lower-case letters (photoblogpost) as web server to test your changes or fi nd errors in defi ned inside the photoBlog/models.py fi le. Other things to try your code. You can always check the live version of 21 when using Django the example Django blog at blog.highiso.net and Last words If you want to see the SQL code used for creating see how it progresses over time. Head over the the 20 You should defi nitely study the project code, one of your database tables, use the following Django website for answers to any questions. run it using the Django test web server and try to make Django command: Getting used to using Django will take some time, small changes to the project. $ python manage.py sql photoBlog but it will defi nitely be time well spent!
Linux & Open Source Genius Guide 141 Developer guide
Create a real-time CPU meter with Node.js and Duino Advisor Get a Node.js-powered Arduino to Sean M Tracey is a display the current processor usage of a creative technologist at a leading digital agency. He spends a lot of his time living computer in real-time inside of Node.js, Python and Arduino In this tutorial we’ll be using Node.js and From the command line execute the following: the Duino library to get real-time CPU data Resources and display it with a series of LEDs. We’ll npm install duino Node.js v0.10.17: also look into adjusting the brightness of nodejs.org those LEDs with a potentiometer, and running the app This will create a folder called ‘node_modules’, which NPM v1.3.8: as a background process with Forever.js. will include the Duino software and all of its listed www.npmjs.org/doc/cli/npm-install.html The normal process of programming an Arduino dependencies. is you write a program (or sketch, as it’s called by We’re now ready to start writing the program for Duino v0.0.9: github.com/ecto/duino enthusiasts), compile it, upload to your microcontroller the Arduino. and then let it run. If you don’t get the result you’re Forever.js v0.10.11: expecting, you amend the sketch, recompile and Prepare the app github.com/nodejitsu/forever re-upload it. Though robust, this method is a tad 02 Using your favourite text editor, create a new Arduino Uno/Duemilanove cumbersome and long – and that’s before we even get fi le ‘app.js’ and insert the following: 5 LEDs into topics like wiring up shields, relays and extra kit. Potentiometer This is where Duino comes in. Rather than going var arduino = require(‘duino’), through all of that, we upload a sketch that allows board = new arduino.Board({debug : false}), 13 prototyping wires the Arduino to become a slave, of sorts, to another spawn = require(‘child_process’).exec, Your favourite text editor computer – an endpoint for data and instructions. pins = [3, 5, 6, 9, 10], Arduino IDE 1.0.5 This is done through a serial port connection and, in //These pin numbers may vary across this instance, a Node.js library. The benefi ts of this different versions of Arduino approach are numerous, but primary among them brightness = 255; is that we can use the calculating ability of a far more powerful computer to crunch problems and send only Here we’re including the Duino library and using the result to the Arduino to enact a function or effect. it to create an instance of Board – the function that provides the methods for interacting with our Prepare the project Arduino. Because Node.js doesn’t have easy/direct 01 The fi rst thing we need to do to get this ball access to our system’s CPU details, our app will rolling is install Duino.js so that we can upload its use the Node.js child_process module to execute software to our Arduino in order for the two to be able the shell programs vmstat and tail to get access to communicate. Let’s start by cloning the GIT repo of to information about user CPU usage. This will be Duino to our computer. covered in detail in a few steps. The pins array contains the numbers of the pins git clone https://github.com/ecto/duino.git we’ll be using in this sketch. If you look at your Arduino, you’ll notice that all of these pins are You should now have a folder called ‘duino’ on capable of reading and writing both digital (HIGH- your system. Open the directory and fi nd the ‘src’ LOW) and analog (0-255) data. folder. Found in this folder is a fi le called du.ino, A little later we’ll be adjusting the brightness of which is the software that will let Duino talk to our our LEDs by adjusting a potentiometer. In order to microcontroller. Open this sketch in the Arduino IDE affect this change across the LEDs, we need to be and click ‘Upload’. The sketch will upload to your able to control the amount of power that goes to Arduino, so that we’re now ready to start talking to it. them, so we’ll be writing analog values with PWM On your system, create a new project folder. You (which is what the ~ symbolises on the Arduino can name it anything you like, and then enter it. board) to refl ect the potentiometer’s value.
142 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
How does it Full code avaliable online: all work? http://bit. ly/1gH6W0S The Duino software is basically an abstracted version of Firmata, with an API for Node.js. At its core, the Arduino becomes an endpoint, always listening for instructions from another machine on what to read or write through the Arduino’s serial port. The beauty of serial data is that, if you wanted, you could go away and write your own library in another language – C++, Python, Ruby, etc – and so long as they send the commands the Arduino is expecting, each language will be able to control it with minimum effort.
Wire the Arduino 03 Before we proceed any further with coding, now would be a good time to wire up and then plug in the Arduinos. In the diagram included, each of our LEDs is powered by its own pin, but are all connected to the ground pin via the ground bus on the breadboard. The potentiometer is not connected to our digital pins, but to one of the free analog pins, it doesn’t matter which one, but this tutorial uses A0 as it’s the fi rst free pin. In order to get a reasonably stable reading from the potentiometer, you must connect it to the 5V power pin. This is done through the power bus or directly, as shown in the diagram. Once all of that is connected, you can plug in your Arduino. Nothing will happen right now as we’re yet to start controlling it.
Get the CPU data with 04 Node.js child processes Before we can start writing information to the board, we need to have information to send it, so let’s get the CPU data. Earlier, it was mentioned that Node. QTo get a stable reading from the potentiometer, it must be connected to the 5V power pin js doesn’t have direct access to CPU information. Depending on the version/fl avour of Linux you’re }, 500); running, the following may vary slightly. This tutorial setInterval(function(){ was tested in Debian 7.2 but if your system runs Using the Node.js child_process module, vmstat and tail, it should be fi ne. If we were to check spawn(“echo $[100-$(vmstat|tail -1|awk we can dispatch Shell commands (among the CPU usage at every possible instance that we ‘{print $15}’)]”, function(error, stdout, other things) and have the standard output could, our program would begin to tax the CPU, which stderr){ passed to callback function. Because we’ve would affect the readings. Irony abounds! If we read used a little command-line magic to shorten the CPU usage values every half-second or so, we’ll var percentage = parseInt (stdout); the output of vmstat, what will be passed get a useful value and a reasonably good measure of displayUsage(percentage); through to the callback in this instance is what’s happening inside the box without causing a the current user CPU usage value expressed measurable impact ourselves. We’ll achieve this by }); as a percentage out of 100. Perfect. So let’s wrapping the following code in a setInterval(). do something with that.
Linux & Open Source Genius Guide 143 Developer guide
Display CPU usage an integer between zero and four. Why is If all fi ve LEDs are lit, we’re running at 80 - 100 05 Let’s create the function displayUsage() this useful? If you look to the circuit you’ve per cent, three LEDs and it’s somewhere between that we just passed the percentage to outside wired, you’ll see that we have fi ve LEDs. This 60 to 80 per cent. Now that we understand what of our interval loop. This part of our process isn’t is a deliberate choice; wiring ten LEDs and ledCount does, let’s use it. particularly time sensitive. controlling them can become complicated and slow in short order, not to mention that we don’t Alternative ways of function displayUsage(percentage){ have ten pins capable of sending PWM values 06 controlling LEDs on the Uno. The formula given will take the Add to displayUsage() the following... var ledCount = ((percentage / 100) * 100) percentage expressed by vmstat and calculate / (10 * (10 / pins.length)); exactly how many LEDs should be lit given function displayUsage(percentage){ } that percentage. In this instance, each LED Right now, all display usage will do is calculate symbolises a 20 per cent block of CPU usage. var ledCount = ((percentage / 100) * 100) / (10 * (10 / pins.length));
for(var z = 0; z < pins.length; z ++){ if(z < ledCount){ board.analogWrite(pins[z], brightness); } else { board.analogWrite(pins[z], 0); }
}
}
Remember the array of pin numbers we created earlier on? Here we’re looping through them and checking if the pin selected from the array represents a value low enough that it can be turned on. We can do this in a variety of ways. If, for example, we only wanted to toggle the pins on and off then we could do something like this…
var leds = [], pins = [3, 5, 6, 9, 10];
for(var x = 0; x < pins.length; x += 1){
leds.push( new arduino.Led({ board: board, pin: pins[x] }) );
}
function displayUsage(percentage){
var ledCount = ((percentage / 100) * 100) / (10 * (10 / pins.length));
for(var z = 0; z < pins.length; z ++){
if(z < ledCount){ QNode aims to offer users an easy way to build scalable network programs leds[z].on();
144 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
} else { leds[z].off(); “If we were to check the CPU } usage every possible instance } that we could, our program would }
...But those methods, while really useful, don’t begin to tax the CPU, which would allow us to set the values being sent. They are HIGH or LOW because we want to be able affect our readings” to adjust the brightness of our LEDs based on user input, so we’re going to address the of the dial. With a little bit of maths, we can board directly with board.analogWrite(pins[z], turn that value into a percentage of 255. If we brightness). The analogWrite method lets set the ‘brightness’ as we have done below, it More benefits us send an analog value between 0 and 255, will now turn the brightness of your LEDs up and The key benefi t of using something like 0 being off (or the analog equivalent of down as we turn it. Duino is that you can offl oad a lot of the LOW) and 255 being on – full power (or, the analog problem crunching to a beefi er machine. equivalent of HIGH). potentiometer.on(‘read’, function (err, Perhaps even more useful is the ability value) { for the controlling computer to navigate Adjust the brightness value = (value / 1024) * 100; the internet on behalf of the Arduino. 07 If you run app.js with brightness = Math.round((value / 100) * Arduino was never actually designed 255); with the internet in mind, by letting node app.js our computer deal with navigating the }); network and sending the result we in the command line, assuming everything don’t have to worry about shields, the is wired correctly, you will now have a fully Run it Forever... .js amount of bytes available or dropping functioning CPU meter. To try it out, why not 08 And that’s our real-time CPU app. We’ve connections. It just works. open a couple of taxing programs at the same managed to quickly put together something time to see how it affects the measure? As pretty funky to use with our system but more of the CPU time is used, the number right now we’re running app.js manually in the CLI. info: Forever processes running of LEDs lit up should increase, and we could CLI is awesome, but what good is a CPU meter that data: uid command leave it there. But, that does mean that we’ve needs a window open to run? This is where Forever. script forever pid logfile only covered writing values. If we want to read js comes in. uptime values, how do we do that? Forever is, at its core, a daemon that lets your data: [0] fwGv /usr/local/bin/ node Head back over to your project fi le. Immediately Node.js scripts run ‘forever’ in the background. app.js 96489 96490 /Users/ seantracey/. after the variables where we set the board Forever will try its best to keep your Node.js app forever/fwGv.log 0:0:1:13.707 information, create a new variable called running – it even handles app crashes and restarts. ‘potentiometer’. This will be the object that Installation is simple: If you want to stop a process, you can enter forever we use to listen for events and access data stop [PROCESS INDEX]. from the potentiometer. $ npm install forever -g Brief recount var potentiometer = new arduino.Sensor({ Notice the ‘-g’ fl ag at the end. This is a global install, 09 Voila! So concludes our whirlwind tour board: board, meaning that if any Node.js app needed to call of Arduino and Node.js. We’ve covered a lot pin: ‘A0’ forever on this system, it wouldn’t need to be saved in a short space of time. We now know how to }); in the node_modules folder. After that’s completed read information from an analog source (our installing we can now enter... potentiometer), writing analog values to various You will notice that when we create the pins, the difference between digital and analog arduino.Sensor object, we pass through an options forever start [PATH TO OUR NODE.JS + DUINO signals, why we’ve used analog signals for this object with a ‘board and ‘pin’ key. Feasibly, this FILE] tutorial, executing child processes within a Node.js means we can control more than one Arduino at a app to enhance our Node’s abilities and making an time, but we’re not going to look at that this time. ...and the CPU meter is now a background process. app run ‘forever’. Pin, of course, is the analog pin we’ll read data from. If you want to check the apps you have running with This should be enough to get you excited about As well as an initiator, ‘potentiometer’ is an forever, you can enter forever list and what Arduinos can do; now it’s time to start event emitter. If we listen for a ‘read’ event on you’ll get an output that looks something like this: going out and making awesome bits and bobs – ‘potentiometer’, we’ll get a value somewhere you can do so much with just a few wires and a between 0 and 1024, depending on the position microcontroller!
Linux & Open Source Genius Guide 145 Developer guide
Build tic-tac-toe with Kivy Ease into the workings of Kivy by creating this pen-and-paper classic in just over 100 lines of Python...
point for the creation of our GUI. We can start by making Advisor the simplest Kivy app possible: Alexander Taylor is a graduate student in Physics, but with a little time for fun with from kivy.app import App Kivy on the side. He makes Kivy screencasts and tutorials at http://inclem.net class TicTacToeApp(App): pass
Kivy is a highly cross-platform graphical if __name__ == “__main__”: framework for Python, designed for the TicTacToeApp().run() creation of innovative user interfaces like multitouch apps. Its applications can run You can already run this, your app will start up and you’ll not only on the traditional desktop platforms of Linux, get a plain black window. Exciting! OS X and Windows, but also Android and iOS, plus We can build our own GUI out of Kivy widgets. devices like the Raspberry Pi. Each is a simple graphics element with some specific QThe classic ‘Hello World!’ in Kivy GUI form, That means you can develop cross-platform behaviour of its own ranging from standard GUI using the built-in Label widget apps using Python libraries such as Requests, functionality (eg the Button, Label or TextInput), to SQLAlchemy or even NumPy. You can even access those that impose positioning on their child widgets the root widget by adding a new method to the ‘App’: native mobile APIs straight from Python using some (eg the BoxLayout, FloatLayout or GridLayout), to those of Kivy’s sister projects. Another great feature is the abstracting a more involved task like interacting with def build(self): Cython-optimised OpenGL graphics pipeline, allowing hardware (eg the FileChooser, Camera or VideoPlayer). return Label(text=’Hello World!’, advanced GPU effects even though the basic Python Most importantly, Kivy’s widgets are designed to be font_size=100, API is very simple. easily combined - rather than including a widget for color=0, 1, 0, 1)) # (r, g, b, a) Kivy is a set of Python/Cython modules that every need imaginable, widgets are kept simple but are can easily be installed via pip, but you’ll need a easy to join to invent new interfaces. We’ll see some of The ‘build’ method is called when the ‘App’ is run, and few dependencies. It uses Pygame as a rendering that in this tutorial. whatever widget is returned automatically becomes backend (though its API is not exposed), Cython for Since ‘Hello World!’ is basically compulsory in any the root widget of that App’. In our case that’s a Label, compilation of the speedy graphics compiler internals, programming tutorial, let’s get it over with by using a and we’ve set several properties - the ‘text’, ‘font_size’ and GStreamer for multimedia. These should all be simple ‘Label’ widget to display the text: and ‘color’. All widgets have different properties available through your distro’s repositories, or via pip controlling aspects of their behaviour, which can be where applicable. from kivy.uix.label import Label dynamically updated to alter their appearance later, With these dependencies satisfi ed, you should though here we set them just once upon instantiation. be able install Kivy with the normal pip incantation. We’ll display the ‘Label’ by returning it as our app’s root Note that these properties are not just Python The current version is 1.8.0, and the same codebase widget. Every app has a single root widget, the top level attributes but instead Kivy properties. These are supports both python2 and python3. The code in this of its widget tree, and it will automatically be sized to fill accessed like normal attributes but provide extra tutorial is also version-agnostic, running in python2.7 the window. functionality by hooking into Kivy’s event system. We’ll and python3.3. We’ll see later how to construct a full GUI by adding see examples of creating properties shortly, and you more widgets for this one, but for now it’s enough to set should do the same if you want to use your variables pip install kivy with Kivy’s event or binding functionality. If you have any problems with pip, you can use easy_ That’s all you need to show some simple text, so run instal via easy_install kivy. “Applications the program again to check that this does work. You can experiment with the parameters if it’s unclear what any There are also packages or repositories available for of them are doing. several popular distros. You can find more information can also run on on Kivy’s website. Our own widget: tic-tac-toe A kivy application is started by instantiating and Android, iOS, plus 01 Since Kivy doesn’t have a tic-tac-toe widget, running an ‘App’ class. This is what initialises our pp’s we’ll have to make our own! It’s natural to create a new window, interfaces with the OS, and provides an entry Raspberry Pi” widget class to contain this behaviour:
146 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
“Kivy comes with all the tools needed to use kv language”
these events to call our own functions when a user presses the button, and can set the button’s ‘text’ property to display the ‘X’ or ‘O’. We also created a new Kivy property for our widget, ‘coords’ – we’ll show how this is useful later on. It’s almost identical to making a normal Python attribute by writing ‘self.coords = [0, 0]’ in ‘GridEntry.__init__’. As with the ‘TicTacToeGrid’, we’ll style our new class with kv language, but this time we get to see a more interesting feature.
As before, this syntax defines a rule for how a ‘GridEntry’ widget should be constructed, this time setting the ‘font_size’ property that controls the size of the text in the button’s label. The extra magic is that kv language automatically detects that we’ve referenced the Button’s own height and will create a binding to QA tic-tac-toe grid now accepting input, adding a O or X alternately update this relationship – when a ‘GridEntry’ widget’s height changes, its ‘font_size’ will change so the text name based on our App class. That is, we should place fits perfectly. We could have made these bindings the following in a file named ‘tictactoe.kv’: straight from Python (another usage of the ‘bind’ from kivy.uix.gridlayout import GridLayout method used later on), but that’s rarely as convenient class TicTacToeGrid(GridLayout):
Linux & Open Source Genius Guide 147 Developer guide
Now all we have to do is replace our root widget (returned from ‘App.build’) with a ‘TicTacToeGrid’ and we can see what our app looks like. def build(self): return TicTacToeGrid() # Replaces the previous label
With this complete you can run your main Python file again and enjoy your new program. All being well, the single Label is replaced by a grid of nine buttons, each of which you can click (it will automatically change colour) and release (you’ll see the printed output information from our binding). We could customise the appearance by modifying other properties of the Button, but for now we’ll leave them as they are.
Has anyone won yet? 02 We’ll want to keep track of the state of the board to check if anyone has won, which we can do with a couple more Kivy properties: from kivy.properties import (ListProperty, NumericProperty) class TicTacToeGrid(GridLayout): status = ListProperty([0, 0, 0, 0, 0, 0, 0, 0, 0]) current_player = NumericProperty(1)
This adds an internal status list representing who has QThe game with final additions, making the grid square and extending the interface played where, and a number to represent the current player (1 for ‘O’, -1 for ‘X’). By placing these numbers in it is updated, doing something special if a player has elif 0 not in self.status: # Grid full our status list, we’ll know if somebody wins because filled a column, row or diagonal. print(‘Draw!’) the sum of a row, column or diagonal will be +-3. Now we can update our graphical grid when a move is def on_status(self, instance, new_value): This covers the basic detection of a won or drawn played (Fig. 02). status = new_value board, but it only prints the result to stdout. At this You can run your app again to see exactly what stage we probably want to reset the board so that the this did, and you’ll fi nd that clicking each button # Sum each row, column and diagonal. players can try again, along with displaying a graphical now places an ‘O’ or ‘X’ as well as a coloured # Could be shorter, but let’s be extra indicator of the result (Fig. 03). background depending on whose turn it is to # clear what’s going on Finally, we can modify the `on_status` method play. Not only that, but you can only play one sums = [sum(status[0:3]), # rows to both reset the board and display the winner move in each button thanks to our status sum(status[3:6]), in a ‘ModalView’ widget. array keeping track of existing moves. sum(status[6:9]), This is enough to play the game but there’s one vital sum(status[0::3]), # columns from kivy.uix.modalview import ModalView element missing... a big pop-up telling you when you’ve sum(status[1::3]), won! Before we can do that, we need to add some code sum(status[2::3]), This is a pop-up widget that draws itself on top of to check if the game is over. sum(status[::4]), # diagonals everything else rather than as part of the normal Kivy properties have another useful feature sum(status[2:-2:2])] widget tree. It also automatically closes when the user here, whenever they change they automatically clicks or taps outside it. call an ‘on_propertyname’ method if it exists # Sums can only be +-3 if one player and dispatch a corresponding event in Kivy’s # filled the whole line winner = None event system. That makes it very easy to write if 3 in sums: if -3 in sums: code that will run when a property changes, print(‘Os win!’) winner = ‘Xs win!’ both in Python and kv language. In our case elif -3 in sums: elif 3 in sums: we can use it to check the status list every time print(‘Xs win!’) winner = ‘Os win!’
148 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
elif 0 not in self.status: winner = ‘D ra w...n o b o d y w in s!’ “Try swapping out the different widget types if winner: to see how other widgets behave” popup = ModalView(size_hint=0.75, 0.5)) victory_label = Label(text=winner, class TicTacToeGrid(GridLayout): Fig 01 font_size=50) def __init__(self, *args, **kwargs): popup.add_widget(victory_label) super(TicTacToeGrid, self).__init__(*args, **kwargs) popup.bind(on_dismiss=self.reset) for row in range(3): popup.open() for column in range(3): grid_entry = GridEntry( coords=(row, column)) Full program This mostly uses the same ideas we already covered, avaliable online: grid_entry.bind(on_release=self.button_pressed) adding the ‘Label’ widget to the ‘ModalView’ then http://bit. letting the ‘ModalView’ take care of drawing itself self.add_widget(grid_entry) ly/1gH6W0S and its children on top of everything else. We also use another binding; this time to ‘on_dismiss’, which def button_pressed(self, instance): is an event dispatched by the ‘ModalView’ when it is # Print output just to see what’s going on closed. Finally, we made use of the ‘size_hint’ property print(‘{} button clicked!’.format(instance.coords)) common to all widgets, which in this case is used to set the ‘ModalView’ size proportional to the window – while a ‘ModalView’ is open you can resize the window to see it dynamically resize, always maintaining these def button_pressed(self, button): Fig 02 proportions. This is another trick made possible by # Create player symbol and colour lookups a binding with the ‘size_hint’ Kivy property, this time player = {1: ‘O’, -1: ‘X’} managed internally by Kivy. colours = {1: (1, 0, 0, 1), -1: (0, 1, 0, 1)} # (r, g, b, a) That’s it, a finished program! We can now not only play tic-tac-toe, but our program automatically tells row, column = button.coords # The pressed button is automatically us when somebody has won, and resets the board so # passed as an argument we can play again. Simply run your program and enjoy hours of fun! # Convert 2D grid coordinates to 1D status index status_index = 3*row + column Time to experiment already_played = self.status[status_index] 03 This has been a quick tour through some of Kivy’s features, but hopefully it demonstrates how to # If nobody has played here yet, make a new move think about building a Kivy application. Our programs if not already_played: are built from individual Kivy widgets, interacting by self.status[status_index] = self.current_player having Python code run when their properties change button.text = {1: ‘O’, -1: ‘X’}[self.current_player] (eg our ‘on_status’ method) or when they dispatch button.background_color = colours[self.current_player] events (eg ‘Button’ ‘on_release’). We also briefly saw self.current_player *= -1 # Switch current player kv language and experienced how it can automatically create bindings between properties. Find the full program at the above URL, which you can reference to check you’ve followed everything correctly. We’ve also added an extra widget, the # Note the *args parameter! It’s important later when we make a binding Fig 03 ‘Interface’, with a structure coded entirely in kv # to reset, which automatically passes an argument that we don’t care about language that demonstrates how to add child widgets def reset(self, *args): this way. You can test it by uncommenting the ‘return self.status = [0 for _ in range(9)] Interface()’ line in ‘TicTacToeGrid.build’. It doesn’t do anything fundamentally different to what we # self.children is a list containing all child widgets already covered, but it does make extensive use of for child in self.children: kv language’s binding ability to automatically update child.text = ‘’ a label showing the current player, and to resize the child.background_color = (1, 1, 1, 1) TicTacToeGrid so that it is always square to fit within its parent. Play with all these settings to see exactly how self.current_player = 1 it fits together, or try swapping out the different widget types to see how other widgets behave.
Linux & Open Source Genius Guide 149 Developer guide
Get started with Installing Gulp Install Node 01 Before you can begin exploring Gulp.js, Gulp.js you will fi rst need to satisfy the most important requirement, which is to install Node. Simply head Explore the many benefi ts of using the over to nodejs.org and download and install the application onto your development machine. This is all Gulp.js task runner in your web projects a very quick and painless process, which will take no to help streamline your workfl ow and time at all. optimise your code Install Gulp 02 With Node installed, you now have access to the Node Package Manager (npm), which we will use to install Gulp as a global Install package onto our system. With your Terminal 01 Node or command prompt window open, use the git repository (github.com/gulpjs/gulp) to install Gulp on your system.
Save as dependency 03 Now you will need to create a ‘package. json’ fi le within your project directory. A skeleton fi le can be created for you using the npm init command from the Terminal. Once created, you will need to install Gulp as a dev dependency for your project, which will update the package.json fi le for you.
Ready to gulp 04 After the Node Package Manager has obtained and installed Gulp as a dev dependency, the reference to the dependency is also automatically added to the package.json fi le as a requirement. Any user or development machine with shared access The chances are quite high that during can then iterate over your directories and combine to this json fi le can install the same dependencies any one web project, you have dealt preselected fi les into one larger fi le, minifying it in quickly and easily. with or written more than one CSS or the process, while still keeping an unminifi ed version JavaScript fi le. These have network as well. overheads when we end up delivering them to the The additional benefi t of a task runner is “Using a task end user and can cause latency and loading issues that it will watch all of your directories and fi les if they are too large. They may also not be as well and only run selected tasks when a fi le or optimised as they could be, perhaps leading to structure has changed. This removes the runner you console errors or slow performance in general. necessity to manually run any task each time Enter Gulp.js, a free open source JavaScript an update has been made. can make the task runner powered by Node.js, which has quickly Let’s also add in the ability to run code gained traction in the web development community validation and tests against your code, most of CSS as a fast, powerful alternative to existing task automatically run as you save the fi le and runners on the market. But what are the benefi ts of now you should start to see the real power preprocessors creating and implementing a task runner tool into behind implementing a task runner procedure into your workfl ow? your web projects. Using a task runner you can make the most of CSS If you have some prior exposure to JavaScript, you to streamline preprocessors such as SASS or LESS to streamline can get started with Gulp.js in a matter of moments how you write stylesheets, as well as coffeescript for and add it into your projects quickly. Let’s have a look how you write your JavaScript fi les. The task runner can compile at how easy it is to work with and create your own these into the desired formats for fi nal delivery. It task defi nitions. stylesheets”
150 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Install and maintain dependencies
Install chosen plug-ins 05 As with any Gulp project, you will fi rst be required to install your plug-ins as dev dependencies to have them automatically added to the ‘package.json’ fi le. You are also able to install multiple plug-ins in one command – just make sure to use the --save-dev command. Install the gulp-load-plugins plug-in as well, which will help manage the references when we code. QTasks should be written in a clean manner, so that code is easy to maintain and manage npm install gulp-concat gulp-jshint gulp- bump gulp-load-plugins--save-dev “Thanks to the high engagement Reading package contents 06 The top of Gulpfi le.js requires the core level of the community, more Gulp library as usual. Directly below this, instead of calling in each plug-in individually we can load them dynamically by using just the gulp-load- plug-ins are being added on a plugins library. This will read the dependencies in the package.json fi le and inject each of them for us. continuous basis” var gulp = require(‘gulp’); page at github.com/jackfranklin/gulp-load-plugins. scope: [“devDependencies”], var gulpLoadPlugins = require(‘gulp-load- replaceString: “gulp-”, plugins’); gulpLoadPlugins({ camelize: true, var plugins = gulpLoadPlugins(); pattern: “gulp-*”, lazy: true, config: “package.js o n”, }); Adjust references 07 The dynamically loaded plug-ins have been assigned and required into the project Set up a project via the gulp-load-plugins extension. They will now be available for use in your Gulpfi le.js by referencing One thing to be mindful of is to try and not continuous basis. Gulp focuses on allowing them from the plug-ins object variable. For example, overload your Gulp confi guration fi le in any one plug-in to handle one task and to any reference to concat() would become plugins. way. This could be through the structure handle it well. Although rather unlikely, it concat(). of your task defi nitions or the general is possible to saturate your Gulpfi le with formatting of the code within. too many plug-ins and tasks that are not gulp.task(‘lint’, function() { Your tasks should be written in a clean necessarily of any relevance to your project. return gulp.src(‘js/*.js’) manner that is simple to understand and Consider the structure of your team (solo .pipe(plugins.jshint()) maintain or manage should you need to. or a larger group of people). Think about .pipe(plugins.jshint.reporter(‘default’)); Your Gulpfi le may potentially be distributed the requirements for the application itself. }); amongst a team of developers working on Do you need minifi cation? The live reload the same project. Should any one individual is useful but does it really fi t in to your Further options wish to amend or revise the fi le, it shouldn’t workfl ow at the moment? 08 This plug-in has a number of take too much time to do so. Think about the bare minimum tasks you confi guration options which you can pass into the There are a vast amount of plug-ins need in order to achieve your optimum Gulp gulpLoadPlugins() method. These include which available to download, implement and use workfl ow, and build upon that as and when fi le to read, which keys in the selected fi le are read within your projects. Thanks to the high necessary. Start with the ‘less is more’ to obtain the plug-in name, and whether or not the engagement level of the development principle. Task runners are here to make plug-ins are lazy loaded. For more information, community, more are being added on a your life easier and more productive. make sure you check out the offi cial repository
Linux & Open Source Genius Guide 151 Developer guide
Minifi cation Apply Gulp to 12 and concatenation Write your own To streamline network resources we should also make sure that any JavaScript fi les are combined plug-in a project into one fi le and then a minifi ed version of each is also created. Once more, set the source directory from which Gulp is highly Create gulpfi le.js to pull the fi les, which are then concatenated and extensible and caters 09 When you run Gulp via the command line, injected with a header value before being placed it will look for the existence of the ‘gulpfi le.js’ fi le. into the destination directory. for the majority of your This fi le informs the task runner which plug-ins to needs, but if you need load and defi nes the available tasks to perform gulp.task(‘scripts’, function() { as well as setting the order of any chained tasks. var headerValue = “Evaluated by to build something Create Gulpfi le.js in the project root and ensure it gulp.\n”; specifi c you can write requires Gulp at the very top. return gulp.src(‘js/*.js’) .pipe(concat(‘combined.js’)) your own plug-in var gulp = require(‘gulp’); .pipe(header(headerValue)) .pipe(gulp.dest(‘dist’)) Hopefully you should be able to see the Reference plug-ins .pipe(rename(‘combined.min.js’)) benefi t of implementing a task runner into your development workfl ow. The 10 Any plug-ins that have been installed .pipe(uglify()) as dev dependencies and are to be used .pipe(header(headerValue)) short amount of time it takes to build up within the Gulp process will need to be defi ned .pipe(gulp.dest(‘dist’)); your task functions and combinations by and made accessible. To do so, simply require }); far outweighs the time it would take to each plug-in by name and assign to a variable manually run some of the processes and that can be called and used throughout the Always watching over features they provide. With so many plug-ins available to confi guration. 13 your fi les One major benefi t of any task runner is having enhance and build your task resources var jshint = require(‘gulp-jshint’); the ability to watch a directory of fi les and then to up into a manageable and benefi cial var concat = require(‘gulp-concat’); have the ability to react to any changes that are workfl ow, you will be hard pressed to var uglify = require(‘gulp-uglify’); made to them. be at a loss for the majority of functions. var rename = require(‘gulp-rename’); Here another task is created to manage this very Having said this, Gulp is still very open var header = require(‘gulp-header’); simple but powerful aspect of the workfl ow. for you to create your own extensions Once a change is detected, Gulp and plug-ins in order to enhance Code linting will pass those fi les through to the lint and scripts your build fi les and give back to the community at the same time. 11 The fi rst task will check any JavaScript methods for processing. code for errors by passing it through the default Writing a plug-in for Gulp is a linting engine. Simply register the task by gulp.task(‘watch’, function() { matter of writing JavaScript that providing a name and the associating function. gulp.watch(‘js/*.js’, [‘lint’, ‘scripts’]); can interact with and process streams Here the source directory is set and all fi les }); and data passed through them as well within it will be piped to the jshint() method as learning about the custom vinyl fi le for parsing and validation. Default tasks object. There may be a learning curve for 14 When starting up Gulp from the command gulp.task(‘lint’, function() line you can optionally choose to pass in the many developers to get up and running { name of the task to run. If the name is omitted, with their own plug-in. There are also return gulp.src(‘js/*.js’) Gulp will instead run a default task if one is some fairly strict guidelines to follow, available. and the code must pass tests before it .pipe(jshint()) You can easily ask Gulp to run a series of tasks is accepted. More information on the in order as part of the default task. process and guidelines can be found online here: .pipe(jshint.reporter(‘default’)); gulp.task(‘default’, [‘l i n t’, ‘s c r i p t s ’, github.com/gulpjs/gulp/tree/master/ }); ‘watch’]); docs/writing-a-plugin.
“Gulp is very open for you to create your own plug-ins to enhance your build files”
152 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Plug-in Resources 15 repository Here are a selection of plug-ins that will add power to your Gulpfi le tasks GULP-BUMP www.npmjs.org/package/gulp-bump Whether you are working within a team of developers, committing open source code for public consumption or building a task runner for your own projects, this plug-in allows you to update and bump version numbers within any Semver JSON fi les following any major, minor or patch changes to the code.
GULP-SASS www.npmjs.org/package/gulp-sass This aptly named plug-in will convert your Syntactically Awesome Style Sheets into processed production- ready CSS fi les. Combined with the watch task, your fi les can be interpreted as soon as they have been updated or amended. This includes success and error callback methods that can be built Gulp plug-ins into the workfl ow process.
Plug-in repository GULP-IMAGEMIN 15 While Gulp doesn’t have a plug-in www.npmjs.org/package/gulp- repository, a client-side application has been imagemin created that reads from the Node Package No limits Images and digital assets can often Manager library and fi lters plug-ins that are 17 There are no limits on how many plug-ins be too large network resource hogs, available for Gulp. This is a fantastic resource you can install into your Gulp process, saving them and the cause of many loading to see exactly what you can implement into into your ‘package.json’ fi le or in a single request issues for applications. This handy your task workfl ow. from the Terminal and the npm install command. plug-in will minify your PNG, JPEG Tailor your workfl ow task processes to suit your and GIF images to a more respectable own preferences and your project requirements. fi le size and help alleviate some of these issues for your production servers.
GULP-LOAD-PLUGINS www.npmjs.org/package/gulp-load- plugins This fantastic plug-in removes the requirement to specifi cally defi ne and require each plug-in individually Update plug-ins into your Gulpfi le. The plug-in can 18 Plug-ins can evolve as new features and read the package.json fi le associated Details and documentation bug fi xes are added. You can update your installed with the project and the scope of 16 Each plug-in page within the NPM repository dependencies easily using the Node Package dependencies contained within holds detailed documentation about the plug-in in Manager via the command line. Simply enter ‘npm and automatically assign them question, the installation procedure and then some update’ from the command line and keep your plug- to a global scope on your behalf. code examples for usage and implementation. ins up to date.
Linux & Open Source Genius Guide 153 Developer guide
All the examples presented in this article are tested for Vim version 7.3.547 on Ubuntu 13.04
Just type :help and Vim displays some Type :help iccf valuable info for and Vim displays newbies, such as information navigation keys and (including weblinks) keyboard shortcuts related to donations to enable the for the needy in mouse in xterm or Uganda as well as for GUI etc the development of the Vim editor Master Vim in easy steps Advisor Take a step ahead, learn some advanced Himanshu Arora is a software programmer, open features and get more productive with the source enthusiast and Linux researcher. His articles have been featured on IBM developerWorks, Vim editor Computerworld and in Linux Journal. He (along with some Thanks to a steep learning curve, many some of the prominent pros include fast text editing other Linux freaks) blogs at mylinuxbook.com people either avoid using Vim or use it on (because of little to almost no use of the mouse) and a very basic level. If you’re guilty of either of easy execution from anywhere on the command these then you’re really missing out. line. There are also many GUI front-ends (like gVim) Resources Vim, fi rst released in November 1991, is a available for Vim. Vim: command-line-based text editor that was developed Although it is command-line based, Vim has a www.vim.org/download.php by Bram Moolenaar as an improved Vi clone for loyal following and is extensively used by system Amiga platform. Besides including all the features administrators and software programmers. Though of the popular Vi editor, it also contains a lot of new it is hard to learn and understand all the features that features and so derives its name – from ‘Vi improved’. Vim provides, in order to reap its real benefi ts it is As well as being a text editor, Vim can be used as an essential to master Vim for your day-to-day work. integrated development environment by software Through this tutorial you will learn how to work programmers. Vim provides a wide range of features, with multiple fi les, how to indent and fold code, how but as it is a command-line utility it has a steep to search and replace keywords, how to split the learning curve. Please note that Vim is not a word Vim window horizontally/vertically, and many other processor, which means that you cannot do layout- related features provided by Vim. related changes with it. Note: This tutorial assumes that you have some While you will get to know about the real strengths basic knowledge (opening, editing, saving and closing of the Vim editor once you master it, on the whole a fi le) of the Vim editor.
154 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Work with multiple fi les line number of the cursor in a fi le, fi le in current mode from within the active Vim window with :split 01 While most regular users use Vim to edit active window (represented by %a), fi le in previous <fi le2-name>. To switch cursor control between the one fi le at a time, it can also be used to edit multiple active window (represented by #) etc. Switch two windows, press Ctrl+w (a couple of times). fi les in parallel. To open multiple fi les with the between current and previous active windows by Copy-paste operation across the fi les can Vim editor, just provide names of all these fi les as typing :b#. be done as follows: open fi les in split mode (as arguments to the vim command. The fi rst fi le is explained above); copy the required text from one displayed by default. Type :n to switch to the next Use wildcard characters fi le; press Ctrl+w to switch the control to the other fi le, :prev (or :N) to switch to previous fi le, :bf to 04 Vim also supports wildcard characters fi le; then paste the copied text. switch to fi rst fi le, :bl to switch to last fi le and :bw (like *, ? etc) while opening multiple fi les at a You can also split a Vim window vertically. All to close the fi le. time. This is especially useful in cases where you need to type is :vsp <fi le-name> from within an multiple log fi les begin with a common set of active window. Work with multiple fi les characters like the current date (2013-08-05-- 02 Sometimes it is required to open a --.log), or where multiple programming source new or an existing fi le while editing an already fi les have same language extension such as “Pros include fast opened set of fi les. Well, this can easily be done .c or .cpp. by typing :e followed by the fi lename (along with a complete path if it’s not located in the current Copy-paste across the fi les text editing and working directory). 05 Another frequent requirement is to copy from one fi le and paste into another. For this, you easy execution Get list of currently opened fi les in need to split the Vim window to open both fi les. This 03 Vim can be done either by using the -o option with the To get the list of fi les currently opened in the Vim vim command (vim -o
Linux & Open Source Genius Guide 155 Developer guide
“While most regular users use Vim to edit one file at a time, it can also be used to edit multiple files in parallel” Close all fi les some blocks of code in order to save window space. 06 To close all the opened fi les in one go, add This helps them to focus on the required code ‘a’ to regular close options like :q, :wq or :q!. So, to blocks. Vim does support block folds. For this, the close all the fi les in a single shot, do something like fi rst step is to bring cursor inside the braces of the :qa or :wqa or :qa!. function defi nition. For example, bring the cursor to the fi rst line of the function body. Once there, just Close fi les without closing Vim type va}zf. As you type this, you will observe that 07 Normally, when the last open fi le is closed, va is used to activate visual mode in Vim, while } the Vim application also exits. But, if it is required selects the complete block including curly braces to close the last open fi le without closing the and zf enables the fold. application, then just type :bd. This closes the To open the fold, press zo; to close it, press current fi le and opens an empty buffer. zc. Make sure that the cursor is within the block while performing fold open and close. Typing va} Fold blocks of code zf creates a fold (that is invisible). zo and zc work 08 Another frequent requirement – which only when a fold is created. An existing fold can be software programmers usually have – is to fold deleted by typing zd.
156 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
indenting for C code, use :set autoindent and :set cindent. Now, try writing some C code and you will see that Vim automatically produces indentation as you write it. Sometimes, depending upon the existing settings of your Vim editor, lines might get indented to a number of spaces other than four spaces or any other default value that you desire. As a solution to this, you can set the number of spaces to indent using :set shiftwidth=
Search words 11 To search for a word in the active window, simply type /[search keyword]. This will enable Vim’s search mode and fi nd the fi rst occurrence of the searched keyword. Use n (just press n) in order to search more occurrences of the keyword. Note Manual indenting that you cannot do a backward search here. 09 Another important feature that “For those If you need to do a backward search then just Vim provides is code indentation. Software type :?[search keyword]. This will enable Vim’s programmers are required to indent their code programmers backward search mode. Again, use n to continue and Vim provides easy steps for code indentation. the search, but backwards this time. To indent a single line of code, you can use >> or << Sometimes the keyword is present in the to indent the line to the right or left respectively. who want text/source fi le and it is desired to fi nd other Multiple lines can also be indented using [number- occurrences of that word in the current text/source of-lines]>> or [number-of-lines]<<; for example, 10>> everything fi le. In this case, just put the cursor below that word or 10<<. This is a better technique than indenting and press * (ie Shift+8). This way, you can search each line, but you still need to count the number for a word without even typing it. of lines to indent. This could prove frustrating, automated, Vim especially if the number of lines is very large. Replace words Fortunately, there is another trick: to select the also provides 12 Vim also supports search and lines by working with Vim in visual mode. In order replace operations. To execute a basic to do this, you fi rst need to bring Vim into visual auto-indenting” replace operation, just type :%s/[keyword- mode. This (as mentioned previously) can be done to-search]/[keyword-to-replace-with] to using va. Once in visual mode, use the arrow keys to Automatic indentation replace keyword-to-search with keyword-to- select all the lines that you want to indent in one go. 10 For those programmers who want replace-with. After this, all you need to do is press > or < and the everything automated, Vim also provides an auto- This trick is good if you want to replace all the whole selected area will be indented accordingly. indenting feature. For example, to enable auto- occurrences of a searched keyword, but what if it is desired to replace only selected occurrences? Well, there is a way through which you can run a search and replace operation in interactive mode. This means that Vim will confi rm before replacing the keyword each time. This can be done through :%s/[keyword-to-search]/[keyword-to-replace- with]/c. The fi rst occurrence is always highlighted and Vim displays a prompt (in green, at the bottom) where you can input your choice of whether you want to replace this occurrence or not. Here, y = Yes, replace this occurrence; n = No, skip this occurrence; a = Replace this and all other occurrences; q = Quit the command; l = Replace this match and then quit the command. Use Ctrl+e (^E) and Ctrl+y (^Y) to scroll the window up and down in this mode. Also, use i and I (along with /c, for example /ci or /cI) for case- insensitive and case-sensitive search.
Linux & Open Source Genius Guide 157 Essential software The best distros and projects
160 Presentation software Choose the best slideshow software as we look over a selection of the best Linux presentation apps on offer 164 Music players We find out the best way to organise and play your music library by testing the four best music players on Linux 168 Cloud storage services Discover which service is the best and easiest way for Linux users to store, share and manage data in the cloud 172 Integrated development We pit four of the best IDEs against each other to find out how Linux users can streamline and enhance their workflow 176 Ultimate IRC client Keep up to date with your community as we find out which is the best IRC client 180 System recovery We compare four live-booting system rescue distros so that you know which one to turn to in a time of need 184 Video editors We discover which video editor tool is the best on Linux for creating and editing clips and films for prosumers and professionals alike 188 Photo managers Sort and edit your photos, from holiday snaps to hardware shots, as we find out the best photo-managing tool on Linux
“We’ve sorted out the cream of the crop in terms of essential Linux software for you”
158 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Linux & Open Source Genius Guide 159 Essential software
Presentation software Choose the best slideshow applications
Successor to KOffi ce and KPresenter, Calligra Stage and direct rival to Impress
Calligra is very impressive, although object, and it takes a little experimentation to fi gure it lacks the same kind of community out how that works. This is not so good for a piece of support and popularity as LibreOffice. software you may not be using regularly – and if you Calligra Stage is the presentation software do need to use it regularly, it may take more time to bundled with the suite, and is a standard desktop learn than you’re willing to give it. presentation application in the same vein as Slide transitions are included in Stage; however, PowerPoint and Impress. animation for different objects on the slides is not Starting up Stage, you’ll notice one thing to begin supported, even if they’re imported from another with – clutter. The main interface is just fi lled with slideshow application. This doesn’t seem to affect the functions that don’t quite fi t the space unless you’re imported presentations so much, though, which we using a 1080p or higher monitor. Even then, there’s a have had problems with in LibreOffi ce, and at most lot on the screen to deal with, leaving little room for it’ll require a little rearranging to make sure everything the preview of the actual presentation. The interface is visible. is somewhat context sensitive at least, changing Presentations created with Stage look absolutely out some of the boxes depending on whether you’re superb, with a wide selection of fonts to choose editing images, text or animations. from, a nice array of image and line tools, and loads Some of the functions aren’t straightforward to of slide transitions to use. It would be a bit better if use, though, even if you can fi nd them in the interface. the interface wasn’t so fi nicky and cluttered, though, The pointer function can link between text boxes and and there’s not much you can do to fi x the latter bar Q There are a few graphical themes to choose titles, but it requires the use of anchor points on each buying a bigger screen. from, but they’re not all that great
SCORES
The bizarre nature of some features makes it a little Ease of use tricky to create your slides 7
It will import many common formats fairly well, but Compatibility animation is not supported 7
Presentations can be made to look good enough for a 9 Aesthetics business environment
A wide variety of features for images, text boxes and fl ow charts, but 8 Features no animations
Calligra Stage is perfectly functional, but there are some interface problems that are difficult to 7 Overall wrestle with Q The interface is very cluttered on non-1080p screens, but there are a lot of features to choose from
160 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
EWIPE A very simple presentation application that does everything in blocks
When we fi rst looked at EWIPE, we were intrigued. Running on a Python base, EWIPE presents itself as a fairly lightweight program that can be used to make presentations and CVs, while also being able to save as a HTML fi le. While this all sounds promising, the entire application is let down by a number of issues that make it, frankly, unusable as Q There’s only a limited workspace in which to design your slideshow a way to create slideshows. Firstly, the interface and slideshow only GIF animations. While it doesn’t allow you to alter Even if you decided to create the basic use a 640x480 window. While this may be an the horizontal position or allow for two things to be presentation in EWIPE to then use elsewhere, inconvenience for the main user interface if you’re side by side, it’s a good way to focus on the fl ow of there’s no way to do that. EWIPE saves to HTML creating the presentation, during the actual the presentation. and its own fi le format, one that isn’t recognised presenting it’s unacceptable to only have a small This is unfortunately when another problem by Stage or Impress. In the same way, it’s unable portion of the screen in use – especially if the comes in – text and images will run off the edge to import any other kind of slideshow fi le, not even display in question is 1920x1080. of the displayed slide, as there’s no automatic its own HTML fi les, and GIFs are the only images it It’s a bit of a shame, though, as the method of image scaling or any way to scale an image within supports as well. constructing your presentation is quite interesting. the tool whatsoever. This means you’ll need to EWIPE is a huge disappointment and not worth Blocks are created in sequence for what you wish change image size in an external application, but your time. The interface is a bit weird, but there are to be placed on the slide. You can have three or so even then you’re still wrestling to fi t items into such some good ideas; however, everything else about different title types, text boxes, images and even a small area. the software is unusable and unprofessional.
SCORES
A simple interface that is sometimes just a little Ease of use too simplistic 6
It only works with its own slideshow fi les, and there Compatibility is no import option 0
It looks like it was designed in the early Nineties, and you can’t 2 Aesthetics make it full screen
The most advanced feature in EWIPE is its ability to play animated GIFs, something even 3 Features LibreOffi ce cannot do
We like the way you build up the presentation with blocks, but it doesn’t make up for the lack of full 3 Overall screen and scaling Q The presentation itself is also limited, and does not have a full-screen option or smart scaling
Linux & Open Source Genius Guide 161 Essential software
LibreOffi ce Impress The slideshow creator from our offi ce suite champion – is it as good as the rest of the suite?
LibreOffice is just an excellent office suite, and a true open source success story. One of the strengths of LO is its compatibility with other software, especially with Microsoft’s versions. While Writer has some of the best compatibility of the lot, Impress is still pretty great at importing a variety of different presentation fi le types with very few Q The layout is clean and simple, giving you access to all the necessary functions issues – none that can’t be tweaked to work with Impress at least. – of course, you can add, remove or reposition It is an amazing piece of software, well worthy of The software comes with just about every functions as you see fi t. Priority is given to making the LibreOffi ce name and a place in the full suite. function you’d see in other slideshow creators, sure you can see the preview of the slides, and the Presentations look great when displayed, while it meaning you won’t need to switch between screen is never overly packed with functions and imports other fi le types reasonably well and is just multiple ones. Whether you’re doing a very customisation. This is in stark contrast to the way nice to use. professional business presentation or something Calligra Stage is laid out, with everything placed The only real issue with the software is that a bit more informal, all the necessary graphs on the screen at once with very little room to you can’t quickly or easily publish presentations and animations and sounds and interactions are manoeuvre around them. Due to Impress’s layout, online, although such a function will probably readily available to use in Impress. we were able to quickly and effi ciently put together be coming sometime in the future. For now, The interface to create your presentations is a presentation without having to hunt down though, Impress is most defi nitely the best offl ine extremely neat and logically laid out by default anything or wrestle with the interface at all. presentation application.
SCORES
Every important function is plainly laid out in an Ease of use easy-to-access fashion 9
While there are some minor issues with various file types, they can usually be 8 Compatibility fixed in the software
Impress will perfectly display your presentation Aesthetics the way you want it 10
From graphs to animations and everything in between, Impress has 9 Features just about everything
LibreOffice is just an amazing suite, and the Impress package doesn’t Overall let it down 9 Q Impress is the only one that supports slide transitions and object animations
162 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Slidifi er An online presentation creator and sharer that uses HTML-like code
Although it’s not open source, even if you are using it through Chromium, Google Docs has a very serviceable presentation tool. It may be a little basic, but it does just enough for a presentation to look as professional as anything Stage or Impress can make. Slidifi er is in some ways the open source equivalent to this aspect Q It’s a very simple way to create a slideshow, almost like editing a wiki page of Google Docs, allowing you to create and share your spreadsheets online without needing any specifi c functions involve a dash, a hashtag or a slash to yet professional, and play on just about any browser. software installed. In fact, the Slidifi er version is signify what they represent. Some of these are also Every presentation you make is saved with two URLs better in that regard, as you don’t need an account accessible from preset buttons on the interface; – one that’s read-only, and another that allows you to for it, and sharing the link is a bit easier than the however, some functions need to be hunted down in further edit the slideshow. Google method. the About section. If you need to create a simple presentation very Unfortunately, that’s just about the only thing it The presentations you make also have some limits quickly, Slidifi er may just be what you’re looking has going for it against Docs. Slidifi er is an incredibly due to this construction method. Like EWIPE, you for. However, there just aren’t enough functions to simple tool that has you build your presentation as can only control the position of items vertically on consider it as a replacement for Impress or Calligra, a piece of code, one with very basic formatting to the screen. You cannot change the alignment of text especially if you plan to use graphs or need some signify titles or a break to the next slide. The most or the position of images, or even have two items form of animation. For now, if you’re happy using an complicated thing is adding an image, which uses side by side. If you are able to play within these rules, offl ine app or Google Docs, there’s really no reason the img tags from standard HTML, while most other though, the presentations you can create are simple to switch.
SCORES
While there are some interface buttons, you’ll have to learn the different code bits to take full 7 Ease of use advantage
There’s no way to import or export a presentation – although for the latter it would be quicker 0 Compatibility to recreate
It generally looks very nice, but you can’t position objects horizontally on the page, or with much 7 Aesthetics precision vertically
A very basic set of features, but no slide transitions or 4 Features other animations
There are some very good ideas in Slidifier, but in the face of Google Docs and Overall Impress, that’s not enough 5 Q There are some interesting features in Slidifier, but it does lack some important ones
Linux & Open Source Genius Guide 163 Essential software
Music players We fi nd out the best way to organise and play your music
The Ubuntu default is a Linux Rhythmbox mainstay, but how does it compare?
There was a bit of a kerfuffle a year or Playback is absolutely fi ne, with fi les, folders and range of online streaming services. If you’re using it, so back with Rhythmbox. Soon after selections being easily added to the now playing perhaps consider an upgrade. Ubuntu One started streaming music, queue thanks to great integration with the standard Canonical opted to switch to Banshee for fi le manager. There’s also good integration with the next release of Ubuntu. These kinds of changes notifi cation areas, particularly with GNOME-based happen everyday in Linux. However, what was more volume controls, allowing you to reduce the number surprising was the instant backtracking by Canonical, of icons and control music and volume from one place. reinstating Rhythmbox as the default music player in So pretty basic but good, then. Unfortunately, that’s the following release of Ubuntu – returning with better about it for Rhythmbox. There’s not a huge amount of music streaming options. customisation available for the interface – either for Rhythmbox is a great default music player for the layout or the way it generally works. Compared a number of reasons. It has a straightforward to Clementine, which has a huge selection of music interface with nice, big, standard buttons that streaming services available, access to Last.fm, Libre. stand out so anyone can use it. The entire music fm and Ubuntu Music is just not enough for Rhythmbox. library is accessible down the side, along with a It needs more. handful of basic streaming services other than Overall, Rhythmbox is a great music player. Music is Ubuntu One, and a dedicated Radio and Podcast list. very well organised and easy to search through, with Playlists are split up from this main Library, although the multi-pane library window aiding in this. Creating these only include playlists you’ve created and playlists and play queues on the go is nice and easy, automatic ones such as recently added or played and playback is easy to control. Smart playlists are music, with no smart recognition. noticeably absent, though, as are the aforementioned Q The interface is simple yet fully featured
SCORES
Pause, skip and more via notifcation bar, media Playback buttons or the interface 7
A nice straightforward interface with good Interface OS integration 9
Great music fi ltering and playlist handling with a Management good search function 8
Very few online services can be streamed, other than normal radio 3 Online and Last.fm
Rhythmbox is definitely good, but it’s in sore need of a big update to make it more relevant in a world of 7 Overall online music Q There’s not a whole lot of customisation in Rhythmbox
164 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Audacious Lightweight yet with a surprising number of features
Audacious is the only media player in this group test that can be described as lightweight – and in fact like many lightweight apps, it’s made that way in mind. Still though, it’s a very popular music player and music manager, thanks in part to a good selection of features that make it more than just a basic music player. Audacious’s interface is simple, with a single pane to list all available media, listed by album by default. Music can be easily added, with a standard folder being used to check regularly for any more content for Audacious to add to your library of media. Interestingly, playlists and play queues are Q The layout of media can make it tricky to find anything handled in separate tabs from the main library, keeping the interface a little cleaner and allowing problem with Audacious: while the interface is quite online services. This is its major downfall compared you to have a specifi c layout of your favourite neat and minimalist, it makes looking for the music to the rest of the applications in this test, as with playlists without needing all of them on show. you want to play a bit of a chore. You’re much better ever growing music libraries in multiple locations, The search function is a bit odd, though. Instead off creating playlists in other music players to then your music may not be all available on your PC or of an omnipresent search fi eld on the interface, use in Audacious, or simply add them from the fi le laptop, or even your home network. a separate column needs to be opened up from manager. There’s at least some decent integration At the very least, Audacious is very customisable. the menus. The results from the search are with the desktop environment once you’ve got From the behaviour of what it plays at startup to not displayed very well, with what seems to be everything playing, though. how it handles playlists and such, there’s an eye a random mash of authors, albums and songs However, due to its lightweight nature, there are for making it convenient or keeping it lightweight. At included in the results. This highlights a small no smart playlists, and there is no way to connect to least in that regard, it performs very well.
SCORES
Plays media perfectly, but lacks auto-generated playlists due to 6 Playback being lightweight
A simple and interesting tab-based interface, although search could 8 Interface be better
Due to the simple interface, actually looking Management for your media is tricky 6
No online services offered Online whatsoever 0 Audacious is fine as a lightweight music player, but for proper media management and integration you’ll need a 5 Overall bit more Q The search function is not the best, but Audacious is highly customisable
Linux & Open Source Genius Guide 165 Essential software
Clementine A Linux favourite, how is the latest Clementine player?
Clementine is based on the KDE music player, Amarok, but with a few improvements and a much better interface. It’s quickly become a very popular media player and the latest version, 1.2, has arrived with a whole host of great new features. These are sure to attract new users while appeasing die-hard fans who still want to use their favourite media player. First of all, Clementine now has access to a lot more music streaming services than before, with new additions such as Dropbox and Ubuntu One Q Clementine has everything but the Kitchen Sink. It even has the Hypno Toad joining the already impressive list of existing ones. These include Google Drive, Spotify, SoundCloud, things like how it might fade between tracks, to actually going. While you can control Clementine Last.fm and Grooveshark. You can easily search tweaking the transcoding settings or even setting from here, you can also control it from the usual within the free services using the built-in a Wii Remote as a remote control device. New in volume control icons if you’re using the right Clementine search functions, and you can log in Clementine 1.2 is the ability to use an Android desktop environment. to do the same with the account-driven services device as a remote, a feature which has been a Clementine basically has it all, then. Its smart such as Spotify and the cloud storage ones. These long time coming. However, instead of using a playlist feature, the dynamic random mix, isn’t settings are easily found in the preferences menu basic HTTP interface, it uses a special app to quite as good as some online equivalents, but it’s under a different section to the vast wealth of make it work. a lot better than any of the other players in this customisation options that Clementine offers. The playback is fantastic, with a special test. It also has the greatest selection of online Through these options you can change just Clementine icon ticking down to the end of the services it connects to, is the most customisable about every way Clementine behaves, from simple song, and showing a play symbol so you know it’s and makes fi nding your music easy.
SCORES
Makes playback as easy as it can be, short of dedicated buttons on the 9 Playback notifi cation area
The interface contains a lot, but does the best it can Interface for the amount of features 8
Easy to navigate and fi nd media, although some of the online services could 9 Management work better
Connects to everything you would probably want to use bar Pandora and 10 Online Google Music
An amazing piece of software that lets you do just about anything you’d want to do with all of 9 Overall your music Q The Android app offers better control than some HTTP interfaces
166 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Banshee It may not be as popular as Clementine, but is Banshee better? Banshee was the one-time default audio player for Ubuntu, replacing and then being usurped by Rhythmbox. Due to this, you would be forgiven in thinking that they are incredibly similar applications – and in some regards they certainly are. They both employ a similar three-pane layout for your media, and they both include a column down the side for navigating your media, videos, podcasts and online services. At the core, they also both run off GStreamer, which is a great media back-end and allows the two to play just about anything with the right codecs installed. The interface for Banshee is nice and easy to use, Q The Banshee interface is very nice and very responsive. Search is instant, bringing up results as you type, and the way results are listed is interval or time for the music library to update. Rhythmbox, though, there’s only a handful like Last. conducive to fi nding the tracks, album or artist you’re These kind of features are sorely missing, especially fm and Amazon. There are a few other, community- looking for. The album pane on the main interface compared to Clementine and Audacious which have built extensions, but none to challenge the features has thumbnails of the album art instead of a list – a whole host of different features and options that of Clementine. although the grid effect can be disabled if you wish. It can help you streamline the experience. At the very So overall, Banshee is pretty good. While it’s all works very well and, like all the others, integrates least, there’s a fairly rich plug-in system and you can easy to compare it to Rhythmbox, it’s generally just fi ne with the desktop environments that allow turn off some of the features of Banshee you don’t a little better, with superior plug-in support that for playback options via volume controls. wish this way, making it much more lightweight than enables it to be more lightweight if you wish, and a Customisation wise, there’s not a whole lot more it is by standard. It’s through these extensions that slightly cleaner and informative interface. It’s still no than Rhythmbox. You can’t even set a specifi c the online services are included in Banshee – like Clementine, though.
SCORES
Good playback options, but lacks its own dedicated 7 Playback notifi cation icon
An easy-to-use interface that is laid out in a Interface logical manner 9
For local content, it’s a great way to keep track of Management all your media of any type 10
Limited online options, and it’s only Online minimally extendable 4
Banshee is a great media player that we’d be very happy to use if we didn’t Q The extensions give Banshee a lot of its features, and turning these off is the main way to have access to any 8 customise the software Overall online services
Linux & Open Source Genius Guide 167 Essential software
Cloud storage services The best cloud services to store, share and manage data
Canonical’s own cloud storage Ubuntu One works on more than just Ubuntu It was a fairly big deal a few years ago across, you can choose to back up and/or sync The client doesn’t allow for much customisation, when Canonical announced Ubuntu individual folders, with quick suggestions of those in though, only speed limitations and whether or not it One – 1GB of storage for all users of Ubuntu, your home directory. should launch at boot time. It’s easy to set up and let to be used as they wish to save and sync You can access your fi les from the web, other run in the background, but Ubuntu One doesn’t quite their fi les to and from the cloud. Since then, Ubuntu computers with the client on, or even from an Android have all the tweaks we’d want. One has upgraded to 5GB of free storage and is client. If you have any music stored on it, you can pay available to anyone, not just those using Ubuntu. As a little more for a streaming service, allowing you to well as offering 5GB for free, you can also add 20GB stream your music from Ubuntu One over the web of storage for $2.99 a month or $29.99 a year – and without having to download any of it fi rst. This is a this stacks so you can continue to add 20GB to your fairly unique feature, although it will cost you an extra account, and add a further $2.99 or $29.99 to your bill $3.99 a month or $39.99 a year. in the process. Syncing is fairly quick and it notifi es you of any The Ubuntu One client is fairly simple to use, changes made. There is no tray icon, though, so you’ll requiring little more than your Ubuntu One login have to hunt down the client to fi nd out if there’s any details and then some basic information on what you syncing going on. If you’re running Ubuntu, it also wish to keep synced. As well as its own Ubuntu One integrates into context menus, allowing you to right- folder that has fi les that are automatically synced click fi les and send them to the cloud – a nice feature. Q Choose extra folders to back up to the cloud
SCORES
5GB free is basically a standard right now, although 20GB increments stack up 7 Space for heavy users
Notifi cations are a bit overbearing and there’s no tray icon, but the context menu entries 7 Integration are good
As well as a standard sync folder, you can stream your own music and back 7 Features up other folders
Rather disappointingly, only the very basic customisations are 5 Customisation available in Ubuntu One
Ubuntu One is a decent, basic service that will happily sync your files without requiring much direction. It should do 7 more, though Q Straightforward access to files though the online portal Overall
168 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Copy The newcomer with a lot of promise
Copy is so named as a little bit of an in- joke – offi cially, of course, it’s because it copies your fi les to the cloud, like all cloud storage services do. However, it also copies other cloud storage services in some of the way it looks and operates, giving its name a bit of a double entendre. This is no reason to look past it, though, as Copy has quickly become a very good cloud storage solution. First of all, you get 15GB of storage free, the Q This is about all you get with the Copy client most out of any of the services we’re reviewing in this feature. It also has the cheapest upgrade path, Copy works by syncing a single folder between benefi ts from already having done the graphical offering 250GB for only $9.99 a month/$99.99 a year, devices and the cloud, and does not include setup, mainly because it doesn’t automatically allow or 500GB for $14.99 a month/$149.99 a year. This is the folder backup function of Ubuntu One or you to log in, but it’s good for headless servers that one of the best prices available on any service and SpiderOak. There is, quite frankly, not a lot of need the cloud storage. while you would think there must be a catch, it isn’t customisation available for the client side of Copy; Copy has defi nitely had a great start, offering a quite what you’d expect. however, it does expose the transfer rate, some fi le competent service for a great price. However, this The client software for Copy is supplied in three management options and allows you to set it as a kind of service comes at a cost to the company and binaries, although you only need to use one of them startup service for your system without having to while 15GB free is a great incentive, it may quickly at a time, or ever. The main one is the graphical client, properly install it. become a double-edged sword if you become reliant allowing you to log in and set up your computer to The other two binaries include a command-line on a service that doesn’t take off in a landscape sync fi les to your copy account. Similar to Dropbox, interface and an online API app. The command line saturated with cloud storage suppliers.
SCORES
A huge amount of free storage compared to rivals, along with very 9 Space cheap upgrades
The binary doesn’t properly install, but allows the client to launch into 5 Integration the notifi cation bar
Only really offers one folder syncing client side, although you can undelete 7 Features fi les from it
You can only change the speed limits and whether or not the service starts at 4 Customisation boot time
Copy is very good, but it’s lacking some of the features of the other cloud storage clients, and it may be too early to jump on 6 Overall board with it Q The website is laid out very nicely at least
Linux & Open Source Genius Guide 169 Essential software
Dropbox No introductions necessary for the king of cloud storage
Dropbox is synonymous with cloud storage; in fact, when describing other cloud services to less tech-savvy people, the conversation usually includes the phrase ‘it’s like Dropbox’. Popularity doesn’t necessarily mean quality, though, which is why we’re comparing it with several other services in this test to see if it truly deserves its reverence. Here are the facts then – Dropbox is a cross- platform service offering 2GB of free data, with small increments available for referrals and such, which is upgradeable for a price. This starts at $9.99 Q Selectively sync specific folders in Dropbox a month for 100GB, which seems to be somewhat of a standard in the cloud storage space and is cheaper case there are fi les that only need to be accessed shared folders that you have ownership of. There’s than Ubuntu One’s 20GB add-on scheme. Dropbox’s by specifi c devices, or just via the web interface. You also a pretty advanced undelete function that lets Linux client is fairly simple compared to the other can also share folders between other users, allowing you browse the last few weeks of fi les you have platforms it works on; however, it offers roughly the you to collaborate on work, or just quickly and easily deleted and restore them as long as you have space same level of functionality without some of the bells transfer fi les between each other. If you also have in your account. and whistles those versions have. another client on the same network, it will allow you The good thing about Dropbox is that it’s available The client only allows for syncing of the main to transfer fi les over the LAN automatically – not a on all desktop and mobile platforms, and it works Dropbox folder, rather than letting you select huge feature, but good nonetheless. pretty much seamlessly between all of them. While different folders to also sync or back up. However, The web interface for Dropbox is also one of it doesn’t offer all that much space for free, it does you can tell Dropbox to selectively sync specifi c the best, allowing you to easily navigate, edit and have reasonably priced upgrade options and is one folders from the main Dropbox directory, just in change settings on fi les on your account and in any of the most stable services available.
SCORES
Not a lot of storage space given away for free, but very reasonable 6 Space upgrade fees
Integrates throughout a network and has a useful Integration notifi cation icon 8
A good set of syncing features and sharing, but no extra folder 7 Features backup options
There’s actually not a huge amount to change in Dropbox, although there 7 Customisation are proxy settings
Dropbox is very good and deserves its reputation. It’s solid, reasonably priced and works very well 8 Overall and very quickly Q Have more control over your files via the web interface
170 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
SpiderOak A veteran in Linux cloud storage
One of the fi rst commercial cloud storage services to make its way to Linux, SpiderOak has been around for a few years. This means a couple of things right off the bat – fi rstly, it has the same 2GB free, $9.99 for 100GB storage deal as Dropbox. Secondly, it also has probably the best and most mature client program out of any in this test, which we’ll shortly go into more detail about. One of the benefi ts of using SpiderOak is that the fi rm behind it is extremely confi dent in the security and privacy of your data on its server. It operates what it calls a Zero Knowledge scheme, where the server doesn’t actually know what you have stored on it because it’s encrypted. This is a good idea and should guard against most types Q You get complete control over the client and how it syncs of intrusion. As mentioned, the client for SpiderOak is client works – such as telling it to only back up fi les is the ability to sync two folders between devices, extremely good on Linux. The main interface of specifi c size range, age range or excluding specifi c or on the same device, without having to use the supplies a lot of information on the current status keywords; setting a backup and syncing schedule in dedicated syncing folder. This is particularly useful of backup and syncing, what the current queue case you don’t want it to do it all the time; and it also for syncing between folders on the same device if of uploading or downloading fi les is, a log of any includes LAN sync like Dropbox. you need some kind of a backup, or multiple users changes made etc. Like Ubuntu One, you can While the website end isn’t as advanced as are working on the same system. select other folders to back up, as well as having Dropbox’s, there is an Android app which allows you Overall, SpiderOak is an extremely mature cloud a SpiderOak Hive folder that syncs between all to access your fi les more easily on the go. One of storage service – and unlike other solutions, it hasn’t devices. You can also heavily customise the way the the neat functions included with SpiderOak, though, skimped on the Linux support.
SCORES
Like Dropbox it only offers 2GB of free storage, but there are decently 6 Space priced upgrades
SpiderOak is able to integrate well with a Integration number of setups 8
It offers the highest number of features we’ve seen for a cloud 9 Features storage service
Just about every aspect of the client is customisable with useful functions that 9 Customisation can be enabled or disabled
SpiderOak is probably the best commercially available storage solution on Linux, and we don’t say 9 Overall that lightly Q The web interface is basic but usable
Linux & Open Source Genius Guide 171 Essential software
Integrated development The best IDEs to streamline and enhance your workfl ow
A more open IDE originally for C++, Code::Blocks now able to do more A relatively new IDE, Code::Blocks While you won’t be able to properly debug test It’s a bit lighter than Eclipse at least, and the interface was released in early 2008 after three code, the interface for Code::Blocks is still very nice is well laid out, but it may still be a few years before it years of release candidates. Although a for actually writing your code. Projects are handled via becomes preferable to the others. C++ IDE by default, it has plug-in support proprietary projects fi les which link to specifi c folders. to extend its functionality to allow for building and Projects don’t need to be created with a specifi c testing of other languages. However, for pure coding, language in mind and, like some text editors, it will Code::Blocks can be used to write scripts in various smartly know to highlight the syntax for different languages thanks to support for syntax of a variety of languages. Browsing doesn’t always work for all different languages. However, you cannot test these functions and classes; however, it will allow you to without add-ons. perform simple code editing to test elsewhere if you Unfortunately, the plug-in scene for Code::Blocks specifi cally like the interface. is fairly immature and there isn’t even proper support Code::Blocks is also quite customisable, allowing for Python and other interpreted programming you to change the look and behaviour of the editor, the languages. Plug-ins are found via the Code::Blocks way the compiler works and even editing the startup wiki and are not particularly well laid out at the script for the application itself. moment. Plug-ins are installed manually, which Code::Blocks is a nice piece of software, although would be fi ne normally, but the other IDEs in the test due its immaturity it doesn’t quite have the best plug- have some interesting ways of managing plug-ins in support of the other IDEs in this test – especially if Q There are a few first-party plug-ins available, that make this method seem somewhat archaic. you want to use it for multi-language programming. but not many third-party
SCORES
Available in repos, but binaries and source code is easily accessible 8 Installation elsewhere
Code::Blocks’ interface is generally good for workfl ow, although it doesn’t support all 7 Workflow languages properly
A great selection of features that can all be Features heavily customised 7
Very limited right now, meaning that not all Plug-in languages are 4 support properly supported
Code::Blocks has a great interface, and some great ideas on how to code and manage projects, but the plug-in selection needs to 6 Overall be improved Q The interface is nice and aids workflow
172 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Geany A fully featured IDE that’s a little more lightweight than most Geany is sometimes mistaken for a glorifi ed text editor. In reality, it’s a full IDE, albeit a slightly more lightweight IDE than most. Geany has multi-language support out of the box, with easy options to create fi les that use different languages as part of the main interface. Projects are kept in specifi c folders, like Eclipse, although there is a fi le to go with them that Geany can read to manage the projects. Multiple projects can be viewed and edited at a time, all in different languages. There are also debuggers and builders Q Geany’s interface is smart and easily breaks down code for workflow purposes for the various languages that support them, and you can run and test stuff like Python that doesn’t usual code-editing tools such as commenting out a ways to even change the characters required for need compiling as such. selection or indenting are all present. autocomplete suggestions. The rest of the interface for Geany is clean and Sadly, Geany’s plug-in support is pretty dire. Geany is without a doubt a very smart IDE, with a well labelled. Code is automatically highlighted Some plug-ins extend the basic functionality for low barrier to getting your project started straight with the correct syntax and there’s a smart tab in a few of the languages, but there’s not the kind of away. It does lack some of the features found in the left column that allows you to track and view depth as Netbeans or Eclipse in the sheer number of some of the bigger IDEs, such as more advanced the different variables, functions and classes in the available plug-ins. unit testing and debugging, although it will let you projects and code. Navigation through the code is There are a lot of ways to customise Geany with browse compile errors. Geany’s plug-in selection fairly simple via context-sensitive menus and nice the standard tools and menus, though. Almost every is also pretty poor, so it may not be extensible for options within the rest of the interface, and the part of the interface and workfl ow is editable, with specifi c functions.
SCORES
Readily available in most repos and requires Installation minimal dependencies 9
Workfl ow is superb for the variety of different Workflow languages included 9
No proper debugger, but a great selection of build and project 7 Features management functions
Although it doesn’t quite need it like the others, Plug-in there are very few 4 support plug-ins available
Geany is a great, lightweight IDE with good native support for a variety of languages, but there’s not much room 8 Overall for expansion Q The native languages are supported quite well with compilers and such
Linux & Open Source Genius Guide 173 Essential software
Eclipse Eclipse is an industry standard – so how does it fare against more community- run efforts? Eclipse is one of the most popular development suites around, and at first glance it’s easy to see why. While created mainly for Java development, Eclipse is highly customisable through plug-ins. Thanks to its popularity and community, this has resulted in Q Plug-in support for Eclipse is top-notch a great selection of add-ons that enables Eclipse to work with just about any language. These plug-ins and warnings for anything that might be ineffi cient and you can even select which extensions to install allow for more than that, with a marketplace full of in terms of the code. There are plenty of editing tools from each of the repositories. interface and behavioural modifi cations alongside too, such as simple indent or dedent options. Eclipse is customisable in other regards, with the language elements. The debug suite in Eclipse is fully featured, with an expansive properties and settings menu that Eclipse has great project management tools as various ways to run, check and unit-test code, lets you edit a huge amount of the native behaviour well, with a smart tabbed interface, and plug-in- although this depends on the plug-ins to some of the IDE, from the way patches are viewed to specifi c menu entries for starting projects in degree. The tools are there, though, and most of the little things like key bindings and other shortcuts. different IDEs if needed. Projects are located in plain major plug-ins seem to use them. Eclipse is a fairly big suite of packages, though, and folders in the location of your choice, allowing for The plug-ins are handled by a repository system, a large resource hog. It does have probably the best easy access of source code if you need it, instead of which lets you keep any add-ons up to date. While extensibility of all these IDEs, however, meaning it being inside a proprietary fi le. The function search there aren’t a huge amount available by default, it’s also probably has the most to offer those who work ability works well and the interface has plenty of tips easy enough to add more to the plug-in manager on a lot of differing projects.
SCORES
Eclipse requires a lot of dependencies; however, it’s available in most 7 Installation major repos
Smart interface design that lets you easily navigate projects 9 Workflow and code
A great set of features by default, although mainly Features for handling Java 8
The best support around, with a great repo system Plug-in offering a wealth of 10 support add-ons
Eclipse is popular for a reason – its got a great selection of features that are easily extensible to 9 Overall suit almost any need Q Code navigation and highlighting aids your workflow
174 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Netbeans The Oracle- developed Java dev platform that multitasks
Netbeans started life as a humble student project to make a Java IDE in Prague. Within a few years, a company had been set up around the IDE, and then had been bought by Sun Microsystems. The rest is history and after over 15 years of development, Netbeans is on version 7.3. Netbeans has a very similar feature-set to Eclipse, with debuggers, compilers, unit testers and the like, although by default Netbeans has support for PHP and HTML as well as its Java-specifi c features. Q Netbeans has a nice welcome screen to start new projects Netbeans also has a huge database of plug-ins to call from, allowing you to extend its features to While the splash screen is nice for starting new the same level of customisation as Eclipse or the just about any programming language. These plug- projects, the rest of the interface is cluttered and others. However, for similar customisation tasks, it’s ins are nicely arranged on the Netbeans website, some functions are hidden away in weird menu a little better laid out than Eclipse. with simple search functions and a fairly easy way options. The code view is not particularly well Overall, Netbeans seems like it’s a little behind to install the plug-ins to the system. There’s no handled, with syntax highlighting a little sparse. The the times, with an interface that needs a small repository function for the plug-ins like Eclipse, but function navigator is fairly nice, though, with the overhaul and a better selection of customisation the database displays enough information to know if sidebar allowing you to track the hierarchy a little options. It’s by no means a bad IDE, with a great suite you’re getting the latest version of the plug-in. better than some. of debugging and testing options to use on code. On The interface is a little confusing – not very Customisation in Netbeans is handled with a the other hand, Eclipse is equal or better in almost well streamlined and aesthetically a little dated. nice-looking options menu, but there just isn’t quite every regard.
SCORES
Packages are readily available for Netbeans and it requires a 8 Installation Java install
The interface is a little cluttered and convoluted, but there are some decent 7 Workflow options to be found
A similar feature-set to Eclipse, but not as much Features customisation 7
A fairly large selection of plug-ins are available Plug-in for it within an easy-to- 8 support search database
Netbeans just misses the mark with a slightly dated interface, but it’s made up for somewhat with a good 7 Overall selection of plug-ins Q The interface is not particularly well laid out
Linux & Open Source Genius Guide 175 Essential software
Ultimate IRC client Keep up to date with your community
A lightweight and customisable Xchat IRC client with a basic interface IRC client Xchat is currently found pre- is limited to fi ve, but should be more than enough in just the way you want it, while still maintaining installed on a lot of Linux distributions, for most servers. You can set default channels to a great, basic interface. Although it may not be the coming with both a graphical and autoconnect to in the servers, which works well best IRC client, it’s defi nitely a great choice to be command-line interface. The graphical with the tabbed interface. Startup is quick and the default in many distros. interface is GTK+ based and incorporates a fairly the handy notifi cation bar icon is omnipresent for minimal and lightweight design, but not so much as multi-desktop workfl ows. you would feel like there are missing features from There’s a lot to customise in Xchat as well, from the interface. the way notifi cations behave, to the graphical Upon startup, a simple connection interface display and font choices. The notifi cations are is presented, enabling you to connect to one of quite varied, with different visual options and many preconfi gured servers, or quickly connect sound options to make sure you know when very to another not on the list. Every major IRC server, specifi c actions have occurred. This works well, and more, is available here, although due to this it although it makes out that there are some default can be a bit tricky to navigate to the one you want. sounds already available for the client, whereas It’s not easy to trim the list either, with many to edit you actually need to add your own. and delete. As well as including logs, Xchat is a very solid It’s also on this screen that you can start editing IRC client, something that should be set as the your identity details, with different priorities minimum standard to compare others. Lightweight, to different nicknames, with some generic easy to use and quick to customise with a wealth Q There are fairly advanced server editing suggestions based on your distro’s username. This of options, Xchat allows you to confi gure it to work options
SCORES
Available in a large number of distributions; requires Installation GTK, though 9
The interface is easy to understand, with a lot of options contained in 9 Ease of use one place
Highly customisable, but not the most customisable Customisation in this test 8
Feature rich for such a lightweight application, with a lot more than the 8 Features basics covered
A fine default, and a great IRC chat client. The inclusion of a command- line interface as well means you don’t need any 8 Overall extra packages for that Q Customisation goes deep into the workings of Xchat
176 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Irssi A functional command-line client with a limited feature set
Irssi is the sole command-line-only IRC client in this section, and is the preferred option for a lot of Linux users that forgo the graphical IRC offerings available. Even though it’s a CLI application, it claims to be the client of the future, and because of this it does have the highest compatibility among the clients in the test and won’t require any extra graphical packages such as GTK+ or Qt. Q The command line limits the amount you can do Being a command-line tool means the interface is extremely limited. There’s no JED-style window lot to edit, again limited by the fact it’s on the the default startup, which will then autoconnect to emulation here, requiring a completely manual command line. channels and such. setup of Irssi to just about everything. The manual to Irssi works, though, and is probably the best While Irssi defi nitely has its unique and positive make it work is only available on the website – which, command-line IRC client around. The extra windows attributes, it’s just not up to par with Xchat or ironically, you will need to access via a graphical stuff, while not bringing it up to the same standard Konversation for the sheer amount of features, web browser. From there you can learn how to as the graphical clients in this test, is fairly novel for customisation and general ease of use. Of course, connect to servers and channels, how to start a command-line-interface IRC client and allows you if you’re only using a command line, we would creating multi-window setups and editing other to have a little more control over it. You can edit fi les recommend this in an instant; however, if you’re in a display preferences. There is actually not a whole to keep these windows, allowing them to be part of desktop environment, the GUI clients offer more.
SCORES
Irssi is available in the repo for most distros and doesn’t require many 8 Installation dependencies
The setup needs to be done manually and does not offer any help unless 3 Ease of use you read the website
While there’s a decent amount to do for a command-line app, there’s still very little 4 customisation compared Customisation to the others on test
There are very few features compared to what you get with the 4 Features other clients
Good for a command-line tool, but just not good enough to abandon one of the much better graphical 4 Overall tools available Q Ironically, you’ll need to look up the website in a graphical browser to make sense of Irssi
Linux & Open Source Genius Guide 177 Essential software
Konversation Take full control of your graphical IRC experience with this KDE app
Konversation, while a KDE app, is not directly tied to that desktop environment. Its development is independent from the DE and it has its own release schedule because of this. It’s used in some KDE-based distros as a default, although it doesn’t quite have the ubiquity of Xchat from earlier in this test. This means that Qt is required for Konversation; however, the necessary dependencies won’t cause a problem in the vast majority of cases. First impressions of Konversation are that it’s a little more complex than Xchat. The menu system and startup screens offer a fair few more options, Q Set up many nicknames and change their priority easily with some of these displayed in separate windows, such as a more advanced nickname selector, away of the interface or behaviour that you cannot ability to edit the buffer, time-out and the fallback messages etc. However, there’s only one server in edit in some way – from showing up any and all interface. It’s very well thought out as well, with all the list to begin with, a complete opposite to Xchat. information available from the IRC server, to setting the default settings for even the most niche options While this means a little more setup on your fi rst go, different fonts for various parts of the interface. It’s set to give a great experience without any extra it does result in a lot less clutter in the server listing. incredibly rich and while the application itself is fi ddling. The same automatic connection settings can be possibly the ‘heaviest’ of the selection, it’s still very Konversation is basically the complete package, made, with the ability to add multiple channels to quick and not particularly weighed down by all the allowing you to have full control over your graphical any server. extra features. IRC experience. It doesn’t include a command-line The amount of features and customisation Even the download manager is highly editable. version, like Xchat does; however, it’s defi nitely the in Konversation is vast. There’s not a single part As well as the basics such as location, there’s the very best you can use on a graphical desktop.
SCORES
Available by default on some distros, get it with Qt Installation on others 9
A little more complex than Xchat, but overall easy to Ease of use use with great defaults 8
Every part of the application is customisable through the 10 Customisation included menus
Everything you could think of, except a command-line Features interface alternative 9
An amazing client and all- round piece of software, taking an IRC client to its Overall logical conclusion 9 Q The amount of options for customising is staggering
178 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Smuxi Smuxi is a simple- to-use application but its interface is too simplifi ed – it doesn’t quite stand up to its rival IRC clients
One of the things we weren’t too keen with in Irssi was how limited it is as a command-line application, and how much extra manual setup you have to perform to get it to anywhere near the Q The interface is more simple than the others functionality of the other clients we’ve tested. Even then, there was only so close you could get to and Konversation, with no welcome window to it’s lightweight. Using Smuxi is fi ne enough and that level of functionality. connect to servers, and some preconfi gured it handles the very basics of what you’d like to do. Smuxi is inspired by Irssi and is a graphical autoconnections to a couple of channels to begin There’s also simple download functionality, and client with a twist. As well as being a standalone with. A quick connect menu is present in the corner, logging; however, it has the weird habit of requiring graphical client, you can also have it as a front-end allowing for just that. This limited main window is you to exit the window and then quitting the entire client for a back-end IRC server. The back-end can indicative of the rest of the application, which has application from its icon on the notifi cation bar. be connected at all times, with multiple front-end very limited customisation options compared to Overall, Smuxi really isn’t as good as Xchat or clients connecting to it at once, allowing for the the others in this test, and nowhere near the same Konversation, especially without the same level of client system to use up very few resources. The amount of features. customisation the other two enjoy. The front-end standalone mode is still very lightweight, though, The interface is at least good, though – the tabbed separation is a nice addition, and could be useful for and can be used just fi ne in that mode. The interface servers and channels works well, it connects and some, but it doesn’t make up for the interface you’ll is a fair bit simpler than the offerings from Xchat runs relatively quickly and, as mentioned before, be using being so simple.
SCORES
It’s available in most distros’ repositories and is otherwise easy 8 Installation to fi nd
Smuxi is a very simple program that is equally as Ease of use simple to use 9
Limited customisation for the interface Customisation and behaviour 6
Aside from the very basic interface features, the front-end separation 7 Features is great
Smuxi isn’t much more lightweight than the others, so its lack of features makes it difficult 6 Overall to recommend Q You can set servers to connect to at startup
Linux & Open Source Genius Guide 179 Essential software
System recovery Live-booting system rescue distros you should turn to
Not the best system recovery tool Parted Magic but offers other useful tools
Parted Magic gets its name from otherwise very poor documentation on how to a bit more focused on hard drive partitioning. There’s two of the partition editors featured perform various maintenance and recovery tasks not much in the way of extra support for if you have in the software suite, which is on the website, something that we will discuss in our it installed to a USB stick or something; however, an indicator of its focus. There’s SystemRescueCD review. there isn’t much customisation you’d need to do to it a lot of partition editing software on the disc, both The ISO itself is pretty tiny at just over 300MB, and between uses. in graphical and command-line form, and there are comes with multiple boot options at startup for 32- even tools and packages to help you diagnose the and 64-bit kernels, alternate graphical modes, and health of your hard drives. Shuffl ed in with these whether you want to load into RAM or not. This extra apps, though, are a few great tools for other system bit of choice gives it a lot more compatibility on older recovery and maintenance tasks. or slower systems, and in general it’s very fast to boot For starters, ddrescue is included with the other and load. partition and editing tools. This data recovery In addition, it has some other maintenance tool is very powerful for fi xing fi le corruption and features, such as some rsync-based utilities for suchlike, with ways to compare it to previous backups, Clonezilla for disc cloning, and even an anti- versions, even if they too are corrupt. Parted virus through ClamAV to scan your drives. There’s also Magic lists all available tools on its website, with access to the internet through Firefox so that you categories and a small description for each, allowing can look up any information or tutorials for the tasks you to look up the relevant package for the task you’re aiming to do. you wish to accomplish. While this is great, there’s Parted Magic is a good recovery distro, even if it’s Q There’s plenty of choice at boot time
SCORES
Quick to boot, with some advanced boot options 7 Boot
Using ddrescue, you can recover and fi x fi les in a Recovery number of ways 7
A lot of graphical tools, although not much documentation on the 8 Ease of use website
Backups, disc cloning, system health and partitioning tools are also 9 Extras available
A great live distro to have to hand if you need to maintain a system, but not the best for 7 Overall recovery Q Parted Magic is full of graphical tools
180 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Finnix 107 The smallest and fastest system recovery distro but can be tricky to use
A Debian-based distro, Finnix is the smallest and fastest one in this test. 107 comes in at about 120MB for both x86 and PPC architectures, yet is apparently able to help you with all your system recovery needs. It manages this by not including a desktop environment, something all the others have, and this also allows it to be incredibly quick due to its command-line-only nature. Before you get to the command line, though, there’s a selection of boot options that seems to be the standard with these rescue discs, with a choice of 32- or 64-bit. There are also a few extra options to Q Boot from the disc or boot from the network boot into FreeDOS, use the Smart Boot Manager and run an iPXE so you can easily install other distros available. However, there’s noticeably less choice It’s not a bad distro in any sense of the word, over the network to the current system. There are than the other distros. It’s still a decent selection, though: if you know what you’re doing, and you need other boot profi les as well, for those who want more though, allowing the system administrators it’s some very specifi c tasks done, Finnix may be your control over how Finnix runs. aimed towards to perform fi le system recovery, best friend. Its extra network-boot functionality It loads very quickly into the command-line GRUB fi xing, fi le transfers, backups, cron jobs and is very useful, and its size and speed make it ideal interface, proudly announcing how many packages so on. It’s just a little too obscure for other users to for a lot of system maintenance jobs. As already are installed. You can browse what’s available use, and there aren’t many great guides for those mentioned, the downside is that it can be diffi cult using ‘dpkg -get-selections’ to get an idea of what’s that need it either. for some to use.
SCORES
Great – a lot of booting options, including ones outside of Linux 9 Boot
Decent recovery apps, but not a huge selection Recovery 7
Diffi cult for those not used to working entirely in the Ease of use command line 3
There are some great extras in terms of net booting, but otherwise it’s 6 Extras limited
A great recovery distro for people who know exactly what they’re doing, but not so great for those who suddenly find themselves 6 Overall needing to use it Q Finnix’s documentation is very light
Linux & Open Source Genius Guide 181 Essential software
Knoppix An all-purpose portable live- booting distro but not our fi rst choice This all-purpose live distro does have sheer fl exibility as a mobile OS. With multiple versions of every kind of app you’d need on the DVD edition, and a great selection of core packages on the CD version, Knoppix is a jack-of-all-trades as a Linux live- booting distro. When it comes to system recovery, though, there are some very specifi c features that need to be included, and Knoppix is clearly not the most focused distro. Here are the things it can do, at least. Obviously, as a live-booting distro, you don’t need to be able to properly boot into your operating system to access Q There’s a lot of software on the Knoppix disc fi les if there’s a problem booting. You can also fi x GRUB using the command line if you know what maintenance tools, such as partition editors and fairly fast at least, but can take a while to download if to do – and of course with ample browsers in the backup utilities, and even the same virus scanner as you need it in a hurry. software list, you can easily look that up. There Parted Magic. Knoppix can be useful in a pinch, but it’s not really aren’t any data recovery or forensic tools, though, so There’s no specifi c disc cloning utility other than designed for this kind of system recovery work. for corrupt data you’ll need to look elsewhere. the standard dd, though, which may not be as fully While it’s a great portable distro – and you should Knoppix is a great portable distro. It’s easy to featured as some would like. probably have a USB stick of it to hand somewhere put onto a USB stick permanently and can then Knoppix also comes in rather a large size – if you’re being that prepared though, you should have programs and packages added to it from compared to the other distros in this test, with the most defi nitely have one of the other distros to there. By default, it does come with other system CD clocking in at 700MB, and the DVD a full 4GB. It is hand as well.
SCORES
Highly customisable boot sequence using cheat sheet 9 Boot
You can browse and copy fi les, but there’s not much Recovery in the way of data recovery 5
The remaining packages are easy enough to use Ease of use 8 It has pretty much everything else you’d ever need, at least 10 Extras
As a system recovery distro, Knoppix doesn’t do so well. All the other distros here are far 5 Overall better Q Once installed, you can extend the functionality of Knoppix
182 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
SystemRescueCD An essential portable distro you should keep to hand
On fi rst impressions, SystemRescueCD seems to tick all the boxes we’d want it to tick as a recovery distro. The live-booting OS comes in at a fairly small 370MB and immediately offers up a huge variety of boot options, with the choice of 32- and 64-bit, languages, alternate boot methods etc. This then quickly loads into a command-line interface for the interest of speed, and it either lets you continue doing work on the command line or gives you information on how to boot into a graphical environment. This environment is fast and gives you some extra graphical tools to do your recovery tasks. All in all, that’s a great start. It’s got plenty Q The online manuals offer great guides of applications to help you with data recovery, including ddrescue that we mentioned earlier, which including partition editors and backup utilities, do. You can quickly work from the command line work mostly on the command line. However, the making it multifunctional as a recovery and if you need to, or boot into a graphical environment SystemRescueCD website has some great guides maintenance distro. You can install it to a USB stick if that’s how you prefer to work, and being able to to get you going on that. This ‘manual’ is set as the from inside the distro with its own tool, or do it via a easily clone itself from within the distro is incredibly homepage of the included web browser (Midori, to Linux distro with a script included with the ISO. You useful as well. keep it fast and within Xfce) so as long as you have can even build it yourself, customising it along the Having a stick or a CD lying around with an internet connection, you can easily access it. way. It’s a well-executed concept that does just SystemRescueCD on it could easily save your There are other maintenance apps as well, about everything we’d want a distro of this kind to computer one day.
SCORES
A huge number of boot options available through a straightforward menu 10 Boot
Multiple recovery tools that can be used in Recovery various ways 9
A lot of command- line tools, but they’re thoroughly explained in 9 Ease of use the manual
Other system maintenance tools round out the selection 9 Extras of apps
SystemRescueCD is a fantastic tool for performing recovery tasks on your system, or doing some general 9 Overall maintenance Q There’s a wealth of boot options for different systems
Linux & Open Source Genius Guide 183 Essential software
Video editors The best tools for creating and editing clips and fi lms
A simple to use video-editing software, but tricky to PiTiVi make more complex edits
PiTiVi is a tool that the developers say is we tried to place audio or visual effects onto the clips, Otherwise, the problems with the effects make it easy enough for anyone to use, and fi rst either not adding them properly or then creating diffi cult to use for anything other than the simplest impressions of this video editing software graphical glitches from the mouse cursor until the editing tasks. are that it has a very straightforward software was closed. Effects have very limited interface that limits the on-screen functionality in customisation as well, and are applied to the entire favour of being clean and understandable. The basics track rather than just a short selection. are present: timeline, some cutting and selection It’s quite a shame, as there are some nice, albeit tools, a preview pane, with options and such fi lling up slightly random features available in PiTiVi. Firstly, the rest of the space. you can change any clip into a sequence of images Sadly, this slightly simplistic presentation limits – great if you want to create an animated GIF of the amount of functionality of PiTiVi, making more something, or edit specifi c frames. than just simple cuts a chore or impossible in general. There’s also a feature to try to join clips using Sections cannot be separated from the main clip – their audio as a guide, and while this requires you to instead, the latter can only be cut, copied or deleted install an extra Python component, it’s an interesting and then placed somewhere else. There’s only one addition that can help string together either broken- video or audio track, making it diffi cult to mix together up clips or dialogue scenes. audio without editing it in a separate audio editor, Generally, though, PiTiVi is too simplistic for its which sort of defeats the point. It’s also a little buggy, own good, even if it has a fairly decent import and with the interface regularly getting confused when export ability thanks to GStreamer. Q PiTiVi has a fairly simple interface
SCORES
PiTiVi is available in most major repos, and also Installation requires GStreamer 8
A little bit too simple, making it not as useful as the other editors in 7 Ease of use this test
There are a few effects, and the image sequence tool is nice, but it’s lacking 6 Features in general
Thanks to a GStreamer base, it has decent Codec decoding for a variety of 8 support video codecs
A bit more basic than we had hoped for, and a little buggy, but you can make some decent videos 6 Q There are a fair number of rendering options Overall with PiTiVi
184 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
OpenShot An all- encompassing video-editing tool Built upon its own media framework, OpenShot is a fairly all-encompassing package that has similar codec support to FFmpeg. It’s one of the newer entries in this group test, with its fi rst release just over four years old now, and it’s defi nitely one of the better editing suites we’ve come across. Like PiTiVi, the interface is nice and straightforward; however, unlike PiTiVi, this interface makes it easier to access a much deeper library of features and editing options. The timeline consists of a track hierarchy, with higher tracks generally having dominance over the lower tracks in terms of what is previewed and encoded. There’s no differentiation between music and video tracks, but it’s smart enough to know not to cut off video if an audio track is placed Q The layout is clean yet hides a lot of depth higher than a video track. These tracks can be moved between by the use of transitions, which are to the entirety of the clip in the timeline rather than compatibility with a wide range of codecs: it’s able to visually represented very nicely with an arrow to a selected area. In this case you’ll need to split up import from MKV containers as well as lots of other determine the direction of play, bridging the tracks tracks to break up the effects. You can split off the fi le formats and codec types. together to give a nice sense of the fl ow of the video audio from a video track as well if you so wish. OpenShot’s slightly more advanced workfl ow itself. There are plenty of transitions available, There are a fair number of rendering options, with over PiTiVi is a fantastic yet still easy-to-understand ranging from dissolves, wipes and fades to either plenty of presets and more advanced customisation addition, which allows people to easily make much emulating a specifi c style or even creating some available for bitrate, fi le format, aspect ratio, more advanced video clips and movies. Overall, impromptu special effects. There are some visual quality and more, so you can create your perfect OpenShot manages to combine ease of use with a and audio effects available as well, which are added video fi le. This goes hand in hand with OpenShot’s good feature set.
SCORES
Widely available in repos, and has its own Installation media framework 9
One of the easiest editors to use in this test, without dumbing down 9 Ease of use the features
A great selection of video editing tools and tricks, as well as effects 8 Features and transitions
We found few issues Codec importing video, and there support were many export options 8
A prosumer-level video editor that is only slightly more difficult to use than Overall your basic video editor 8 Q There are a great selection of effects and transitions that are fully customisable
Linux & Open Source Genius Guide 185 Essential software
Kdenlive A video editor suitable for prosumers Whereas OpenShot is a straightforward video editor with a deceptive amount of depth and large number of features, Kdenlive could prove a little more intimidating. With a more utilitarian interface and workfl ow, Kdenlive is deceptive in its design, however, and generally just as easy to use as OpenShot. The KDE video editor has been updated in recent years to work a lot better on other desktop environments and is also based on the popular FFmpeg media framework, giving it a lot of compatibility with various containers, formats and codecs. Like OpenShot, Kdenlive works on a hierarchy of separate tracks, with higher tracks having priority, and transitions being used to go between the clips up and down the tree. However, in Kdenlive there is a differentiation between video and audio Q The multi-track interface is very effective tracks, allowing you to perform some audio- centric manipulation to the tracks and clips in noise tracks, countdowns and clicks among many Kdenlive really is a very complete package, your timeline. Effects and transitions are generally others – it’s a pretty advanced piece of software. with a great amount of attention to detail to a lot easier to get to as well, with the right-click menu There’s also the ability to create a video DVD, taking of its core and extended features that make sure bringing up a list of both to access. These effects already DVD/MPEG2 rendered material either it does just about everything you’d want a video can also be heavily customised to your exact liking. from Kdenlive or another DVD to create your own. editor to do. There’s a whole host of other video editing Rendering itself is fairly quick, with a queue so you With everything highly customisable, including features in Kdenlive to use as well, with a can have multiple things ready to go at once, or the presets and encoding profi les, Kdenlive is a stabilisation algorithm and the ability to create even transcode other fi les to use elsewhere. fantastic prosumer-level video editor.
SCORES
Built on FFmpeg and Installation easily obtainable 8 There is some sacrifi ce to usability for the sake Ease of use of features 8
A wealth of options to edit together any video project Features 10 Due to the FFmpeg Codec core, its video support support extends far 9
With a little more to offer than most, and decent compromise on the interface for it, Kdenlive is one of the best Linux video 9 Overall editors around Q Video exporting is highly customisable
186 Linux & Open Source Genius Guide Linux & Open Source Genius Guide
Cinelerra An out-dated video- editing tool for professionals Cinelerra is an interesting program. Having previously won awards for its video editing ability, it was a bit confusing to fi nd that it was not readily available in all major distros. It’s promoted as an incredible advanced video editor, with a lot of the tools used in more professional scenarios, including a compositing engine and the ability to work in both RGBA and YUV colour spaces. The website offers links to binaries and PPAs so you can install it on your system, albeit only for a small selection of distros. The interface for Cinelerra really needs an overhaul. With a lot of low-resolution artwork and bad colour choice, it can be a little jarring at fi rst. At the very least, it’s comprised of many smaller windows for the timeline, compositor, preview, options etc, with a couple more windows you can add and arrange as you please. The timeline has the same layout as we’ve seen in OpenShot and Kdenlive, and you can also now select sections of track to manipulate. There are of course effects Q The layout is configurable and complex and transitions; however, there doesn’t seem to be quite the amount that comes with some other not useful to everyone. This same limitation is there attention to its display and import features shows. editors in this test. for encoding as well. The software also seems very While it may have been a great professional tool Perhaps due to its status as more professional slow, chugging along trying to preview video, and in the past, it’s far past its prime now, meaning software, it is unable to import a fair few fi le formats would grind to a halt when encoding a fairly simple more professional video makers will have to look we tried to throw at it, such as MKV, MOV and some video that hadn’t caused much issue to the other elsewhere, and others will have to stick to the AVIs. These are not uncommon fi le formats and video editors. Cinelerra has only received minor prosumer-level applications also featured in this while it will handle more RAW style formats, that’s updates in the past few years, and this lack of group test.
SCORES
The software is not well maintained, and not Installation readily available 6
Cinelerra forgoes usability in favour of offering a Ease of use professional experience 5
Huge variety of features, although there isn’t much Features depth in some of them 10
Very few fi le types can be Codec imported and exported support 3 Cinelerra may once have been very good, but right Overall now it’s slow and unwieldy Q There are limited presets, but everything is customisable 7
Linux & Open Source Genius Guide 187 Essential software
Photo managers Discover the very best photo-managing tool on Linux
This popular photo-managing tool covers all Shotwell the basics, but you might want more
Shotwell is the one application on operations such as tagging, placing in events, and great function that allows you to publish photos our test that is used by default in the using the ‘enhance’ button to do some autocorrection online, though – this includes posting to Facebook likes of Ubuntu or Fedora. It’s popular on lighting and colouring – it won’t instantly make or Flickr accounts, and is a good way to sort out all in GNOME-based distributions and things look amazing, but it’s subtle enough that it your photos before uploading them. other GTK desktop environments, although of doesn’t overdo anything. course it works fi ne in KDE and the like. It’s a Events are treated like albums in the interface and straightforward photo manager that requires little you can only assign one to a photo, while multiple tags confi guration and is fairly lightweight to begin with, can be assigned. This is enough to differentiate their much like the other Yorba applications. usage, as you might have hundreds of tags but only a When we say the interface is straightforward, few events. what we mean is that there’s only a simple Image editing in Shotwell is also very basic, thumbnail view of your pictures. While you can allowing you normal things such as cropping and increase the size of the thumbnails in this view, this rotating, along with red-eye removal and a manual of course reduces the amount of images you can colour balancing tool. That’s about it, though, and if see at once, and is not a true preview like you would you want to do some other effects or manipulations get with something like a fi lmstrip view – that you’ll need to fi re up the GIMP. kind of view is great for quickly scanning through Shotwell is a very serviceable photo manager, pictures for either yourself or showing off to others, doing the semi-basics well enough so that while it so its absence is a bit odd. The interface does might be easier to use other apps in this test, you’ll Q There’s only a thumbnail view for photos have its perks, though, allowing you do a few batch be able to get the job done. It does come with a SCORES
A lot of dependencies required; however, it’s a default application on 8 Installation some distros
A fi lmstrip view would be nice, but otherwise it’s Ease of use very straightforward 9
The most basic tools to crop and rotate images, along with a magic 4 Editing enhance button
Quite basic; however, the online publishing feature stands out among all the 6 Features other photo managers
A decent photo manager that can do all the basic tasks you’d want from such an application, but 7 Overall not much more Q Publishing online is a great feature
188 Linux & Open Source Genius Guide Linux & Open Source Genius Guide gThumb A straight-forward photo manager to rival Shotwell We were actually a little surprised by gThumb, especially as it’s a GNOME- made application. With the recent move to simplifying all parts of GNOME, we were wondering whether its popularity was misplaced; however, it looks like all the changes to the GNOME Shell haven’t affected the way gThumb works. It’s not to say that gThumb a super-advanced, though – it features a fairly straightforward interface that we’d want from a photo manager. By default, it’s a thumbnail view, displaying the Picture folder in the home directory. Instead of importing photos from specifi c albums, you merely navigate Q gThumb lets you navigate the directory structure through the directory structure. This means that instead of having specifi c albums to create, you’ll as anti-blurring, desaturation and negative fi lters. Images that have been edited in this way get a date need to organise those photos yourself in a fi le There isn’t any red-eye removal, though, and no added to them, letting you know when they were last manager or terminal. paintbrushes or anything to do that, meaning you’ll modifi ed by gThumb. Clicking on the Edit fi le button in the top right need to open GIMP for those kind of operations. Like Shotwell, gThumb also has online publishing opens up the image editor, and a sort of limited You also can’t do batch autocorrection for the tools, with similar services such as Facebook, fi lmstrip view as well. You can click between images, but you can do batch tagging. You can drag Twitter, Flickr etc. It’s nice that these tools exist, as photos in the stream on the bottom to edit them a box over the images to select them, or press Ctrl/ these services don’t have touch-up options, allowing individually, although you can’t move between them Shift with click, and you’ll be able to add, remove or for mass uploading of holiday photos and the like. with arrow keys. The editor is a little more advanced assign tags in batches. gThumb is a great little application and while it than what you get with Shotwell, allowing for more You can edit information in batches as well; doesn’t quite have everything we want, it’s defi nitely manual colour and balance corrections, as well however, you’ll likely want to do that individually. a step up from Shotwell.
SCORES
Easy to obtain, but also requires a few Installation GNOME packages 8
Generally very easy to use; however, the edit/fi lmstrip Ease of use view is confusing 8
Decent amount of editing tools, although no Editing red-eye removal 7
While not particularly feature-heavy, it’s got enough to properly 7 Features manage your images
A very strong showing from gThumb, a photo manager with plenty of features, although it could do with some batch image 8 Overall touch-up options Q Plenty of photo information can be changed in gThumb
Linux & Open Source Genius Guide 189 Essential software
F-Spot A little out-dated tool but with some interesting features F-Spot was replaced by Shotwell in Ubuntu a few years ago. However, the application has stuck around, albeit without an update in over two years. This simple image editor actually comes with an interesting variety of features, although they’re not all that useful in the long run. The fi rst time you start it up, you’ll need to import photos from a directory. This is a pretty straightforward process, although it initially looks like you’ll need to change settings on each individual image in selected folders. It quickly imports into the library, displaying a thumbnail view, some basic Q The timeline can help filter some images metadata, a histogram and a sort of timeline for basic fi ltering of your photos. The timeline breaks which you can also select from the top bar, and this you need to completely reset the fi lters to remove are used to show a graph of photos taken per month includes a fi lmstrip view that you can key between any. Tagging is a little weird as well, since you need along the top, allowing you to select each month with arrow keys on the keyboard. The editor is a little to create the tags before you can even apply them from there. simplistic, with some basic autocorrection tools, to anything, even if you have the images you need to You can also fi lter by tags and such in the main although they seem to do very little to the photos we tag already selected. view, as well as adding extra tags and dates to the tried them on. While there are some good parts to F-Spot, in fi lters to fi nd exactly what you need. The main browsing interface can be a little general it’s not as complete or as easy to use as we The thumbnail view is quite basic, and you can confusing as well, especially once you start fi ltering would have liked. The interface has a few ease-of- increase the size of the thumbnails in the view. images. It’s not always clear how you’re supposed use issues that are not encountered with the other Double-clicking a picture enters the edit mode, to get back to the standard view if you want to, and photo managers, too. SCORES
Generally easy to get, although it’s a little older now and requires older 7 Installation dependencies
Basic navigation is great, but can get a little counter-productive with 5 Ease of use more advanced searches
Has a small selection of autocorrection buttons, Editing but they do very little 6
A decent variety of features, but not all of them work quite as well as 6 Features they should
F-Spot is showing its age and is just not as good as the others in this test. A newer update could fix the problems, but that may 6 Overall never come Q The editor and file strip view are quite nice, but could do more
190 Linux & Open Source Genius Guide Linux & Open Source Genius Guide digiKam A fully- comprehensive photo manager The KDE-developed image organiser has recently released the digiKam Software Collection version 4.0.0 which has lots of new features. DigiKam has received a few upgrades over the past few years since it hit version 2.0 and became one of open source’s premier photo managers. It comes with a full range of functions, including the standard preview and fi lmstrip view we’re looking for, batch tag and data editing, editing software, and even a light-table function. Q digiKam has a selection of different views Although built for KDE, digiKam works just fi ne with GTK-based desktop environments and has a Any changes you make during this wizard can of Selecting an image then allows you to enter the more neutral design aesthetic than KDE, allowing course be altered in the main program’s settings and image-editor window or the light-table view. In the it to not stick out on other desktops. It can also be such, and there’s a lot of extra customisation you image editor, there are a lot of functions for adding found in all major package managers thanks to can do to digiKam as well. effects and general image manipulation, although its popularity and ties to KDE, although options to The main window allows you to view your photo it’s mostly hidden away in drop-down menus, compile from source via a tarball or GitHub are just library in a stream, with thumbnails and previews, making it a bit tricky to use. The light table is a great as available. as well as by their geotags if they have any. It’s here feature, allowing you to compare and contrast a Initial startup involves a quick but thorough setup you can edit tags, geotags etc, as well as looking couple of photos in case you need to select from wizard, going through basics such as photo library up metadata and colour maps for the images. multiple images of the same angle and subject. locations, the manner of importing RAW fi les and a The interface aids in this, allowing you to select It’s a really comprehensive piece of software, few more options. While a lot of people can mostly multiple images with the plus symbol that appears offering just about everything we’d want from a ignore these and have a decent default selection, as you hover over thumbnails, although Ctrl and photo manager and more, especially for those that they are useful for people with more specifi c needs. click also still works. need it in a professional capacity.
SCORES
Requires some extra packages from KDE, but Installation easy to install otherwise 8
The main window is easy enough to use, although the image editor is a 7 Ease of use little confusing
Not quite at the level of GIMP, but more than good enough for a 9 Editing photo manager
digiKam is overfl owing with a variety of features to make it easier and 9 Features better to use
A fantastic overall package that offers specialised tools to organise, analyse and edit 9 Overall your photo library Q Light table allows you to compare photos to find which is best
Linux & Open Source Genius Guide 191 Special trial offer Enjoyed this book? Exclusive offer for new
Tr y 3 issues for just £ 5*
* This off er entitles new UK Direct Debit subscribers to receive their fi rst three issues for £5. After these issues, subscribers will then pay £25.15 every six issues. Subscribers can cancel this subscription at any time. New subscriptions will start from the next available issue. Off er code ‘ZGGZIN‘ must be quoted to receive this special subscriptions price. Direct Debit guarantee available on request. ** This is a US subscription off er. The USA issue rate is based on an annual subscription price of £65 for 13 issues, which is equivalent to $102 at the time of writing compared with the newsstand price of $16.99 for 13 issues, being $220.87. Your subscription will start from the next available issue. The magazine for the GNU generation Written for you Linux User is the only magazine dedicated to advanced users, developers & IT professionals In-depth guides & features About Written by grass-roots developers & industry experts 4.5GB DVD every issue the Four of the hottest distros feature every month - mag insert the disc, reboot your PC & test them all! subscribers to…
Try three issues for £5 in the UK* or just $7.85 per issue in the USA** (saving 54% off the newsstand price) For amazing offers please visit www.imaginesubs.co.uk/lud Quote code ZGGZIN Or telephone UK 0844 249 0282 overseas +44 (0) 1795 418 661 Pass our knowledge off as your own
TM Know more with world-leading features and tutorials on everything from WordPress to Photoshop BUY YOUR COPY TODAY Print edition available at www.imagineshop.co.uk Digital edition available at www.greatdigitalmags.com
Available on the following platforms
facebook.com/ImagineBookazines twitter.com/Books_Imagine
VOLUME 5