Michael Bolton in NZ Pinheads, from the Testtoolbox, Kiwi Test Teams & More!

NZTester The Quarterly Magazine for the New Zealand Software Testing Community and Supporters

ISSUE 4 JUL - SEP 2013 FREE

In this issue: Interview with Bryce Day of Catch Testing at ikeGPS Five Behaviours of a Highly Effective Time Lord Tester On the Road Again Hiring Testers Michael Bolton in NZ Pinheads, From the TestToolbox, Kiwi Test Teams & more!

NZTester Magazine

Editor: Geoff Horne [email protected] [email protected] ph. 021 634 900

P O Box 48-018 Blockhouse Bay Auckland 0600 New Zealand www.nztester.co.nz

Advertising Enquiries: [email protected]

Disclaimer:

Articles and advertisements contained in NZTester Magazine are published in good faith and although provided by people who are experts in their fields, NZTester make no guarantees or representations of any kind concerning the accuracy or suitability of the information contained within or the suitability of products and services advertised for any and all specific applications and uses. All such information is provided “as is” and with specific disclaimer of any warranties of merchantability, fitness for purpose, title and/or non-infringement. The opinions and writings of all authors and contributors to NZTester are merely an expression of the author’s own thoughts, knowledge or information that they have gathered for publication. NZTester does not endorse such authors, necessarily agree with opinions and views expressed nor represents that writings are accurate or suitable for any purpose whatsoever. As a reader of this magazine you disclaim and hold NZTester, its employees and agents and Geoff Horne, its owner, editor and publisher, harmless of all content contained in this magazine as to its warranty of merchantability, fitness for purpose, title and/or non-infringement.

No part of this magazine may be reproduced in whole or in part without the express written permission of the publisher .

IN THIS ISSUE…

Click on title...

30 On the Road Again Geoff Horne, NZTester Magazine

6 Interview With Bryce Day, Founder & CEO, Catch Software 34 Hiring Student Testers, Ian Wells, Telogis 14 Testing at ikeGPS

Jeremy Gold, ikeGPS REGULAR FEATURES 18 Five Behaviours of a Highly 4 NZTester Magazine Effective Time Lord Tester Announcements Richard Boustead, Statistics NZ 11 Testing Events 20 The Pinheads 22 Michael Bolton in New Zealand 21 From the TestToolBox Adam Howard with Katrina Edgar & Aaron Hodder 29 Kiwi Test Teams

28 When John Was a Tester Great Bugs takes a break this issue.

The Journal For New Zealand Test Professionals

Welcome to Issue 4 of NZTester Magazine. Richard Boustead on the perils of being a Time Lord Tester! A lot has been happening since our last issue, mainly in the form of conferences, summits and In May, Colin and I put out a combined Newsletter meetups. After the launch of of OZTester in April, in which we wrote about our personal experiences editor Colin Cherry & myself embarked upon a with the various schools of software testing. Our conglomeration of conference attendances which initial idea was that this was to be a "taster" for a encompassed the Cognizant Test Summits, STPCon, more detailed analysis of the subject in the next StarEast, ANZTB 2013 and will also take in KWST, (current) issues of OZTester and NZTester. OZWST, Let’s Test Taster, STANZ, StarWest and However, at this point in time we’re still gathering ignite over the next few months. These “road information and comment and therefore the full trips”enable us to keep abreast with the very latest article is still a work-in-progress. in testing trends and ensures that we continue to Anyway, happy reading and as always, we do establish and maintain a healthy wordwide appreciate any feedback you may have around our network of testing experts and personalities as publications. well as with professional testing practitioners from all over.

In this issue we welcome Ian Wells, Adam Howard, Katrina Edgar, Aaron Hodder and Jeremy Gold to the NZTester writers fold plus we hear again from 3

NZTester Announcements

NZTester Survey NZTester Auckland Meetup

We’d still like to know a bit more about your The inaugural NZTester Auckland Meetup will be thoughts on NZTester Magazine so we can held in August. Details are as follows: continue to shape it according to what you would like to see. Wed, August 14 2013, 4:00pm—6:00pm

Our survey with SurveyMonkey is still active. It’s Venue: mostly multiple choice questions which should Catch Software take no more than 4-5 minutes to complete. Level 3, 300 Queen St Auckland We’d certainly appreciate you taking the time out to complete this survey, we’ll hold the results over Agenda: until the next edition of NZTester (October). 4:00pm Meet and greet 4:30pm Presentation: Geoff Horne, QA Click on the logo below to go straight to the Transformation - the Journey Continues survey. 5:30pm NZTester Magazine discussion

To sign up to the NZTester Meetup Group and register, click on the logo above.

To register for the session without signing up through Meetup, click here. VOTE…. for your favourite article in this edition of NZTester Magazine. Next issue……! Send in your vote for the article you think is the best in this issue and the winning author will receive a $100 JBHiFi voucher.

Send to [email protected]

Last issue winner was Nele Nikovic of TradeMe for his Project vs Product Testing article.

Well done Nele, a $100 JBHiFi voucher will be on its way to you shortly.

Announcements cont.

Over the Tazzie, in the latest issue of…. NZTester Directories

Later in the year, we will be looking to publish three testing industry directories:

 Interview with Brendan Fitzgerald of TestIT 1) Testing Companies - New Zealand  The Joolery Diaries: An Aussie Abroad 2) Testing Companies - Australia  ANZTB Conference & SIG reviews 3) Testing Tools - management, automation,  Confessions of a Test Strategist code coverage etc.

 Starting a New Role as a Test Manager We encourage companies wishing to appear  Authority Bias: The Bane of Testing in either of the Testing Companies directories

Click on the cover to download. to register interest with us below.

There will be no charge for placing an entry

and full advertising opportunities will be available at our standard advertising rates.

The Tools directory will be product-based and researched by NZTester personnel.

Planit TestingI ndex

Now in its seventh year, the Planit Testing Index is an industry-wide benchmark for software development projects from across Australia and New Zealand. It provides a solid foundation for strategic planning, budgeting and execution of quality assurance activities in IT. Our commentary for last year’s Index can be found in NZTester 2.

The 2013 Index will represent over 10,000 software development projects from Australia and New Zealand. To participate, click on the graphic

5 This issue’s interview is:

Bryce Day

Founder & CEO, Catch Software, Auckland

Our interview this issue is with Bryce Day, Founder The first with Atlassian, has developed numerous and CEO of Catch Software. Catch is one of New tools including JIRA (defect management) and Zealand’s success stories with its flagship product, Confluence (enterprise wiki). Secondly with Sparx Enterprise Tester, being used by such organisations Systems, creators of Enterprise Architect (CASE as PayPal, US Navy and eBay among others. tool for analysis, design, and architecture). We have partnered with both companies for about seven years and run Sparx Systems New Zealand, which NZTester: Can you please describe Catch? gives us responsibility for Enterprise Architect in New Zealand and global responsibility as one of Catch is a Kiwi software company, that has been in seven sister companies. We are also one of twenty- operation for nine years. Our aim is to develop the five global Atlassian Enterprise Experts that work world's best test management platform and for us with customers to train, consult, and implement that's not just about the software, but the services Atlassian products globally and are recognised to and support that wrap around it. We have an have the expertise to do this at an enterprise level. extremely high-performing team, that is passionate about delivering better software for test teams globally, and a solutions team that provides training, NZTester: What product and services does consulting services, and support to our customers Catch offer? to ensure that they have everything they need to be Our product offerings centre around Enterprise successful. While we're based in New Zealand, over Tester, but we also sell Enterprise Architect and 90% of our software sales occur overseas, with the associated plugins, Atlassian software (JIRA, majority being in North America to the likes of the Confluence, Team Calendars, etc.), and numerous Intel, PayPal, US Navy, LG, eBay, Stanford University, plugins for these products. On the solutions side of Scotiabank and Philips. Closer to home organisations the business, we offer training, mentoring, and like Air New Zealand, Orion Health, eRoad, consulting around our core products; Enterprise University of Auckland, Navman, and APN are using Tester, Enterprise Architect, and the Atlassian our test management software, Enterprise Tester, software. This includes both small pieces of work to ensure they are producing high quality products. around the software and end-to-end Enterprise Tester is not only a great test implementations across an organisation using both management tool, but also allows organisations traditional waterfall approaches and agile practices. to create an end-to-end integrated tool chain by integrating with our partner products; Atlassian NZTester: What do you believe makes JIRA, Rally, Sparx Systems Enterprise Architect, Catch different? and Microsoft TFS. With this in mind we aim to provide a total product experience for our customers From a soft skills point of view we are passionate and as such provide training, mentoring, consulting, about what we do, have worked hard to put in place support along with the product to ensure they have a great culture, are globally focused, and we care everything they need to be successful. about our customers. From more of an analytical stand point, Enterprise Tester sales have doubled In terms of partnerships, we recognised early on a year-on-year since we released back in 2008 and our commonality with two great Australian companies. 6

customers include some of the biggest companies in more than manually running tests for them. Is it in a the world, so we must be doing something right. pre-packaged set of tools, processes, and templates? Strategic leadership for test? or other things? Finally, the fact that we are based in New Zealand and have been marketing, selling, and supporting Commoditisation - Growing Kiwi organisations to our customers from here is a differentiator in the think of testing as more than just the last chance to software testing market. We are proud to be a Kiwi catch bugs is a challenge. Testing is a strategic company and international customers love working investment that can differentiate businesses from with us, which is great! others in their industry, provide cost reductions and faster speed to market. Testing isn't a commodity and testing responsibility can't just be abdicated to NZTester: What do you think makes a Test outsource partners, it's an integral part of the Manager or Analyst come to work for Catch? development cycle. I didn't want to put words into the mouth of our team, so I asked them why they enjoy working here. Skilled Resource - It's easy to find a person to run Here's what they said: tests. It's much harder to find skilled testers that fully understand how testing works and the key metrics  I really enjoy testing - I've been involved with and techniques that can and should be used to make commercial software development for a few your testing successful. That's not to say running years and, although there's lots of fun stuff tests is not valuable but more about running the right to do in this field, testing is the activity I test and knowing what the right test is, as apposed to enjoy most. just running a test to run the test.  In every company, you occasionally need to do things that are outside the tight focus of 'the NZTester: Where do you believe NZ’s approach to thing you love doing'. That's true here too, but testing is going well? the trick is that even things that are not strictly testing are inevitably linked to testing, because New Zealand is an innovative nation and I think that's what our core product is all about. that carries through into everything we do, including how we test and how quick we are to pick up new  I like recursion. I get to test the tool that testers testing techniques and try new approaches. We also use for testing other tools. Neat. have some fantastic testing conferences and groups  I really like making tools. Here I get the support (NZTester included) that are active and support the I need to get on with making little tools (think testing community well. Because we are a small Ruby scripts and batch files) that help me prod community ideas are relatively easily discussed and and poke at Enterprise Tester in new and debated across the community, and with our limited unusual ways. resources we tend to be good at trying different approaches. Sometimes they work and sometimes  I really like learning new things. That they don't, but either way the help the community happens on a daily basis here and it's learn and grow. extremely stimulating (or that could just be the caffeine talking). NZTester: Where do you believe NZ’s approach  The people are awesome - that should could improve? probably be number one! New Zealand, in comparison with the rest of the world, is very small. Techniques like large NZTester: Where do you believe the challenges automation frameworks, that work in large global for NZ testing companies lay? organisations, won't work here. We just don't have Differentiation - New Zealand is such a small the scale and man power to successfully implement country and historically we supplied test resources. and maintain these types of heavy testing The challenge is how do you supply more than just frameworks. I believe New Zealand testers need to resource so that we add value to our customers in focus on limiting automated frameworks to being

used on very stable parts of an application, to coverage automatically. This meant that if we got the minimise the maintenance cost, while managing requirements and use cases / stories correct in the quality well and continuing to innovate with analysis phase we could save significant time in the manual testing. testing phase and have fewer translation issues within the process (faster and better!).

NZTester: Do you believe that the overall As part of the initial development, we worked with standard of testing in NZ is improving? Auckland University and produced the a white paper Overall yes, but there are still a lot of small teams and on the Enterprise Tester project: start-ups that don't understand the value of testing their products to ensure it's high quality. There are http://bit.ly/12JAXDc shocking statistics around customer quit rates, that show exponential quit rates based on the number of NZTester: In a cluttered market could it have bugs found when they first use a product. been seen as a me-too exercise, what do you think are its differentiators? NZTester: Where do you believe the next This question fascinates me. Is testing really a initiatives in testing lay? What’s coming next? In cluttered market place for tooling? HP, IBM and NZ? Internationally? Microfocus still have something in the region of If you ask me for a "wow factor" next initiative then 90-95% of the traditional market share and the rest I'd say mobile testing and automation frameworks. of the tooling vendors make up the other 5% or so. Though personally these sound great, but are hard to scale and in a small team are costly to maintain. So, looking at the test management software market Practically, I believe there are more gains to be we see that the top three tools, by market share, are made through organisations throwing agile testing large tools, with little or no innovation, and old techniques into the mix and doing the basics better technology which take a silo'd approach to test i.e. actually managing their testing, being able to see management information. In addition, they all use coverage and traceability across their development enterprise pricing levels that limit test tool usage to life cycle, and being able to better reuse what they large organisations. Of the other tools in the market, already have. there are numerous ones that can be found, but if you look closer you'll find that they are often developed by individuals not companies, with the NZTester: How did Enterprise Tester inherent challenges that come with that; support come about? response times, frequency of releases, technology, etc. With Enterprise Tester we have worked to Catch started life as a pure services business. We counter a lot of those challenges. grew rapidly and had a lot of success, being one of the fastest growing companies in the Asia Pacific  We have a full product team working on region for three years running. features and the product roadmap.

As we grew though, we found that inefficiencies in  Reporting of features requests and bugs can projects became more and more costly as the scale of be done within Enterprise Tester and you'll the projects grew. We knew the inefficiencies lay in receive a response within 24 hours of it the hand-off between the business analysis and being logged. testing teams and already had an analysis and design tool (Enterprise Architect), so went to market for a  New versions are released every 10-12 weeks testing tool. We couldn't find anything that met our and are always striving to enhance Enterprise needs, so decided to build it ourselves. Our aim Tester, and test management in general, so initially was to provide a test management tool that that not just traditional testing techniques integrated with Enterprise Architect and auto- are catered for but also newer more agile generated test cases from the use cases ("user techniques. stories" for the agile crowd) and provide the

 Enterprise Tester uses enterprise technologies It was1999 and I was working in the UK for one of - it uses Oracle, MS SQL, MY SQL or postgres the major banks. The bank decided to completely database technologies and work on any replace their main banking system and decided that browser, not just IE like other tools. the best branch to start with was their branch in Budapest, Hungary. I was brought on to be part of  We know we can scale. Independent a three person test management team, based out of performance testing from LoadVerify showed London. This role saw me and another team Enterprise Tester to be "the most stable error members taking turns flying to Budapest, so every free platform that [they] have load tested since fortnight I was on a flight across. 2008" and we support large customers; Paypal (2,500 testers) and another organization with Like any good test management team, we decided 15,000 users that we cannot disclose, to name we needed a tool to manage all the test cases and a couple. keep track of the test stages and results. We chose the Rational toolset and with help from the System In addition to these points we are an open platform, Administration team had it set-up and the client which means you chose the tools you like and we'll installed within a couple of days, we thought… play nicely with them, our pricing makes it accessible but two weeks later we still didn’t have a working to all teams (we start at $10 for a single user), and we system. As it turned out, after much wailing and integrate with requirements management and defect many things getting "lost in translation" we finally management software to provide customers with a found that the Budapest office had a login script fully-integrated tool chain from analysis through to that set all client machines up with the standard development, testing, and release management. network drives and mapped the standard MS SQL ODBC connections up. This meant that every time

NZTester: Do you have a testing horror story we logged off, all of our connection strings were to share? overwritten, and configuration files were updated - ya!

Now we had testing software sorted we had to find our environments were different and code was concentrate on the testing team - a team of fifty that being deployed directly into our production didn’t speak English, so we had two English-speaking environment, so tests that were passing in TEST staff that could translate for us. Little did we know failed in PROD, while tests passing in PROD failed in that our translators didn’t like our approach, so were TEST. To top it all off, I flew over on a Sunday night, instead instructing the test team to use a different as I would normally, got into the Budapest office and approach. This meant that bugs that were found and was working away reviewing the prior weeks testing deemed minor were never logged (since they just and pulling together the management reports for the weren’t worth worrying about...grr!) Once we found standard Monday afternoon meeting when, at about this out, there we a number of heated discussions midday, one of the system administrators walked with our interpreters until they finally understood passed and casually said “You do realize that the that they were jeopardizing the whole QA program project got shutdown Friday, don’t you?” by not following our instructions. Yep, on ringing the London office to verify this I On other fronts we were doing just as well. We had received the response “Oh, we forgot to let you know. the lead development team based out of the US and Just finish up and get on the first plane tomorrow Canada, a secondary development team based out and come back". of India, our servers were located in Germany, but the system administrators were based in Greece, we had auditors from Australia, operations were based Thanks Bryce. It is good to see another NZ software in Hungary, and the management team based out of product carving out a name for itself in overseas London. With all this communication across cultures, markets- Ed. timezones and languages it wasn’t a huge surprise to

Click to link to NelsonHall’s Software Testing Analysis site

Testing Events

Sydney 5 Aug 2013

September 23-25 Arlington, VA

October 14-15 Sydney

Testing at ikeGPS

by Jeremy Gold

What is ikeGPS? ikeGPS combines a complex range with a bit of trigonometry (sorry build a development team that’s of sensors with a mobile computer Mr Langston, you were right, it inspired by the motto, 'Getting into a rugged, integrated solution is useful in real life) to perform paid to make cool s**t that’s a that can be used to remotely further measurements from the pleasure to build and sell with the measure items with only a few image itself, such as the height of best team you’ve ever worked in'. button clicks. items on the object being (Try explaining that laser cut photographed. plaque on your desk when you’re I won’t describe all of the trying to teach your seven year old measurement capabilities here Who is ikeGPS? that swearing is inappropriate). (that’s what www.ikegps.com is However, we take our products ikeGPS, the company, is a global for), but for example a typical very seriously – we understand team of dedicated staff striving to ikeGPS measurement involves that when one of our customers help people work smarter and the user aiming the camera at the drives several hours to a site to faster by providing elegant object of interest and pressing the use our device, it has to work. So solutions on how to measure 'Go' button, at which time the testing is extremely important to things remotely and capture geo- software will: us. But, before I delve into that, a spatial data efficiently and safely. brief diversion into scrum ...  Determine the location of We are also very proud to have the device using the GPS won the Innovative Hi-Tech Crouch, Touch, Set!  Measure the range to the Hardware Product award at target using the laser the 2013 New Zealand Hi-Tech Here at ikeGPS, we’ve been distance meter Awards! practicing scrum for about a year  Use the compass and now and I’m still absolutely in my inclinometer to measure the Since we’re still a fairly small honeymoon phase with the azimuth and inclination of company, we get to make up our process. Scrum has really helped the device own rules, so I’m working hard to us to focus on the highest value  Request an image from the 5 MP imaging sensor  Calculate the position of the object of interest  Save the resulting image along with a synchronised recording of the raw instrument readings and the calculated target location.

Also, since we know the angle of view of the calibrated camera and the laser range and inclination to the object of interest, this information can be used along

development work for the ideally each user story will be -at-the-last-minute-with-a-critical- company and through it we’ve completed according to our bug stories. Then the team will developed a strong disciplined agreed 'Definition of Done' from quickly identify testing as a approach to the craft of software highest priority story to lowest bottleneck during our regular development that empowers the and at any time we could release retrospectives and generate team to build great software. the software including the stories SMART goals to address completed so far. But in reality, this problem. However, as the trainer on my we are very tempted to batch up Scrum Master course reminded The solution that the team has our stories and then do the build us many times, ‘Scrum is Hard come up with over the past year and test at the end to minimise and Disruptive’ – it sends ripples or so involves a custom test the overhead (the dreaded mini throughout the company and system that checks out the latest waterfall). holds up all of your thorniest code from version control every problems right in your face and However, once you’ve had a few night, builds the release, deploys says, 'This is slowing you down – sprints where the testing shows across to two separate hardware what are you gonna do about it?'. up a critical bug right at the last configurations and runs all of the minute, it becomes painfully automated tests. An email For us, one of the biggest pain obvious that you need to test and summary is sent to each member points that scrum highlighted release at the end of each story. of the team and the PASS/FAIL was testing. Every release, we I take on the product owner role lights on the wall are illuminated have a stack of manual test for our team and I would much appropriately. One of the procedures that need to be rather take the hit on velocity and interesting points to note about performed and these take several have three fully-complete-tested- the system is the use of Arduinos days for the team to complete. and-ready-to-ship stories than six and some basic electronics to Throughout our fortnightly sprint, nearly-complete-but-failed-testing allow the control PC to power the

device under test on and off as needed. Also, thanks to an Internship grant from MBIE, we have recently integrated Borland SilkMobile into the system for remote user interface testing as part of the nightly tests.

As much progress as we have made on automating the testing and simulating our instruments, at the end of the day we still need to do plenty of manual system testing – someone needs to stand outside and use the ikeGPS to validate that we can receive tiny radio signals from a mesh of GPS satellites orbiting at 20,000 km above the earth and correctly calculate our location on the earth; fire a laser pulse at the speed of light to measure the time taken to receive the reflection from an object and ensure that we can correctly calculate the distance; use a precisely calibrated configuration of magnetometers and accelerometers to measure the direction of the earth’s tiny magnetic field to an accuracy of one degree; simultaneously capture a 5 MP image and process it through an embedded Linux multimedia processor; and finally summarise all of this information in a Google Earth KML file with Placemarks at the correct position. And that, even if I do say so myself, is some pretty cool s**t.

I want one! - Ed.

Jeremy Gold is a Software Architect at ikeGPS in Wellington. He can be contacted at [email protected]

Five Behaviours of a Highly Effective Time Lord Tester by Richard Boustead

The Doctor, last Time Lord of built to do isn’t the only use you has companions accompanying Gallifrey graces our TV screens can actually put them to. him on his travels. More often each year, and with a wave of his than not some observation of 2. Exploration sonic screwdriver banishes the theirs will point out a mistake or monster of the week threatening At the beginning or end of an alert him to a danger. Be it a huge to destroy Earth, delete all adventure, and sometimes both, number of testing tasks or a humans, or exterminate all life. the Doctor gives a grin, flips a troublesome Invasion of the Bugs, Such superheroic thwarting is lever on the TARDIS console and testers can turn to their colleagues often beyond the concerns of a vanishes into a random place in and ask for assistance. Problems simple tester hunched over a time and space. He explores, goes can often be solved easier, faster keyboard, but there are five off the beaten track and invariably and better if you get someone else behaviours of the Time Lord comes across some problem, to take a look as well. Like The that testers should take note of. disaster or distress. Testers Doctor, we are very good at verbal shouldn’t be afraid to go and processing, so just the act of 1. Experimentation explore. Exploratory testing may explaining a problem aloud can be scoffed at by those who prefer cause a realisation of where we The Doctor has a vast suite of the tried and true planned haven’t quite got something right. tools at his disposal – from the method. But once in a while, don’t ubiquitous screwdriver to rarer 4. Time Travel be afraid to let the TARDIS drive, one-shot scanners, locators and and go look at somewhere you Well that’s silly – we can’t actually devices, all bundled away in the normally wouldn’t go. Because travel in time. But what we can do TARDIS. That said, he often those jumps to the unknown are is take lessons learned from our resorts to kludging a solution often where you can find the projects and incorporate those together with whatever is on really interesting problems. suggestions into future projects hand. Testers also strive to or use them to understand past explore the possibilities of 3. Companions errors. Documentation may be whatever tools they are using – the scourge of the universe, but it from Excel spread-sheets to As Exposition excuses, audience also tends to be very helpful when full Quality Testing Programs. avatars or just the occasional examining the whys and Sometimes what they’ve been badass, the Doctor almost always

and wherefores of programming there is always one enemy who decisions made years previously. escapes to cause more problems Scripts can be invaluable in later on. The enemy of the week picking out pitfalls and traps usually gives an ultimatum and around a new piece of The Doctor has to let them get functionality in an old system, away in order to save people. But or even just to identify a blind for us testers, when we see a tiny spot that newer test suites may bug hiding out in some corner of not be covering sufficiently. the specification document, we should see it as the last Dalek. Let 5. Recurring Villains it go and it will return with an Richard Boustead is a Test Let’s face it. This is one example army at its beck and call. Analyst at Statistics New Exterminate it as early as possible. where it’s best to not copy The Zealand in Wellington. He can Doctor. Be it Cybermen, Daleks, Bad drama but good practice. be contacted at Autons or The Master himself, [email protected]

The Pinheads

Trying out session-based testing....

OK, well try a Um....OK negative value then...

swear words

more swear words

What type of testing shall we call that then?

From the TestToolBox

by NZTester Staff Writer

I’ve come across quite a number of useful utility- type toolsets in the past however none as useful as Snagit from TechSmith. While there are many screenshot capture products out there both commercial and freeware, the $50 I spent on Snagit Microsoft Windows and while overall probably is without a doubt one of the best investments in more flexible than Snagit, it is nowhere near as software I have ever made. straight forward & easy to use. When doing things on the fly, Snagit enables very quick editing without Snagit is a screenshot capture tool however is so some of the clumsiness of Paint and others. In more much more than that. As well as simple screenshots, recent times I’ve had a few folk tell me that the scrolling screens, regions, menus, objects, videos, Snipping Tool that now also ships with Windows fixed regions and a whole lot more can be captured does pretty much the same thing however once and edited. Along with the usual range of drawing I show what Snagit can do, they’re usually lining and commenting facilities, Snagit also provides the up with their fifty. facility to overlay, expand/reduce, filter, partial capture, colour adjustment and many more tools I have to say Snagit is particularly useful when for playing around with your content. I find it producing these magazines as time is of the essence particularly useful when I don’t want to include a between receiving articles, having them proof read full screenshot as these can take up valuable disk and getting each edition onto the drawing board for

space & email bandwidth (especially when sent publishing. Author’s photos and other supporting to all & sundry). I can trim down, cut out, custom imagery is easily captured & edited without reading select any part of the image I want plus save it if files or saving them. Simply capture from a need be in a myriad of different formats. I find it screenshot, edit on the go then simply copy & paste great as a tool to quickly update or amend an image into the magazine layout. Easy! or any other screenshot with the minimum amount of fuss and hassle. Snagit is also very useful for demonstrating & providing a step-by-step inventory of testing where Prior to discovering Snagit, like a lot of folk I simply each action a test analyst has taken can be added used the Paint application that comes with and highlighted on a single screenshot (where

relevant). It can save a lot of time documenting in resolution, none of them match Snagit for speed Word, Excel or your favourite testing tool when you and ease of use. can add these to an application screenshot and No doubt there are other tools out there like Snagit simply attach it to the defect log. It also adds the and I’m sure everyone has their favourite. However benefit of the developer seeing from the screenshot how the application behaved as opposed to mere if you find yourself looking for something better dialogue included on a defect log. than you have, I suggest it would certainly be worthwhile checking Snagit out. Like me, you Lastly, I find that Snagit’s flexibility covers just may find the $50 well spent. about anything I would want to do with screenshot captures, short of providing some of the more Click on the TechSmith logo for more details on Snagit. complex editing facilities found in commercial image manipulation tools. While I still occasionally use these, especially if an image is at a lower

Guess who’s the tester….

contributed by Melissa Neale, Test Analyst, TradeMe

Michael Bolton in NZ by Adam Howard with Katrina Edgar and Aaron Hodder

In April, Assurity Consulting hosted improving how we can go about This is because they can’t Michael Bolton, industry leader in testing software. encapsulate all the factors that must context-driven testing and co-creator inevitably inform such a call. (alongside James Bach) of the Rapid The major theme that runs through Software Testing methodology. the ideas he presents is that testing This awareness of the fallibility of is becoming rather too much like a testing is another theme that runs During his week in Wellington, game – that we spend too much time through the Rapid Software Testing Michael taught the Rapid Software keeping score and competing against course. Bolton stresses that the goal Testing course and a one-day session one another. of testing is only ever to find as many on Critical Thinking Skills for Testers. bugs as possible in the time available He also presented at the Test Testing, Bolton argues, ought not to and aims to maximise such returns – Professionals’ Network forum and be judged by the number of test cases but in the knowledge that exhaustive WeTest workshop. we run or the number of bugs we testing will never be achievable. find. The goal of testing should only And so he proposes that we make the Aaron Hodder, Adam Howard and ever be to find as many problems as best possible use of our time and that Katrina Edgar share their insights possible in the time available. when we are engaged to test a of these events presented by such a product, we should spend as much prominent figure in the testing One of the key distinctions here is the time as possible doing precisely that. community. difference between a problem and a bug. Bolton suggests that quality is This means that he argues for less Rapid Software Testing “value to some person who matters, emphasis on creating lengthy and (15–17 April) at some time” and that a bug is exhaustive test plans and reporting – something that threatens this value, although by no means suggests these I’ve heard Michael Bolton described according to the person who matters. are banished altogether. In particular, in many different ways – ‘testing he disputes the sense of investing our superstar’, the oft-cited ‘testing In that context though, testers don’t precious time in creating test scripts. specialist’, even ‘that singer with the matter so we cannot truly identify funny hair’. But having been lucky bugs. Problems may only potentially Bolton argues that a test script can enough to attend his Rapid Software be bugs if someone who matters – cause unintentional blindness in a Testing course, hosted by Assurity, if such as a project manager or tester, causing them not to see issues I was to describe him I could only business owner – decides that they that may be right in front of them use one word… ‘passionate’. threaten the value of the product. because they become reliant on the script they’re following. His passion for testing really shines Building on this, Bolton identifies through during the course, which is testing as a lens that enables the As such, scripted testing ought to be designed to promote methodologies project to see more clearly than it used for what Bolton and Bach term and practices that enable software otherwise could. Testing is a learning ‘checking’ – the process of making testing to be done more quickly and activity, its purpose being to identify evaluations by applying algorithmic less expensively while delivering as much information as possible and decision rules to specific excellent results. allow the business owner or project observations of a product – while manager or whoever matters, to manual testing time is better used in Developed in tandem with associate make informed decisions. This is not conducting structured exploratory James Bach, the course pays homage to say that the tester has no role in testing sessions. to the school of Context-driven identifying bugs or threats to the Testing, to which they are both project’s value. To be able to identify This places greater emphasis on the strongly aligned. But what struck such potential problems, Bolton tacit knowledge of the tester – the me most of all was the realism with suggests we identify and utilise things we know without necessarily which Bolton approaches the idea of oracles to help us form judgments knowing that we know them. These improving the way we do testing. about the software. inform our understanding of the world and our interactions with it, This is no mutiny, no cult, no Oracles are ways in which we judge guide our instincts and expectations revolution against the established a product and they are heuristic in and may alert us to the presence of a order – although the fervour and the sense that they are fallible and potential problem. passion inspired in those who had not absolute. They act as a guide to recommended the course to me may enable us to infer the existence of a Bolton is quick to dismiss the suggest otherwise. Instead, it’s a potential problem, but ultimately common view of exploratory testing considered and practical approach to they can’t guarantee that an issue will as an unstructured and non- undermine the value of the product. repeatable activity that cannot be 24

accurately reported on. In his testing more efficient. Instead of long security and usability. Something argument for session-based and wordy documents, Bolton argues we don’t hear much about though exploratory testing, he stresses for the construction of a simple is ‘charisma’. the need for each test session to narrative about the product, the be strictly time-boxed, have a set testing and the value of that testing. During Michael’s visit, he presented mission or charter to govern its a thought-provoking and inspiring objectives, generate a reviewable However, the real value in the course interactive session at the Wellington session sheet of what was observed – lies in the fact that Bolton is not Test Professionals’ Network (TPN) to be followed by a debriefing session selling solutions. The Rapid Software that explored this concept and how with an experienced test lead or Testing course makes no claims to it applies to testing. manager to introduce accountability solve all your testing problems and and trust in to the exercise. help you produce perfect software. Charisma as a quality attribute was first introduced in 2011 by Rikard Such an approach to testing allows Instead, Rapid Software Testing is Edgren in his Little Black Book on a more hands-on testing engagement a mindset and a skillset that helps Test Design. It has since been – maximising the time spent actually make testing faster, cheaper and incorporated into the heuristic test testing the software, rather than more effective. It is also a living thing. strategy model taught by Bolton writing test scripts that become a Bolton regularly pointed out that he and his colleague James Bach. representation of the tacit knowledge and Bach continuously debate a tester would use to guide their various elements and are constantly The presentation started with a exploration. introducing new ideas. broad overview of the three pillars of the heuristic test strategy model Organising a testing effort in this The ideas behind Rapid Software (HTSM): Product Elements, Project session-based way also doesn’t Testing are consistently developing Environment, and Quality Criteria. discount the act of planning as a and evolving to keep pace with a After a brief, but broad introduction crucial testing activity. Initial survey rapidly developing industry. to the HTSM, we delved deeper to and strategy sessions can be used focus on how charisma can be to help familiarise yourself with the This doesn’t mean that I’ll need to considered as a quality attribute. system under test and define the re-take this course in a few years to processes to be utilised to test it. keep up with those developments. As explained by Rikard Edgren, More than anything, you take a charisma is something that software Indeed, Bolton argues that such mindset away with you – a new way can or should possess. When we look early engagement with the product of looking at testing and the way we at a product, we often ask questions is crucial in building a model of the do things and a passion to improve like, “Does it work?” and “Is it user- software that then allows us to define the way we work. friendly?”. These relate to more our test approach and focus our functional attributes, but to ask, energies on those areas of the system That word ‘passion’ again. It’s what “Does the product have ‘it’?” – where most important to the business or drives the course and makes Bolton ‘it’ is that X-factor or charisma that that present the greatest risk. such an engaging and inspirational makes something stand out from teacher. He has a genuine love for the competition – is an equally valid Such a model can take any form, but what he does – as illustrated by the assessment to engage in. during the course we used mind- way he gleefully showed us his mapping software to great effect. collection of software bugs found We then broke into groups to It’s a skill I’ve never employed before, in the world’s hotels and airports. brainstorm what we thought but I quickly found it to be a charisma – with reference to wonderful method of visually Bolton’s passion is for software computer software – might entail and representing the extent of a system testing. For finding bugs and for what the elements of charisma were. and the relationships within it. finding them in the methods and processes we use to do that. While This generated some really Presentation of such a model to a talking to him about having pride in interesting discussion and ideas. business owner can be extremely what we do, he looked at me and said I was especially intrigued with an effective in demonstrating the something that I think sums up the idea that one group came up with vastness of a system. It also ethos of Rapid Software Testing and about ‘tribe values’ and how the encourages them to make a call on Bolton himself: application’s ability to conform to those areas your testing should cover the values of the tribe could give it – building in an awareness of the “You know what, I just think that we this sense of charisma. inherent limits and helping the can do better” people who matter to define the After we came back and put all the test approach. Test Professionals’ Network ideas on the wall, Michael outlined Forum (16 April) what Rikard Edgren had proposed: Reporting outcomes in a more efficient manner is another way that As software testers, we’re usually Uniqueness: the product is the Rapid Software Testing approach called upon to discover threats to distinguishable and has something attempts to make the business of functionality and often to things like no one else has.. 25

Satisfaction: how do you feel after for the session on ‘Regression The consensus was that although using the product? Obsession’. regression testing is often prescribed Professionalism: does the product as a distinct phase, it is actually an have the appropriate flair of With everyone settled, beverages activity that occurs – or should occur professionalism and feel fit for in hand, notebooks at the ready, – within all testing. purpose? Michael ripped in to a 20-minute Attractiveness: are all types of summary of a presentation that Assurity’s Aaron Hodder presented aspects of the product appealing usually takes an hour. the analogy of a roast dinner. The to eyes and other senses? diner doesn't approach the plate Curiosity: will users get interested Michael set about challenging the with a phased schedule: one minute and try out what they can do with disproportionate focus placed on for eating peas, three minutes for the product? regression testing, queried how carrots, two minutes for chicken. Entrancement: do users get hooked, many bugs it really finds and asked What if they want more peas? Or have fun, in a flow and fully engaged why regression merits its own testing would like to eat the chicken and when using the product? phase. This was the perfect opening carrots together? Hype: should the product use the to the evening’s main event – Open latest and greatest technologies/ Season – a moderated discussion of Instead of specifying the items on a ideas? the topic on the table. plate, a person would simply say that Expectancy: the product exceeds they are ‘eating a roast’. Similarly, expectations and meets needs you One attendee started the debate testers should label their activity as didn't know you had describing how he felt that the testing, performing the activities they Attitude: do the product and its obsession with regression generally attribute most value to at any given information have the right attitude comes from the top of an time, rather than allocating time and speak to you with the right organisation and that this is against perceived phases. language and style? prompted by fear. He pointed out it is Directness: are (first) impressions more embarrassing for a manager to Towards the end of the evening, the impressive? release a product with a bug in a topic of cost vs. benefit for regression Story: are there compelling stories piece of functionality that used to testing was raised. Though this had about the product’s inception, work, than to release a bug in a piece previously been touched upon with construction or usage? of functionality that had never Michael stating that regression worked. testing found very few defects and Since the TPN session, I've been Nigel Charman of Assurity presenting ruminating on the idea of 'charisma' This idea of re-testing what’s the ‘Goldilocks problem’ of how to in relation to software. The most known to work, lead us to a determine the correct amount of important insight was that the value discussion around the important idea regression to run, the passion of the of charisma isn't just applicable to that regression and repetition are room sparked a red card or two and the Apples and Googles of the different things. That is, we don't took the debate to another level. software world. necessarily need to look for regression problems the same way The consensus was, that the earlier That internal banking app you're each time. in the SDLC automated functional testing? A charismatic application checks are present, the better. Unit engages users and makes them happy Michael stated that automated checks testing as a means of basic regression to use it. A happy, engaged user should be used to ask "Is there a was seen as cost effective and makes fewer mistakes than someone problem here?" rather than "Does it essential, but it was more difficult to who feels 'forced' to use your work?" A passing test doesn't mean pin down definite answers around application and will likely respond that the product is ready to ship. It costs vs. benefits later in the lifecycle better to the experience and your means that the single problem we and at the systems testing level. business as a whole. are looking for in the particular space that the check is being made doesn’t If regression testing was finding a As such, there are definite, tangible actually exist. large number of bugs, this may be business outcomes to this idea of an indicator of an endemic treating charisma as a quality Checks should be used to grant development issue which would be attribute when we test software. testers the opportunity to investigate very valuable to fix. Alternately, the product, rather than focusing on regression testing may find very few WeTest Workshop: Regression confirmatory testing activities that bugs – but of high importance – so Obsession (18 April) automated checks are suited to. would still be a valuable exercise. Another member of the group had Wellington’s WeTest workshops also experienced a regression As always, it was interesting to hear are monthly gatherings of passionate obsession from higher levels – in her how different testers from different testers giving us a chance to learn case a distinct two-week phase of organisations do things, as well as from each other’s experiences and to regression testing was mandated their struggles and triumphs. The hone our craft. With Michael Bolton by the Steering Committee within animated discussion during the break as the special guest for the April meet the organisation. and after the event – even with the -up, the attendees eagerly gathered double distraction of pizza and beer – 26

was testament to the energy One part I found particularly for overcoming these fallacies, and passion of the attendees interesting was the difference enabling us to think about testing leaving everyone buzzing and between system one and system two problems in a more considered, deep in thought. thinking and the importance of being logical way. able to recognise the difference and Thanks again to Assurity for apply it to different situations as Everyone I spoke to was energised sponsoring the event, allowing appropriate. and enthusiastic. I highly recommend testing practitioners to share their this workshop for anyone who wants stories with one another for free System one thinking is fast, to clarify and enhance their thinking, and able to continue to discover and instinctual and emotional and often not just with regards to their testing progress the craft of software testing. serves us well for everyday thinking work, but in their everyday when speed is important. However, life as well. Critical Thinking Skills for Testers it can also mislead us. This is when (19 April) we need to move into system two ------thinking, which is slower, but more To round up the week, Michael deliberate and logical. All in all, a whirlwind week for the presented a one-day workshop on man behind Rapid Software Testing. I Critical Thinking Skills for Testers – I also found it very valuable hearing think it’s fair to say he took his definition of ‘critical thinking’ how to elicit hidden assumptions Wellington a little by storm! It’s being “thinking about thinking with from the things people say. As impossible to be around someone the aim of not getting fooled”. testers, we often deal with multiple with Michael’s energy and passion sources of truth and multiple and not have it rub off on you a little. In this workshop, Michael taught interpretations of requirements and A huge thanks goes to Assurity for different ways of deconstructing objectives. Being able to think clearly giving us – and the Wellington testing claims made about testing and and recognise the differing community – the chance to be how to structure our own thinking interpretations and assumptions inspired. to avoid traps. This leads to more that may occur is a crucial skill. careful thinking and more Adam Howard with Katrina Edgar careful testing. By the end of the day, we had learnt and Aaron Hodder about common-thinking fallacies and had picked up tools and techniques

When

Imagine - to the tune of John Lennon’s Imagine (obviously!) John

Was a

Tester Imagine perfect projects It's easy if you try No risks impede us No issues make us sigh Imagine all the software Faultless every day.... Imagine there's no triage It isn't hard to do No bugs to fix or pay for And no debugging too Imagine all developers Living life in peace…

You may say I'm a tester But I'm not the only one I hope someday you'll join us And the world will be as one

Imagine clear requirements I wonder if you dare No need for code inspections No rework or repair Imagine all your users Happy with their apps…

You may say I'm a tester But I'm not the only one I hope someday you'll join us And the world will be as one

Kiwi Test Teams

Harpal Singh and the Test Team from Auckland Council

Back Row: l. to r.- Pieter Smuts, Amarsingh Shinde, Jimmy Djunaedi, Craig Saunders, Tamati Stevens, Harpal Singh

Front Row: l. to r.- Arulmozhi Selva Durai, Arasi Rajendra Devan, Lianne Pimentel, Shweta Bansal, Mary Joy Magcale, Laly George, Sadaf Ansari, Anand Gudimanchi

On the Road Again (a travelogue based on a true story apparently)

by Geoff Horne

I always knew April would be busy this year. software entrepreneur from way back. He related Looking at the calendar in late February, I realised a couple of tales of software disasters in the USA in that within the space of four weeks I was due to the mid-sixties and I realised that the only thing participate in no less than four events delivering that has really changed since then is the technology. four presentations, one half day tutorial and a Some of the disasters John described are still being 'Lightning Talk’. The evenings of preparation were seen today. plentiful and I looked forward to the time when The Wellington Summit was well attended with everything was done and I could unwind before guest speakers including Steve Wenzel (Air New heading out on the road. The fact that I sent the Zealand), Matt Mansell (Ministry of Internal Affairs) wrong slide sets, got presentation dates incorrect, and Ian Wells (Telogis) along with US-based missed planes and turned up at the wrong hotel industry analyst Theresa Lanowitz of voke inc. My ensured that the adrenaline kept pumping right session was straight after lunch where I spouted until the time I landed back in good ol’ 'Godzone' forth on all aspects of QA Transformation as well in the first week of May – and all this was as on the latest editions of NZTester Magazine and exacerbated by a wrist in cast thanks to a beaut OZTester Magazine, which had recently been double fracture received whilst playing squash with released; there was much interest around these. my youngest daughter a few days before I departed (a story for another time … and another magazine).

First stop was the Cognizant Quality Engineering and Assurance Summit in Wellington. Just a short one hour non-eventful flight from Auckland – if one can ever call flying into Wellington Airport non- eventful (all due apologies to Wellingtonians). In the cab on the way to the hotel, I got talking to John the driver who turned out to be an American Matt Mansell makes a point (Wellington)

Min Xu outlines BankWest’s approach (Sydney)

Next day, it was off to Sydney for the next leg of the summits, which again was extremely well attended and, like Wellington, enjoyed full participation. Some of the topics generated a myriad queries and discussions, in particular mobile application testing and virtualisation. Seems like testing of mobile applications, both manual and automated, is on everyone’s lips, mainly due to the added challenges presented by multi-platforms, e.g. Android, iOS,

Windows 8 and all the different versions thereof, enjoyable stay in the hotel, which happened also causing testing requirements to be amplified many to be the same hotel I stayed at on my very first fold. Guest speakers included Min Xu (BankWest) business trip with Hewlett-Packard in 1987; it and Yaron Nahmias (Optus) and as in Wellington, hadn't changed much. I drew the apre s dejeuner slot, endeavouring to With a few days to myself, I flew the ‘stang to keep everyone awake whilst again negotiating the Tombstone, AZ. where I fulfilled a childhood wilds of QA Transformation. I had the weekend to dream of visiting the OK Corral (that of the famous relax in Sydney before heading back to Auckland. gunfight). Did you know that the gunfight did not The few days spent at home were a welcome break, take place at the OK Corral? It was actually fought however that came to a quick end as I boarded the three doors down the street in a vacant lot between next flight. Managing to stick my grubby little hands Harwood’s Store and CS Fly’s Photography Studio, on a Ford Mustang after a little haggling with Hertz, where Doc Holliday rented a room. In fact, the I headed off to the STPCon event. I had been to only connection with the OK Corral was that the STPCon in 2012 and whilst not the largest bad guys walked through it in search of Holliday conference around, it does host a good variety of and once at Fly’s, happened upon the goodies speakers and topics. I particularly enjoyed the panel coming the other way. However, I guess Gunfight session with Bob Galen, Cem Kaner and Rex Black at Harwood’s Store or at CS Fly’s Photography where I got to ask my usual curly questions that I Studio doesn’t sound as sexy as Gunfight at the OK knew I would probably get curly responses to. I Corral. Anyway, I came away from Tombstone with wasn’t disappointed. enlightenment, an awesome pair of cowboy boots

and an almost 1,000 km drive, which I totally under Matt Heusser, who is scheduled to speak at this year’s -estimated, back to Los Angeles. STANZ, leads one of the expert sessions at STPCon. I turned up at the airport early next morning to find my next flight had not been ticketed. Whilst trying to sort it out, the plane left without me and I subsequently ended up in Chicago where I finally caught a flight in the right direction. The next gaffe was waiting twenty minutes for the Hertz bus at Orlando Airport in the early hours of the following morning before realising that Hertz is now on- airport rather than off-airport, as it was when I was last there. To round off a day of set backs, I turned up at the Rosen International Hotel where the StarEast summit has traditionally been held to find that this year it was being held at the Rosen Shingle Creek, some 8 km away. Perhaps I should I was speaking at STPCon on Test Estimation and have read the brochure after all. Forecasting Trends, a subject dear to my heart as a Test Manager. The response from attendees was Next day, I delivered my very first ever half-day great and hopefully I was able to clear up a few tutorial - on Data Warehousing Testing. I was still misunderstandings that some folk had regarding rather jaded from the previous day’s anomalies, the tracking of testing. however I needn’t have feared about not having enough material. In fact, I had too much and I As is often the case, whilst the quality of the think I fell into the trap of trying to cram it all in. presentations was very high, the greatest benefit I In addition, I realised ten minutes or so before realised was in the contacts I made and the the start that I had previously forwarded the wrong networking with other test professionals. I took slide set and after profuse apologies from me, my away a stack of business cards and a number of poor attendees had to negotiate through incomplete possible opportunities. Afterwards, I had to engage documentation whilst I rabbited on. Despite that, a little with the hotel management on checking out the response was reasonably good and I had plenty over a mischarge, however it was quite an of requests for further information. What is 31

apparent is that the advent of Big Data (suggest you do a Google search for details) is certainly going to increase the requirement for data warehousing as more and more data, much of it unstructured, is going to be captured and used in ways that perhaps current technologies may not be able to cope with. Already, data storage is now commonly measured in petabytes - what are we going to do when we get to yottibytes?

StarEast was huge this year, over 1,200 attendees I’m told. After 9/11, many US-based conferences were hit hard due to reticence about flying, so it is good to finally see StarEast back to its old self. Whilst I was there, I also caught up with Theresa Lanowitz who was presenting with Microsoft and if you thought my itinerary was wild, you should have seen Theresa’s.

The StarEast Exhibition was packed to capacity again this year with the most popular offerings being automated testing tools for mobile platforms. There was a definite downturn in testing services- only providers and the tool vendors outnumbered them by, I would guess, four to one. I hope to produce a test tool directory under the Tester Magazine banner later in the year, so you might Michael Bolton delivering his Lightning Talk at StarEast like to keep an eye out for this. The curtain call for me was the presentation of I spent an evening as one of a number of guests my Testing Risk Management Methodology, which of SOASTA inc., a software company that develops as usual generated much discussion and I was automated testing tools for both mobile and web pleasantly surprised by the number of queries I applications. I was able to ask my usual bunch of received after the session officially closed. Just curly questions and somewhat to my surprise, goes to show that testing challenges are pretty I found that the SOASTA folks had already much the same the world over albeit perhaps on encountered many of the 'bumps' that I asked different scales. about and have come up with solutions. I suggest watching out for the SOASTA toolset, it looks If I’m to summarise from a testing perspective what quite exciting. I picked up 'on the road' or at least had reaffirmed, it is as follows: I also participated in the StarEast 'Lightning Talks' which is where keynote speakers and tutorial  Software issues are with us to stay – my presenters are given an opportunity for a ten conversation with John in Wellington minute rant about anything related to testing. I showed me that (as if I did know already). was tempted, as I have done in this article, to give  New technologies and needs such as an exposition around 'what I did in the holidays'. mobile applications, Big Data, social media However I decided that professionalism should take integration, virtualisation, etc., are coming priority and consequently delivered a diatribe on upon us and some are here already. We will positive mental attitude and overcoming situations need to rise to the challenges of learning where people tell you 'you can’t' – which I know about these if we are to i) overcome the new will surprise many of the project managers I have issues they will present and ii) re-address worked with in the past! the old ones they will no doubt exacerbate.

 Test automation has become a necessity as Epilogue opposed to a nicety and there is a challenge The after-party for this little ball was the annual to rethink the way we develop and deliver ANZTB Conference held in June for which OZTester testing solutions using automation. All testers Magazine provided sponsorship. Colin and I were would do well to brush up on or get both present; Colin delivering a Lightning Talk and themselves into some sort of education yours truly merely observing (for once). I will leave process around automation. the conference tales on this one to Colin (see his  Performance Testing is still a major review in OZTester on page 26). requirement and the need for understanding around Performance Engineering will only increase. Again, upskilling in this area will only be to a tester’s benefit in the future.  There is still immaturity around estimating and forecasting testing efforts based on something other than percentages. These may be a good starting point however in light of all of the above we will need to be much smarter around how this is performed and reported.  All testing needs to be viewed in context. Whilst I’m sure many of us have had the good ol' stuff drummed into us from an early testing age, with so many technological advances being made in recent years we can no longer assume that the same approach will necessarily suit every situation. I suggest if not already doing so, that all testers gear up around all approaches, ‘cos as sure as eggs we will be using them, if not now then certainly in the future.

Furthermore, with the increasing reliance upon digital technology, the incidence of disruption is going to impact more and more upon business continuation and opportunity. Enterprise architects will need to be continually reviewing emerging technologies to assess and address the likelihood and impacts of failures. Therefore, it stands to reason that IT professionals experienced in spotting and predicting possible failure points will bring a unique perspective to the design, development and deployment of solutions based on emerging technologies. The 'tester’s nose' - who’d have thought? Top: Rick Craig (SQE) “checks in” (StarEast) Finally, as usual I thoroughly enjoyed myself on the Middle: Bob Galen, Cem Kaner and Rex Black man the road and I’m looking forward to doing it all again panel (STPCon) in September at StarWest where I’ll be delivering Bottom: Theresa Lanowicz in full flight at the Cognizant the Data Warehouse Testing tutorial and Wellington Summit (hopefully) I will have built in the lessons learned from the first time around – such is the case for continuous improvement!

Hiring Student Testers

by Ian Wells

I am currently the QA Manager at Telogis in to build a stronger link between universities and Christchurch. Prior to this I managed testing for the growing New Zealand technology industry. It’s the Trimble Mapping & GIS Division and for Hewlett- a world market for talent. Already universities have Packard compiler products. longer internships and co-op programmes that are incorporated to provide even greater learning and In my current and previous positions, I’ve contribution opportunities for students. interviewed over a hundred candidates for various testing roles and hired over twenty. What to ask at an interview. There have recently been plenty of online discussions about the value of the “technical As Laszlo Bock found out by sifting through all interview”1. Is the technical interview dead for Google’s data , its very difficult to predict how testers? Is there a best way to find, interview and someone will perform in a role based purely on an keep top testers? interview. Nevertheless there are three points I have found, through testing interview techniques, As a manager, choosing who to hire is one of the key to be effective in determining whether the candidate areas of decision making. As per recent comments might be a good fit for testing. in the New York Times2 by Google’s People Operations VP Laszlo Bock, illustrate it is extremely Ask behavioural questions. Behavioural questions hard to do even for a company like Google. can help you determine how the candidate approaches actual rather than hypothetical Testing is now recognised as a career path in itself situations. and top companies know that hiring top testers can be crucial to meeting quality and schedule objectives Debug a real application. Have the candidate test an in today’s complex software environments. As we all actual software application. know, testing can be a very challenging job and rewarding career path for the right people. I am Pattern identification. Provide the candidate with an always looking for top testers and the best fit to exercise where the pattern of inputs to outputs needs company culture. to be established. One avenue I have found to be particularly beneficial I always review results with the candidate and in finding and growing top software testers is from discuss them. What matters is not so much how the educational institutions. candidates do on these tests rather their thought processes.

Where to find candidates. Four Types of Testers Recruiting talent at universities and technical There are four types of testers I tend to look for institutes is an excellent starting point. Testing amongst students and indeed when hiring internships provide students with an excellent experienced testers as well: introduction to the software development lifecycle where they get to see first hand the impact of quality, The Accountant Tester: Accuracy and methodical whether it be positive or otherwise. Students bring approaches are highly beneficial when keeping a wonderful enthusiasm that invigorates test teams. track of the myriad of combinations of tests that can Internships also provide both company and be run. Accountant-testers are meticulous and make candidate with the opportunity to assess each other, sure no details are missed. an extended interview if you like. The Tester Developer: Building up an automation Of course there will be the requirement for time off facility is fundamental for regression testing. In my for study and exams and accordingly, I am a keen opinion, automated test suites need to be higher supporter of longer internships, as in other countries, quality than the code they are testing so ideally the 34

most experienced developers build quality Editor’s comment: automated test suites. The temptation is to have student developers create test code - avoid Many years ago, I was working with a well known this misconception. New Zealand software company that was looking The Breaker: The Breaker tester is usually tickled to merge the different versions of its product into pink when a defect is discovered. The “special one parameter-driven single source code set and mindset” of knowing where to look for bugs, thus avoid multiple versions of each new release. uncovering them, prioritising those that matter and interacting constructively with developers for The testing scope was sizeable and required 25- resolution is a skillset that requires careful odd testers for a period of 3-4 months to document development and nurturing. Introducing candidates and execute some 2,500 test scenarios. The project with such potential to exploratory testing manager hit on the idea of hiring students and a approaches provides extremely useful grounding. notice was placed on the local university job The Doctor Tester: The skills of a top tester can be search board. Fifty applications were received likened to those of a doctor in that a doctor has a and all of them put through an aptitude test to limited amount of time to investigate what is wrong determine the most suitable. The top twenty five with a highly complex “system” ie. your body, what were offered roles as student testers over the tests to run and to suggest remedial action. Of course university Christmas break. a doctor has been trained for years how to debug the human system. However as testers become more They were placed into one of four teams, each experienced and rise up in their profession, the covering specific modules of the application and organisation relies on them to diagnose a complex software release so it’s “healthy” when it ships. Like alongside more experienced testers and a test lead all testers, the Doctor-Tester must also be a for coaching and mentoring. perpetual learner. Each of the students flourished under tutelage Top testers are crucial to the success of any software and we found that those who really excelled were development team. I hope you find these suggestions the music and psychology students. This was helpful in finding and growing testers, and increasing somewhat of a surprise as our anticipation was our pool of top testers in New Zealand. that it would be the IT or engineering folk who would become the stars. References: The outcome was a highly successful exercise with 1 https://www.nytimes.com/2013/06/20/business/ only one of the students deciding it wasn’t for him. in-head-hunting-big-data-may-not-be-such-a-big- At the end of the project, those who indicated they deal.html?pagewanted=all had finished their studies were interviewed for full 2 http://techcrunch.com/2013/06/22/the-technical time positions and most hired. Others continued to -interview-is-dead/ work part time for the company. 3 http://www.sandwichyear.co.nz/ Yours truly on the other hand moved on to one Ian Wells is Quality Assurance Manager at Telogis. of the company’s clients and became a recipient of He can be contacted at [email protected] the new release. I can bear first hand testimony as to its quality and it was top notch.

Fast forward 15 years or so, from time to time I still see some of the student’s names pop up on LinkedIn and other places as senior developers, consultants, project managers and yes, testers. It’s good to see so many have carved out solid careers in IT for themselves. – Ed.

And now it’s your turn…

If you would like to be involved with and/or wish to put out there. You don’t have to be a great contribute to future NZTester issues, you’re writer as we have our own staff writer who is formally invited to submit your proposals to me at always available to assist. [email protected] Please remember to provide your email address Articles should be a minimum of ½ A4 page at which will be published with your article along Cambria 11pt font and a maximum of 2 A4 pages with any photos you might like to include (a for the real enthusiasts. If you wish to use names headshot photo of yourself should be provided of people and/or organisations outside of your with each article selected for publishing). own, you will need to ensure that you have As NZTester is a free magazine, there will be no permission to do so. financial compensation for any submission and the Articles may be product reviews, success stories, editor reserves the sole right to select what is testing how-to’s, conference papers or merely published and what is not. some thought-provoking ideas that you might Please also be aware that your article will be proof- read and amendments possibly made for readability. And while we all believe in free speech I’m sure, it goes without saying that any defamatory or inflammatory comments directed towards an organisation or individual are not acceptable and will either be deleted from the article or the whole submission rejected for publication.

Feedback

NZTester is open to suggestions of any type, indeed feedback is encouraged. If you feel so inclined to tell us how much you enjoyed (or otherwise) this issue, we will publish both praise and criticism, as long as the latter is constructive. Email me on Catch Software 9 [email protected] and please advise in your email if you specifically do not want your comments Seapine Software 12 published in the next issue otherwise we will Tasting Let’s Test 13 assume that you’re OK with this.

Assurity Consulting 17 Sign Up to Receive NZTester Software Education 19 Finally, if you would like to receive your own copy

KJRoss & Associates 22 of NZTester completely free, even though we’re real low tech right now, there’s still two easy ways: Qual IT 23 1) go to www.nztester.co.nz, or 3) simply click here Seapine Software 27 - Ed.