Testing Strategies for Complex Environments: Agile, SOA

May/June 2009 Volume 2

SOFTWARE TESTING Testing strategies for complex environments: Agile, SOA

• CHAPTER 1 A SOA VIEW OF TESTING • CHAPTER 2 AGILE TESTING STRATEGIES: EVOLVING WITH THE PRODUCT EDITOR’S LETTER 1 Testing in the new worlds of SOA and agile

p UNLESS YOU LIVE in a sci-fi novel, Karen N. Johnson in “Agile testing a there’s one rule of thumb for any new strategies: Evolving with the product.” EDITOR’S LETTER place you go: some things are differ- Agile is a more collaborative process ent; some are the same. Usually, suc- and calls for seizing “iterations as a cess, fun or survival in that new place chance to evolve test ideas,” Johnson a depends upon how well you handle writes. Fundamental testing tasks, like CHAPTER 1 A SOA VIEW the differences. In this issue of exploratory and investigative testing, OF TESTING SearchSoftwareQuality.com’s Soft- stay the same, but the productivity of ware Testing E-Zine, the new “places” completing those jobs can increase. are a service-oriented architecture Johnson and Kelly discuss the finer a (SOA) and an agile software develop- points of testing in agile and SOA, CHAPTER 2 AGILE TESTING ment environment. respectively, in this issue’s articles. STRATEGIES: EVOLVING Examining the ins and outs of soft- They also describe revelations they’ve WITH THE ware testing in SOA environments in had while working in those environ- PRODUCT “An SOA view of testing,” consultant ments and best practices they’ve Mike Kelly focuses on the “subtle dif- learned and continue to use. ferences.” What’s the same is that The theme of connectivity runs testers here must start with the through these discussions of SOA and basics. In SOA, the basics are connec- agile testing, as the former focuses on tivity, basic capacity, and authoriza- system connectivity and the latter on tion and authentication—not that dif- human collaboration. Both approach- ferent from other environments. The es, when done well, can help develop- level of complexity of data models, ment and IT organizations achieve however, is very different from that of lower costs and better software. I other architectures and requires some new methods of testing. JAN STAFFORD While agile development requires Executive Editor different testing methods than the [email protected] waterfall model, those differences lie as much in human behavior as in technology, according to consultant

2 SOFTWARE TESTING MAY/JUNE 2009 CHAPTER 1 1 A SOA view of testing The complexity of a service-oriented architecture can complicate testing and make choosing and implementing the right testing tools more difficult. BY MICHAEL KELLY

a EDITOR’S LETTER p SERVICE-ORIENTED architectures test. This means people testing SOA (SOAs) are stealthily becoming ubiq- need to be comfortable with varied uitous. Large and small enterprises and changing technology, need to a leverage them to integrate wide understand the various models for CHAPTER 1 A SOA VIEW arrays of disparate systems into a SOA, and should be familiar with OF TESTING cohesive whole. Most companies and the risks common to SOA. project teams that implement SOA do a so to reduce the cost, risk and difficul- CHAPTER 2 ty of replacing legacy systems, acquir- WHY SERVICE-ORIENTED AGILE TESTING ing new businesses or extending the ARCHITECTURES? STRATEGIES: EVOLVING life of existing systems. Large companies typically turn to WITH THE PRODUCT Testing a SOA requires a solid SOA because their existing systems understanding of the SOA design and can’t change fast enough to keep up underlying technology. Yet, SOA’s with the business. Because business complexity makes figuring out the operations don’t exist in discrete or functional purpose of the SOA diffi- finite units, there are a lot of depend- cult and choosing and implementing encies built into existing systems. the right testing tools and techniques SOA is an attempt to separate the to use with it more difficult. system from the business operations SOA is not simply Web services. it supports. This allows companies to According to iTKO (the makers of the incrementally build or decommission TechTarget award-winning SOA test- systems and minimizes the impact of ing tool LISA), nine out of 10 Web changes in one system to changes in services also involve some other type another. of technology. In addition, they state Implementing a SOA removes that most testing for SOA isn’t done the requirement of direct integration, at the user interface level. It’s done which results in business operations using either specialized tools or as that are easier to understand. That part of an integration or end-to-end translates into a more testable system

3 SOFTWARE TESTING MAY/JUNE 2009 overall. Over time, this makes it dards such as application-oriented easier to support a broader technolo- networking (AON), WS-Security, gy base, which in turn makes it easier SAML, WS-Trust, WS-SecurityPolicy, to acquire and integrate new systems, or other custom security schemes for extend the life of existing systems data security as well as authorization and replace legacy systems. and authentication. There are a number of different model architectures for implementing I Load balancing: SOAs are often SOA, including publish and subscript designed to spread work between (often called “pub/sub”), request resources to optimize resource and reply, and synchronous versus utilization, availability, reliability asynchronous. In addition, they can and performance. a be implemented in a broad range EDITOR’S LETTER of technologies, including message- I Translation: As data moves oriented middleware (MOM), through a SOA, it’s often converted simple HTTP posts with DNS routing, or changed based on business rules a Web services, MQ series, JMS (or or reference data. CHAPTER 1 A SOA VIEW some other type of queuing system) OF TESTING and even files passed in batch I Logging: For monitoring and processes. auditing, SOAs often implement If you’re testing a SOA you need multiple forms of logging. a to understand the implementation CHAPTER 2 AGILE TESTING model and technologies involved, I Notification: Based on the model STRATEGIES: EVOLVING because these typically combine to of SOA implemented, different notifi- WITH THE give you some common features of cations may happen at different PRODUCT a SOA. These features often become times. a focal point for your testing. They commonly include: I Adapters: Adapters (both custom and commercial) provide APIs to I Transformation and mapping: access data and common functions As data moves through a SOA, it’s within a SOA. often transformed between data formats and mapped to various standard or custom data schemes. FIGURING OUT WHAT TO TEST The first thing you need to do when I Routing: The path information you start figuring what to test for your takes as it moves through a SOA SOA is to develop (or start develop- is often based on business rules ing) your test strategy. When I’m embedded at different levels of faced with a new project and I’m not the architecture. sure where to start, I normally pull out the Satisfice Heuristic Test Strate- I Security: SOAs often use stan- gy Model and start there. Each sec-

4 SOFTWARE TESTING MAY/JUNE 2009 Feeling the Agile Rush? Test Faster With TestComplete7 The Easiest 7 TestComplete Ever!

Agile development moves fast, too fast for manual testing alone. In an agile environment you need automated testing tools to succeed. Check out TestComplete from AutomatedQA. TestComplete lets anyone automate tests for the widest range of technologies like Web, Ajax, Windows, .NET, Java, Flash, Flex and Silverlight. TestComplete has won four Jolt awards, been voted Best Testing Tool Script-Free Test Creation and yet it's still a ordable. Simple to learn and easy to extend

TestComplete's extensive feature set and Download and Customize Extensions Pre-built and custom add-ons make great price has long made it the choice of complex tests point and click easy expert testers. Now version 7 adds script-free test creation and a simplied Unmatched Technology Support user interface so new testers can get New technology? No problem! Rapid productive fast. TestComplete 7's easy support for the latest software automated testing lets your entire QA team Price and Performance Leader test more, test faster and ship on time with Unsurpassed features and a low price condence. to make you smile.

Start Testing In Minutes! Download TestComplete now and start Download testing today. TestComplete FREE www.testcomplete.com AutomatedQA(978) 236-7900 tion of the Satisfice Heuristic Test I Project environment: What fac- Strategy Model contains things you’ll tors will be critical to your success need to consider as you think about and the success of the in-house team your testing: as you take over this work? How will you take in new work, structure your I Test techniques: What types of release schedules, or move code testing techniques will you be doing? between teams, phases or environ- What would you need to support ments? What are the business drivers those types of testing in terms of to moving to a SOA and how will people, processes, tools, environ- those come into play while you’re ments or data? Are there specific testing? things you need to test (like security, a transformation and mapping, or load Recognize that as you think of EDITOR’S LETTER balancing) that may require special- these questions, it’s a matrix of con- ized techniques? cerns. A decision (or lack of decision) in each of these categories affects the a I Product elements: What product scope of the decisions in the other CHAPTER 1 A SOA VIEW elements will you be covering in your three. Therefore, you will most likely OF TESTING testing? What’s your scope? To what find yourself approaching the problem extent will different data elements be from different perspectives at differ- covered in each stage? Which busi- ent times. a ness rules? Will you be testing the Once you have an understanding CHAPTER 2 AGILE TESTING target implementation environment? of what features you’ll be testing and STRATEGIES: EVOLVING How much testing will you be doing you know which quality criteria you’ll WITH THE around timing issues? How will you be focused on, you’re ready to dig in. PRODUCT measure your coverage against those Don’t be surprised if in many ways elements, track it and manage it from your testing looks like it does on a a documentation and configuration non-SOA project. SOA isn’t magic, management perspective? and it doesn’t change things all that much. However, I’ve found that there I Quality criteria: What types of are some subtle differences in where risks will you be looking for while test- my focus is when I’m working on a ing? Will you focus more on the busi- SOA project. ness problem being solved or the risks related to the implementation model or technology being used? Will you COVER THE BASICS AS SOON look at performance at each level? (AND AS OFTEN) AS YOU CAN What about security? How will you The first area to focus on is typically need to build environments, integrate connectivity. Establishing a successful with service providers, or find tools round-trip test is a big step early in a to do all the different types of testing SOA implementation. The first time you’ll need to do? connecting is often the most difficult.

6 SOFTWARE TESTING MAY/JUNE 2009 Once you have that connectivity, development and test environments don’t lose it. Build out regression had different security controls than tests for basic connectivity that you production, which led to confusion can run on a regular basis. While a and rework once we deployed and SOA evolves, it is easy to break con- tried to run using an untested permis- nectivity without knowing it. If you sions scheme. If you use LDAP in pro- duction, use it when testing. If you’re handling authorization in your SOAP As data moves request, do it consistently through through a SOA it’s your testing, even if you think it’s easier to disable it while doing translated, transformed, “functional” testing. a reformatted and EDITOR’S LETTER processed. That means you’ll have FOCUS ON THE DATA Service-oriented architectures a a lot of tests focused aren’t just cleaner interfaces between CHAPTER 1 on testing data. A SOA VIEW systems, they are also complex data OF TESTING models. As data moves through a SOA it’s translated, transformed, break it, you’ll want to find out as reformatted and processed. That a soon as possible so it’s easier to means you’ll have a lot of tests CHAPTER 2 AGILE TESTING debug any issues. focused on testing data. These tests STRATEGIES: EVOLVING Next you may want to look at basic typically make up the bulk of the WITH THE capacity. Capacity at this level can be regression test beds I’ve seen on PRODUCT anything from basic storage to con- projects implementing a SOA. nection pooling to service throughput. Tests focused around testing data I’ve been on a couple of projects in an SOA typically happen at the where these elements weren’t looked component level and leverage stubs at until the end, only to find that we and harnesses to allow for testing as had to requisition new hardware at close as possible to where the change the last minute or end up spending takes place. This simplifies debugging days changing configuration settings and allows for more granular tracking on network devices and Web servers. of test coverage (based on a schema, Figure out what you think you’ll need mapping document or set of business early on, and run tests to confirm rules). SOA teams are constantly those numbers are accurate and thinking about regression testing for consistent throughout the project. these tests. Small changes can have In addition, don’t put off testing for a large impact on the data, so these authorization and authentication until tests become critical to refactoring you get into the target environment. or when adding new features. I’ve worked on several teams where For many teams, these tests can

7 SOFTWARE TESTING MAY/JUNE 2009 be part of the continuous integration agers and testers alike. That means process. While building these regres- faster communication, clearer sion test beds can be very easy, once requirements and better tests. At you have libraries of XML test cases some level, every SOA implementa- lying around, maintaining them can be tion needs to think about perform- very time consuming. That makes a ance, availability, reliability and team’s unit testing strategy a critical capacity. Usage models help focus the part of the overall testing strategy. testing dialogue on those topics (as What will be tested at the unit level, opposed to just focusing on function- component level, integration level and ality and data). end-to-end? Starting as close to the When testing for SOA performance, code as possible, automating there, I focus on speed, load, concurrency a and running those tests as often as and latency. My availability and relia- EDITOR’S LETTER possible is often the best choice for bility scenarios for SOA often focus data-focused testing. on failover, application and infrastruc- ture stress, fault tolerance and opera- a tions scenarios. And conversations CHAPTER 1 A SOA VIEW UNDERSTAND around capacity often begin with a OF TESTING YOUR USAGE MODELS basic scaling strategy and spiral out When I test a SOA, I develop detailed from there (taking in hardware, soft- usage models. I originally used usage ware, licensing and network concerns a models solely when doing perform- as appropriate). Because the scope is CHAPTER 2 AGILE TESTING ance testing, but I’ve also found them so broad, effective usage modeling STRATEGIES: EVOLVING helpful when testing SOAs. When I requires balancing creativity and WITH THE build usage models, I use the user reality. PRODUCT community modeling language (UCML) developed by Scott Barber. UCML allows you to visually depict DEMONSTRATING complex workloads and performance YOU CAN IMPLEMENT scenarios. I’ve used UCML diagrams BUSINESSPROCESSES to help me plan my SOA testing, to At the end of your SOA implementa- document the tests I executed, and to tion, you will have delivered some- help elicit various performance, secu- thing that implements or supports a rity and capacity requirements. UCML business process. If the bulk of your isn’t the only way you can do that; if testing up to that point has focused you have another modeling technique on various combinations of service- you prefer, use that one. level processes and data functions, Usage models allow us to create then your end-to-end acceptance realistic (or reasonably realistic) test- tests will focus on those final imple- ing scenarios. The power behind a mented business processes. These modeling approach like this is that it’s tests are often manual, utilize the end intuitive to developers, users, man- systems (often user interface sys-

8 SOFTWARE TESTING MAY/JUNE 2009 tems), and are slow and costly. All the already tested. If you’ve done a good testing outlined up to this point has job of facilitating discussions around been focused on making this testing performance, availability, reliability more efficient and effective. and capacity, then you likely already reduced most of the risk around those quality criteria and have clear plans Make sure you’re for migration to production. selecting tools to fit If you’re testing your first SOA the types of testing implementation, take some time to learn more about the underlying tech- you want to do. Many nologies and implementation models. tools will do more than Once you’re comfortable with what’s a you need, or may force being done, start looking at the tools EDITOR’S LETTER you to do something that are available to support your testing needs. There are many great in a way that’s not commercial and open source tools a optimal for your available, and there is always the CHAPTER 1 A SOA VIEW project. option to create your own tools as OF TESTING needed. Just make sure you’re select- ing tools to fit the types of testing you If you’ve successfully covered the want to do. Many tools will do more a basics, early and often, then you than you need, or may force you to do CHAPTER 2 AGILE TESTING should have few integration surprises something in a way that’s not optimal STRATEGIES: EVOLVING while doing your end-to-end testing. for your project. WITH THE When you deploy the end-to-end Finally, remember that whatever PRODUCT solution, you don’t want to be debug- you’re building now will change over ging basic connectivity or authoriza- time. Make sure you’re working close- tion/authentication issues. If you’ve ly with the rest of the team to ensure successfully covered the data at the regression capabilities have been unit and service level, then you won’t implemented in the most effective have thousands of tests focused on and cost-effective ways possible. subtle variations of similar data. When you come back a year later to Instead, you’ll have tests targeted at make a small change, you don’t want high-risk transactions, or tests based to have to re-create all the work on sampling strategies of what you’ve you’ve done the first time around. I

ABOUTTHEAUTHOR: MICHAEL KELLY is currently the director of application development for Interactions. He also writes and speaks about topics in software testing. Kelly is a board member for the Association for Software Testing and a co-founder of the Indianapolis Workshops on Software Testing, a series of ongoing meetings on topics in software testing. You can find most of his articles and blog on his website, www.MichaelDKelly.com.

9 SOFTWARE TESTING MAY/JUNE 2009 Making the decision to outsource the development If you do outsource, you owe it to yourself, your of your critical application is not easy. There are organization, and most importantly your customers pros and cons, but, once the decision is made, it to know the answers to these questions. is imperative that the quality you paid for and expected is the quality you receive. McCabe IQ provides those answers using advanced static and dynamic analysis technology Ask yourself these questions: to help you focus your attention on the most complex and risky areas of your code base. 1. What is the quality of the code that has been developed? McCabe IQ’s breakthrough visualization techniques, enterprise reporting engine, and 2. Is it too complex, making it unmaintainable executive dashboard provide you with a complete and unreliable? picture of what is being produced. 3. How thoroughly was it tested prior to delivery back to you? It’s time you made the most of your outsourcing investment and benefitted from our 30+ years of 4. Are you convinced that the most complex software quality research and development. areas of your critical application are being tested? Don’t just think your code is good. Be sure of it, with McCabe IQ. 5. Is the code quality and test coverage trending in a positive direction? CHAPTER 2 1 Agile testing strategies: Evolving with the product Software testing in an agile environment is the art of seizing iterations as a chance to learn and evolve test ideas. BY KAREN JOHNSON a EDITOR’S LETTER

a CHAPTER 1 A SOA VIEW p YOU CAN ACHIEVE successful than I do when I’m forced to wait OF TESTING testing in agile if you seize upon itera- through a full waterfall lifecycle to see tions as a chance to learn and evolve an allegedly more polished version of a test ideas. This is the art of software an application. CHAPTER 2 testing strategy in agile. The chal- But there is an opposing point of AGILE TESTING lenge: How can testing be planned view that sees the iterative approach STRATEGIES: EVOLVING and executed in the ever-evolving as frustrating. Early iterations could WITH THE PRODUCT world of the iterative agile methodol- ogy? This article discusses some of philosophies, pros, cons and realities Iterations provide of software testing in the agile itera- a preview opportunity tive process. One aspect of an agile software to see new features development project that I believe and a chance to watch benefits testers is working with itera- features evolve. tions. Iterations provide a preview opportunity to see new features and a chance to watch features evolve. The be viewed as a hassle; the fact that early iterations are such sweet times features will evolve and morph is to see functionality flourish. I feel as potentially maddening if a tester though I’m able to watch the process, thinks their test planning could to see the growth and the maturity of become obsolete. What if the fea- a feature and a product. I feel more a tures morph in such a way as to crush part of the process of development the test planning and even crush the with an iterative approach to building spirit of a waterfall-oriented tester? I

11 SOFTWARE TESTING MAY/JUNE 2009 think it’s a matter of making a mind tions that help the development shift. process move along. I especially enjoy Early iterations of functionality asking development a question and provide time to learn and explore. receiving a pensive look from a devel- Learning is opportunity. While I’m oper, telling me I’ve just found a bug learning the product, I’m devising in code that I’ve not even seen or that more ideas of how to test and what to hasn’t even been built—talk about test. Cycles of continuous integration finding your bugs upstream. Frankly, are chances to pick up speed with manual testing and fine-tune test automation. I’ve long loathed The iterative process also gives me the idea that a fat a time to build test data. I’m able to EDITOR’S stack of requirements LETTER take advantage of the time. I also see my learning grow in parallel to the could give me maturity of a feature. As I’m learning, everything I needed. a and the functionality is settling down, CHAPTER 1 I can’t help but feel that both the A SOA VIEW OF TESTING product as a whole and I as a tester there’s also a greater chance that my are becoming more mature, robust own ideas will be incorporated, since and effective. I’m learning and devis- agile gives me the opportunity to pro- a ing more ideas of how to test and vide feedback about features early CHAPTER 2 AGILE TESTING what to test and potentially building enough that my ideas can actually STRATEGIES: test data to test. We’re growing make it into a product. This is far EVOLVING WITH THE together. more likely with agile than if the prod- PRODUCT Cem Kaner outlined the approach uct was fully baked before I had my of investigatory testing, and Scott first whiff. Ambler discussed it in an article I’ve long loathed the idea that a fat called “Agile Testing Strategies”. stack of requirements could give me Having been through a couple of agile everything I needed. “Here you go,” projects before reading Scott’s article, I’ve been told. “Now you can write it was interesting to find myself nod- those test cases you testers build, ding in agreement throughout. One of because if you only test to require- Kaner’s ideas on investigative testing ments, the requirements and specifi- that I especially like and have experi- cations are all you need.” I want to enced is finding both “big picture” holler, “No, it isn’t so!” An hour at the and “little picture” issues. keyboard has more value to me than I enjoy the early iterations to a day reading specifications. I want stomp about an application so that time with an application, I want time I can understand it. I like seeing my to play, and I want time to learn. And knowledge escalate to the point the agile iterative process gives me where I’m asking good, solid ques- that.

12 SOFTWARE TESTING MAY/JUNE 2009 .PUTTING STRATEGY INTO record an idea at any time by having PRACTICE: OUTLINE TEST IDEAS a notebook, index cards, a voice Review the user stories created for recorder—whatever the medium, upcoming features. Read each story make the process easy. closely, envisioning the functionality. Write questions alongside the user stories; if possible, record your ques- GAINASENSEOFPRIORITY tions on a wiki where your comments After reviewing all of the user stories, and the user stories can be viewed by develop a sense of priority within the team. Sharing your early thoughts each story and then across all the sto- and questions on a team wiki gives ries. What do you want to test first? the entire team a chance to see the What have you been waiting to see? a questions being asked. Raising more And what test idea is most likely to EDITOR’S LETTER questions and conversations helps the evoke the greatest issue? These are entire team’s product knowledge the first tests. evolve. Also record your test ideas. The theory behind the first-strike a This gives the developers a chance to attack is the old informal risk analysis. CHAPTER 1 A SOA VIEW see the test ideas that you have. Mis- Risk analysis doesn’t go away with OF TESTING communications have an early chance agile. for clarification. When the next iteration comes, I’m When I write my early test ideas, not in pause mode. I know where I a they’re usually formatted as a bullet- want to be, I know which tests I want CHAPTER 2 AGILE TESTING ed list—nothing glamorous. I raise my to run. I don’t have any long test STRATEGIES: EVOLVING questions with the team and the scripts to rework because of changes. WITH THE developers on team calls or at times I I expect change. As I test these early PRODUCT arrange for that purpose. I openly releases, I take notes. I adjust my share my test ideas and hunches at ideas as I learn the application. I find any possible time that I can. I tell that some early test ideas no longer every developer I work with that my make sense, no longer apply. I gener- work is not and does not need to be ate new ideas as I learn with each mysterious to them. I share as much iteration. as anyone is willing to listen to or review as early and as often as I can. To be honest, often the developers are USE SWEEP TESTING busy and ask for less than I would pre- In 2007, I blogged about my approach fer, but that’s OK. to exploratory testing through a con- I sometimes mix how I record my cept I call sweep testing. The practice test ideas, but the recording isn’t the is one of accumulating test ideas, point. The idea is to be constantly often using Excel. I could say I build thinking of new and inventive ways to test ideas into a central repository, find a break in the product. Find a way but the term “repository” has such a to record test ideas. Be prepared to heavy sound to it, when what I really

13 SOFTWARE TESTING MAY/JUNE 2009

mean is that I take the time to collect approach. I write in bulleted lists. The those scraps of test ideas and early lists are ideas, not steps or details. hunches and pool them together in Now it’s not just the product that one place. As the early iterations con- has evolved through the iterations—I tinue and my test ideas come to me at have evolved as a tester as well. random, unplanned times, I’ve jotted down ideas on index cards in my car, as voice recordings on my BlackBerry COLLABORATIVE SPIRIT: or as notes on the team wiki. PEOPLE WORKING TOGETHER As the project continues, early test The shortage of people, money and ideas fall by the wayside and new time, and the twisted situations each ideas come into place. Taking the time of these can present—agile doesn’t a to list my ideas in one place helps me cure all the blues. So it’s best not to be EDITOR’S LETTER rethink ideas and gain that revised, too naïve, to think a process change better informed sense of order and or a change in approach is going to importance. Once I’ve built my list suddenly resolve all the murky events a and my list sweeps across all the and behaviors that can happen on a CHAPTER 1 A SOA VIEW functionality, I’m ready. I feel pre- project. OF TESTING pared, and I can launch into testing I still believe that the most powerful rapidly once the build hits. advantage on any project is people I function as the only tester on proj- working together who want to work a ects fairly often, so I’m able to use my together. This is one of the core prin- CHAPTER 2 AGILE TESTING concept of sweep testing without ciples of the context-driven school STRATEGIES: EVOLVING needing to share, do version control of testing. WITH THE or address the issues that come into Bret Pettichord discusses several PRODUCT ways to build a collaborative spirit in his presentation “Agile Testing The idea is to be Practices.” The concepts of pairing constantly thinking together and no one developing spe- of new and inventive cialized knowledge are two constructs that I find especially insightful. I just ways to find a break seem to work alone more often than in the product. not and have had fewer chances per- sonally to put some of his collabora- tive test team concepts into practice. play when I’m not testing alone. On But as the solo tester, I can certainly projects where I have had other relate to and find his thoughts on test- testers, I have used the concept of ing “half-baked” code highly practical. sweep testing to assign different A small benefit I’ve seen in working worksheets or sections of a sheet with scrum has been a side effect of from the same Excel file and had no the daily scrum meeting that I hadn’t issues with the divide-and-conquer expected. The frequency of the meet-

15 SOFTWARE TESTING MAY/JUNE 2009 ings helps to drive a friendly familiari- to be integrated with the team. This ty. Given the short duration of the concept resonates deeply with me; meetings, people seem less inclined I have long felt it is the team and not to open a laptop and ignore each a single stakeholder who will know what value I have brought to the proj- ect. As an independent consultant, These daily meetings this has been interesting; I often have bring people together; a stakeholder who owns my contract, but it is the team that knows what I they build a sense deliver. of team and, for that reason alone, add a CONCLUSION EDITOR’S great value. LETTER While some of the test practices in agile may be different than working on a waterfall project, the fundamen- a other as they check emails and the tal ideas in testing still apply. Explor- CHAPTER 1 A SOA VIEW daily news. The forced frequency atory and investigative testing pro- OF TESTING seems to help. A more practical duce more bugs that matter in a aspect is hearing from team members timelier manner than detailed test what activities people have been scripts can provide. Risk-based think- a pulled into, how time has been spent ing helps steer testing to find the CHAPTER 2 AGILE TESTING or misspent. People have a place to defects that matter most. Testing will STRATEGIES: EVOLVING explain how hours have been burnt always be affected by the reality that WITH THE over disruptive, unexpected and time at the end game is tight, and the PRODUCT expensive tasks. These daily meetings best way to cope with tight time is to bring people together; they build a be flexible. And people who want to sense of team and, for that reason work together can produce great alone, add great value. products, so finding ways to build In his “Agile Test Strategies and healthy, constructive relationships Experiences” article, Fran O’Hara talks is likely the most valuable strategy about “independent testers” needing of all. I

ABOUTTHEAUTHOR: KARENJOHNSON is an independent software test consultant with 14 years of experience in software testing and software test management. She views software testing as an intellectual challenge and believes in the context-driven school of testing. Karen frequently speaks at soft- ware testing conferences and participates in software testing workshops, in addition to author- ing articles in software testing publications.

16 SOFTWARE TESTING MAY/JUNE 2009 FROMOURSPONSOR

q Learn More and Download TestComplete Free

a About AutomatedQA: AutomatedQA has been making award-winning soft- EDITOR’S ware products for quality assurance and software development worldwide LETTER since 1999. AutomatedQA’s flagship product is TestComplete, the automated soft- a ware testing solution that’s easy to use and affordable for any size team. CHAPTER 1 TestComplete makes testing fast and simple for all Windows software, A SOA VIEW OF TESTING including Web, .NET, Java, Windows Desktop, Flash/Flex, Ajax and Sil- verlight. TestComplete’s easy script-free keyword testing and unified interface a enable testers to learn just one product and automate the full range of test CHAPTER 2 types including functional testing, load testing, distributed client/server AGILE TESTING STRATEGIES: testing and unit testing for Windows and Web. EVOLVING WITH THE PRODUCT

17 SOFTWARE TESTING MAY/JUNE 2009 FROMOURSPONSOR

q Improved Software Testing Using McCabe IQ Coverage Analysis

q More Complex = Less Secure: Miss a Test Path and You Could Get Hacked

q Using Code Quality Metrics in Management of Outsourced Development and Maintenance

About McCabe Software, Inc.: McCabe Software provides Software Quality Management and Software Change & Configuration Management solutions worldwide. “McCabe IQ” is used to analyze and visualize the quality and test coverage of mission, life, and business critical applications, utilizing a comprehensive set of advanced software metrics including the McCabe- authored Cyclomatic Complexity metric. Our configuration management solution, “McCabe CM”, utilizes exclusive Integrated Difference technology to manage software changes faster and more efficiently, ensuring quality throughout the Application Lifecycle. McCabe Software has offices in the United States, distribution worldwide, and can be found online at www.mccabe.com.

18 SOFTWARE TESTING MAY/JUNE 2009 FROMOURSPONSOR

q iTKO Whitepaper: Service Virtualization in Enterprise Application Development

a iTKO Whitepaper: Minimize IT Outsourcing Risk with Collaborative EDITOR’S q LETTER Quality

a q Free Analyst Report from Butler Group: LISA 4.6 Technology Audit CHAPTER 1 A SOA VIEW OF TESTING

a About iTKO: iTKO helps our customers transform the software development CHAPTER 2 and testing lifecycle for greater quality and agility in an environment of con- AGILE TESTING stant change. iTKO’s award winning LISA(tm) product suite can dramatically STRATEGIES: EVOLVING lower quality assurance costs, shorten release cycles, reduce risks, and WITH THE PRODUCT eliminate critical development and testing constraints by virtualizing IT resources to provide accessibility, capacity and security across interdepend- ent teams. LISA enables test, validation, and virtualization solutions opti- mized for distributed, multi-tier applications that leverage SOA, BPM, inte- gration suites, and ESBs. iTKO customers include eBay, American Airlines, Allstate, Time Warner, SwissRe, Bank of America and the U.S. Department of Defense. Visit http://www.itko.com.

19 SOFTWARE TESTING MAY/JUNE 2009