INTRODUCTION

evOps is no doubt one of the buzzwords for today’s agile company culture. D It has been widely adopted in variety of different types of software companies building yet more different types of products. What DevOps actually mean and what are the implications of this approach for mobile app development and testing? As defined by Wikipedia:

“Practices that emphasize the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes.” –Wikipedia

What drives companies towards this type of DevOps thinking? First of all, there are always business requirements and demand that drive the adoption of any sort of process or culture changes within companies. With DevOps approach the apparent benefits for business are the full adoption into agile methodology, getting things done faster and with focused mentality on support, operations and services. And when these two are combined businesses can get both as efficiently as possible.

Although the bar has been set high for mobile apps to be successful, there are lots of different characters, things that should be considered when adopting DevOps approach for mobile app development and testing. These requirements are driven by app developers, business, and most importantly, the end users.

As the beneficiary of DevOps practices, we would like to share what we have learned from adopting DevOps for mobile projects and what we see is valuable for your mobile teams.

Share this eBook! CHAPTER 1

Deep Dive on DevOps in Mobile App Testing

Share this eBook! obile is more important than ever. The increasing competition with mobile apps M and user acquisition has pushed companies to act quickly with its development and operations. Getting that app ‘right’ in the first place, and working properly with a myriad of different device configurations is a task that should be considered way before mobile app is published. In this the DevOps culture, process and tools have a significant role.

What is Mobile DevOps? The DevOps thinking has become an established part of companies and development teams relying on agile methodologies. The combination of agile methodologies and DevOps are in essence to provide and continuous deployment to speed up the integration, deployment and testing, plus improve the quality of application. Getting the mobile app instantly from the build system to real device where application can be thoroughly tested improves the overall process velocity and makes the bug fixing easier and faster.

What is Mobile DevOps?

The DevOps thinking has become an established part of companies and development teams relying on agile methodologies. The combination of agile methodologies and DevOps are in essence to provide continuous integration and continuous deployment to speed up the integration, deployment and testing, plus improve the quality of application. Getting the mobile app instantly from the build system to real device where application can be thoroughly tested improves the overall process velocity and makes the bug fixing easier and faster.

Share this eBook! The DevOps in mobile means incorporating the continuous integration and testing process tightly with the delivery and deployment, and wrapping up everything in continuous operations. Developers – and development in general – focus on building the actual product while relying on their own relevant tools for them. The quality assurance (QA) is the stakeholder that is interested (and responsible for) in testing those regressions, performance of the product, and preferably automating as many aspects of the testing flow as possible. The operations maintain the continuous build, integration, deployment and delivery environments, and take care of those releases. When combining all these three different functions into one holistic approach Mobile DevOps can be used as a definition.

3 Rule of Thumbs in Mobile DevOps

Mobile DevOps is an approach that focus on communication, integration, deployment, automation and measurement of collaboration between app developers, testers (QA folks) and the operations. The whole thing is to acknowledge the interdependence nature of this approach and rely on the right tools, methods and infrastructure in order to develop, deliver and test apps faster and more efficiently.

Communication Integration Deployment

Develop and Test Against Real Environment

The most important thing for mobile developers is that the end-product works across on all its target devices. Using emulators or simulators are acceptable in the earliest phase of the development, but when app becomes more sophisticated and have the features that end-users will see, the only acceptable way is to test mobile apps on real devices.

Share this eBook! Nowadays, it’s not a cost issue and nevertheless organizations spend significant sums in their hardware infrastructure. It’s well justified to not to try to save in the most important hardware cost – the real mobile devices – to get the app thoroughly and comprehensively tested out. As said, not everything should be acquired by organization itself, but using a cloud service that contains the right set of mobile devices is a perfect fit for today’s mobile toolchain.

Deploy and Test Frequently (Many Times, Every Day)

Agile methodology praises the use of continuous integration and delivery tools. There are plenty of great open source as well as commercial tools available that help organizations to build, deploy, test and release their apps. The most meaningful value- add to this flow is the mobile that can accommodate frequent builds to testing context and enable instant bug hunting.

The continuous development and testing integration can enable companies to respond to market and competitive changes in timely manner. Reusing the code, back-end implementations, APIs and other infrastructure to build a new version of the mobile application is easier, as well as testing of that when infrastructure is in place. The frequent deployment and testing also enhances ability to automate functional, compatibility and performance testing of the application, against an array of devices and different form factors.

Continuously Consider and Validate Quality Characteristics

The modern mobile devops is (all) about continuous everything. The DevOps approach should always strive to improve things, process, culture, and even reconsider what devops

Share this eBook! tools should be in use. The continuous evaluation of the efficiency and productivity can get insights to real performance issues and how to make things smoother and work seamlessly in the next regression.

Changing the devops tools should be considered every now and then, especially if there are any doubts about the current environment, its capability t o deliver and work as efficiently as possible. And this doesn’t apply only for devops tools but also how mobile devices and device infrastructure is used. The requirements are constantly moving and that primarily due to demand by end-users. The quality characteristics of mobile app are set from the first line of code until it’s on hands of end- users, and everything that happens between that should be measured, validated and fixed – if needed.

3 Business-Critical Mobile DevOps Considerations

From the business perspective, the ability to quickly introduce new app, features in it, and bug fix releases is one of the most critical tasks for companies with mobile-first strategy. In essence, the core of DevOps methodology aims at speeding up the app development, delivery and process by getting developers and operation specialists to collaborate throughout the end-to-end app development and deployment process.

Continuous Integration and

Continuous integration and delivery can be measured with various different metrics. However, what are the most important ones for mobile app development and testing matter the most. For example:

Share this eBook! • Frequency of deployment and tests executed (against each regression). • How many new features, lines of code or other integrations have been deployed by each build. • What is the actual time between the development (when it started) and test finalization. • Percentage and total number of failures vs. successful tests. • Urgency to get mobile app published. • Performance of automation. Do you need to tweak in something manually?

Now, these metrics are valid for every regression of the app. Each and every time when source code is modified and new build is deployed, it should be thoroughly tested. With test automation that’s not a challenge but really a smooth process that gets instant results without any manual intervention.

Application and Infrastructure Testing and Monitoring

When talking about mobile apps it’s not only about the APK or IPA deployed on device, but more often there is tremendous number of dependencies to other applications, backends, and network(s). Developing, deploying and testing in the infrastructure context also improves insights of how well the entire environment does and how application performs. In this context, it’s also much easier to drive iterations to make usability, UI and user experience aspects better: If something lags or performs bad in the system it will have an impact to all other pieces of the system.

Monitoring mobile apps, websites, APIs and all other relevant parts is also important. While app may perform just fine there are other entities that doesn’t necessarily provide the same level Monitor of performance. Accurate performance analysis with the real data helps improving the overall development flow.

Share this eBook! Comprehensive Dev & Test Process and Mobile App Delivery

It’s always important to remember that organizations that use agile methodology should reduce things that get done manually. Things that get done manually (especially testing) are never agile. The agile development sprints that include test automation will be faster and generally the adoption of the right tools, methods and technologies speed up the development and testing tremendously. Agile development is the epicenter of today’s mobile app development process, culture, and should be reflected in tools used by developers.

Share this eBook! CHAPTER 2

Effective Mobile DevOps Strategy and Typical Goals

Share this eBook! he effectiveness of adoption Mobile DevOps depends on your goals and metric T that naturally vary across companies. That being said, in general there are four characteristics that by maximized can deliver the best outcome for Mobile DevOps approach and yield as significantly improved results in your mobile app quality.

“The highest priority is to satisfy the customer through early and continuous delivery of valuable software.” — Agile Manifesto – Principle #1

Overall, Mobile DevOps adopters should strive for four goals. These goals are the cornerstones in creation of Mobile DevOps strategy and a culture that helps to achieve the expected quality and faster turnaround to market. Let’s break it down.

Maximize the Delivered Value

The trend – and perhaps motivation towards DevOps adoption for many companies has been in a more efficient way to get things done. This is typically seen as cost savings and enables people to get things done faster (and more efficiently). Today, for large enterprises, DevOps, mobility and mobile apps used by its employees are the value drivers as they boost productivity, make the daily-work more efficient and drive the business value.

Share this eBook! For many enterprises maximizing the value from technology investments has changed the way they spend money on services. As proposed in the article, spending smart but quickly to deliver results is yet more efficient than just keep spending big with no measurements on return-on-investment.

For enterprise IT, mobility and mobile apps in general hold the potential to boost their productivity and drive business value of the entire organization. And to make the most out of mobile apps, enterprises need to put in place a comprehensive mobility strategy that can work as a holistic approach, encompassing all aspects from mobile app development to security and connectivity.

Maximize the Effectiveness, Efficiency and Productivity

Comparing and measuring productivity, efficiency, and effectiveness equals to the faster time-to-market, speed of actual delivery, deployment and turnaround with testing. And all of these cumulate as higher quality in mobile apps which means the most crucial quality factor.

“ The effort of maximizing the efficiency and effectiveness can improve the overall productivity. ”

The Mobile DevOps effectiveness, which can be seen as doing the right things, means that there is a great chance to achieve expected results. But doing the right things doesn’t mean products will be successful. Doing the right things alone is not sufficient to achieve to the ultimate success. DevOps strategy needs to consider efficiency side of things as well to complement the overall outcome.

Share this eBook! The Mobile DevOps efficiency, which will be seen as doing things right, is the end-result of the effort with minimal costs, time and overall contribution. From the business perspective DevOps efficiency can mean the use of minimal resources to accomplish the goals.

The effort of maximizing the efficiency and effectiveness can improve the overall productivity. What this means in mobile DevOps strategy is that test automation must be included in the testing of each build of the application. Test automation can provide tons of benefits from making sure the device compatibility is met, defects/bugs are instantly spotted out, and those get – with great details of why failures happen – in hands of capable developers in order to get fixed. This itself can control the cost as earlier fixed problem doesn’t drag on with the build by build, but also it will cumulate as higher quality. Finally, as problems are solved earlier and verified against real devices, the overall efficiency is improved and individual productivity will be higher.

That being said, efficiency without effectiveness doesn’t guarantee the best results. “If ain’t broken, don’t fix it” means that the DevOps must focus on the right items; if the

Share this eBook! problem with a mobile app is in general compatibility across device variants, OS versions and other characteristics, it really doesn’t matter what sort of features are built in. The distinction between the best app and average app are typically the general compatibility, great performance, and fully exploited hardware.

To maximize the effectiveness, efficiency and productivity starts with the full adoption of mobile test automation, with automated, continuous integration, deployment and testing, that is exercised with the every application regression coming out from the development pipeline.

Maximize the Quality and Robustness of Apps

The most critical thing for companies building a mobile app is the quality. Indeed, the process, approach or used methodologies are all secondary if the quality isn’t up-to-snuff and product cannot be showcased for its excellence and robustness. To meet and exceed this requirement many enterprises have adopted DevOps culture and many of those that

“The most critical thing for companies building a mobile app is the quality.”

did it early have been very pleased with the results. Clearly, a well-established and well- executed DevOps strategy has been the key to the successful adoption.

What are the key tasks or things to focus in the Mobile DevOps approach or Mobile DevOps tools? First of all, all used DevOps tools must be collaborative that enable the seamless automation and are technically possible to be integrated with each others. Non-collaborative DevOps tools can make the entire process and flow challenging and make the effort of building better quality and more robust apps difficult.

Share this eBook! The Mobile DevOps approach itself creates a culture that helps in achieving those strategic goals by employing the following aspects:

Collaborative Process, Development, Testing and Release – and Support. The Mobile DevOps approach requires continuous collaboration between all teams involved, from developers to testers and support (post-release).

Continuous Integration and Continuous Testing. Regardless of if agile methodology adopted continuous integration and testing are must to have for any organization building mobile apps. Using these standard DevOps tools efficiently will have an impact on quality, robustness and app compatibility across devices.

Continuous Deployment and Continuous Release. The quality process should start as soon as the first line of code has been written. After each regression and stage until the final product, the quality concerns should be the most highest priority for Mobile DevOps to ensure application quality criteria is met and product is deployable and ready for the release. Furthermore, testing effort should continue even application is published as post- release issues come up quickly – and developers needs to jump on them.

Share this eBook! Continuous Monitoring. Performance of mobile application is important in every step of the product development lifecycle. Getting a comprehensive understanding of how well application does on different devices, what are the bottlenecks and how application can be adjusted/optimized to provide better user experience for its users.

Maximize the Support, Service & The Delivery

In agile methodology, support, service or product, and deployment are highly important to determine whether the delivery can be successful. The Mobile DevOps is constant alignment and collaboration between different teams, all focusing and aiming for better quality, and yes – continuously enhancing the process.

For instance, continuous testing as part of the development flow aims to deliver customer- facing, ready, high-quality applications. While starting to do this when application is still under development and continue it for all different stages of the product development the quality criteria can be met. This also has an implication on support and how the product is treated after by support after it has been published. Including unit testing, UI functional testing, API testing and all possible integration testing with different entities can ensure that there will be less surprises while the mobile gets mature.

Finally, effective testing and quality process as part of the Mobile DevOps thinking can enable much faster turnarounds, ensure that developers and QA folks can do constant releases with confidence, and minimize risks related to quality concerns.

Share this eBook! CHAPTER 3

How Mobile DevOps Can Make A Difference

Share this eBook! he DevOps approach have had notable differences for enterprises striving to streamline T their IT operations. This positive impact on companies that adopt DevOps (and Mobile DevOps) approach has enabled them to build collaboration across teams, something that typically wasn’t that obvious in the past, just be focusing on adopting DevOps practices with organization. There are lots of great success stories about how DevOps mentality changes the way companies were doing things, making process yet more faster and streamlined, and enabling companies to save money.

Despite of many of these panegyrical stories about the positive impact of DevOps for enterprises there are also challenges and other things organizations must tap in. During the past years, use of agile methodologies and impact of it for development, operations and all other parts of the organizations have been accelerating the DevOps movement and breaking down those boundaries between these functions. The question is – how Mobile DevOps can make a difference to an organization? And is the distinctive difference even between regular DevOps and Mobile DevOps?

Building Mobile DevOps culture isn’t necessarily easy. Like any process, culture or behavior changes within organizations it needs to be implemented properly and adopted step-by-step to encourage all stakeholders. It simply doesn’t happen overnight, but modernizing of process, DevOps tools and practices will respond to business demand.

Share this eBook! How, Why and Where Mobile DevOps Makes A Difference

The discipline for Mobile DevOps requires few important technical and support things to be completed to make difference for any organization. In order to get all pieces together and make the Mobile DevOps tools and process seamless for companies the problem and potential bottlenecks must be identified. These are the top things that came up in our survey and companies should focus on either fixing or adopting the better Mobile DevOps tools and process to get these challenges tackled.

(Way) Too Long Dev-Test-Feedback Cycles

Comparing fully adopted Mobile DevOps approach for development, testing, and deployment the old way of doing things lack a shortcut to reveal, document, and eventually fix found issues. If the development cycle including building, testing, and deployment is too long the development progress might advance faster and it’s definitely harder to keep up.

Even with agile methodology in use, each step in the Mobile DevOps must be carefully evaluated and measured for efficiency and effectiveness. That eventually makes the process more productive, delivers value, and maximized the critical business advantages. If the process is too slow some things can be boosted up, paced up and even eliminated if those do not have any impact on the results. Mobile test automation can really help in enabling continuous, frequent and instant releases that can be then quickly integrated, deployed and testing in automation context, and in timely manner. Share this eBook! Bug Catching/Reporting Too Slow to Developers

When the right tools are in use everybody gets notified about problems pretty much instantly. Then there is only motivational and attitude that prevents developers to jump on those issues. Getting as rich and detailed results as possible will help developers to solve issues. You know, developers get the DNA of the wrong behavior, bugs or failures from log information, so this is critical to be delivered and preferably from as many devices as possible.

Non-Functional or Not Well Utilized Test Automation

Relying on non-agile methodologies, or not fully adopted agile methodology with recommended functions as a mandatory can make things very difficult in the testing phase. Many of failures with test automation has been due to lack of knowledge and skills to adopt agile test automation for mobile app testing. There are still stories why many developers see test automation as something great, but just haven’t got real-life cases to get things running on variety of different devices. Because of this, test automation has been doomed by those who haven’t got it fully implemented and working. Instead of fully functional test automation many users have done basic unit/system tests.

Use of Non-Real (Emulators) Test Environment

Testing on emulators gives you something result-wise. But again, are those results trustworthy? No end-user will run your application on emulator that is typically executed on x86-powered desktop, with much more memory, with different network infrastructure, and lack of all critical hardware components (different GPU, screen, its resolutions & orientation, WiFi components, and many others). There are very good reasons to stick with real devices and test only using as authentic environment as possible.

Share this eBook! Setting Up & Maintaining Real Devices

This has been an issue and one remarkable reason why developer organizations haven’t seen on-premise labs as worth it. Based on the facts, the need for different mobile devices (with all diverse software and hardware characteristics) has gone significantly up. For example, when the typical mobile device farm built internally was about 20-50 devices in 2014, the requirement has already exploded to 500+ devices. That’s significant growth number when it comes to use of real devices for testing.

Nowadays, getting and setting up that level of device farm is no problem. For example, our Testdroid PrivateCloud caters mobile app developers with any number of devices team must test their app on. Depending on how many devices are reserved and dedicated for customer comes from the market demand, and for what purpose is the application targeted for. Typically, the answer for this is “globally”.

Share this eBook! Access to Relevant Mobile Platform / OS version / Devices

Fragmentation is not a new phenomena for either major OS platforms – Android and iOS – and devices. There are lots of users that are passionate to upgrade to the latest one – and the other half who are just happy how things currently work on the device. In the middle of this, there are also users that have upgraded their OS version at some point but do not want to go the latest or simply their device OEM doesn’t provide upgrade for that specific device model. That’s very common and is also known as device/user/platform fragmentation.

By defining what is a relevant OS version, as well as finding out devices that people use, can be typically a challenging task. Companies building professional apps may need to investigate both Android Dashboard and Apple’s iOS Dashboard to get better understanding what OS versions are actually used.

Use of Manual Testing – The Weakest Link in Agile and DevOps Process

Manual testing is still (by far) the most used way to test anything on mobile. And that’s the biggest resource hog. Sinking effort and resources to manual doing while there are awesome test automation to help is a bad choice. Organizations understand this and want to build up that test automation competence to get things moving faster, cheaper and getting better results.

Share this eBook! No Geographically Relevant Testbed – Devices, Networks, Back-End Integration

The target market matters you the most. If (and typically yes) app developers are targeting their app globally as potential with global app market is simply huge. Instantly, there are hundreds of millions of potential users for the application, game or web product – and making sure it will work well on all possible devices these users use.

According to the results of our survey, there are three major things that app developers are concerned about: 1) devices, 2) networks, and 3) back-end.

Share this eBook! CHAPTER 4

5 Critical Daily Tasks for Mobile DevOps

Share this eBook! any companies that have adopted Mobile DevOps as part of their product M development lifecycle are also using internal check lists to ensure the expectations of their software and delivery are met. However, these types of check lists also form a great baseline to measure what are the critical areas of improvement for the future and what parts of the process may need to be potentially enhanced.

When looking at Mobile DevOps process there are only few top categories that need daily attention. But those critical daily tasks must be taken care to ensure the development, testing, deployment and delivery have a smooth flow and teams get things done. If we look at what Mobile DevOps is really about, it can be roughly categorized in 6 different subsets to evaluate how well the process is established and probably what parts of it could be improved. The checklist was divided into 6 different topics:

– DevOps Alignment. DevOps is an organizational shift, not just a technical one and unifying group and individual direction and goals around the singular vision of the organization, can make a difference. – DevOps Context. This could be also ‘DevOps participation’. Is everyone participating and is the process adopted by all critical stakeholders? The DevOps context is actually making all relevant information available to those who need it, and when they need it. – DevOps Learning. Eventually, everyone involved should be learning to execute and by learning those best practices DevOps life can be easier. It’s all about how to empower your team and your personal growth and how to foster understanding through continuous improvement. – DevOps Lifecycle. Focus on software as a product with care, attention and reflection, and be a part of the ever-changing ecosystem.

Share this eBook! – DevOps Organization and Resources. Providing structure for interactions and ground rules on supporting collaboration and productivity. – DevOps Process. Habits crafted to foster consistency, confidence and collaboration, provided within Mobile DevOps framework to achieve continuous improvement.

Let’s see what are those 5 critical targets for Mobile DevOps everyday.

5 Daily Tasks of Mobile DevOps

Here are the top 5 tedious things for mobile devops – if you don’t rely on sophisticated device farm or have invested in devices to run on a local environment – that should be taken care of every day:

Mobile Devices

Handling mobile devices can be a cumbersome task. Depending on how large on-premise device farm company may have it’s still number one the most important daily check- up target for mobile devops. This is the top reason why mobile device farms are getting significant popularity among mobile-first companies.

Share this eBook! For example, have you ever thought how to enable a new Android device in your local test lab? Here are few tips and tricks – and the actual checklist we go through every time we add a new device in our device farm:

1. Set WiFi network. – Without network, devices and testing capabilities are limited.

2. Add Google account to device. – And good rule of thumb is to uncheck if auto-sync for account is enabled. If you leave it checked your daily tasks will be quickly all about checking why devices aren’t ready for testing…

3. Keeping Google Play Store relevant. – It’s actually important and for sure makes your life easier to have the Google Play store updated and device installed with the latest version. Otherwise, you’ll start getting notifications and again, the circle of ‘why devices are stuck and my tests are flowing’ will hit your fan. Few tips: • Without Google Play Store installed, device won’t have required services which could be critical for some of the testing scenarios. • Start Play Store once and check possible notifications. Typically there are few so check them and close the app.

4. Install Google Play Services. – Services provided with Google platform will be in use and you can rely that there is much less thing to worry about. Also, it’s recommended to disable this functionality occasionally to test how apps work WITHOUT these types of services: • Open Google Play Store and search “Google Play Services” • NOTE! Google Play Services are updated quite often. Without upgrading your device will get notifications about a need to upgrade Google Play Service

Share this eBook! 5. Install Google Play Games. – Similar to Play Services, and especially if your company is developing a mobile game. Absolutely must to have: • This is required if – for example – you are testing games or your app depends on it. • Open Google Play Store, search “Google Play Games” and install it.

6. Disable security checks for installing applications through USB connection. – Android is not like Apple and one of its greatness is that apps CAN be installed from various locations, by various developers and sources, without confirming everything. My take, I think Apple will shy away this sort of features in time, as it can really make this tedious for mobile devops: • Enter “Development options” on Device settings • Check “USB debugging”, “Stay awake” and “Allow mock location” • Uncheck “Verify apps over USB” • Enter “Security” on Device settings • Check “Unknown sources” • Uncheck “Verify apps” (This is not necessarily available with all versions of Android)

7. Disable Screen Lock. This will save the day. – If screen locks for any reason (not used for some time) with some test scripts tests may be • Enter “Security” on Device settings • Go to “Screen lock” settings and select “None” • If this is not possible, you need to configure device to be unlocked from Device settings on Testdroid Admin pages.

8. Set correct Date & Time to device. – If you get questions why tests were executed yesterday or tomorrow, you know what is wrong and how to fix it.

Share this eBook! 9. Set all “Location Services” on.

10. Install Chrome Browser and additional browsers. – Eventually people want to test on different browsers than what was delivered as part of the device. It’s good rule of thumb to add few of them, and preferable the most latest and the most used ones: • Install Google Chrome Browser, if your tests needs it • Sign in to Chrome browser to make it work robust on tests.

11. Set Screen timeout to maximum value. – The same things as with screen lock, if the screen stays up for longer period time, the better: • Enter “Display -> Sleep” in Device settings • Select maximum timeout value for sleep.

Now, these were the tips for adding a new device. But as you can see, if that is not done ‘the right way’ mobile devops guys will have a lots of things to do on these devices, every day.

Infrastructural Hardware

Let’s put devices aside and think about what else is required to run a mobile device lab. As there are a myriad of moving components, attributes, that all must be maintained and checked whether everything is up in speed, many of the mobile device farms rely on either some internal monitoring implementation or commercial ones to keep everyone up-to-date with the status. For mobile devops, the monitoring tools and features provide the most fastest way to spot if something is (seriously) wrong.

Share this eBook! Ready-to-go infrastructure is typically a feature on premium mobile device farms and easily delivers what it promises: no delays for development, testing, deployment, and always focusing on result-driven approach. In long run, this is one of the biggest factor to save money and make things run faster and be more robust.

Support for Developers, Testers, QA – and Customers

The Mobile DevOps culture relies heavily on communication and collaboration across teams, and very often it is the DevOps guys that provide internal support for devs, testers, QA folks – and sometimes even to customers. No, it’s not a full blow customer support division, but when operating large scale labs for different counterparts, they are the best resource to collaborate with external customers as well.

Support provided by Mobile DevOps can be just helping to get test runs progressing, running faster or making sure that (test) results are delivered. Software developers in test (SDET) and QA engineer need different type of support when using the infrastructure, devices, and basically the agility produced with a DevOps approach can help a lot. And get those important business metrics met (and exceeded).

Share this eBook! Operational Side of Things

Isn’t this no-brainer? Actually, it’s again and again making sure that all infrastructure is working properly. And as devices, infrastructural hardware (servers, databases, monitoring tools, additional physical hardware, among all the other things) and integration points work out, there is still the connectivity. Power supply and making all infrastructure powered is very critical, but so is the connectivity of all these things together. Typically there are a dozen of WiFi networks device farms must use and those monitoring software must continuously scan what is the current service level and how much can be gained from the service provider.

Integration with DevOps Tools and Test Automation Frameworks

There is no doubt: open source frameworks, technologies, and tools are the most successful when it comes to use of those in today’s agile processes. It all starts from the nature of those products. When something is open it basically means these products are always available and usable, can be easily changed to something else, and this gives the reliability and freedom for organizations to change those any given day.

Proprietary frameworks and tools may offer better support, but that’s just another side of the coin. These proprietary tools, frameworks platforms have disadvantages in that you are dependent on the limited capabilities that a vendor can provide and if you need

Share this eBook! to do something outside of the box you don’t have many options other than begging for enhancements and having to wait.

The maturity of open source testing frameworks such as Appium, Calabash, UI Automation and Espresso have come a long way. There are vibrant communities surrounding these platforms and lots of contributors. Most use cases are covered and you have the ability to branch for your own needs. There are a few cloud vendors supporting open source, but make sure that you’re not having to use custom libraries or methods in order to make things work, as this limits the ease of executing locally on your own devices and creates vendor lock in.

What does this mean to Mobile DevOps and does it need to happen everyday? Well, making sure that open source enablement is part of the process is crucial and helping hand is required every day to ensure test automation scripts run smooth on a set of devices. Integration flow can break easily if not monitored as people tend to bring their own habits (and yet often, proprietary, non-compatible) tools to be part of the process.

Share this eBook! CHAPTER 5

7 Process Steps with Mobile DevOps Tools

Share this eBook! ools required to develop and test against production-like environment, deployment T done with a repeatable and reliable process, and monitoring and constant validation of the process flow are those principles that vary depending on organization’s scope and level of Mobile DevOps adoption. As there are plenty of Mobile DevOps tools and all have different use case and sometimes can be overlapping each other in the overall Mobile DevOps process, we’ll take a look at different phases of the process and what tools are used in each one of them.

Today’s mobile app developers and their organizations have been increasingly investing in monitoring systems that can capture production, development, and bunch of other metrics all in real time. This is a great trend, but if monitoring is done only for specific silos of the overall flow, results may not expose how to optimize the most important resource hogs and therefore monitoring only produces imperfect results.

There has been lots of discussion about ‘shift-left’ concept where operations focus more on earlier phases of the delivery process and entire production lifecycle. Especially in mobile app testing, this approach and goals are highly useful as app should be always tested against real environment – not ‘close-enough’ to production environment, but actually using the production environment, which in case of Mobile DevOps would mean real physical devices. In addition, it would easily expose issues in earlier phases of the development and issues would be more cost efficient and easier to fix.

Whereas testing has been traditionally seen as one of the final steps of the process, it is nowadays in the middle of the

Share this eBook! process, and right after each build. This way the agility of the development and testing isn’t compromised and more data is acquired after each test cycle. This test data with all possible details can significantly help development to fix issues and create more robust apps.

In general, there are 7 widely accepted and used steps in the Mobile DevOps process flow.

7 Process Steps with Mobile DevOps Tools

Instead of traditional 4 step process – plan, develop & test, deploy and operate – there are actually more flavors in the process that should be highlighted. Especially when it comes to considering what tools are used, how those fit together and enable seamless tool flow from planning to production. Automation is without shadow of doubt critical part of the

flow and enables process to be highly iterative, repeatable, and robust. Automation is actually an enabler for continuous everything, in each step of the process.

Code

First and foremost, code development, code reviews and anything to do with actual

Share this eBook! development is closely related to continuous integration. The agile approach made continuous integration very popular as it enhanced the way developers work, collaborate and integrate their doings together, in natural way. The continuous integration with regular daily routines produce lots of data about the mobile application, regressions and compatibility of each developer’s contribute, and discovering issues early exposes integration flaws, even incompatibility of developer’s doings, and makes better time consumption and schedule estimating easier and more accurate.

Build

Building application is an everyday job. In Mobile DevOps, organizations use variety of version control, source code management (SCM), different practices for merging code and notifying everyone about the build status.

There are generally few different branching strategies when it comes engineering team to work as part of the Mobile DevOps approach:

• No branching. Team works only from the main source tree. This kind of case is typical for smaller teams that do not need isolation for development, releases and builds. • Branching for release. For example, development team can create branches to support an ongoing release. In this scenario, team creates a branch before release to stabilize the release and then merges changes from the release branch back into the main source tree after the application is deployed. • Branching for maintenance. This is not so typical case but development can create a branch to maintain some older application version. Branch is created to support maintenance of specific version and current development and regressions are not jeopardized. Sometimes it’s just a quick fix for certain user-base that are not using the latest versions. • Branching for feature(s). Engineering team can also create branch based on certain feature set. This is quite typical when certain individuals or teams work on specific feature that development branch is create, the coding takes place, and the feature

Share this eBook! based branch is merged to main source. This makes parallel feature creation possible.

For branching and merging strategy, there are plenty of great repository products, version control tools and source code management software available. However, this is critical as it needs to go inline with entire development team and the same procedures must be used by all contributors.

Test

As said, testing has been one of those functions that had “shift left” due to DevOps thinking. Testing is also one of those key areas that define the quality aspects of the software; despite no error, bug or issue is fixed by QA, all or at least majority of those can be discovered during this phase. As continuous integration has several goals for testing phase, testing should be always automated to ensure seamless flow, rich and detailed results, and adequate verification of the application.

Share this eBook! Continuous testing basically means testing each regression, build, and continuously practice this throughout the development lifecycle. Early testing, early validation and frequent testing ensures quality built-in to application can be verified and issues to be tackle early. When Mobile DevOps is concerned of production-like or real environment, the testing should always go on with real mobile devices. There are hundreds of good reasons why Mobile DevOps process should not include in this stage any use of emulators/simulators.

Package

The package step includes tools for package repositories and other storage mechanism for the binaries created in the build step. Repositories, also known as artifact or asset repositories – contain all assets required and associated with binaries to facilitate deployment, such as scripts, configuration files and additional infrastructural files for deployment.

The continuous deployment adoption may not be ideal for every company (and developers), but the practice that enables releasing and deploying can also enable a seamless of a delivery pipeline and flow for Mobile DevOps. This flow facilitates continuous deployment of applications to testing (or QA) and then all way to production, in automated and efficient manner. The package step clearly supports the goal of continuous deployment to package and release new versions of applications (with new features) to end-users.

Share this eBook! Release

Most of the tooling and processes that make up the core of Mobile DevOps technology exist to facilitate continuous integration, continuous release, and continuous deployment. Part of this process is application release automation tools that help up packaging and deploying application from development to production (to be readily available for download / users), using automation.

These sort of tools aid Mobile DevOps flow to produce capabilities to implement continuous delivery for even large number of releases. In addition,

and deployment planning with every release requires collaboration across stakeholders. Release management tools allow organizations to plan and execute releases, provide a single collaboration portal for all stakeholders in a release, and provide traceability for a release and its components across all stages of the build and delivery pipeline.

Share this eBook! Configure

Configuration can be also seen as (IaC) or Infrastructure as a Service (IaaS). IaC has been claimed to be a key attribute to enable full utilization of DevOps into organizations. With this, developers can be more involved in defining configuration and operations teams can get involved earlier in the development process.

Tools and methods that utilize IaC and IaaS can bring transparency to configuration of process and provide the visibility to users within the organization, aiming to bring teams together to maximize their efforts. Automation as part of the process can remove error- prone aspects of manual doings and processes, and make things more efficient and productive.

Configuration tools aim to allow creation of better applications with flexibility, less downtime, and enhancing an overall cost effectiveness for the company. This approach is intended to reduce as many things creating complexity as possible by removing manual configurations. Automation and collaboration are the epicenter in Mobile DevOps and this is reason why configuration is also widely automated across development flows.

Monitor

There is also important aspect of making sure organization gets (only) valid data about the process, results in each step, and the outcome. Continuous monitoring provides that sort of data and metrics to organization to evaluate all different functions of the process: development, operations, testing/QA, personnel and other individuals involved in the process. It’s important to remember that these metrics should not be measured only in production but during the process itself. When done properly as part of the process, enhancing and changing the daily-doings is easier and will have an impact for all steps.

Share this eBook! Conclusion

obile DevOps has significantly different requirements, expectations and criteria M for usefulness when it comes to monitoring mobile performance and general health-level of mobile apps, websites and APIs. Whereas Mobile DevOps approach can be highly useful to integrate with, it breaks down barriers between mobile development and operations, and also provides clarity to process and steps for mobile product development lifecycle.

Over the years the DevOps approach has produced variety of principles that are constantly evolving and new tools will be adopted in growing numbers. When looking into Mobile DevOps process and its steps, things don’t happen in waterfall, but process itself is highly iterative and all these tasks (and use of devops tools) happen in cycles.

At Testdroid, we have implemented DevOps approach for couple of years and gained lots of positive results. We hope this ebook would give you enough info and insights on how to implement similar mindset and turn it into your own style for your unique situations in your teams.

Share this eBook!