What is Agile? DevOps? How does it impact audit?
October 2019 Why is this important? Agile and DevOps methods are increasingly used by IT functions
Of respondents stated that they currently have a DevOps initiative in their organization or are planning one in the next 12 months.
71%
Of respondents cited reducing Of surveyed organizations used risk as a top reason for going some form of Agile practices in 87% 41% DevOps, while only 17% the past year cited ensuring “Success in Disruptive Times,” compliance/governance PMI’s Pulse of the Profession, (Respondents could select 2018 multiple responses)
Source: Version One, 12th Annual State of Agile
What is agile? DevOps? How does it impact audit? October 2019 PwC 2 Agile is an umbrella term that means many things..
AGILE
Agile Methodologies Individuals & Interactions Development & Testing
Extreme Feature Driven Scrum Crystal Drive Cynefin TDD/ATDD Context Driven Programming (XP) Development (FDD) Lean Scrumbut Theory of Constraints /BDD/SBE Testing Programmer Dynamic Systems Kanban (Modern Leadership Mikado Method Adaptive Software Anarchy Development Method Management Methods) Development (ASD) Beyond (DSDM) Holacracy Mob Personal Kanban Budgeting DevOps Programming Deming System of Lean Software Lean Startup Rightshifting Management 3.0 Profound Knowledge Development Vanguard Radical (Product Development) Flow Method Management Agile Extensions Scaling Agile
Hybrid Agile (Scrumban, Agile Unified Disciplined Agile Enterprise Unified Extreme Manufacturing Certifications (BABOK, PMI Xanpan, Nonban, Process (AUP) Delivery (DAD) Process (EUP) (Wikispeed) ACP, icAgile) Water-Scrum-Fall) Scale Agile @Spotify/ Large Scale Scrum Framework (SAFe) Squadification (LeSS) ScrumPLOP Enterprise Transition (Pattern Languages of Accelerated Agile XSCALE *Note: many other Framework (ETF) Enterprise Scrum Programs) (AgileTNG) methods not included
What is agile? DevOps? How does it impact audit? October 2019 PwC 3 How does traditional compare with Agile?
What is agile? DevOps? How does it impact audit? October 2019 PwC 4 Deliberately Agile could also mean this… overly simplified
Waterfall Hybrid Agile
What is agile? DevOps? How does it impact audit? October 2019 PwC 5 Impact to Audit?
Internal Audit Trigger uses agile methods to Audit Agile Audit Agile health checks
A significant IT program uses Agile Ensuring a controlled, compliant Agile Agile and/or DevOps becoming an adoption enterprise standard
Agile risks and controls
What is agile? DevOps? How does it impact audit? October 2019 PwC 6 What does traditional waterfall “look like”? Deliberately overly simplified
18 months
Signed Sep 2018Signed Signed Signed Signed Sep 2018Signed Si gned Signed Jan 2018 Mar 2018 Jun 2018 Sep 2018 Jun 2019 July 2019
Project business Requirements Design Document Test Strategy, Test Results Production Change case Specification Plans, Approval (lengthy, detailed (lengthy, all Requirements scope, requirements locked Traceability requirements) down, approved)
What is agile? DevOps? How does it impact audit? October 2019 PwC 7 So, what does a common agile project “look like”? Deliberately overly simplified
18 months Every 2-4 weeks
Si gned Si gned Jan 2018 July 2019
Lightweight A backlog of The team Evidence is Stories are Production business case User Stories commits to captured that marked Done Change (lengthy, (analogous to complete each story’s - i.e. completed Approval detailed scope, ‘requirements’) certain stories Acceptance to the requirements) in an increment Criteria (~‘test satisfaction of (e.g. “release”, plan’) was met the Product “PI”, “sprint”) Owner
What is agile? DevOps? How does it impact audit? October 2019 PwC 8 ..and it could look like this in your Agile tool Deliberately overly simplified
‘Requirement’ in the form of a This might* be used user story: “As a to evidence …” This might* be used authorized business as the location for review and approval test evidence (of ‘requirements’ and ‘tested’ / ‘done’) Acceptance criteria
Allocation of the *These are common patterns from one story to a example tool. It is up to Release + management to define Sprint their standards and controls.
What is agile? DevOps? How does it impact audit? October 2019 PwC 9 The program team says they’re using Agile for the implementation
Global Bank - IA case study
PwC 10 What is DevOps? The contraction of “Dev” and “Ops” refers to replacing siloed Development & Operations to create multidisciplinary teams that work together with shared and efficient practices and tools. - Sam Guckenheimer
Continuous Continuous Monitoring 06 Planning 01 ServiceNow, Jira, Trello, Slack, Splunk, Sumo Logic, Fluentd, Stride, CollaboNet VersionOne, Prometheus, ITRS, Moogsoft, Remedy, Agile Central, OpsGenie, Logstash, Nagios, Zabbix, Zenoss PagerDuty Monitor infra, app and Product vision, release network and delivery plan
Continuous Continuous 05 Testing Integration 02 FitNesse, Selenium, Gatling, Cucumber, JUnit, JMeter, TestNG, Core Jenkins, Bamboo, Travis CI, Circle Mocha, Karma, Jasmine, Tricentis Dimensions CI, Codeship, VSTS, TeamCity, Tosca, Locust.io, Soap UI, Sauce AWS CodeBuild Validate the changes Build and Packaging to Labs, Perfecto, MicroFocus UFT through automated make the code ready for test scripts deployment
Continuous Infrastructure Deployment Management Containers: Docker, Kubernetes, XL Deploy, Octopus Deploy, AWS Mesos, Rancker, Docker Enterprise, CodeDeploy, ElasticBox, UrbanCode GKE, AKS, AWS ECS, Rkt, Deploy the changes Provision the Deploy, GoCD, ElectricCloud, CA automatically to the deployment Codefresh, Helm Automic 04 infra provisioned infrastructure 03 Cloud: AWS, Google and Azure’s suite of tools
Version Control/Source Code Management underpins all of the dimensions and will likely be 07 at the center of the process and control structure. Examples: Gitlab, BitBucket, Subversion What is agile? DevOps? How does it impact audit? October 2019 PwC Tools displayed above are examples only and neither an exhaustive list nor endorsed. 11 All production changes go through our Continuous Deployment tool which has restricted access. Isn’t that enough?
Large cloud ERP - case study
PwC 12 Some common controls and audit concerns with Agile and DevOps
The changing nature of risk ... Considerations (not an exhaustive list)
Agile tends toward a different, or less, documentation Consider what can be extracted from tools and/or automated into the process. Look for defined minimum acceptable standards
Increased reliance on tools to “control” the process Restricting user access to the tools and the configuration of the automation within the tools will be key.
Increased reliance on automated testing Consider what is being done to ensure quality of test scripts and overall coverage
Segregation of duties between DEV and PRD difficult to obtain Look at secure pipelines and/or code reviews to “cleanse” code prior to release into production
Traditional traceability from requirements to testing to release can be Create linkages and traceability between the tools within the tool chain, difficult to obtain without a detailed spreadsheet essentially automating traceability
Demonstrating business involvement and sign-off Co-locate the product owner, consider automated controls within tools to demonstrate sign-off
Traditional status reports and other communications tools are not Educate stakeholders on automated “status” like information in tools, produced allowing for real time information and decision-making
Tips Tips Agile and DevOps will primarily impact program development and program Performing a detailed walkthrough of the processes, tools, risks and controls change. However, it may not be limited to these areas so you need to is the 1st step. Take the time to build your understanding. understand the risk.
PwC 13 What are other typical business issues we are seeing from the adoption of Agile?
PwC 14 Agile and DevOps tend to come naturally to new, digitally-native technology companies with low regulatory burden. Following are other topics that we hear at our clients.
Using agile buzzwords, but in Surrounding processes Lack of a coherent plan for reality: clash: adoption of agile and • Agile as an excuse for no • Annual budgeting DevOps (processes, culture, documents • Capex/opex systems, org structures, etc) • Excessive overhead • Timely involvement of (satisfying old and new cyber, risk, controls controls) advisory, etc • Inconsistent and/or • … and many more conflicting approaches • They’re ‘doing agile’ vs ‘being agile’
What is agile? DevOps? How does it impact audit? October 2019 PwC 15 Where can I get more guidance on establishing effective controls, and auditing them?
Internal Audit: Thinking differently Our perspective on the areas of friction Our perspective on the audit & controls in an agile organization that emerge when organizations adopt impacts when organizations adopt agile agile Publication available on request https://www.pwc.com/us/en/services/risk- https://www.pwc.com/us/en/services/risk- assurance/library/controlled-compliant-adoption-agile- assurance/library/effective-controls-agile- devsecops.html environment.html
What is agile? DevOps? How does it impact audit? October 2019 PwC 16 Applying Agile methods to Internal Audit Methodology
17 Imagine if..
Traditional internal audit lifecycle Annual Risk Annual Audit Audit Audit Audit Audit Issue Assessment Plan Preparation Planning Fieldwork Reporting Management
Definition of Ready guides when an area Sprint is a time boxed Definition of Done (DoD) The Audit Backlog is a is ready to be audited. Considerations (2 weeks) set of activities guides the quality of the audit continually updated list include agreement with Business where audit tasks should or business value delivered of areas to be audited Stakeholders, Audit resource availability be completed during the Sprint
Agile internal audit lifecycle …. Ongoing Risk Prioritized Sprint 0: Sprint 1: Sprint 2: Sprint x: Sprint: Once Issues are accepted Assessment Audit Planning Planning, Planning, Planning, Formal by the auditee, Issue Backlog fieldwork, fieldwork, fieldwork, Reporting management activities Review Review Review can start early. Retrospective Retrospective Retrospective Risk Assessment is an ongoing Higher level audit areas are activity and the risk view gets more refined on outcomes and updated from each audit timing. Lower level areas can Each Sprint involves early be vague discussion of observations Issue Management with Business Stakeholders.
What is agile? DevOps? How does it impact audit? October 2019 PwC 18 Feedback from the trenches
“We saved 100’s of hours in risk assessment and audit planning”
“We were able to pivot to new risks as needed”
“New templates and processes reduced workload and increased speed”
“Our customers loved the overall audit experience”
What is agile? DevOps? How does it impact audit? October 2019 PwC 19 Thank you
pwc.com
© 2019 PwC. All rights reserved. PwC refers to the US member firm or one of its subsidiaries or affiliates, and may sometimes refer to the PwC network. Each member firm is a separate legal entity. Please see www.pwc.com/structure for further details.