What is Agile? DevOps? How does it impact audit? October 2019 Why is this important? Agile and DevOps methods are increasingly used by IT functions Of respondents stated that they currently have a DevOps initiative in their organization or are planning one in the next 12 months. 71% Of respondents cited reducing Of surveyed organizations used risk as a top reason for going some form of Agile practices in 87% 41% DevOps, while only 17% the past year cited ensuring “Success in Disruptive Times,” compliance/governance PMI’s Pulse of the Profession, (Respondents could select 2018 multiple responses) Source: Version One, 12th Annual State of Agile What is agile? DevOps? How does it impact audit? October 2019 PwC 2 Agile is an umbrella term that means many things.. AGILE Agile Methodologies Individuals & Interactions Development & Testing Extreme Feature Driven Scrum Crystal Drive Cynefin TDD/ATDD Context Driven Programming (XP) Development (FDD) Lean Scrumbut Theory of Constraints /BDD/SBE Testing Programmer Dynamic Systems Kanban (Modern Leadership Mikado Method Adaptive Software Anarchy Development Method Management Methods) Development (ASD) Beyond (DSDM) Holacracy Mob Personal Kanban Budgeting DevOps Programming Deming System of Lean Software Lean Startup Rightshifting Management 3.0 Profound Knowledge Development Vanguard Radical (Product Development) Flow Method Management Agile Extensions Scaling Agile Hybrid Agile (Scrumban, Agile Unified Disciplined Agile Enterprise Unified Extreme Manufacturing Certifications (BABOK, PMI Xanpan, Nonban, Process (AUP) Delivery (DAD) Process (EUP) (Wikispeed) ACP, icAgile) Water-Scrum-Fall) Scale Agile @Spotify/ Large Scale Scrum Framework (SAFe) Squadification (LeSS) ScrumPLOP Enterprise Transition (Pattern Languages of Accelerated Agile XSCALE *Note: many other Framework (ETF) Enterprise Scrum Programs) (AgileTNG) methods not included What is agile? DevOps? How does it impact audit? October 2019 PwC 3 How does traditional compare with Agile? What is agile? DevOps? How does it impact audit? October 2019 PwC 4 Deliberately Agile could also mean this… overly simplified Waterfall Hybrid Agile What is agile? DevOps? How does it impact audit? October 2019 PwC 5 Impact to Audit? Internal Audit Trigger uses agile methods to Audit Agile Audit Agile health checks A significant IT program uses Agile Ensuring a controlled, compliant Agile Agile and/or DevOps becoming an adoption enterprise standard Agile risks and controls What is agile? DevOps? How does it impact audit? October 2019 PwC 6 What does traditional waterfall “look like”? Deliberately overly simplified 18 months Signed Sep 2018Signed Signed Signed Signed Sep 2018Signed Si gned Signed Jan 2018 Mar 2018 Jun 2018 Sep 2018 Jun 2019 July 2019 Project business Requirements Design Document Test Strategy, Test Results Production Change case Specification Plans, Approval (lengthy, detailed (lengthy, all Requirements scope, requirements locked Traceability requirements) down, approved) What is agile? DevOps? How does it impact audit? October 2019 PwC 7 So, what does a common agile project “look like”? Deliberately overly simplified 18 months Every 2-4 weeks Si gned Si gned Jan 2018 July 2019 Lightweight A backlog of The team Evidence is Stories are Production business case User Stories commits to captured that marked Done Change (lengthy, (analogous to complete each story’s - i.e. completed Approval detailed scope, ‘requirements’) certain stories Acceptance to the requirements) in an increment Criteria (~‘test satisfaction of (e.g. “release”, plan’) was met the Product “PI”, “sprint”) Owner What is agile? DevOps? How does it impact audit? October 2019 PwC 8 ..and it could look like this in your Agile tool Deliberately overly simplified ‘Requirement’ in the form of a This might* be used user story: “As a to evidence …” This might* be used authorized business as the location for review and approval test evidence (of ‘requirements’ and ‘tested’ / ‘done’) Acceptance criteria Allocation of the *These are common patterns from one story to a example tool. It is up to Release + management to define Sprint their standards and controls. What is agile? DevOps? How does it impact audit? October 2019 PwC 9 The program team says they’re using Agile for the implementation Global Bank - IA case study PwC 10 What is DevOps? The contraction of “Dev” and “Ops” refers to replacing siloed Development & Operations to create multidisciplinary teams that work together with shared and efficient practices and tools. - Sam Guckenheimer Continuous Continuous Monitoring 06 Planning 01 ServiceNow, Jira, Trello, Slack, Splunk, Sumo Logic, Fluentd, Stride, CollaboNet VersionOne, Prometheus, ITRS, Moogsoft, Remedy, Agile Central, OpsGenie, Logstash, Nagios, Zabbix, Zenoss PagerDuty Monitor infra, app and Product vision, release network and delivery plan Continuous Continuous 05 Testing Integration 02 FitNesse, Selenium, Gatling, Cucumber, JUnit, JMeter, TestNG, Core Jenkins, Bamboo, Travis CI, Circle Mocha, Karma, Jasmine, Tricentis Dimensions CI, Codeship, VSTS, TeamCity, Tosca, Locust.io, Soap UI, Sauce AWS CodeBuild Validate the changes Build and Packaging to Labs, Perfecto, MicroFocus UFT through automated make the code ready for test scripts deployment Continuous Infrastructure Deployment Management Containers: Docker, Kubernetes, XL Deploy, Octopus Deploy, AWS Mesos, Rancker, Docker Enterprise, CodeDeploy, ElasticBox, UrbanCode GKE, AKS, AWS ECS, Rkt, Deploy the changes Provision the Deploy, GoCD, ElectricCloud, CA automatically to the deployment Codefresh, Helm Automic 04 infra provisioned infrastructure 03 Cloud: AWS, Google and Azure’s suite of tools Version Control/Source Code Management underpins all of the dimensions and will likely be 07 at the center of the process and control structure. Examples: Gitlab, BitBucket, Subversion What is agile? DevOps? How does it impact audit? October 2019 PwC Tools displayed above are examples only and neither an exhaustive list nor endorsed. 11 All production changes go through our Continuous Deployment tool which has restricted access. Isn’t that enough? Large cloud ERP - case study PwC 12 Some common controls and audit concerns with Agile and DevOps The changing nature of risk ... Considerations (not an exhaustive list) Agile tends toward a different, or less, documentation Consider what can be extracted from tools and/or automated into the process. Look for defined minimum acceptable standards Increased reliance on tools to “control” the process Restricting user access to the tools and the configuration of the automation within the tools will be key. Increased reliance on automated testing Consider what is being done to ensure quality of test scripts and overall coverage Segregation of duties between DEV and PRD difficult to obtain Look at secure pipelines and/or code reviews to “cleanse” code prior to release into production Traditional traceability from requirements to testing to release can be Create linkages and traceability between the tools within the tool chain, difficult to obtain without a detailed spreadsheet essentially automating traceability Demonstrating business involvement and sign-off Co-locate the product owner, consider automated controls within tools to demonstrate sign-off Traditional status reports and other communications tools are not Educate stakeholders on automated “status” like information in tools, produced allowing for real time information and decision-making Tips Tips Agile and DevOps will primarily impact program development and program Performing a detailed walkthrough of the processes, tools, risks and controls change. However, it may not be limited to these areas so you need to is the 1st step. Take the time to build your understanding. understand the risk. PwC 13 What are other typical business issues we are seeing from the adoption of Agile? PwC 14 Agile and DevOps tend to come naturally to new, digitally-native technology companies with low regulatory burden. Following are other topics that we hear at our clients. Using agile buzzwords, but in Surrounding processes Lack of a coherent plan for reality: clash: adoption of agile and • Agile as an excuse for no • Annual budgeting DevOps (processes, culture, documents • Capex/opex systems, org structures, etc) • Excessive overhead • Timely involvement of (satisfying old and new cyber, risk, controls controls) advisory, etc • Inconsistent and/or • … and many more conflicting approaches • They’re ‘doing agile’ vs ‘being agile’ What is agile? DevOps? How does it impact audit? October 2019 PwC 15 Where can I get more guidance on establishing effective controls, and auditing them? Internal Audit: Thinking differently Our perspective on the areas of friction Our perspective on the audit & controls in an agile organization that emerge when organizations adopt impacts when organizations adopt agile agile Publication available on request https://www.pwc.com/us/en/services/risk- https://www.pwc.com/us/en/services/risk- assurance/library/controlled-compliant-adoption-agile- assurance/library/effective-controls-agile- devsecops.html environment.html What is agile? DevOps? How does it impact audit? October 2019 PwC 16 Applying Agile methods to Internal Audit Methodology 17 Imagine if.. Traditional internal audit lifecycle Annual Risk Annual Audit Audit Audit Audit Audit Issue Assessment Plan Preparation Planning Fieldwork Reporting Management Definition of Ready guides when an area Sprint is a time boxed Definition of Done (DoD) The Audit Backlog is a