2020 DevOps Trends Survey by Atlassian & CITE Research Foreword Over the last 15 years, tens of thousands of organizations have adopted a DevOps way of working with the help of our tools. We’ve seen how DevOps has grown from a term only familiar to technical teams to becoming part of the C-suite vocabulary. Practices like CI/CD and automation have become the norm in every engineering organization. In February of this year, we conducted a DevOps trends survey to understand where DevOps is going. We surveyed 500 professionals about their success with DevOps, the barriers they faced, and the impact of tools and culture on their work. The three major trends we found all speak to one fact. The path to DevOps is a matter of when, not if - but there are still significant obstacles along the way.
The research contained in this report has been completed with the advice and assistance of CITE Research. This survey was conducted in a pre-pandemic world and may not be reflective of the realities of DevOps today.
2 Table of Contents
Background & Executive Detailed Appendix methodology summary findings
Page 4 Page 6 Page 11 Page 36 Background & methodology
4 OBJECTIVE METHODOLOGY
Explore how companies are managing DevOps: ‣ CITE Research, on behalf of Atlassian, conducted ‣ What practices companies have in place for DevOps an online survey among 500 Developers & IT Decision Makers in February 2020. ‣ How do companies measure DevOps success ‣ Barriers to DevOps transformation ‣ Impact of DevOps practices ‣ Differences in perceptions of DevOps by level, team, etc.
‣ Employed full-time ‣ At companies with 101+ employees RESPONDENT ‣ Be in Software Development or IT ‣ Work at an organization that practices DevOps CRITERIA ‣ Manager-level or above
5 Executive summary
6 Executive Summary / Survey Highlights
DevOps is now a There are still Executives and corporate term. obstacles. practitioners don’t
‣ More than half of orgs ‣ 85% of respondents have see eye to eye. have been practicing faced barriers in their ‣ Confidence in DevOps for over three DevOps implementation measurements of DevOps years ‣ Lack of skills, legacy success vary ‣ Over 90% said DevOps infrastructure, and ‣ Practitioners emphasize had a direct impact on adjusting corporate collaboration culture while business metrics culture are top complaints executives value individual mindset
7 Executive Summary / The positive impact of DevOps
Impact on career 78% had to learn a new skill The top factors in 48% of respondents say it helped implementing 99% them get a raise DevOps successfully are the right tools of respondents Better deliverables and the right people. say DevOps has 61% say it helped them produce The top factors in a had a positive higher quality deliverables DevOps team impact on their performing well organization Faster deliverables are collaboration and the ability to 49% say they see a faster time to market problem-solve. 49% say it improved their deployment frequency
8 Executive Summary However, most face issues with DevOps implementation
Barrier to 85% Implementation 37% Lack of skills in employees of DevOps
36% Legacy infrastructure Nearly all (85%) of organizations face some type of hurdle when implementing DevOps, with lack of skills in employees, legacy infrastructure, and adjusting 35% Adjusting corporate culture corporate culture being the top complaints.
9 Executive Summary While most feel they can effectively measure DevOps success, executives feel much more confident in their measurements than the practitioners.
Compared to Executives, the Practitioner is: Most are measuring DevOps success/impact More likely to agree it is difficult to measure the and feel they are doing it effectively… impact of DevOps progress and success – 62% agree compared to 49% of decision-makers Have a way to measure DevOps More likely to agree my organization has no clear 74% success (most commonly way to measure DevOps success – deployment frequency) 58% agree compared to 47% of decision-makers
More likely to agree I’m unsure how to improve my organization’s DevOps processes – Feel they are very or 47% agree compared to 34% of decision-makers somewhat effective in 97% measuring DevOps success
10 Detailed findings
11 DevOps Practices Detailed findings
PART 1 DEVOPS PRACTICES
12 DevOps titles are the norm…
Organizations use DevOps in titles – DEVOPS PRACTICES 69% have a team with DevOps in the name and/or employees with 52% 49% DevOps in their titles. 42% ‣ Larger organizations with 500+ employees are especially likely to have a team with DevOps in the title (57%). 20% ‣ Organizations that have been using DevOps for 3+ years are also more likely to have a team with DevOps in the name (57%).
We have a team with We have We have We have the name DevOps in employee(s) with dedicated teams employees who the title DevOps titles responsible for are responsible Which of the following best describes your DevOps, but for this, but organization’s management of DevOps? without DevOps in without DevOps Please select all that apply. their title in their title
13 …but almost half of organizations are just starting with DevOps, practicing it for less than 3 years.
LONGEVITY OF DEVOPS PRACTICES ‣ 46% of organizations are relatively new to DevOps 39% with under 3 years of doing it. 36%
‣ 54% of organizations have been practicing DevOps for 3 years or more.
18% Approximately how long has your organization used DevOps practices?
7%
1%
Less than one year 1 year to 3 years to 5 years or more Unsure less than 3 years less than 5 years
14 Source code management and code development are the most widespread toolchains used
TOOLCHAIN USAGE Respondents are using an average of 10.3 toolchains, with 80%+ using Source code management 100% source code management, code development and review and Code development and review 90% applications performance Applications performance monitoring 82% monitoring. The least common is Agile planning 79% artifact repository, although almost Continuous testing tools 79% half are using this toolchain as well. Incident response and management 79%
Continuous integration 79% Which of the following DevOps toolchains does your company currently use? [N=417 Infrastructure configuration and… 77% who have use source code management toolchain] Application release automation and… 76%
End-user experience 74%
Change and release management 72%
Packaging 59%
Artifact repository 53%
15 Detailed findings
PART 2 PERCEPTIONS OF DEVOPS – IMPACT & CHALLENGES
16 DevOps has a resoundingly positive impact on organizations…
Nearly all respondents say the implementation of DEVOPS IMPACT DevOps will have a positive impact on their organization on the future.
‣ Interestingly, decision-maker employees are more likely to say very positive (75% compared 30% Somewhat to only 60% of practitioners).
‣ Those who measure DevOps success are more likely to say its very positive as well (76%).
69% Very Generally, what impact do you expect implementing DevOps will have on your organization in the future? 1% Positive Negative
17 ...across a variety of metrics...
‣ Organizations see a multitude of positive results from IMPACT OF DEVOPS ON ORGANIZATION DevOps implementation, most frequently higher
quality deliverables (61%). Higher quality deliverables 61%
‣ About half see faster time to market/lead time, Faster time to market / lead time 49% improvement deployment frequency, better team Improved deployment frequency 49%
culture and increased collaboration across teams/ Better team culture 48% departments. Increased collaboration across 48% different teams and departments ‣ Those who work on both Developing and Operations Faster recovery times 47%
are significantly more likely to see various impacts, Building better products and services 46% such higher quality deliverables (65%), faster recovery Lower failure rate of new releases 40% times (52%), building better products and services Increased collaboration with 36% (48%), and lower failure rate of new releases (45%). non-technical departments Less incidents 35% ‣ Those who have been practicing DevOps for longer None of the above 1% (3+ years) are more likely to have seen higher-quality deliverables (66%), lower failure rate of new releases Other (please specify) 0% (45%) and less incidents (40%). Which of the following, if any, have been the impact of implementing DevOps practices on your organization? Please select all that apply. 18 ...and it leads to career transformation.
9 out of 10 respondents say DevOps impacted their career IMPACT OF DEVOPS ON CAREER
‣ The majority of respondents say DevOps implementation has required them to learn new skills. I have had to learn new skills 78% ‣ Almost half (48%) say it helped them get a raise.
‣ A third had to shift roles. It helped me get a raise 48%
What impact has DevOps implementation had on your career, personally? I have had to shift roles 31%
It has had no impact on my career 8%
19 Collaboration is the key for DevOps success…
Overall, Collaboration and Ability to MOST IMPORTANT TRAITS FOR problem solve are ranked first among DEVOPS TEAM TO PERFORM WELL traits of a successful DevOps team
‣ Ops respondents are more likely to rank Ability to problem solve and autonomy #1 at 32%.
‣ Devs are more likely to rank Collaboration #1 36%.
‣ Decision-makers are more likely to
rate Forward-thinking as #1 at 18%. Collaboration Ability to Alignment/ Forward- Visibility Autonomy Openness Empathy problem-solve coordination thinking within the with other organization Which of the following are the most teams important traits for a DevOps team/ department to have in order to perform well? Please rate the top 3 traits, with 1 being the most important to a successful DevOps team.
20 …And the right people and culture lead to successful implementation.
The right people and/or company MOST IMPORTANT TRAITS FOR culture is the top trait for successful SUCCESSFUL DEVOPS IMPLEMENTATION DevOps implementation. The right tools and support from senior leadership are also important.
‣ Devs respondents are especially likely to say A way to measure DevOps success is #1 (19%).
‣ Software Developers are more likely than IT professionals to feel they need The right tools The right people / The right tools Support from A way to Company Culture senior leadership measure DevOps (38% rank this #1). success
Which of the following, if any, are the most important factors to a successful DevOps implementation? Please rate the top 3 factors, with 1 being the most important to a successful DevOps team.
21 Lack of skills, legacy infrastructure and corporate culture are biggest hurdles to DevOps implementation.
The majority of respondents (84%) BARRIERS TO DEVOPS IMPLEMENTATION have faced barriers to their DevOps implementation. Most common hurdles are lack of skills in 37% employees, legacy infrastructure 36% 35% 30% and adjusting corporate culture. 26% 24%
‣ C-suite respondents are more 16% 16% likely to see inability to cleanly 12%
coordinate well with other 1% teams (38%) as a barrier. Lack of Legacy Adjusting Silos Inability to Wrong / No clear None No clear Other skills in infrastructure corporate between cleanly insufficient way to of the plan (please employees culture Developers coordinate tools measure above specify) and well with DevOps Which of the following, if any, have Operations other success been barriers to implementing DevOps teams teams practices into your organization? Please select all that apply.
22 Detailed findings
PART 3 MEASURING TEAM SUCCESS
23 Three quarters measure DevOps success, most commonly through deployment frequency
Unsure 7% ‣ Most measure DevOps success or impact (74%). No Does your organization 19% ‣ Deployment frequency is by far the most have a way to measure common method of DevOps success DevOps success or impact? measurement as companies want to be more agile and move faster. MEASURE DEVOPS Yes ‣ Nearly half of respondents leverage the four key 74% SUCCESS / IMPACT DevOps metrics identified by DORA.
‣ Interestingly, 79% of decision-makers say they Which of the following does your organization use to measure DevOps success? measure success compared to only 67% of Please select all that apply. [N=372] practitioners. Decision-makers are more likely to say they use MTTR to measure success at METHODS OF DEVOPS SUCCESS MEASUREMENT 60% vs. 45% of practitioners. Deployment frequency 75% Mean time to restore (MTTR) 55% Lead time 53% Change fail percentage 48% Other (please specify) 1%
24 Most agree it is important to measure the impact of DevOps, but don’t have a clear way to do so
‣ Respondents agree it is important AGREEMENT WITH DEVOPS to measure DevOps progress/ MEASUREMENT / IMPACT STATEMENTS success (95%), DevOps has an impact on important metrics (94%) It is important to measure the impact 58% 37% 4% 1% and has an impact on business of DevOps progress and success metrics (93%). It is clear that DevOps has an impact on mean time to repair, cycle time and other important metrics 50% 44% 6% 0% ‣ Three quarters prefer to use best of It is clear that DevOps has an impact on business metrics, such as revenue and profitability 51% 42% 6% 1% breed tools versus one toolchain. I prefer using best of breed tools 36% 41% 20% 3% ‣ Half of respondents say that it is over one toolchain for DevOps difficult to measure the impact of It is difficult to measure the impact of DevOps progress and success 21% 33% 28% 17% DevOps progress, and that their My organization does not have a clear organization does not have a clear way to measure DevOps success 21% 30% 22% 26%
way to measure success. I’m unsure how to improve my 26% organization’s DevOps process 13% 26% 35%
How much do you agree or disagree Completely agree Somewhat agree Somewhat disagree Completely disagree with each of the following statements?
25 Organizations are frequently shifting priorities
‣ Organizations are changing priorities FREQUENCY OF CHANGE (60%) and projects (49%) daily or weekly. The Decision-maker perceives Daily Weekly Monthly Quarterly Annually more change than practitioners – they are more likely to say all of these items Priorities 27% 34% 20% 15% 4% change daily or weekly. Tools 22% 21% 19% 18% 20%
‣ People and Budgets have a much People 20% 15% 12% 22% 31% lower frequency of change than Projects 18% 31% 29% 18% 4% priorities and projects. Technology stacks 16% 30% 20% 20% 14%
Business targets 14% 27% 26% 24% 9% How often does your development team change each of the following? Budgets 11% 18% 21% 27% 24%
26 Detailed findings
PART 4 TRACKING MODERN SOFTWARE TRENDS
27 Most organizations continue to spend up to 30% of their time on updates
‣ There are no statistically TIME SPENT ON UPGRADES/UPDATES significant changes in the amount of time spent on upgrades and 2018 2020 updates since the 2018 report.
28% 28% ‣ Seven in ten (70%) are spending 27% 26% up to 30% of their time on upgrades and updates. 19%
15% 15% Approximately what percentage 14% of your team’s time each month is spent on updates or upgrades 10% to self-hosted software you use? 7% 5% 5%
1% 1%
Zero 1-10% 11-20% 21-30% 41-50% 51-60% 70% or more
28 The frequency of being asked status and updating status may be declining...
Since 2018, the frequency respondents are asked status and the frequency they update status in a tool is statistically less often.
‣ In 2020, respondents are less likely to say they are asked a few times a day or once a day to provide their work status.
‣ They are also less likely to stay they need to update status in a tool a few times a day or once a day.
FREQUENCY ASKED STATUS FREQUENCY UPDATE STATUS IN TOOL
Approximately how often are you asked by someone to How many times a day do you have to update status in a provide the status of your or your team’s work? software development tool? 29 …But the number of tools to understand a project status is on the rise.
The number of tools respondents use to NUMBER OF TOOLS CONSULTED TO understand the status of their software UNDERSTAND PROJECT STATUS development projects are on the rise: 2018 2020 ‣ Significantly less say they need 3-4 tools to know their status, while 39% 39% significantly more say they need 5+ 35% 33% tools to understand their status. 32%
How many tools do you need to consult to understand the status of your software 21% development projects?
1 to 2 3 to 4 5 or more
30 Bugs and delays remain the top issues with releases
Issues with releases are similar to 2018 in ISSUES WITH RELEASES that defects / bugs are the most common 2018 2020 issue, followed by unexpected delays and 57% outages/downtimes. Defects / bugs 48%
‣ Significantly less say they deal with 50% Unexpected delays 46% bugs, but it is still the most common 34% release issue. Outages / downtime 29%
Significantly more say they deal with Over budget 30% none of the issues listed. Notably, this is 27% still a minority of 12%. 30% Customer complaints 26%
Missed deadlines 29% Which of the following issues has your 32% current team faced with a release? 21% Please select all that apply. Dropped important features 21%
4% None of the above 12%
31 Organizations are measuring customer satisfaction on new releases less often
‣ Organizations are statistically less MEASUREMENT OF CUSTOMER likely since 2018 to be measuring SATISFACTION OF NEW RELEASES customer satisfaction on new releases now. The proportion that say 2018 2020 they do measure customer satisfaction has decreased significantly, while the proportion that 84% do not has increased significantly. 76%
‣ Even so, the majority continue to measure customer satisfaction on new releases. 20% 11% Does your organization have practices 5% 4% in place to measure customer satisfaction on newly released features? Yes No Unsure
32 Having too many manual processes is becoming a more pervasive issue
‣ Respondents are reporting fewer overall ISSUES PREVENTING GOING FASTER issues preventing them from going faster in FROM CODE TO DEVELOPMENT moving code from development to production 2018 2020 since 2018. Significantly more say they We lack sufficient automated test 32% experience none of the issues tested. coverage to avoid manual testing 24% We have people who understand 27% ‣ The proportion that say they lack sufficient how the build pipeline works 26% automated test coverage to avoid manual We haven’t automated the whole 27% 23% testing and that say we have too many tools build and deployment pipeline 27% has integrated in a fragile way have We have too many manual processes 31% decreased significantly. 26% We have defects caused by code merges 22% ‣ Meanwhile, the proportion that says we have We have too many tools, 25% 18% too many manual processes has increased integrated in a fragile way We lack visibility into 22% (although not at a statistically significant dependencies across code bases 19% level) to become the most common issue. 21% We have a slow build process 21%
Which of the following prevent your current team We lack tracking for code 18% from dev to production 19% from going faster in moving code from development 8% to production? Please select all that apply. None of the above 18%
33 Customer satisfaction remains the chief means of measuring team success
While customer satisfaction remains the METRICS MEASURING TEAM SUCCESS most commonly used metric for software 2018 2020 teams to evaluate and track success, 46% usage of several metrics have increased: Customer satisfaction 49% 33% Mean time to recover/repair 35% ‣ Team velocity 31% Application crash rate 36% ‣ Revenue 30% Issues closed 31% ‣ Mean time between failures 29% Open / close rate 31% 28% Feature usage 29% Which of the following metrics does 25% your team use to evaluate and track Team velocity 32% your team’s success? Please think of 25% these measurements specifically Number of story points 21% related to the software development Revenue 25% team, rather than the organization 36% overall. Please select all that apply. 22% Mean time between failures 30% 20% Features shipped 22% 1% None of the above 2% 34 IT may be more responsible for resolving incidents now
The percentage of respondents who say IT is responsible for resolving major incidents and the percentage that prefer IT to resolve major incidents has increased since 2018. However, this may be due to IT professionals being a larger part of the sample in the 2020 survey.
PRIMARY RESPONSIBILITY FOR PREFERRED PARTY RESPONSIBLE FOR RESOLVING MAJOR INCIDENTS RESOLVING MAJOR INCIDENTS 2018 2020 2018 2020 60% 59%
51% 51% 49% 49%
41% 39%
1% 0% A central IT Application Other (please specify) A central IT Operations team Application development team Operations team development team
For major incidents, which team is currently primarily For major incidents, which team would you prefer to be responsible for responding and resolving them? primarily responsible for responding and resolving them? 35 Appendix
36 DEMOGRAPHICS & FIRMOGRAPHICS
GENDER LEVEL TITLE
Male 72% Chief Executive Officer 4% DevOps Engineer 1% Female 28% Chief Information Officer 8% Team Leader of Application 1% Chief Technology Officer 13% Engineering AGE President 1% Director of IT 30% 18-24 2% Vice President 3% IT manager 40% 25-34 24% Director 31% IT Ops manager 3% 35-44 47% Manager / Team Lead 39% Manager, Operations and 45-54 1% 19% Release 55-64 7% DEVOPS RESPONSIBILITY CTO 6% 65 or older 1% Building apps 15% Infrastructure Engineer 1% Operating apps 17% REGION Director of Development 2% Both 66% Northeast 24% Other 1% CIO 8% South 18% Architect 1% Midwest 33% DEPARTMENT Director of DevOps 1% West 25% IT 90% Senior Software Developer 4% Software Development 10% 37 FIRMOGRAPHICS
REPORTING TO REPORTING TO, CONTINUED NUMBER OF EMPLOYEES REVENUE
DevOps Engineer 2% Senior Developer 1% 101-249 8% $0 - $5 Million 3% Team Leader of Application Build/automation manager 1% 250-499 12% 1% $5.1 Million - $10 Million 7% Engineering Director of Developer Tools 2% 500-1,000 24% $10.1 Million - $25 Million Director of IT 21% CIO 14% 1,001-4,999 30% 8% IT manager 23% Architect 1% 5,000-9,999 11% $25.1 Million - $50 Million 8% IT Ops manager 2% DevOps Lead 1% 10,000+ 14% Manager, Operations and Director of DevOps 2% $50.1 Million - $100 Million 2% 11% Release Senior Software Developer 1% ENGINEERING TEAM SIZE DevOps architect 1% Other 4% $100.1 Million - $250 Million Less than 10 employees 6% 9% None of the above 2% Lead developer 1% 11-20 employees 16% $250.1 Million - $500 Million 9% CTO 16% 21-50 employees 24% ORGANIZATION TENURE Infrastructure Engineer 1% 51-100 employees 24% $500.1 Million - $1 Billion Less than one year 1% 19% Director of Development 2% 101+ employees 29% 1 to less than 3 years 6% $1.1 Billion - $5 Billion 13% 3 to less than 5 years 13% $5.1 Billion - $10 Billion 5% 5 to less than 10 years 24% $10.1 Billion + 6% 10 years or more 56% Don’t Know 2% 38