Nbcuniversal

Business challenge NBCUniversal sought to engage a DevOps approach to application development across its complex enterprise, consisting of 17 business units, in order to improve code quality, streamline development and lower costs.

Transformation NBCUniversal utilizes IBM® UrbanCodeTM Build, IBM UrbanCode Deploy and IBM Cloud for Skytap Solutions as the engine of DevOps, combining continuous integration, delivery, testing, feedback and monitoring into one automated workflow, bridging process, culture and technology across the organization.

Business benefits “…the code quality has NBCUniversal improved, it’s going faster 75% and we’re able to meet reduction Scaling DevOps across a the business requirements in time required for new application much more rapidly.” releases large, multi-speed IT —John Comas, Manager of Decreases Platform DevOps, NBCUniversal regression testing time enterprise from weeks to hours Satisfies NBCUniversal is a multinational, multimedia conglomerate headquartered in business requirements New York City. The company, with revenues of approximately USD30 billion, is involved primarily in the media and entertainment industries; among its most more rapidly and improves code quality significant divisions are the NBC television network and the major film studio, Universal Studios. Through its Universal Parks & Resorts division it is also the third-largest operator of amusement parks in the world. Share this Uniting silos NBCUniversal has grown largely possible, so we could standardize The IBM UrbanCode platform— through acquisition and, at present, is support and training. We also wanted UrbanCode Build (application Until recently there’s been a made up of 17 individual companies to improve the quality of the code build automation software) and general perception that only the and business units, including its developed, and we wanted to UrbanCode Deploy (application new generation of cloud-native international footprint. Characteristic develop it faster, more efficiently and release automation software)—acts organizations, those conceived as of most large organizations, its at lower cost,” he says. “Our builds as the engine of DevOps, enabling market disruptors and built for rapid application environment is diverse and deployments had high failure NBCUniversal to combine continuous innovation, have embarked on the and multi-speed—with a mix of rates, and because our application integration, continuous delivery, con- DevOps journey. Not so. For every on-premises, private and public teams did little or no testing there tinuous testing, continuous feedback Etsy, Uber or Netflix there are scores cloud environments, with core busi- were releases that probably should and continuous monitoring into one of established enterprises, like ness applications integrated with never have gone into production, automated workflow, bridging pro- NBCUniversal, seeking to consistently new web and mobile applications. resulting in too many outages.” cess, culture and technology across streamline time-to-market for new The company’s DevOps team the organization. applications, increase application needed to bridge wide variability in Undertaking the DevOps journey code quality and reduce develop- speed, process, tools, skills and became the impetus for building a The UrbanCode platform supports a ment, testing and deployment costs culture between System of Record workflow across four groups that diverse set of IT environments and through DevOps. (often mainframe) developers and were not historically well integrated— delivers comprehensive audit capa- System of Engagement (cloud-based, the Development, QA, Run and SDLC bilities, which is critical to a complex These corporations represent a customer-facing applications) devel- teams. The journey encompasses enterprise, like NBCUniversal. “With cross-section of industries. And opers to achieve integrations across technology as well as mindset, the UrbanCode products we’re able regardless of organizational and IT these systems. speeding innovation by breaking to provide build-as-a-service for the complexity they’re engaging a down complex processes into the organization. I can reproduce any DevOps approach because agility— John Comas, Manager of Platform smallest units of work and running build that comes into UrbanCode at the ability to speed innovation and DevOps, is responsible for the them in parallel whenever possible. the click of a button,” say Comas. respond to their business’ require- processes and tools underpinning “We have a single, unified dashboard ments faster—is vital. the DevOps software development Delivering where I can see all of the activity. And lifecycle (SDLC) at NBCUniversal. we have clear records as to what was The DevOps path that a large enter- “We needed to provide our applica- innovation faster built, the binaries that relate to that prise takes likely looks quite different tion customers with a single, stan- Comas and NBCUniversal’s Senior build, what was done with it, how was than that of a born-on-the-cloud dardized SDLC for a number of Director of Service Strategy and it tested, what environment it was business, to be sure. Most significant reasons: first, to reduce application Collaboration, Charles Williams, deployed to—all of that. Give me a are questions about how best to licensing costs by eliminating multiple positioned DevOps internally as a date, and if you want to know what usher in widespread cultural change instances. Secondly, we wanted to shared platform/framework for was deployed to production on that and transform long-standing pro- provide a standard, on-premises delivering innovation faster. date I could tell you everything.” cesses and paradigms. enterprise-hardened system and migrate as many users over to it as

2 NBCUniversal uses IBM Cloud for tools, managing the tools themselves In that sense, tools and technology, environment. Once they deployed it Skytap Solutions (ICSS), developed and managing the best practices in like UrbanCode, drove culture and to an environment it was basically left by IBM Business Partner Skytap, in the use of those tools. The team also process change. “I think the tooling to the QA team to handle,” he says. conjunction with UrbanCode Deploy creates the automations for the helped fortify what the culture was “So a week or so would go by and to automate test environment Development teams. shifting towards; more agile develop- they’d get some results, do another configuration and management ment, more shift-left testing,” says build and send it back over to QA, throughout the software development Advocating for DevOps wasn’t easy Williams. “The tools underpinned the and on and on. So the time-to- lifecycle and parallelize regression at first; not everyone was receptive, DevOps process we envisioned, market for an application was very testing as part of QA automation. so Comas and Williams had some which is an end-to-end workflow.” high—20 weeks to get a release out. cultural hurdles to overcome. One reason it took so long is that the In one case, NBCUniversal’s “John and I have been talking about Automating for QA team would do full-scale testing Television Group has used ICSS to DevOps for a while but it was kind of on a release that may or may not reduce its regression testing from falling on deaf ears. It was when we speed and quality have met the business requirements.” 6-8 weeks to as little as 3 hours for started talking about capabilities and NBCUniversal’s Television Group Compass, a large application it uses then showing capabilities to the app The Compass application team’s goal in New York stands as an example for program scheduling. Associated owners and then into the QA environ- was to reduce that time by 75 per- of how the organization’s DevOps with Compass is an enormous ment that we started to see traction,” cent, to five weeks. “Once we moved journey has taken shape. That regression test suite consisting of Williams says. “At one point I think them to the UrbanCode platform they particular business unit is the televi- 1000 test cases. Previous to utilizing John and I even decided to stop were able to hit that goal. In fact, sion distribution arm of NBCUniversal ICSS, the organization’s QA team using the word ‘DevOps’, because it we’re now looking to reduce time-to- and is, in part, responsible for sched- would run these tests locally—each didn’t mean anything to most people. market to two weeks,” Comas says. uling on-air content for all NBCU- test run in series. Now they’ve set up Improved speed of delivery in appli- First, his team integrated QA automa- owned stations. As noted previously, an environment of 50 test machines, cation development; being able to tion with the Compass workflow, the Compass application is central to all pointing to the Compass QA run tests repeatedly and consistently eliminating the need to manually that task. environment on premise, with and get results fast—that’s what’s execute numerous QA tests for 20 tests running per machine important. When we spoke in those Comas and his team migrated every deployment. “We started with in parallel. terms we started changing the Compass—a massive .net-based shift-left testing. We unit-tested the culture. And when we had a couple app—to the UrbanCode platform application release and did code Through DevOps, NBCUniversal has of opportunities to demo our agile some 18 months ago. “I would coverage analysis on it, while also broken down what was previously a development capabilities we started describe the previous Compass pro- starting a small functional test monolithic process into component hearing, ‘Oh, so this is what DevOps cess as like a ‘manu-matic’ process; where we would observe the different parts, with Comas’ DevOps team is. This is what I’ve been wanting.’” they would commit their code to the business requirements and create taking the lead by providing the infra- repository, manually click the build tests which would exercise those. So structure that supports the DevOps button and then deploy it to an before the release would even get to the QA team to be fully functional and

3 regression tested we could get From a technology standpoint, For Williams and Comas, DevOps is immediate feedback if the release Comas says the UrbanCode platform about building trust and confidence Solution components was good or not. That, in and of itself, gives his team the ability to integrate and about instituting predictability, ●● IBM® UrbanCode™ accelerated things greatly.” a wide range of tools into one speed and quality in software ●● IBM Cloud for Skytap Solutions automated workflow, eliminating delivery lifecycle. “We’re providing a (ICSS) NBCUniversal’s DevOps approach over-reliance on developers. “With robust infrastructure guaranteeing also broke down silos in deployment. UrbanCode, when a developer a level of quality and a level of service “Historically, releases could only take commits code without adequate unit to our Application teams,” say Connect with us place on certain days of the week— tests, it’s going to be flagged. If a Comas. Indeed, the DevOps team is typically Tuesday and Thursday developer didn’t have that as part of earning enough trust that business evening—and if you missed that the workflow you’re not going to units are now asking them to help window you couldn’t do a release know that. There’d be no feedback with larger projects. “Because we’ve until the following window,” Comas into the process telling you the status shown that the capabilities are flexi- says. “So by working with the of the application. Our Compass ble and expandable and that they Take the next step To learn more about UrbanCode Compass Run team we can now do workflow is probably the most allow us to deliver on a very tight Build software, UrbanCode Deploy deployments to production much comprehensive automation we have, timeline we’ve seen a six-fold software and IBM Cloud for Skytap more frequently, resulting in smaller, and the technology is helping us increase in project volume, from ten Solutions please contact your more frequent releases. So they have drive the application quality, improv- to more than 60 applications. We’re IBM sales representative or much greater agility and can meet ing build quality by an order of mag- providing a path to application IBM Business Partner, or visit the the business requirements much nitude. Not only that, we’re delivering production that engenders a level of following websites: faster. In short, the code quality has the changes the business wants confidence we’ve never had before.” improved, it’s going faster and we’re much faster. Those are the ultimate • ibm.com/cloud-computing/ able to meet the business require- goals; the reasons why we’re on this solutions/continuous-deployment/ ments much more rapidly.” DevOps journey.” index.html • ibm.com/cloud/skytap

Produced in the United States of America, November 2018. IBM, the IBM logo, ibm.com, and UrbanCode are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. It is the user’s responsibility to evaluate and verify the operation of any other products or programs with IBM products and programs. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. Statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Please Recycle RAC14389-USEN-01