24 x 7 Borland StarTeam A practical look at high availability A Borland White Paper Randy Guck Chief Scientist, Borland Software Corporation January 2005 24 x 7 Borland StarTeam Contents Overview ..................................................................................4 High-availability fundamentals .................................................4 How available is highly available? ............................................................................................5 A distorted term .......................................................................................................................................5 Availability by the numbers ....................................................................................................................5 The myth of the nines ..............................................................................................................................6 A better approach to availability.............................................................................................................7 High availability at what cost? ...................................................................................................8 Availability vs. investment......................................................................................................................9 ALM high availability in perspective..................................................................................................10 Enemies of high availability .....................................................................................................11 Infrastructure failures ............................................................................................................................11 Application outages ...............................................................................................................................13 Plan of attack .............................................................................................................................15 StarTeam high-availability best practices...............................15 Administrative practices............................................................................................................15 Practice #10: Don’t be cheap ................................................................................................................15 Practice #9: Enforce security ................................................................................................................16 Practice #8: Centralize your servers .....................................................................................................16 Practice #7: Enforce change control .....................................................................................................16 Practice #6: Document everything........................................................................................................16 Practice #5: Test everything..................................................................................................................17 Practice #4: Design for growth .............................................................................................................17 Practice #3: Choose mature software ...................................................................................................17 Practice #2: Choose mature hardware ..................................................................................................17 Practice #1: K.I.S.S................................................................................................................................17 Flash control ..............................................................................................................................18 StarTeamMPX .......................................................................................................................................21 StarTeamMPX Cache Agent.................................................................................................................22 Backup procedures ....................................................................................................................23 Backup procedure prior to StarTeam 2005 ..........................................................................................24 Backup procedure with StarTeam 2005 ...............................................................................................25 Redundancy: Reducing SPOFs.................................................................................................28 Failover management ................................................................................................................30 Failover prerequisites ............................................................................................................................30 Active/passive failover configuration...................................................................................................31 Process monitoring with Borland Deployment Op-Center .................................................................34 2 24 x 7 Borland StarTeam Disaster Recovery......................................................................................................................35 Replication for DR.................................................................................................................................35 Synchronous replication ........................................................................................................................36 Asynchronous replication......................................................................................................................37 Batch replication ....................................................................................................................................38 Other StarTeam 2005 high-availability features .....................................................................38 Summary ................................................................................39 Glossary .................................................................................40 References .............................................................................42 3 24 x 7 Borland StarTeam Overview Software configuration management (SCM), change management, and other application lifecycle management (ALM) processes are quickly becoming mission-critical processes within the enterprise. Correspondingly, organizations that depend on Borland StarTeam are seeking ways to maximize its availability and resiliency to keep development teams running uninterrupted. In this white paper, StarTeam high-availability topics are discussed, such as online backups, failover techniques, and disaster recovery. Specific StarTeam capabilities and techniques that afford continuous operation in the global enterprise are described, including features introduced in the 2005 release. The goal of this white paper is to help you prepare a comprehensive high-availability plan for your organization’s use of StarTeam. Because there are many factors that contribute to a comprehensive plan, the first part of this paper discusses general high availability issues such as what high availability means and costs vs. benefits. The second part focuses on specific strategies and options that you can use with StarTeam to provide the level of availability needed by your organization. You will find that high availability can be achieved with a graduated series of measures, each of which improves availability at an additional cost. High-availability fundamentals As you probably already suspect, there are no true 24 x 7 applications. Even the most critical and downtime-sensitive systems experience glitches and even failures. The real goal for critical applications is to maximize availability, coming as close to 24 x 7 operation as possible. In this section, we’ll take a practical look at what high availability means, what it costs, and the factors that make high availability a challenge. 4 24 x 7 Borland StarTeam How available is highly available? In this section we take a look at what high availability means and how we measure it. A distorted term Back when mainframes ruled, high availability had a well-defined meaning. Unfortunately, with today’s myriad hardware and software products, the term has been hyped, marketed, and distorted to mean many things. Depending on whom you ask, high availability could mean any of the following: • 24 x 7 uptime: A worthy goal, but not absolutely attainable • Clustering: One of many technologies that can improve availability • Failover: Any of several techniques for recovering from an outage • Online backups: Another technique that can improve availability by reducing scheduled downtime • “Five nines” or “Six sigma”: 99.999% or 99.9999% availability, measured or predicted over time • Currently not dating: As we said, the term has lots of meanings. Availability by the numbers Most commonly, the availability of a given service is measured as a percentage of time that the service is actually or theoretically available, on average, compared to the desired availability of the service. The following chart translates uptime (and downtime) percentages into real time: 5 24 x 7 Borland StarTeam % Uptime % Downtime Downtime per year Downtime per week 99% 1% 3.65 days 1 hour, 41 minutes 99.9% 0.1% 8 hours, 45 minutes 10 minutes, 5 seconds 99.99% 0.01% 52.5