IBM Rational Software Conference 2009 Agenda
Goals of Presentation
Rational Team Concert Concepts
CRMA41: Large Scale Enterprise Deployment of Deep Dive on Rational’s Deployment & Self-Hosting (R) (TM) Rational Team Concert 2.0 Best Practices and Lessons Learned Doug Weissman Sr. I/T Architect, IBM, SWG, Rational Brand Recent and Upcoming Enhancements dweissman@us.ibm.com
Scott Rich Distinguished Engineer - Jazz Foundation Development, IBM, SWG, Rational Brand [email protected]
CRMA41 Select View/Master/Slide Master to add Session Number Here
© 2009 IBM Corporation CRMA41 2
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Goals of Presentation Agenda
Use Rational’s deployment experience – Over 5,000 users Goals of Presentation and 250 Projects as an example to … Rational Team Concert Concepts
Help you think about a Rational Team Concert Deep Dive on Rational’s Deployment & Self-Hosting deployments Upgrades, including new things to consider with RTC 2.0 Best Practices and Lessons Learned Give some suggestions as to possible architectures and Recent and Upcoming Enhancements configurations Discuss best practices, pitfalls, and lessons learned
CRMA41 3 CRMA41 4
1 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Rational Team Concert Concepts RTC Simple Configuration Server OS Simple Configuration Server Operating System Windows / *Windows 64 Application Server Repositories vs. Project Areas Linux / *Linux 64 Rational Team Concert Authentication vs. Authorization *AIX Application Server Project Area Private Project Configuration (RTC 2.0) WebSphere Application Server (WAS) Team Area 250 Users & Beyond - RTC Edition – Standard vs. Apache Tomcat Application Server Team Area Enterprise? Database Derby Multiple RTC Servers / Repositories DB2 Directory (LDAP / Tomcat User / Other) C/ALM RTC+RQM+RRC Test Configurations (RTC 2.0) Microsoft SQL Oracle ClearCase / ClearQuest Bridge or Connectors Directory Services Database High Availability (RTC 2.0 Enterprise) Rational Team Concert
* RTC 2.0
CRMA41 5 CRMA41 6
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Repositories vs.. Project Areas Authentication vs. Authorization Application Server Authentication = RTC Repository Mechanism to RTC Repository verify identity RTCRTC ProjectProject AreaArea A Server OS RTC Project Area Application Server Team Area(s) Workitem RTC Project Area B Categories Rational Team Team Area(s) Workitem Category(s) Repository Concert Core Team Core Authorization Project Area Application Core Team Core Application = Permission to Team Area RTC Project Area C Web Team Web access a repository Team Area Application Web Team Web Application Client Access Server Team Server License (CAL) Directory (LDAP / Process-Level Tomcat User / Other) Server Team Server Authorization = Permission to access Project & Database Team Area resources
CRMA41 7 CRMA41
2 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Authentication Repository-level Authorization Apache Tomcat Application Server (Tomcat) Repository Roles and Access Can I get into the server? Supports authentication using either Tomcat Users or LDAP (but not Each role maps to one or more groups (LDAP Groups) in the registry (J2E concurrently) Roles) Tomcat Users JazzAdmins Less secure JazzDWAdmins Stores account passwords lightly encrypted on the application server JazzUsers Difficult to manage Password Policies JazzGuests Other forms of authentication require custom coding Licensing What can I do when I get there? WebSphere Application Server (WAS) Developer / Floating Developer License Supports Enterprise level authentication Contributor / Floating Contributor License Provides federated repository capability (See Useful Links ) Licenses for system processes like Build and Connectors Can authenticate with more than one authority (i.e. LDAP(s), AD and local operating system accounts) concurrently
CRMA41 CRMA41
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Process-level Authorization Private Project Configuration (new in RTC 2.0) Process Roles and Access Within my repository and Access Control license permissions, what can I do when I get there? Options This is where teams in Rational take over responsibility from IT… Everyone You may choose to give more or less freedom to teams No one (JazzAdmin only) Configured in the process configuration per Project and Team Area Members of the project hierarchy Access to specific Project Areas – Project level ACLs (RTC 2.0) Members of the project Refined access control to components and functionality hierarchy plus access list Role-Based Process Configurations Members of an access list Configurations support Timeline / Iteration Based Processes only
Development Milestone Phase * Note: Need to configure process roles Stabilization Milestone Phase to control how members of a Private Project will work
CRMA41 11 CRMA41 12
3 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009
Private Project Example Multiple Application Servers Application Server (OS) RTC Repository Application Server B Multiple RTC Repositories Application Server A RTC Project Area A Users Rational Team Concert Configured against a single Project AreaRational A TeamQuality Concert Manager RTC Project Area A Database server with Multiple Team AreaProject A Area A Team Area(s) Workitem Category(s) Database Instances Team AreaTeam B Area A Core Team Core Application Team Area B No longer necessary to Project Area B Web Team Web Application achieve project isolation Team AreaProject A Area B Server Team Server (private projects) or more Team AreaTeam B Area A than 250 users (Enterprise Team Area B RTC Project Area B RTC Project Area B Users Edition) Directory Team Area(s) Workitem Category(s) (LDAP) Core Team Core Application Good Candidate for C/ALM DB Server (OS) Web Team Web Application Configuration Server Team Server DB RTC A DB RTC B
CRMA41 13 CRMA41 14
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 C/ALM & CC / CQ Bridges and Connectors Topology High Availability – Idle Standby-server
Application Server (OS) External Interoperability ClearCase / This Basic HA solution is implemented for purposes of Application Server A ClearQuest between Jazz Rational Team Concert Environment providing a failover server. Bridge / Foundation Products ClearCase / Connector ClearQuest Bridge / The secondary server only becomes active when the primary Connector Application Server (OS) server fails. Synchronize and Project Area A Application Server B replicate CC & CQ Team Area A Rational Quality Manager It is not a configuration for load balancing. Team Area B or Rational Requirements artifacts into RTC Composer Project Area B Project Area A Team Area A Bridge workitems with C/ALM Team Area A Team Area B CC & CQ artifacts Team Area B
Project Area B DB Server (OS) Team Area A
DB(RTC) A Team Area B Directory (LDAP) DB(RQM/ RRC)B
CRMA41 15 CRMA41
4 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Agenda Deep Dive on Rational’s Deployment & Self-Hosting
Goals of Presentation Numbers and adoption rates over time
Rational Team Concert Concepts Our Current Topology
Deep Dive on Rational’s Deployment & Self-Hosting Jazz.net Topology
Best Practices and Lessons Learned Our Authentication & Authorization Story
Recent and Upcoming Enhancements How do we maintain availability?
Upgrades
Other Aspects of our Environment
CRMA41 17 CRMA41 18
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Total Number of Projects Hosted by RES Total Number of Unique Users of our Infrastructure
CRMA41 19 CRMA41 20
5 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Current RTC Topology Rational Internal Deployment – Raleigh, NC Mix of Jazz Foundation Products 24 Rational Team Concert Repositories (RSDC 2008 – 10) 2 Rational Quality Manager Repositories (RSDC 2008 – 0) 1 Rational Requirements Composer Repositories (RSDC 2008 – 0) Application Servers 4 Production Applications Servers Running AIX 6.1 (RSDC 2008 – 1) Combination of Tomcat and WebSphere Application Server (RSDC 2008 Tomcat only) 1 Production Application Server Running RHEL 5 (RSDC 2008 – All AIX 5.x) WebSphere Application Server Database Servers 4 Production Database Servers Running AIX 6.1 (RSDC 2008 – 1) All Running DB2 9.5 (RSDC 2008 DB2 9.1) Self-Hosting ClearCase and ClearQuest Bridges 2 Repositories Connected, 5 more in the pipeline (RSDC 2008 – Non in production) 10+ product teams in use or in the pipeline for connecting to CQ for customer APAR / RFE handling
CRMA41 21 CRMA41 22
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 GBS Internal Deployment – Littleton, MA Jazz.net Self Hosting – Toronto, ON
CRMA41 23 CRMA41 24
6 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Our Authentication & Authorization Story How do we maintain availability? Enterprise LDAP Environment Our Configuration Tied to IBM HR Data Not HA at the middleware layer Password Management – Aging, Complexity Rules, Lockouts, Resets Solid pSeries & xSeries servers Self Service LDAP Group management (JEE Roles) Intelligent power setup User Account Management Tool for Account Requests Solid disk subsystem Distributed repositories on multiple physical servers Manual Processes outside of RTC Workflow engine – Audit trail for tracking management approvals Monitoring Generates Requests for Adding users to specific Repositories and Projects IBM Tivoli Monitoring 6.2 for OS and DB2 Validate quarterly against Project membership Home Grown Scripts for Application Availability Central Application Management team owns JazzAdmin Role Rapid notification of Issues via Pager Import New users based on request system Backup / Time to Restore Assign CALs – We use a Floating Licenses Nightly database snapshots Add users to a Community Team in each project (used for quarterly validation) Distributed schedule of snapshots per database server that host more than one DB instance, to prevent over taxing the database server Investigating the use of Tivoli Identity Manager for automated account We store 3 copies to NAS storage – Backup to tape using TSM handling
CRMA41 25 CRMA41 26
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Upgrades Agenda
Major version upgrades happen on a preview server Goals of Presentation Upgrade based on a previous snapshot and make available prior to production upgrade Allows for testing of functionality, plug-in compatibility, build configurations Rational Team Concert Concepts Upgrade Schedule based on type of deployment Standard – eGA deployments only Deep Dive on Rational’s Deployment & Self-Hosting Cutting Edge – Upgrade schedule varies based on RTC milestone / beta release schedule We Always perform an offline backup of the repository before an upgrade Best Practices and Lessons Learned Repository Upgrades vs. Migrations Upgrades are quick deployments – 30 minute scheduled outages Recent and Upcoming Enhancements Migrations require export and import to new version – time can vary based on size of repository – 1 to 20 plus hours (not including backup) RTC 1.0.x to 2.0 SCM Considerations – Workspaces are not compatible between versions
CRMA41 27 CRMA41 28
7 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Best Practices Lessons Learned Centralized vs. Distributed Deployments Centralized vs. Distributed Deployments Rational’s deployment is biased to a centrally managed deployment System Configuration / Considerations May be geographically distributed infrastructure Allows us a higher level of certainty that Security is being Virtualization maintained for the critical assets stored in RTC Networking Easier to manage continuity / disaster recover plans Ensures a well tuned infrastructure environment Disaster Recovery & Continuity Planning Take advantage of economy of scale Project Partitioning / Process Configurations System resources – Servers / Storage / Data Center Space Staffing – SME’s, SPOC Staffing Considerations
Other Considerations
CRMA41 29 CRMA41 30
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 System Configuration & Considerations Virtualization Application Server Types of Virtualization Supported Memory Intensive – Monitor Heap Size Application Server Consider 1 to 2 GB of RAM per Application Server as a starting point WebSphere Application Server Profiles Fast storage for Reports Temp directory Apache Tomcat Application Instances Report snapshots build to temp storage before being stuffed in the data warehouse Database Instances Users viewing reports build to temp storage OS Level Images Enterprise Edition supports Standby Configuration with WAS VMware – supported in RTC 2.0 Database Server Considerations for Virtualization Memory, CPU and Disk intensive Availability – VMware VMotion Technology Configure storage in a High Available configuration – Striped Volumes (RAID) Consider HA options – RTC can utilize it Partitioning of high-end server to take full advantage of systems resources Bulk of the storage, including Source Code and Attachments are stored in the database Consider Database server virtualization closely Consider 1GB starting and 250MB per Developer as a starting point pSeries LPARs is a solid choice, but other technologies may reduce performance High availability configurations require more servers Virtualization is great, but monitor closely to ensure you don’t oversubscribe the environment to the point that performance is unacceptable Keep Database on same LAN as Application Server Virtualized images want high performance disk subsystems
CRMA41 31 CRMA41 32
8 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Networking Disaster Recovery & Continuity Planning Firewalling Backup Considerations Client to Application Server – No problem HTTP(S) Database backups or resiliency Consider nightly online snapshots Work with firewall vender if placing a firewall between the Application and – Schedule snapshots for off peek time Database Server to ensure supportability – Distribute schedule if multiple database instance on a server High latency between the Application and Database Server is not Ensure archive logs are enabled recommended Application Server Backup Considerations /conf/* directory (RTC 2.0) End User Proximity to RTC Infrastructure Profile backups (for WAS) RTC is well optimized for the WAN Tomcat-users.xml (Tomcat) Traffic flows over standard HTTP protocol RTC Recovery process CQ Gateway Placement Restore the database; roll forward using archive logs Utilizes Team API to communicate with CQ – Not WAN friendly Deploy RTC to your application server Must deploy the same release of RTC that the database snapshot was taken of LDAP or Directory Infrastructure Rebuild Workitem Index If the Application Server can not communicate with the Directory environment Replace configuration files and reapply licensing files if needed then you can’t access RTC Consider a recover site using CNAMEs to redirect traffic
CRMA41 33 CRMA41 34
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Project Partitioning / Process Configurations Staffing Considerations Lots of decisions to be made here… Centralized or Distributed management Who Owns What Responsibilities Start with a natural mapping of projects to ongoing efforts Servers / Storage / Tape Libraries Additional timelines to support parallel dev/maintenance efforts Middleware / Applications Team areas map to sub-teams of the project Security Corporate Policies Work Item categories and SCM components to further Patches – OS, Database, Application Server, RTC categorize work and organize related code User Access Validation / Employment Validation Account Management Review the Everyone (default) role, you may inadvertently be What’s your Corporate Directory Services giving everyone the ability to do things you don’t want them to Who manages the Groups that will map to your J2E Roles Who is managing Import of IDs and License Allocations for RTC End user Support see slides: SDP20 – “Getting Started with IBM Rational Interoperability – Connectors and Bridges Team Concert” Migrations
CRMA41 35 CRMA41 36
9 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Other Considerations Other Considerations … Continued Why would you consider deploying multiple RTC repositories with RTC Compartmentalization of Sensitive Intellectual Property 2.0? Multiple Project Areas for a Single Product Consolidating Existing Repositories Private Project for Source Control Management Public Project for Workitems / Planning / Dashboards Geographically Distributed Teams See Jean-Michel’s video on using private projects to control source code Maintenance Windows / Upgrade Flexibility visibility Different Authentication Realms Front End web site for aggregating RTC Instances Security zones Reverse Proxy to RTC Instances Migrating from Other Tool Establish Rules of the Road Early Know your stakeholders – Moving tools can be painful Roles and Responsibilities Should I … Import, connect or bridge? Infrastructure Teams Create Sandbox Environments Project Leads Test Drive before settling in on a final process configuration SPOC (Single Point of Contact) Ensure Imports work as planned Scheduled Outages Ensure Connectors and/or Bridges work as expected Service Level Agreements
CRMA41 CRMA41 38
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Agenda RTC 1.0.1.1 Enhancements
Goals of Presentation Incrementally Supporting Larger Teams Floating License support – server limits only count against concurrent users Rational Team Concert Concepts RTC Contributors excluded from server limits HTTP proxy support – scale and WAN-friendliness Deep Dive on Rational’s In House Deployment & Self-Hosting Native BuildForge Integration Best Practices and Lessons Learned
Recent and Upcoming Enhancements
CRMA41 39 CRMA41 40
10 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 RTC 2.0: Enhancements 250 Users & Beyond - RTC Edition – Standard vs. Read permissions – private projects Enterprise? High Availability Support for 64-bit server platforms (Windows, Linux, AIX) Higher performance OS, increased memory capacity Repotools support for users creation and licensing repotools - createUsers fromFile=users.csv Enterprise Edition removes the 250 user limit User limit 250 unlimited ClearCase / ClearQuest C/ALM integrations Synchronizer and Bridge IBM Developer Cloud Floating licenses LDAP import High Availability
CRMA41 41 CRMA41
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 RTC 2.0 Enterprise Configurations: RTC 2.0 Enterprise Configurations: up to 750 Users - Dual Tier Solution up to 1500 users on latest server technology 2 IBM System X 3550 Dual CPU Intel Xeon 5160 2.4 GHz or Single Tier - IBM System x3650 M2 - Dual CPU Intel Xeon higher 64-bit 5500 2.4 GHz or higher 64-bit Memory - 4GB* or higher (recommend 8 GB) Memory - 16GB or higher Storage – High Performance SAS (15K), RAID, SAN or NAS recommended Storage - NetApp 3050C provides a Fault Tolerant High Availability solution snapshot backup (near zero downtime backup support)* Software Recommend Fiber or ISCSI connection to NAS* Operating System – Red Hat Release 5.3 or Windows 2003 Server Software Application Server - Tomcat 5.5 or WAS 6.1.0. 23 or higher – Operating System – Red Hat Release 5.3 or Windows 2003 Server Database - Oracle 10GR2, DB2 9.1 , DB2 9.5 FP 3b, SQL 2005 and 2009 – Application Server - Tomcat 5.5 or WAS 6.1.0. 23 or higher – Database - Oracle 10GR2, DB2 9.1, DB2 9.5 FP 3b , SQL 2005 and 2009 * bold = Load-Tested Configuration
* bold = Load-Tested Configuration
CRMA41 CRMA41
11 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Developer Cloud Within IBM Cloud IBM Cloud Strategy • Public Cloud with multiple service “on-ramps” for clients, hosted in key centers worldwide • Common infrastructure to provide computing and storage resources • Common platform, BSS and OSS services to operate and manage • Range of IBM and partner services • Implementation and Hosting Services to build “Private Clouds” for SO data centers, other clients
Common Cloud User Experience – “The IBM Cloud” Portal
Develop & Information Collaboration Desktop Applications Test Protection On-demand Developer applications IBM Info Lotus Live! Desktop as a (horizontal and Cloud Protection Service vertical industry) Services
Common Cloud Platform Operational Support Services Business Support Services Platform Products Metering, Monitoring, Provisioning, Billing, Reporting, Contract Mgmt, Web Application Server, Infrastructure Mgmt, Capacity Planning, SLA Mgmt, Order Mgmt, Account Database, Tools, Systems Infrastructure Security Mgmt Management
“Compute” Cloud Storage Cloud
45 CRMA41 10/06/09 CRMA41
IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 5 mins later -> RTC Running in the Cloud
CRMA41 CRMA41 48
12 IBM Rational Software Conference 2009 IBM Rational Software Conference 2009 Concluding Thoughts Useful Links
RTC 2.0 Enterprise takes Scaling to a new Level Jazz.net 64-Bit Platform support on the Server Enterprise performance and scalability testing HA Options DB2 Online Backups Try it out WAS Configuration with Federated Realms Download a Trial version and give it a Try Basic Deployment can be deployed in an hour Using read permissions in RTC 2.0 to protect source code
Private Projects or Project ACLs is a game changer on deployments
Know your Customer / Stakeholder
Participate on jazz.net – Great resource
CRMA41 CRMA41 50
IBM Rational Software Conference 2009
© Copyright IBM Corporation 2009. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
CRMA41 51
13