Sequent’s NUMA-QTM Architecture Overview Platform architecture This document overviews the path-break- alternatives for the enterprise ing technologies introduced by Sequent’s® Sequent’s target market is the commercial NUMA-Q™ architecture. It describes data center solving mission-critical prob- how the current enterprise-class system lems. Computer systems in these centers architectures are driven by usage models have several characteristics in common. such as on-line transaction processing They need to be highly available (just (OLTP), decision support systems (DSS), minutes of downtime per year), highly and business communications. It also reliable, capable of meeting ever increasing describes Sequent’s work to develop a performance demands, highly scalable, common building block for all enterprise- and finally integrated into a heterogeneous class system architectures. This includes systems management environment. The descriptions of the 4x Intel® Pentium® Pro primary applications, or usage models, processor SMP system (quad) building found in data center computing today block, a Sequent-developed system inter- fall into three major categories: connect for linking these quads, the ■ OLTP: On-line transaction processing architectures to which they can be refers to the day-to-day management applied, and the benefits realized from of business functions using a relational these applications. database. ■ DSS: Decision support systems refer Sequent’s new NUMA-Q (Non Uniform to the extraction, analysis, and pre- Memory Access for Quads) architecture sentation of data from databases to yields new levels of performance, avail- enable decision-making based on ability, and manageability in enterprise- operations. class systems. NUMA-Q is not so much ■ Business communications: Refers to a family of products as it is a quantum messaging, web servers, document leap in symmetric multiprocessing (SMP) retrieval, and workflow. and clustered systems architectures, and the realization of highly available and manageable enterprise-class networked server architectures. Networked Servers Large SMP systems MPP, Shared Nothing systems Clustered Shared Disk SMP systems Four fundamental architectures for enterprise-class computing 1 Architecture Usage Model Pros Cons Networked servers Small Digital Libraries Inexpensive Management, availability SMP DSS, OLTP, Bus. Comms Easy to program Limited in size by backplane Clustered SMP DSS, OLTP, Bus. Comms High Availability Requires more management MPP DSS Can be very large Data skew problems OLTP, DSS, and business communica- networked-servers model is unsuitable tions systems designers currently have for implementing large OLTP, DSS, and four architectural options for their com- business communications applications. puting platforms: It forces an arbitrary distribution of ■ Small networked-servers: Multiple data across servers, leading to the diffi- small standalone servers connected cult problems of migrating processes over a network. or replicating data across a network ■ Large SMP nodes: Many processors of servers. and resources running under one operating system. The primary difficulty with implement- ■ Clustered SMP nodes: Multiple ing a vast network of small servers is instances of an application running in management and availability. Most on separate nodes under separate commodity server companies put more instances of an operating system, emphasis on cost than reliability and but sharing some storage devices manageability. As a result, networked- and data. server solutions are often driven by a ■ MPP (Massively Parallel Processor) low-cost requirement and suffer avail- systems: Many unique instances of ability and manageability problems. an operating system and application, on separate nodes, usually without SMP any shared resource, and communi- Large single-node SMP systems have cating by passing messages. gained popularity, because they are ideally suited to large DSS and OLTP Each of the usage models has different applications. Data managed by an SMP requirements with respect to I/O, mem- system is centrally located, users share ory, processor, and connectivity. Thus, a pool of resources, and SMP systems each architecture has characteristics are easy to manage. Additionally, single that can be a help or a hindrance SMP nodes make it easy to measure depending on the usage model. The peak performance, and project and plan choice of architecture is, therefore, for future performance needs. Another largely dependent on the usage model. reason why SMP has become the domi- nant enterprise architecture is because Networked servers it provides a smooth migration path for The networked servers model suggests sophisticated uniprocessor applications that many large computing problems to high-performance multi-processor can be solved by a network of small systems. computers or servers. It is true that a collection of networked servers can be One future drawback for large single- successfully and economically applied node SMP systems is that the number to some problems such as a small of processors will be increasingly limited World Wide Web service, or a digital by the size and speed of the backplane library for presentations and documents and the shared system bus. Physics is in large corporations. However, the the largest contributor to future band- 2 width limitations. As microprocessor application be running on all nodes performance continues to dramatically simultaneously and that nodes commu- increase, computer system designers are nicate before making changes to shared forced to make a bus length/bus speed data. The latter point is, in fact, what tradeoff—electrons travel at near light controls cluster scalability. It is also one speeds, and no amount of encourage- of the many factors that limit the appli- ment will speed them up! Large SMP cation of MPP to business problems. system designs must include shorter The industry has learned how to make backplane/system buses to meet the 4-8 SMP nodes communicate effectively, needs of faster processors and I/O. The while MPP architectures attempt to smaller backplanes, while faster, support pass messages between hundreds of fewer processors simply because of nodes. The immaturity of MPP message- packaging constraints. This architectural passing software, and the associated limit will constrict the amount of I/O overhead, limits MPP’s applicability into and out of single-node SMP sys- of DSS and OLTP problems. “Out of tems. In the future however, DSS and box” clustered SMP performance is still business communications applications improving, especially with the advent will continue to require increasing of software that can take advantage of amounts of I/O. Another downside to reflective memory technology such as large single-node SMP systems is that Sequent’s Scalable Data Interconnect there are single points of failure, which (SDI). can cause application interruptions. The downside of clusters is that they Clustered SMP require more thought in management The solution to the latter problem is the and load balancing. The more nodes, interconnection of single SMP systems the more complex the problem. The into a cluster of nodes. When imple- speed and latency of the message passing mented to gain availability, clustering interconnect is key to improving the provides enough performance on one scalability of clusters. or more nodes-and access to common resources-to completely replace the MPP unplanned loss of another node. In the The one overwhelming advantage of event of a single node outage, the other MPP architectures is the ability to con- nodes continue to operate and may nect hundreds of processor/memory automatically assume the load of the cells (individual nodes with their own failed node in a period of minutes. copy of the OS and application). This Open-systems relational database man- is also the overwhelming disadvantage. agement system (RDBMS) companies For problems that require an enormous are evolving their software to support amount of I/O followed by localized clustered environments to dramatically computation, and where the intervening improve availability beyond what has results and original data do not have to been practical on traditional single-node be shared across the pool of processors, SMP systems. MPP systems can offer satisfactory results. A video server is just such an Clusters can also achieve far greater application. However, for applications performance and scalability than a single that need to scan large data sets in an SMP node. This “out of box scaling” unpredictable fashion (DSS) or applica- is generally due to the increased number tions that require many updates (and, of users that can be connected, the therefore, locking) like OLTP, the cum- increased I/O bandwidth, and the bersome messaging of MPP becomes a increased amount of processors and bottleneck. memory. It also requires that the 3 This is where SMP’s single large memory To overcome the architectural limitations and processor pool excels. In an SMP of virtually all the current approaches system, message-passing between proces- to building enterprise-class computing sors is implicit through shared memory systems today, Sequent launched a and as such is orders of magnitude massive design project in 1992 with faster than MPP. The equally short the following ambitious goals: memory access latency of SMP systems ■ Creating a new set of CPU and makes optimizing performance quite memory-interconnect building straightforward. The “distributed every- blocks for enterprise-class