Calix E7 Platform Release 1.0 Product Planning Guide

f CUSTOMER ADVISORY Calix E7 Platform Release 1.0 BULLETIN NO: CAB-09-025 Product Planning Guide DATE: 11/06/2009 ORIG: Field Marketing DIST: External

The information contained in this product planning guide is not a commitment, promise or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion. CALIX CONFIDENTIAL CUSTOMER ADVISORY BULLETIN 11/06/09 PAGE 1 OF 87 CAB-09-025

Table of Contents

Calix E7 Platform Release 1.0 Product Planning Guide ...... 1 Calix E7 Platform Overview ...... 6 Product Description ...... 6 E7 Line Cards Ensure Flexibility ...... 7 E7 CONVERGED Network Solutions...... 8 High Availability System Design ...... 10 E7 ARCHITECTURE AND KEY ATTRIBUTES ...... 10 Calix Unified Access Infrastructure ...... 12 Subscriber Services ...... 13 Residential Triple Play Services ...... 13 Voice ...... 13 IPTV Video ...... 13 High-Speed Internet Access ...... 14 Business Services ...... 14 MEF Data services ...... 14 T1 Service ...... 14 CATV Services...... 15 RF Video Delivery ...... 15 RF Return ...... 16 E7 Platform Details ...... 17 System Components ...... 17 E7 Line Cards ...... 18 Subscriber and Network Ports ...... 18 Pluggable Optics ...... 18 Card Status Indicators ...... 19 Port Status Indicators ...... 19 FAN TRAY ASSEMBLY (FTA) ...... 19 System Management ...... 20 Management Interfaces ...... 20 System Status Indicators ...... 21 E7 2-Slot Chassis ...... 21 Rack Mounting Options ...... 21 Power and Ground Connections ...... 22 Alarm I/O ...... 22 BITS Timing ...... 23 Copper Connectors ...... 24

Power consumption and Heat Dissipation ...... 24 E7 Dual Line Card Operation ...... 25 Backplane Port configuration ...... 25 Backplane port Use Cases ...... 26 Case 1: Default for single card in E7 shelf ...... 27 Case 2: Send all 10GE ports to front (single card system)...... 27 Case 3: Adding a second card to E7 shelf ...... 27 Case 4: Changing which ports are backplane ports ...... 27 Case 5: Using 2 backplane ports with 20Gbps bandwidth ...... 28 Case 6: EAPS domain spanning across two E7 cards ...... 28 Case 7: Mis-configuration during commission of 2nd card ...... 28 E7 Ethernet Features...... 29 Ethernet Interfaces ...... 29 VLAN Support ...... 29 VLAN per Port Provisioning Model ...... 30 VLAN Stacking / Q-in-Q ...... 32 VLAN per Service Provisioning Model ...... 33 Hybrid VLAN Provisioning Model ...... 34 MEF Compliant Transparent LAN Services (TLS) ...... 34 Ethernet Traffic management ...... 35 Traffic Management and Rate Limiting on Ingress ...... 36 Traffic Shaping ...... 36 Rate Limiting ...... 36 Ethernet Topology Protocols ...... 36 Rapid Spanning Tree Protocol (RSTP) ...... 36 Link Aggregation ...... 37 ITU G.8032 ERPS / Enhanced EAPS ...... 39 E7 GPON FEatures ...... 43 GPON Overview ...... 43 Calix GPON FTTP Solution ...... 43 Calix ONT Interoperability ...... 44 Optical Distribution Network ...... 45 ONT Ethernet Services ...... 46 VLAN Service Model ...... 46 GPON MAC Learning and VLAN Switching ...... 46 Classification ...... 47 ONT Tag Action ...... 47 Bandwidth Policing and Shaping ...... 48 GPON Class of Service (CoS) ...... 49 IGMP ...... 50 Multicast Service Profile ...... 50 IP Hosts ...... 51 The information contained in this product planning guide is not a commitment, promise or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion. CALIX CONFIDENTIAL CUSTOMER ADVISORY BULLETIN 11/06/09 PAGE 3 OF 87 CAB-09-025

ONT Pre-Provisioning and Activation ...... 51 ONT Pre-provisioning ...... 51 ONT Arrival and Activation ...... 52 Video Services ...... 53 IGMP Snooping Operation ...... 53 IGMP Scalability ...... 54 E7 Management ...... 55 System Configuration ...... 55 Commissioning ...... 55 Shelf Controller Configuration ...... 56 Software Upgrade ...... 56 Configuration Management ...... 56 Performance Monitoring ...... 57 Realtime PM and Diagnostics ...... 57 MAC Table Display ...... 58 VLAN Monitoring ...... 58 Threshold Crossing Alerts ...... 58 Port Mirroring ...... 58 Fault Management ...... 58 Security Management ...... 59 SNMP...... 59 E7 applications ...... 61 CO AND RT Deployments ...... 61 Central Office FTTP Configurations ...... 61 Remote Terminal FTTP Configurations ...... 62 Point to Point Ethernet Services on Every E7 Card ...... 65 Ethernet Transport and Aggregation ...... 66 Ethernet Point-to-Point Aggregation ...... 66 Ethernet Ring Topologies ...... 66 Ethernet Aggregation Topology ...... 67 Protected Uplink with Virtual Router Redundancy (VRRP) ...... 68 Diverse Fiber Route and RSTP Node Protection ...... 69 T1 Pseudowire applications ...... 71 Calix 766GX ONT ...... 72 T1 Synchronization ...... 73 Traceable clock source ...... 74 T1 Pseudowire Aggregation ...... 74 T1 Pseudowire Applications ...... 75 Network Compatibility ...... 81 The information contained in this product planning guide is not a commitment, promise or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion. CALIX CONFIDENTIAL CUSTOMER ADVISORY BULLETIN 11/06/09 PAGE 4 OF 87 CAB-09-025

A Complement to the Calix Access Portfolio ...... 81 E7 Specifications ...... 82 System Specifications ...... 82 E7 port capacity ...... 83 power and Heat dissipation ...... 83 Ethernet switching capacity ...... 84 Protocols and standards Supported ...... 85 SNMP...... 86 Ordering Information ...... 87

CALIX E7 PLATFORM OVERVIEW

PRODUCT DESCRIPTION The Calix E7 Ethernet Service Access Platform (ESAP) integrates IP service delivery and Ethernet transport into a compact, high availability (HA) and carrier-class, modular chassis that delivers extensible network solutions for service providers. The E7 ESAP delivers Gigabit Passive Optical Network (GPON) and point-to-point Gigabit Ethernet (GE) services with integrated, standards based, 10-Gigabit Ethernet (10GE) transport and aggregation within a single 1RU, 2-slot chassis. The E7 allows service providers to deliver differentiated triple play services, advanced business services, and mobile backhaul from a single converged network that revolutionizes the economics of networking by enabling new services or market expansion with a flexible, scalable, pay-as-you- grow solution.

Figure 1: Calix E7 ESAP

Ethernet eXtensible Architecture (EXA): Standards-based Ethernet kernel built for access The Calix E7 ESAP is EXA Powered. It’s built on an Ethernet kernel optimized for the access network, leveraging the latest generation of deployable Ethernet standards --- ensuring your network is highly extensible and ready for an all-video world. IP Services Access Network: The differences between residential and business services are disappearing as more subscribers work from home offices with premium service level agreements and more corporations rely more heavily on distributed social-networking applications common to residential subscribers. Meanwhile, internet “over the top” video services are consuming an ever increasing percentage of the network capacity. High performance applications demand high performance networking solutions and the Calix E7 is extremely well suited to fulfill the needs of the evolving IP services access network. The E7 is designed to deliver a wide array of high performance applications, including 10GE Ethernet transport, delivery of high density residential triple play services over GPON and point-to-point Ethernet, Metro Ethernet Forum (MEF) compliant business services, mobile backhaul, and protected GE aggregation of other E7 chassis as well as Calix C7 and E5 platforms..

MAXIMUM INTERFACE COUNT PER 1RU E7 CHASSIS

10GE Transport 8 ports 10GE with 24 ports GE aggregation GE Aggregation 24 ports GE with 8 ports 10GE transport and uplink GPON Service 8 GPON OLT ports,16 GE ports for service or aggregation, and 10GE transport and uplink Point-to-Point (Active) Ethernet 24 ports GE with up to 4 ports 10GE uplink

E7 LINE CARDS ENSURE FLEXIBILITY The E7 platform supports a flexible lineup of high capacity line cards offering the latest generation of unified, carrier grade IP services & technologies. All line cards integrate non-blocking, full duplex switches to meet the evolving needs of residential, business and transport applications. All E7 line cards use pluggable modules for optical and copper-based interfaces. Industry standard SFP, SFP+ and XFP modules ensure cost effective scalability and interoperability while enabling a pay-as-you-grow business model. A common set of Ethernet services and network topology protocols are supported on all E7 line cards and individual cards can be used interchangeably create a redundant system configuration. Further, while each E7 card has been designed to meet the needs of a specific application, many network deployments require a mix of applications. The unmatched strength of the E7 platform architecture is such that each card can be deployed alongside any other to provide “just right” network design for mixed-application configurations.

The Calix E7 10GE-4 card combines four 10-Gigabit Ethernet 10GE ports with twelve GE ports to provide high speed Ethernet transport with integrated aggregation of lower speed Ethernet devices. The E7 10GE-4 line card can be plugged into one or both of the two universal slots within a Calix E7 shelf to create a compact, high availability (HA) Ethernet transport switch ideal for aggregation and delivery of IP services across the access network.

The Calix E7 GPON-4 card provides multiservice capability over four GPON OLT ports that can subtend up to 64 ONTs each for a card capacity of 256 GPON ONTs, 512 per E7 1RU chassis. An additional eight GE ports per card can provide high-bandwidth, point-to-point Ethernet services to individual subscribers or be used to aggregate other Ethernet devices. Four 10GE ports per card provide integrated transport and uplink capability.

The Calix E7 GE-12 line card provides twelve GE interfaces, along with two ports of integrated 10GE. The E7 GE-12 card can be plugged into one or both of the two universal slots within a Calix E7 shelf to support up to 24 point-to-point Ethernet Optical Network Terminations (ONTs) per 1RU E7 chassis. The GE-12 supports bidirectional, single fiber GE optics modules that match the Calix 700GX ONTs.

E7 CONVERGED NETWORK SOLUTIONS With a high availability architecture and ubiquitous 10GE support on all line cards, the Calix E7 system bridges the gap between traditional IP service access nodes and Ethernet aggregation switches. The E7 delivers the most demanding converged voice, video and data services over a variety of optical network topologies and technologies with a resilient, future-proof IP platform.

HIGH AVAILABILITY ETHERNET TRANSPORT: 10GE Uplink / Redundancy Designed to provide transport and aggregation for any Ethernet-based access network, the 10GE-4 card is the core element of the E7 Ethernet transport solution. Ring protection protocols and link 10GE 10GE aggregation are used along with the E7’s native dual- 10GE Ring card redundancy to provide unparalleled Ethernet transport redundancy. The 10GE-4 card includes four 10GE ports for use as uplink and transport for the local E7 system; or to subtend additional E7 chassis from the primary E7 shelf. The GPON-4 card includes 10GE10GE Transport / GE Aggregation / Aggregation the same lineup of 10GE ports and can be used interchangeably with the 10GE-4 card. Multiple E7 systems can be linked together using low cost 10GE 10GE NxGE SFP+ copper cables to create a high-speed 10Gbps transport ring with aggregated, high-density GPON GE 10GE 10GE and point-to-point Ethernet subscriber services. Each 10GE-4 card provides twelve GE ports that can be GE used to provide protected aggregation links to other Calix E7, C7 and E5 platforms.

GPON RESIDENTIAL SERVICES: The Calix E7 IP Unified GPON / GE services platform is ideal for delivery of GPON and GE FTTP-based services. With the GPON-4 card, network operators can support GPON residential services and 10GE GPON dedicated point-to-point GE service delivery in a single 10GE platform. The flexibility of the Calix E7 allows service providers to maximize revenues and customer GE satisfaction while reducing both operational and capital costs.

POINT-TO-POINT ETHERNET SERVICES: Point-to- Residential GE point Ethernet is a complimentary network design to point-to-multipoint GPON access networks. As a GE dedicated service, point-to-point Ethernet allows 10GE service providers to provide individual customers with 10GE very high-bandwidth, dedicated symmetric services. It also has the greatest deployment flexibility and OSP GE simplicity, with the tradeoff of requiring additional fiber management in the CO and OSP.

MOBILE BACKHAUL: With integrated network Mobile Backhaul synchronization, hierarchical QOS and support for T1 services, the E7 provides uncompromised transport of mobile broadband traffic while also supporting triple 10GE play residential and MEF certified business services 10GE from a single platform. A powerful collection of classification, policing, and scheduling algorithms let operators manage per-subscriber and per-service traffic flows to maintain priority/delay/loss differentiation within the E7 network.

METRO ETHERNET BUSINESS SERVICES: The E7 Business Services can be used to deliver Metro Ethernet Forum (MEF) compliant business services. All E7 cards supports up to 4,092 Ethernet Virtual Connections (EVCs) and can 10GE GPON be configured for E-Line (point-to-point) and E-LAN 10GE (multipoint-to-multipoint) services over both GPON and GE networks. GE

HIGH AVAILABILITY SYSTEM DESIGN The Calix E7 combines the most advantageous attributes of a small fixed form factor product with a large chassis-based system, while mitigating the disadvantages of each. • 1RU design is small yet can expand with an extra slot for very low first install cost • Additional chassis are added with subscriber growth yielding a near linear cost curve • Line cards are managed as a single chassis for operational efficiency • Mix and match line cards in a common chassis • Line cards can be added or removed without affecting other units or cards • Each chassis unit can be physically added or deleted without affecting services on the other units • Subscribers are aggregated and network resources efficiently shared across protected trunk facilities • Resilient, hot-swappable fan tray requires less power

With the Calix E7, service providers no longer need to decide between a single service product and a high growth chassis solution when considering a new service market. The E7 provides low cost deployments, operational efficiencies and near linear incremental costs per subscriber. The E7 system grows in step as the subscriber base grows, enabling Calix customers to maximize the return on their investment.

Figure 2: High Availability Converged Network Design

E7 ARCHITECTURE AND KEY ATTRIBUTES IP has become the dominant networking protocol and all services – voice, data, and video – are rapidly becoming 100% IP-based. The E7 enables multi-service convergence over IP and Ethernet, providing key advantages for service providers. • Protocol aware aggregation and transport • Application specific Service Level Agreements (SLA) • Dynamic, VLAN based traffic engineering • Synchronization based on application

HIGH PERFORMANCE ETHERNET SWTICH: Each E7 line card includes a high performance Ethernet switch that powers network transport, aggregation, and service delivery. All services and packet flows within the E7 are managed with native IEEE 802.1Q VLAN tagging and IEEE 802.1ad VLAN stacking (Q-in-Q) support. DELIVERING QOS WITH “QUALITY OF EXPERIENCE”: The E7 provides per-subscriber and per-service hierarchical QOS to deliver uncompromised triple play and business services. A powerful collection of classification, policing, queuing and scheduling algorithms let operators manage per-subscriber and per-service traffic flows to maintain priority/delay/loss differentiation within the E7 network. SCALABLE IPTV SUPPORT: IPTV services are by far the most demanding in terms of quality, and user expectations are very high. The E7 supports industry standard IGMP snooping to identify and replicate multicast video sent between the set-top box and the video distribution network, providing efficient, scalable, high-quality IPTV distribution on both GPON and Ethernet interfaces INTEGRATED HIGH-CAPACITY AGGREGATION: The E7 is built on a core Layer 2 and Layer 3 switch capable of full-duplex, wire speed forwarding at all frame sizes and traffic types across all interfaces. This capacity makes the E7 ideal for aggregation and transport of IP/Ethernet services across the access network. PLUGGABLE OPTICS: The E7 platform supports industry standard pluggable modules for all service and network interfaces, including ITU G.984 compliant GPON, Small Form-Factor Pluggable (SFP) Gigabit Ethernet, XFP 10GE ports, and SFP+ 10GE ports. NETWORK RESILIENCY: The Calix E7 supports a flexible set of network topology protocols for use in aggregation, ring-based transport, and uplink applications. • ITU G.8032 Ethernet Ring Protection Switching (ERPS) • IEEE 802.1w Rapid Spanning Tree Protocol (RSTP) • IEEE 802.3ad/802.1AX Link Aggregation

SERVICE AWARE MANAGEMENT: The E7, along with the Calix Management System (CMS), allows operators to manage services while understanding their relationship to the network infrastructure. Service- oriented management includes rapid service provisioning, service templates and policies, and service assurance. Comprehensive network management tools let operators create physical and logical topology maps, engineer traffic flows, and manage network commissioning and software upgrades. Network inventory, alarm surveillance and PM collection are enabled by the E7 element management systems. The E7 provides equivalent, locally hosted Web GUI and CLI interfaces.

Calix Management System (CMS) Unified Access Management

Figure 3: Calix CMS service aware management of E7 networks

CALIX UNIFIED ACCESS INFRASTRUCTURE The E7 has been designed as a complementary extension to the Calix product portfolio and is managed by the Calix Management System (CMS). This includes full interoperability testing with the Calix broadband portfolio, which includes the C7 multiservice access platform (MSAP), the E5-400 Ethernet transport and E5-100 Ethernet service platforms, and the F5 GPON OLT platform.

Figure 4: Calix E7 Plays a Key Role in the Calix Unified Access Infrastructure

The E7 supports a broad portfolio of Calix 700 ONTs, including Single Family Unit (SFU), Small Business Unit (SBU), Multi-Dwelling Unit (MDU), and rack-mount models. Calix 700GX ONTs support both GPON and GE network interfaces.

SUBSCRIBER SERVICES

Calix E7 delivers a full spectrum of ONT-based IP access services over GPON and Point-to-Point Ethernet optical distribution networks. • IPTV – broadcast and Video on Demand (VoD) • MEF compliant business services • High-Speed Internet (HSI) access • Voice – Native SIP/VOIP and TDM Gateway support • T1 services • CATV video: RF video overlay with RF return

RESIDENTIAL TRIPLE PLAY SERVICES

Voice The E7 supports switched voice services from the Calix 700 / 700G / 700GX ONTs. An Integrated Access Device (IAD) function is embedded in the ONT and interfaces to the GR-909 metallic POTS ports on the ONT. The ONT performs the call handling and IP packetization required for VOIP implementation. The integration of VOIP at the ONT uses Session Initiation Protocol (SIP) as specified in RFC-3261, with supplemental RFCs related to SIP employed on the ONT for performance enhancements. In addition to the GPON OLT function, the E7 provides QOS for voice traffic throughout the Ethernet transport network and uplink interfaces. ONT VOIP traffic is tagged with an 802.1Q VLAN tag and prioritized using 802.1p priority bits. E7-based voice services are delivered in conjunction with either or both of the following: • Calix C7 TDM switch interfaces (GR-303, TR-08 Mode II) using a Voice over IP Resource (VIPR) card as the SIP server and VOIP to TDM gateway. The Calix C7 R6.1 VOIP Services Guide provides a complete service description and provisioning instructions.

• Third party softswitches from BroadSoft, MetaSwitch, Nortel, and others. Calix conducts softswitch interoperability testing through its engineering organization and design verification testing (DVT) process. Calix CAB-08-016 includes softswitches test results and is continuously updated as new switches and ONTs are added to the Calix VOIP portfolio.

The ONT SIP solution supports the standard voice service offerings of a Class V switch, so the ONT VOIP implementation is transparent to the subscriber. These features include Caller-ID, Call Waiting, Call Transfer, Three-Way Calling, Distinctive Ring, Ring Splash, and E911 support. The suite of supported CLASS features depends on the type of SIP server (softswitch or C7 TDM gateway) that the ONT operates with because feature support varies by vendor. Consult the above referenced documents for a complete set of ONT voice service attributes.

IPTV Video The E7 supports both broadcast IPTV and Video on Demand (VOD) video services. On Demand video streams are simply unicast traffic that is no different than any other data service being delivered by the E7. VOD streams are merged with multicast IPTV at the ONT Ethernet interface or the residential gateway/set top box.

The E7 supports broadcast IPTV service using IGMPv2/v3 snooping (RFC 4541). The E7 sits between a Layer 3 IGMP-enabled multicast router and the ONT and optimizes the channel bandwidth and IGMP join/leave requests to provide a high quality video experience with minimal network resources. IGMP snooping occurs internally to the E7 and Calix ONT and is not a networking protocol. The following describes the flow of IPTV video through the E7 GPON and Ethernet network: • Within the E7, only a single copy of a broadcast channel watched by multiple subscribers is placed on the PON. The ONTs of users watching the channel will pick up the channel from the PON. Equivalent performance is provided for ONTs when using a point-to-point GE link to the E7.

• If multiple users on the same ONT are watching the same stream, the ONT provides the channel replication function.

The E7 supports up to 800 IGMP multicast groups, enabling up to 800 independent broadcast video and music channels.

High-Speed Internet Access The E7 and 700 ONTs provide data interconnection between Internet service providers and subscribers. The method of transport over the PON is GEM (Ethernet) format. Service profiles in the E7 and ONT are used to provide tiered service offerings. Calix 700 ONTs provide both Fast Ethernet (100 Mbps) and Gigabit Ethernet (1000 Mbps) ports.

BUSINESS SERVICES

MEF Data services The E7 provides flexible, high-capacity Ethernet business services in addition to traditional residential voice, video, and data. Metro Ethernet Forum (MEF) compliant services (MEF 9 and 14) are supported over GPON and point to point Ethernet. Calix 700 ONTs provide both Fast Ethernet (100 Mbps) and Gigabit Ethernet (1000 Mbps) ports. All E7 cards supports up to 4,092 Ethernet Virtual Connections (EVCs) and can be configured for E-Line (point- to-point) and E-LAN (multipoint-to-multipoint) services over both GPON and GE networks.

T1 Service The E7 and Calix 766 ONT provide T1 services over GPON and point-to-point Ethernet networks using pseudowire emulation technology. IETF RFC 3985 and RFC 4197 define the pseudowire emulation edge to edge (PWE3) architecture and provide a standards-based approach to TDM / T1 service delivery over a packet switched network.

Figure 5: Pseudowire Emulation Edge to Edge (PWE3)

Calix 700 ONTs are available in wall mounted and rack mounted versions. Integrated T1 ports provide: • Pulse amplitude: 3V base to peak • Line coding: AMI, B8ZS • Frame formats: Unframed • Line build-out: 0 to 660 feet (0 to 201.2 m)

CATV SERVICES

RF Video Delivery The E7 system and GPON-4 cards allow overlay components to add RF video functionality to the PON. The RF signals do not traverse the E7, and therefore do not consume system resources. The basic components of the E7 GPON architecture with RF overlay are shown in Figure 6. At the headend, video signals are analog modulated or digitally encoded into RF signals. These signals are combined and sent to a transmitter and optically modulated onto a 1550 nm wavelength. An Erbium-doped fiber amplifier (EDFA) takes in the optical signal from the transmitter and amplifies it to the necessary power level.

Figure 6: One-way RF Video Overlay

The optical signals are combined with the GPON OLT 1490 nm forward path and 1310 nm reverse path wavelengths using Coarse Wave Division Multiplexer (CWDM) passive optics and transported over a single strand of fiber into a PON. Each network is then split using passive optical splitters into serving nodes of 16, 32 or 64 subscribers, with feeder fibers connecting an ONT at each customer site. At the ONT, the optical interface converts the 1550 nm signal back to RF and forwards the signal to the coaxial output via an integrated amplifier. The ONT operates as the network interface device (NID) connecting the service provider network with the in-home wiring. To the subscriber, this service offering looks exactly like standard cable TV does today. For the service provider, the RF video over PON system costs less to maintain due to elimination of HFC active components and noise immunity of fiber.

RF Return The E7 can be used with Calix 700 ONTs featuring integrated RF Return supporting a 5 to 42 MHz return path frequency band. The ONTs provide wide-band transport of return path signals from a set-top box, consumer electronics device or cable modem. Because the E7 and ONT do not sample or process the return path signals, the solution is transparent to the signals or protocols being sent. The system supports the SCTE 55-1 standard used by Motorola, the SCTE 55-2 standard used by Scientific Atlanta, and the DOCSIS/DSG standard. The design of the system uses a similar topology and identical set-top box hardware for interactive services as current cable operators’ HFC technology and practices. The product is unique in that it allows a single ONT for transport of both the forward and reverse RF paths, incorporates power management per FSAN power shedding standards, and can be operated without the GPON data link as a PON RF node for video and interactive set-top box services.

Figure 7: Two-way RF Video Overlay with RF Return

The Calix RF Return solution uses the 1610 nm wavelength as the return path from the ONT to the headend. The ONT generates the optical signal and multiplexes the wavelength with the PON signals. At the headend the 1610 nm wavelength is separated using the 4-wavelength CWDM module. The 1610 nm signal is terminated on a standard high-gain HFC reverse path optical receiver which demodulates it into an electrical RF signal, amplifies and sets the signal level. At this stage the signal may be combined with other return path signals depending on the aggregate number of set-top boxes to be supported. These combined signals are fed into the RF return path demodulators located within the headend and converted back into return path messages. Refer to CAB-07-025, RF Return System Features and Planning Guidelines, for additional detail regarding RF video overlay and RF return network characteristics and planning.

E7 PLATFORM DETAILS

The Calix E7 is a pure Ethernet platform, utilizing the latest wire speed Ethernet switching fabric, hardened for Outside Plant (OSP) field deployments. It is based on a layer 2 VLAN/MAC switching architecture, but has been designed to support robust Layer 3 IP forwarding, enabling service aware security and deployment scalability. This architecture allows rapid deployment of the E7 platform for GPON and GE FTTP as well as Ethernet aggregation, transport or business service delivery with any available 10GE, NxGE or GE interfaces.

SYSTEM COMPONENTS An E7 system consists of the E7 2-slot chassis, a removable fan tray assembly (FTA) and up to two line cards that are inserted into the universal card slots. All line cards and the FTA are installed from the front of the E7 shelf, where all fiber connections are also terminated. The removable fan tray and line cards are all secured in the E7 shelf by ejector or slide latches with positive retention force locking mechanisms.

Figure 8: Front View of E7 Chassis with Two Line Cards

Power, BITS timing, alarm I/O, network management, and copper connectors (RJ-21) are located on the rear of the E7 shelf. A clear cover protects the power connectors.

Figure 9: Rear View of E7 with Power, Alarm, Management, Timing and RJ-21 Copper Connectors

E7 LINE CARDS

Subscriber and Network Ports The E7 subscriber and network port lineup is determined by the selection of line cards placed in the E7’s two universal card slots.

E7 GPON GE SFP 10GE XFP 10GE SFP+ Card Ports Ports Ports Ports 10GE-4 0 12 2 2 GPON-4 4 822 GE-12 0 12 0 2

Table 1: E7 line card port type and count

Pluggable Optics The E7 platform supports industry standard pluggable modules for all service and network interfaces, including ITU G.984 compliant GPON, SFP GE, XFP 10GE ports, and SFP+ 10GE ports. SFP ...... 1 GE optical and copper Small Form-factor Pluggable (SFP) modules SFP+ ...... 10GE optical and copper Small Form-factor Pluggable (SFP+) modules GE SFP modules may also be used in SFP+ ports at a 1Gpbs rate XFP ...... 10GE optical 10 Gigabit Small Form-factor Pluggable (XFP) modules GPON OIM ...... 2.5Gbps GPON (Class B+ ODN with minimum 28dB link budget, up to 1:64 splits) ER-GPON OIM ...... 2.5Gbps Extended Reach GPON (up to 40 km with 1:8 split)

NOTE: Calix GPON OIM, 10GE SFP+ and 10GE XFP modules have been qualified to provide high performance and reliability and must be purchased from Calix. The E7 GE-12 line card supports a selection of GE modules in the SFP sockets that are matched to the Calix 700GX ONT and must also be purchased from Calix. GE SFP modules used in the GE-12’s SFP+ sockets are not restricted.

Card Status Indicators Each E7 line card has three LED status indicators. 1. Fault (FAIL) – a solid RED LED indicates a fault has occurred in the card that should be addressed 2. Control (CTRL) – a solid GREEN LED indicates that the card is the active E7 Shelf Controller; a solid AMBER LED indicates that the card is the standby E7 Shelf Controller 3. Service (SRVC) – a solid GREEN LED indicates that at least one port has been enabled on the E7 line card and is capable of carrying service. Service may be interrupted if the card is removed from the E7 shelf while the service LED remains lit.

Port Status Indicators Each GPON Optical Interface Module (OIM), GE SFP, 10GE SFP+, and 10GE XFP module sockets have combined link status/activity LED below the socket. 1. GREEN LED (Ethernet ports) is used to indicate Ethernet Link Status / Activity. The link LED will be solid green when a link has been established. The green LED will then blink at a variable speed to indicate traffic load. 2. GREEN LED (GPON OLT ports) will blink at a steady cadence when the first ONT is ranging on a GPON port. After the first ONT is in service, the LED will remain lit while at least one ONT is in service. 3. RED LED is used to indicate a module has been inserted into a socket which is directed to the backplane for connectivity between two line cards. The LED will be turned off when a Technician removes the module or reconfigures the port. Only the second XFP port (X2) and the second SFP+ port (X4) can be directed to the backplane. 4. When a valid module is inserted in a valid socket, the LED will blink three times GREEN to indicate that the inserted module is recognized and allowed to operate in the specific socket. 5. When an invalid module is inserted, the port LED will remain off. An alarm is posted against a port which has an invalid module. Examples of an invalid module include a 10GE SFP+ module inserted in an SFP or GPON socket or a non-Calix 10GE module inserted in either the XFP or SFP+ sockets.

FAN TRAY ASSEMBLY (FTA) The E7 FTA resilient design includes 4 individual variable speed fans that maintain system cooling even with one fan failure. Airflow is from right to left (toward the line cards).

Figure 10: Calix E7 Fan Tray Assembly Front Panel

Each FTA is hot-swappable and can be quickly replaced by unlocking the sliding latch, removing the failed unit and sliding in the new FTA. Each FTA includes a field replaceable fan filter that slides in from the top of the FTA once it is removed from the E7 shelf.

NOTE: To increase airflow within the E7, the FTA fan filter should be removed from the FTA when the E7 is operated in a remote terminal that includes integrated cabinet filters.

SYSTEM MANAGEMENT

Management Interfaces The E7 local user interfaces, along with the Calix Management System (CMS), allow operators to manage services while understanding their relationship to the network infrastructure. The E7 provides equivalent, locally hosted, Web-based Graphical User interface (GUI) and Command Line Interface (CLI) interfaces. User interfaces and external management systems access the E7 CPU over one of four management interfaces.

MGT-1 10/100 Base-T Ethernet (RJ-45) on E7 FTA front panel typically used for local craft access to management user interfaces

MGT-2 In-band management interface configurable within any VLAN on any E7 Ethernet interface

MGT-3 10/100 Base-T Ethernet (RJ-45) on back of E7 shelf typically used for permanent network connection to back office management systems

MGT-4 RS-232 serial: 38.4 kbps serial interface on FTA front panel typically used for local craft management

The two Ethernet interfaces (MGT-1 and MGT-3) and in-band VLAN interfaces (MGT-2) provide equivalent access to all user interfaces. The serial interface (MGT-4) supports the Calix CLI only.

System Status Indicators The E7 system includes multiple visual status indicators located on the front of the E7 Fan Tray Assembly. The FTA status indicators will remain dark until at least one card is inserted in the E7 shelf. 1. Critical (CR) Alarm – RED LED indicates a critical alarm is present in the system 2. Major (MJ) Alarm – RED LED indicates a major alarm is present in the system 3. Minor (MN) Alarm – AMBER LED indicates a minor alarm is present in the system 4. System Controller (MGT) – GREEN LED indicates E7 shelf has an active shelf controller 5. 7-segment LCD display – not used in R1.0 An Alarm Cut-Off (ACO) button is also integrated on the front on the FTA.

E7 2-SLOT CHASSIS The E7 rear panel features two power connectors, frame ground lugs, Ethernet management ports, the Alarm Interface Module (AIM), BIT timing interface, and copper subscriber interfaces. Rack Mounting Options The E7 is designed to be rack mounted in either 19” or 23” equipment racks. Figure 11 shows both a 19” and a 23” rack mount bracket, which ship with the E7. The service provider should choose the bracket that works best for their installation requirements. In addition, the E7 has several mounting holes along the side of the unit allowing it to be flush or mid mounted. This is most important in cabinet retrofit applications, where airflow may be blocked if the unit is mounted in the forward position.

Figure 11: Calix E7 Mounting Brackets

In some 19” racks, it may be necessary to flip the 19” mounting brackets and protrusion mount the E7 if mid- mount is not an option.

Power and Ground Connections The E7 supports two redundant DC power inputs, located on the back of the unit, which can be connected to A and B power sources for automatic switchover. Equipment ground is also located on the back of the unit. The E7 supports an input power range of -42.5 VDC to -72 VDC. The following diagram shows the location of the power and ground connections on the rear panel.

Figure 12: Rear View of E7 with Power and Ground Connectors

Alarm I/O The E7 supports 7 input alarms and 1 output alarm, located on the back of the unit as wire wrap pins.

Figure 13: Calix E7 Alarm Input / Output Interfaces

The alarm interface pins are mapped as follows:

OUTPUT INPUT OUT+ AL1+ AL2+ AL3+ AL4+ AL5+ AL6+ AL7+ OUT- AL1- AL2- AL3- AL4- AL5- AL6- AL7-

Table 2: Alarm input / output pin mapping

Alarm input (AL1+/- through AL7+/-): pins detect contact closure only and will raise an alarm. Alarm output (OUT+/-): normally open, this pair of pins can be configured to interface with various alarm management devices such as lights or horns located in the central office.

BITS Timing The E7 can be synchronized to a local traceable clock using the BITS timing wire warp pins located on the back of the E7 shelf. Traceable clock synchronization is recommended for all GPON applications.

Figure 14: E7 BITS External Clock Input / Output Pin Field

The BIT pins are mapped as follows:

BITS Timing In (A) SHLD TIP+ RNG- In (B) SHLD TIP+ RNG- TIP+ RNG- TIP+ RNG- Out (A) Out (B)

Table 3: External BITS Timing Input / Output Pin Mapping

Up to ten E7 shelves can be connected together to share a local BITS input. A BITS Chaining Cable available from Calix includes connectors to attach the redundant BITS output pins from one E7 shelf to the redundant BITS input pins on a second E7 shelf.

Copper Connectors The E7 shelf includes two standard 25-pair RJ-21 connectors per slot. These connectors may be used by any card installed in the E7 system.

POWER CONSUMPTION AND HEAT DISSIPATION The power consumption and heat dissipation of an E7 system are determined by the card lineup and the location of the E7 system. In a typical central office (CO) environment, the E7 fans run at low speed and draw less power than in a remote terminal (RT) location where temperatures may peak at 65° C.

E7 System Power Heat Component Draw Dissipation (Watts) (Watts) 10GE-4 55 55 GPON-4 75 75 GE-12 50 50 FTA (CO nominal) 22 6 FTA (RT max) 65 18

Table 4: Power consumption and heat dissipation for E7 components

The following tables provide nominal power consumption and heat dissipation values for E7 system configurations deployed in typical CO environments and RT with maximum temperature.

E7 Shelf E7 Shelf Typical CO Typical CO RT Maximum RT Maximum Slot 1 Slot 2 Power Heat Power Heat Draw Dissipation Draw Dissipation (Watts) (Watts) (Watts) (Watts) 10GE-4 Blank 77 61 120 73 GPON-4 Blank 97 81 140 93 GE-12 Blank 72 56 115 68 10GE-4 10GE-4 132 116 175 128 GPON-4 GPON-4 172 156 215 168 GE-12 GE-12 122 106 165 118 10GE-4 GPON-4 152 136 195 148 10GE-4 GE-12 127 111 170 123 GPON-4 GE-12 147 131 190 143

Table 5: Power consumption and heat dissipation for E7 system configurations in CO and RT locations

E7 DUAL LINE CARD OPERATION

E7 line cards are designed to operate at full port count and capability when deployed in a “single-card” E7 system, and also operate as a protected pair when deployed in a dual-card E7 system. In an E7 system one or two 10GE ports on each card must be dedicated to inter-card communication when used in dual-card configurations. Within the E7 shelf, inter-card communication takes place over backplane traces, which are capable of 100Gbps. When two cards are present in an E7 shelf at least one 10GE port on each card must be directed towards a backplane link in order for the two cards to switch traffic between cards.

BACKPLANE PORT CONFIGURATION The selection of backplane ports within the E7 system is configurable, within a range of options, and a single user action affects the backplane ports on both cards, ensuring that the settings are compatible. Users can choose between “First Ports”, “Second Ports”, “Both Ports” or “No Ports”. The system default is “First Ports”.

First Second Backplane Backplane Ports Ports

10GE-4 XFP1 XFP2 SFP+1 SFP+2

GPON-4 XFP1 XFP2 SFP+1 SFP+2

GE-12 SFP+1 SFP+2

Default Optional backplane backplane connection connection

Figure 15: Backplane Port Options on E7 Line Cards

For the 10GE-4 and GPON-4 cards, the “First Ports” value is the rightmost 10GE XFP port (X2 within the user interface). The “Second Ports” value is the rightmost 10GE SFP+ port (X4 within the user interface). “Both Ports” would assign both X2 and X4 on both cards to the Backplane interface, establishing a 20Gbps connection between line cards. “No Ports” would set both X2 and X4 ports on both cards to be “forward facing” user ports, with no backplane connectivity. For a GE-12 card, which has no 10GE XFP ports, the “First Ports” connection can be established without redirecting any ports to the backplane. That is, a GE-12 card always has one 10GE backplane port available for use. The “Second Ports” value is the rightmost 10GE SFP+ port (X4 within the user interface). “Both Ports” and “No Ports” have similar operation as the 10GE-4 and GPON-4 cards.

The following table shows the aggregate number of ports available in dual card E7 systems with default “First Ports” uses as backplane ports, also the optional configurations shown in parenthesis.

E7 Shelf E7 Shelf GPON GE SFP 10GE XFP 10GE SFP+ Slot 1 Slot 2 Ports Ports Ports Ports 10GE-4 Blank 0 12 2 2 GPON-4 Blank 4822 GE-12 Blank 0 12 0 2 10GE-4 10GE-4 0 24 2 (or 4) 4 (or 2) GPON-4 GPON-4 8 16 2 (or 4) 4 (or 2) GE-12 GE-12 0 24 0 4 (or 2) 10GE-4 GPON-4 4 20 2 (or 4) 4 (or 2) 10GE-4 GE-12 0 24 1 (or 2) 4 (or 2) GPON-4 GE-12 4 20 1 (or 2) 4 (or 2)

Table 6: E7 System Port Capacity with Dual Card Configurations

When a user modifies the system backplane port configuration of an E7, the system will automatically update the appropriate 10GE Ethernet ports on both line cards by changing their configuration to/from the system backplane interface. When a new E7 card object is created (either by user action or card arrival) its 10GE ports are created with interface configurations appropriate to the current value of the system backplane port configuration. All VLANs created in the system are available for membership/association on all other interfaces and are automatically switched across the backplane connections. A user may create an ERPS/EAPS domain spanning two E7 cards; that is, with one interface on card 1 and the other interface on card 2. Internally, the ERPS/EAPS domain will span across the two line cards through the backplane connection(s). The E7 subscriber and network port lineup is determined by the selection of line cards placed in the E7’s two universal card slots. A 1 rack unit (RU) E7 can be configured as shown indicated below.

BACKPLANE PORT USE CASES The following use cases illustrate how backplane configuration works for various scenarios. The 10GE port numbers in these examples presume the cards are 10GE-4 or GPON-4 cards. For GE-12 cards the port numbering would be different as described above. The use cases considered are: 1. Default for single card in E7 shelf 2. Send all 10GE ports to front (single card system) 3. Adding a second card to E7 shelf 4. Changing which ports are backplane ports 5. Using 2 backplane ports with 20Gbps bandwidth 6. EAPS domain spanning across two E7 cards 7. Mis-configuration during commission of 2nd card

Case 1: Default for single card in E7 shelf When a new E7 is commissioned and a default database is created, the backplane port configuration will default to “First Ports”. When an E7 card object is created in the user interface, port X2 will be configured as a backplane port. The remaining 3 10GE ports will be available for association with an interface or link aggregation group. A user may not change port X2’s interface association directly, as validation will prevent this based on conflict with the E7 system-level provisioning.

Case 2: Send all 10GE ports to front (single card system) A desired a user may change the system’s backplane port configuration to “No Ports”. This change causes the interface association of port X2 to be updated to the matching Ethernet interface. At this point, no ports are configured as backplane ports. The user may then add VLAN associations to Ethernet interface X2 to add services to this port.

Case 3: Adding a second card to E7 shelf The user may wish to add a second card to the E7 shelf. If the E7 system backplane port configuration has been configured (or remains at the default configuration) to “First Ports”, then the second card’s X2 port will default to a backplane port. At this point the two cards are connected by the backplane trace. As the user provisions services on the two cards, the system will determine which VLANs are common to both cards and switch these VLANs across the backplane. If the user had changed the backplane port configuration to “No Ports” before adding the second card, then no 10GE ports on the second card would be assigned to the backplane. In this case the E7 system would raise an alarm indicating that there is no data path between the cards.

Case 4: Changing which ports are backplane ports The user may change which ports are acting as backplane ports, by changing the system backplane port configuration, even after service provisioning is in place. The following are the possible transitions: A. “First Ports” to “Second Ports” In this case, the user is changing the Backplane port on each card from X2 to X4. The X2 port on each card is re-set to forward facing Ethernet interface X2. The X4 port on each card will be configured as a backplane port. If the X2 port had previously been associated to its Ethernet interface, any services that had been provisioned against the Ethernet interface remain, but are disabled. If the X2 port had previously been associated to a LAG Interface, any services provisioned against that LAG interface may be degraded (due to reduced bandwidth) or disabled. B. “Second Ports” to “First Ports” This case is similar to the previous case. With this transition the X4 ports are returned to their corresponding Ethernet interfaces and the X2 ports are configured as backplane ports. C. From Any Setting to “No Ports” In this case the user is reassigning all 10GE ports to front facing, with no ports connected to the backplane traces. If more than one card is provisioned, the E7 system will raise an alarm indicating No Data Path between the cards. Any ports which were configured as backplane ports are re-assigned to their corresponding Ethernet interface. D. “No Ports” to Any Other Setting In the case the user is re-configuring one or both ports from front facing to backplane ports. As with the earlier cases, any services on the X2 or X4 interfaces to which the ports were previously assigned may be left stranded. If the No Data Path alarm had been raised in the system, it is cleared by this change.

E. From Any Setting to “Both Ports” In this case the user is configuring both available 10GE ports on each card to backplane ports. As with the earlier cases, any services on the interfaces to which the ports were previously assigned may be left stranded. F. From “Both Ports” to “First Ports” or “Second Ports” In this case the user is re-configuring one of the two backplane ports on each card to front facing. No alarm is raised, as the system continues to have a backplane data path between the cards. The ports which are no longer configured as backplane ports are re-configured to their Ethernet interfaces.

Case 5: Using 2 backplane ports with 20Gbps bandwidth When two ports on a card are directed to the backplane we must implement Link Aggregation (LAG) between the ports to avoid routing loop issues. Since we are modeling the Backplane Interface using the LAG-Interface construct in the object model, the user need take no special action to enable Link Aggregation. Simply directing “Both Ports” to the backplane will be sufficient. As noted above, this option may be disabled for the first release.

Case 6: EAPS domain spanning across two E7 cards The system will support an EAPS domain spanning the backplane link. This is configured by the user as a single EAPS domain with one Interface on each card. Any VLANs associated to the EAPS domain will be carried on the Backplane link. Internally, the EAPS provisioning will be sent to the SIT on each card, which will configure EAPS between the specified card local Interface of the EAPS provisioning, and the Backplane Interface. In this fashion there will be two functional EAPS domains (one on each card) acting as a single EAPS domain across the pair of cards. Any VLANS associated to the EAPS domain will automatically be carried across the Backplane Interface on each card. However any VLANs not associated with the EAPS domain and which have a presence on both cards will need to be added to the Backplane Interface just as if there were no EAPS involved.

Case 7: Mis-configuration during commission of 2nd card The system will raise a system alarm (i.e. “No Data Path”) when a system has a backplane port configuration of “No Ports” and a second card is provisioned, either by user action or by card arrival. The presence of this alarm will not prevent provisioning of services on the shelf, since the user may wish to move services from a 10GE port on the first card onto a port on the second card before switching the first card’s port to the backplane. This alarm will remain until the user changes the backplane port configuration setting to another value, or until the provisioning for the second card is deleted.

E7 ETHERNET FEATURES

The E7 cards share a common set of Ethernet, quality of service (QOS), and network topology features that allow the cards to share E7 systems with expansive and consistent behavior. The Ethernet VLAN, traffic management, and network topology feature descriptions in that follow apply to the E7 cards’ GE and 10GE Ethernet interfaces specifically. Ethernet features on GPON OLT and ONT interfaces are described in the GPON Features section of this planning guide.

ETHERNET INTERFACES Each GE and 10GE port, as well as link aggregation groups, is mapped to an Ethernet interface within the E7. An interface is a logical construct to which VLANs membership and tag actions are assigned, QOS is applied, and topology protocols are enabled. Two interface roles are defined to bundle common configuration options and improve usability of the system. Trunk Interface – An interface typically connecting to other network equipment belonging to the service provider or to another service domain with consistent VLAN tagging definitions. Trunk interfaces may also be referred to as Network interfaces or Provider interfaces in common industry usage. These interfaces support outer VLAN tag plus MAC switching. Examples of a Trunk interface role are 10G ERPS/EAPS transport, GE uplinks, etc. Trunk interfaces can be configured for link aggregation, RSTP or ERPS/EAPS. Edge Interface – An interface typically facing untrusted customer equipment or facing reduced functionality devices, alternative administrative domains, or managed CPE. Generally, this interfaces is expected to be the interface on which all DOS/Security/Classification is first performed on ingress traffic (if customer facing). It also is expected to add/replace/remove one or more VLAN tags on edge traffic. Examples of an Edge interface role would include GE interfaces to managed CPE or a GE/10GE interfaces to external equipment which may use different VLAN tagging levels. ONT Ethernet ports are also Edge interfaces. Split horizon, a feature that isolates Edge interface traffic from other Edge interfaces within the same E7 card, is enabled by default on E7 Edge interfaces. Edge interfaces can be configured for link aggregation or RSTP.

VLAN SUPPORT The E7 provides standards-based IEEE 802.1Q VLAN tagging, and Q-in-Q VLAN stacking support based on IEEE 802.1ad Bridged VLANs - 802.1Q Amendment 4, Provider Bridges. VLAN tagging was developed as a means to allow multiple networks to transparently traverse the same physical network. This is sometimes called Ethernet trunking or VLAN trunking. In addition, 802.1Q can be used to create transparent LANs, virtual LANs and to separate traffic by VLANs, ensuring end user traffic segregation and network security. When VLANs are provisioned as a unique customer or port identifier, VLAN C-tags (customer tag) are utilized to create a VLAN per customer / port association, which is described in Figure 16. The VLAN tagged frames are identified as having a tag by utilizing the Ethertype field, which is part of a standard Ethernet frame and is part of the Ethernet packet overhead. The Ethertype field is set by default for VLAN tagging (0x8100). This value is a software provisionable value in the E7 since some switches and routers require specific values that don’t follow the standard recommendation for Ethertype designations. In addition to support of IEEE 802.1Q VLAN tagging, the E7 supports IEEE 802.1ad Provider Bridging, which describes the ability to add multiple tags onto an Ethernet frame. This allows for support of multiple provisioning models as described in the Broadband Forum’s TR-101 standard. The ability to add multiple VLAN tags (called VLAN stacking or Q-in-Q) enables the following:

• Expands the addressable VLAN space from 4094 VLANs to well over 16 million VLANs • Allows logical separation and trunking of VLANs through a network by using a VLAN tag to group a larger range of VLAN tags together

The most common way to use VLAN stacking is by inserting two tags on the traffic. These tags are often referred to as the inner tag or C-tag and the outer tag or S-tag. As stated previously, the C-tag, also known as the customer tag as it is used to uniquely identify a customer, typically is used on a per port basis. The S-tag, also known as the service provider tag as it is used to logically group C-tags together, typically is used by the ISP but can be used for any logical grouping. In order to ensure a wide range of access deployments, the E7 was designed with an extremely flexible VLAN support model. In order to enable future development, the E7 reserves 4 VLAN values. The E7 requires these VLANs always be identified. The default value for these VLANs is 1002, 1003, 1004 and 1005. It is possible to change these 4 values to be some other set of 4 consecutive VLANs if required.

VLAN per Port Provisioning Model The VLAN per port provisioning model, also called VLAN per customer, is based on each port of a broadband access platform being mapped into a unique VLAN prior to Ethernet aggregation and transport. This deployment method is very similar operationally to the traditional ATM model, where a layer 2 ATM PVC maps each customer (VCI) on a DSLAM’s ATM uplink (VPI) for transport back to a broadband RAS for protocol conversion and subscriber management.

Figure 16: E7 VLAN per Subscriber (Port) Model

As shown in Figure 16, the E7 supports the VLAN per subscriber (port) provisioning model and it allows the full range of VLANs to arrive on a downlink port and be forwarded to the upstream ports. The diagram above shows the VLAN per port provisioning model, but unless VLAN stacking is utilized, there are only 4094 useable VLANs per the IEEE 802.1Q standard. Thus it can easily be seen that the aggregation capabilities in this scenario are limited, making VLAN stacking essential. The E7 supports the following VLAN capabilities when configured for VLAN per port provisioning: • VLAN switching based on the outer VLAN tag received from the downstream device with no modifications made to the VLAN tags. • Add an outer tag (S-tag) to the VLAN per port traffic, then switch based on the newly added S-tag. • Add an outer tag (S-tag) on all received VLANs on an E5-400 interface regardless of originating subscriber port, then switch based on the newly added outer S-tag. • VLAN tag translation – change VLAN tags to the same value or a new value based on pre-identified criteria. The E7 limits the number of VLANs that can be translated to an aggregate limit of 768. • Add a VLAN tag using the 802.1p bit value to classify the incoming frame. • Set the value of the Ethertype on a per port basis. The default Ethertype value will be set to 0x8100 for the VLAN S-tag. The default value can be changed to be 0x8100, 0x9100 or 0x88a8. The ability to support the provisioning of the Ethertype value enables support for devices that implemented Q-in-Q prior to the standard being fully specified.

VLAN Stacking / Q-in-Q As shown in Figure 17, the E7 allows the combination of the VLAN per subscriber (port) model and the ability to add S-tags to incoming traffic to simplify VLAN transport / switching. Some 3rd party devices may not have the ability to add S-tags before E7 handoff. In these cases, the E7 can add the tag. The tag added to this traffic can be defined to be different based on what port the E7 receives the traffic on or be common for all traffic sent upstream.

Figure 17: E7 VLAN Q-in-Q Single tag Model

As shown in Figure 18, another way VLAN stacking can be used is when the aggregated device has the ability to send traffic with two VLAN tags. In this example, the E7 will only need to switch the traffic to the appropriate port without any modifications to the VLAN tags.

Figure 18: E7 VLAN Q-in-Q Double tag Model

VLAN per Service Provisioning Model As shown in Figure 19, the VLAN per service provisioning model separates traffic onto VLANs based on the type of service carried in the traffic. For example, IPTV video traffic is carried on a separate VLAN from data and voice traffic. This provisioning model is the easiest model to configure and use but does not provide segregation of user’s traffic, which is often desired especially for data traffic to ensure security and tracking of subscriber activity. In this VLAN provisioning model, the VLANs may have an S-tag added to the traffic by the E7 as shown in the diagram. The main reason for the S-tag would be to segregate traffic on the network into smaller groups, which provides added security benefit. For example, if something in a network causes a broadcast storm the amount of traffic that is affected is limited because the network has been logically divided into smaller groups. The E7 supports the following VLAN capabilities when configured for the VLAN per service provisioning: • Addition of an S-tag to traffic on a per drop basis. • Addition of an S-tag to traffic based on the value of the C-tag. • Switching of traffic based on the newly added S-tag. • Switching of traffic based on the S-tag on the traffic received from the downstream port.

Figure 19: E7 VLAN per Service Model

Hybrid VLAN Provisioning Model The E7 allows VLAN per port traffic to be on the same interface as VLAN per service traffic. This configuration is typically used by service providers that want to maintain data traffic security and customer isolation, yet utilize the VLAN per service model for IPTV and VOIP. As a general rule, IPTV networks will place all multicast channels on a single VLAN, thus enabling a simplified provisioning model and enabling interoperability with multiple middleware vendors. In this scenario there might be different S-tag stacking requirements for a single drop based on the type of tag received. VLAN per service aggregation may require different S-tags to be added for different C-tags. In either scenario, the E7 supports the VLAN provisioning model.

MEF Compliant Transparent LAN Services (TLS) The E7 supports flexible transparent LAN service (TLS) architecture to deliver business services. Metro Ethernet Forum (MEF) compliant services (MEF 9 and 14) are supported over GPON and point to point Ethernet. All E7 cards supports up to 4,092 Ethernet Virtual Connections (EVCs) and can be configured for E-Line (point-to- point) and E-LAN (multipoint-to-multipoint) transparent LAN services over both GPON and GE networks. As shown in Figure 20, TLS services can be used to transparently trunk business traffic across a network to other locations, typically a remote offices or secondary business location. The traffic received from the business may be untagged, single tagged or double tagged. The E7 will add an outer tag to all frames to create a private switched LAN with two or more end points.

Figure 20: TLAN Business Service Delivery

When the E7 is utilized for TLS services, it supports the following VLAN capabilities: • Addition of a single VLAN tag to untagged, single tagged or double tagged incoming frames • VLAN/MAC switching of TLS traffic from any designated TLS interface to any other TLS interface • Transparent transport of all TLS traffic • Peer, discard or tunnel of RSTP BPDU and Link Aggregation control plane traffic

NOTE: In E7 R1.0, multiple GPON ONTs within a single PON cannot be part of the same multipoint Transparent LAN Service (TLS). To participate in a multipoint TLS service, GPON ONTs must be on separate GPON ports within the same GPON-4 line card or on different GPON-4 line cards in the same E7 shelf.

ETHERNET TRAFFIC MANAGEMENT The E7 supports the ability to classify traffic based on the P-bit or VLAN values of the ingress traffic. Additionally, the ONT can classify traffic based on the MAC OUI value. The P-bit values are defined by IEEE 802.1p which is a specification used to give layer 2 Ethernet switches the ability to prioritize traffic. The E7 supports standard queuing structure, which defines 8 priority queues (0-7) and specifies that the highest priority queue is 7, with the lowest priority queue being 0. The quality of service scheme will allow service providers to determine the priority of traffic within the network and to properly traffic manage the entire Ethernet network. As part of the classification of traffic the E7 will examine the P-bit or VLAN value depending on the provisioned classifications. The egress queues on the port then deliver the traffic to the network using the strict priority queuing scheme (with minimum COS bandwidth guarantees if desired). Strict priority queuing means that all traffic (in the queues) with the highest priority will be delivered to the network first, then lower priority queue traffic will be delivered.

Traffic Management and Rate Limiting on Ingress In order to ensure proper Ethernet network management, the E7 has been designed to support rate limiting and traffic shaping. The primary application for this functionality is to ensure TLS service level agreements and provide control over the amount of bandwidth utilized for service delivery. This functionality allows the E7 to limit the rate at which a subscriber can send traffic to the network and prevents a negative impact on the network from one high bandwidth user. This also enables the service provider to offer tiered services and gain additional revenue from customers requiring more bandwidth. The E7 uses two traffic control constructs that are well known in the data switching industry and originally developed by Cisco, called the class map and policy map. The class map defines traffic into groups, such as VOIP traffic. The policy map then matches the classes from the class map with information about how much bandwidth or what priority is desired for that grouping of traffic.

Traffic Shaping Traffic shaping ensures that traffic leaving the E7 network does not exceed a particular bandwidth requirement. When traffic is shaped it is sent at a particular rate by controlling the queuing of packets. If the queue overflows, the traffic is dropped at the tail of the queue. The E7 supports the ability to shape traffic on a per class of service (COS) basis with up to 8 COS queues supported. The ability to shape on a per VLAN basis allows for a service provider to use the same physical uplink port for different businesses or residential customers and shape each customer’s traffic to their allotted limit. The benefit of shaping is that the bursting nature of the traffic is reduced and as long as there isn’t a substantially large burst that overflows a queue, traffic is not dropped.

Rate Limiting The E7 also supports rate limiting, which is similar to traffic shaping and is sometimes called traffic policing because it ensures traffic does not exceed a specified bit rate. Rate limiting is used for traffic entering the E7 network from a downstream device. It enables traffic to be controlled at a specified rate and keeps a subscriber from exceeding their data rate contract. Traffic shaping is also used to smooth traffic (not drop); enabling a downstream device to process the traffic without overflowing traffic queues. A side effect of rate limiting is that connection oriented protocols, such as FTP, will send an acknowledge (ACK) to indicate receipt of a packet. In certain scenarios, the ACK can actually be dropped causing a retransmission of the original packet even though it may have been successfully delivered. If FTP is being used to determine the actual throughput of an interface, the results can be distorted by the retransmission of traffic making the throughput look substantially lower than the actual capability of the interface.

ETHERNET TOPOLOGY PROTOCOLS The E7 supports multiple network topology protocols, each with its benefits and reasons for use within the access network. The following sections describe these protocols and the capabilities of the E7 platform with respect to each protocol.

Rapid Spanning Tree Protocol (RSTP) The E7 has been designed to support the IEEE 802.1w Rapid Spanning Tree Protocol (RSTP), which represents the evolution of the IEEE 802.1D Spanning Tree Protocol (STP) to deliver improved network convergence time. If a device connected to the E7 only supports STP, the E7 will fall back to STP, thus ensuring seamless protocol interoperability.

RSTP allows a network design to include redundant network uplink / downlinks and provides a backup path to an Ethernet device if one of the active links fails. This is achieved without having to worry about network loops or manually enabling the backup links. If allowed, bridge loops would allow traffic with the same MAC address to be seen on multiple ports, causing a networked device to not know where to send traffic and leading to a network broadcast storm. Broadcast storms can occur when a large volume of broadcast and multicast packets are forwarded in an endless loop between Ethernet switches if multiple paths exist. Broadcast storms consume CPU resources and can take a network or connected device down. Based on the design of the E7, bridge loops are NOT allowed to occur, thus avoiding this troublesome scenario. When two E7 ports are configured for RSTP, one port will be designated as blocking and the other designated as forwarding (active). This behavior is true whether the ports are on the same E7 card or on different cards within a single E7 shelf. This designation happens automatically as part of the RSTP protocol and does not require special provisioning. When connecting devices to the E7 and using RSTP, the devices should have both of its ports connected to like ports on the E7. For instance, two GE or two 10GE ports should be used, not one GE and one 10GE port.

Figure 21: Industry Standard Platform / Port Protection Enabled with RSTP

The advantage of RSTP as an E7 network redundancy protocol over link aggregation is the ability to protect rings of subtended devices, such as an E7 aggregating multiple E5-100 units in an Calix ODC remote terminal. E7 RSTP does not interoperate with PVSTP (Cisco proprietary STP).

Link Aggregation The E7 uses IEEE 802.3ad Link Aggregation to bond multiple GE or 10GE links into a single link aggregation group with a single logical Ethernet interface. Link aggregation expands the bandwidth of the logical interface beyond the capacity of a single link and also provides link redundancy between devices even if one of the links fails. Ports that are configured for link aggregation are all part of a link aggregation group. The E7 supports the ability to support link aggregation across multiple GE or 10GE ports on the same E7 card. An example of link aggregation is shown in Figure 22.

Figure 22: Link Aggregation Enables Industry Standard Port Protection & Bandwidth Expansion

NOTE: A link aggregation group must be fully contained within a single E7 card. A Link Aggregation interface may NOT contain ports on different cards within the same E7 shelf.

Link aggregation is supported with the following GE and 10GE port configuration limitations: • Up to 6 link aggregation groups are allowed across the 12 GE ports of a 10GE-4 or GE-12 card; four groups on the GPON-4 eight GE ports. • Up to 8 GE ports are allowed in a single link aggregation group • Up to 2 link aggregation groups across the 10GE ports; one group on the GE-12 10GE ports • Up to 4 10GE ports allowed in a single link aggregation group; two 10GE ports on the GE-12

Any device connected to the E7 using link aggregation must be connected to multiple ports on the same E7 card. Therefore, link aggregation does not provide equipment redundancy within the E7, but does providing link failure prevention. Link aggregation is often used with RSTP to provide a combination of equipment redundancy and increased bandwidth. Figure 23 shows how link aggregation and RSTP are used together to provide a redundant, high speed uplink interface using GE interfaces.

Figure 23: Uplink redundancy with RSTP-protected link aggregation interfaces

ITU G.8032 ERPS / Enhanced EAPS The E7 has been standards aligned with the recently approved ITU G.8032 Ethernet Ring Protection Switching (ERPS). The ITU ERPS specification is an enhanced version of Ethernet Automatic Protection Switching (EAPS) defined in the IETF in RFC 3619 (an “Enhanced EAPS” protocol). Other vendors have implemented similar types of ring recovery protocols, based on RFC 3619 or proprietary ring recovery protocols, which are not compatible with the Calix C-Series or E-Series products. Calix has chosen this implementation due to our standards-based focus, and in order to reach full ITU-T compliance with a future software upgrade, pending industry / vendor adoption of the ITU G.8032 ERPS protocol. The initial implementation of G.8032 protocol on the E7 (called Enhanced EAPS) is closely aligned with the ITU specification. ERPS / Enhanced EAPS Planning Guidelines • The E7 ERPS feature is compatible with all other Calix products running ERPS and can be mixed within the ring. • Only a single ERPS ring is supported per E7 system, regardless of whether the system has 1 or 2 cards installed. • For high availability dual-card E7 configurations, the two ERPS ring interfaces should be on separate cards within an E7 shelf. • Up to thirty-two (32) E7 systems are supported in a logical ring. In order to provide E7 system redundancy, two E7 cards can be placed in the E7 shelf. In R1.0, a dual-card E7 configuration counts as two E7 ring nodes. If all locations on a ring are configured for shelf redundancy, the ring will support 16 locations (32 E7 cards in 16 shelves). This configuration and node count matches the E5- 400 capacity and planning guidelines. With ERPS / Enhanced EAPS one node in the ring is designated as the ring master and all other nodes are considered transit nodes. The master node is responsible for protecting the Ethernet ring from network loops as well as coordinating the flushing of bridging tables. A group of VLANs can be assigned to an ERPS / Enhanced

EAPS ring. During normal operation the entire VLAN group is enabled on the primary port and blocked on the secondary port of the ring, as shown in Figure 24.

Figure 24: ERPS / Enhanced EAPS Ring Topology – Normal Operational Model

The ERPS / Enhanced EAPS protocol allows three methods of detecting a ring failure: • Loss of signal from the link which generates a link down message. • Master node detects loss of network / ring heartbeat. • IEEE 802.1ag connectivity check message can generate a link down message.

Figure 25 illustrates the change in traffic flow when a network link failure occurs. When an ERPS / Enhanced EAPS ring is created, a control VLAN is created on each node in the ring, as shown in Figure 26. This control VLAN is used to pass control packets between all the nodes of the ring. The master node uses this control VLAN to send health packets around the ring. Health packets are used to determine if a ring failure has occurred. The control VLAN must be selected by the service provider. Once the control VLAN is provisioned and the ring is operational, a change to the control VLAN will affect service. Per the ITU G.8032 specifications, the health heartbeat is sent around the ring at 1 second intervals and cannot be counted on to cause a ring failover within 50 milliseconds. This method of detecting a ring failure is used as a failsafe if the other methods do not detect the failure. When a node is provisioned as the ring master, it blocks the forwarding of data VLANs on the secondary ring port when the ring is in a normal operational state. The control VLAN is not blocked on the secondary port as it must be received by the master node to verify the health of the ring. However, control VLAN packets received by the node master on the secondary ring port are never forwarded. The master node sends a health PDU onto the primary ring port. The rate at which the health PDU is sent by the master node is software provisionable. The master node then looks for the receipt of the health PDU on the secondary port. If the health PDU is not received in the designated timeframe, a ring failure is declared. When a ring failure occurs, the master node flushes its bridging table for the ring-based VLANs, except the control VLAN. The master node then enables the secondary port for the ring-based VLANs and sends a LINK_DOWN PDU out both the primary and secondary ports, instructing the transit nodes to flush their local bridging tables and begins learning the new topology.

Figure 25: ERPS / Enhanced EAPS Ring Topology – Network Link Failure Mode

When an ERPS / Enhanced EAPS ring is provisioned, the VLANs used to carry traffic are provisioned on each node of the ring. Multiple VLANs can be configured across the same physical ring for further separation of traffic. As shown in Figure 26, two logical VLANs are configured for voice and video traffic. The diagram also illustrates the control VLAN, which is provisioned as VLAN 2 in this example. All nodes on the ring send their traffic in the direction of the master node’s primary port with the exception of the Control traffic on the ring.

Figure 26: ERPS / Enhanced EAPS Ring VLAN Configuration

The E7 is extremely flexible and provides the ability to interconnect with multiple protection protocols to multiple devices at any given node in the ring. As shown in Figure 27, device protection can be accomplished using the E7 redundant card architecture, while network protection is accomplished using ERPS, RSTP or link aggregation.

Figure 27: Supported E7 Link and Node Protection Topologies

Any pair of GE ports or 10GE ports can be configured to be part of an ERPS / Enhanced EAPS ring. The two ports can be on a single E7 line card or one each on the two line cards in a redundant E7 configuration. ERPS rings must be of the same port speed; an ERPS / Enhanced EAPS ring cannot be configured with one GE link (West) and one 10GE link (East) as part of the same ring.

Figure 28: High Availability Configuration using ERPS Transport and RSTP-protected Link Aggregation Uplink

E7 GPON FEATURES

GPON OVERVIEW With a wire speed Ethernet architecture supporting delivery of a full 2.5 Gbps to every GPON OLT port, the Calix E7 GPON-4 card provides limitless capacity to meet the demanding requirements of today’s and future business and residential services. Each E7 GPON-4 card provides 4 GPON OLT ports that subtend up to 64 ONTs each, for a card capacity of 256 ONTs; 512 per 1RU E7 shelf. Multiple E7 systems can be linked together using cost-effective industry standard 10GE SFP+ copper cables resulting in a very high-density, high bandwidth configuration of over 2500 potential subscriber endpoints in a 10RU space (1:32 split).

Calix GPON FTTP Solution The following diagram depicts a high level architectural view of the Calix E7 GPON FTTP solution.

Figure 29: Calix E7 GPON Solution

E7 GPON FTTP solution key attributes • GPON technology in compliance with the ITU G.984 family of standards • 2.5 Gbps (2.488 Gbps) downstream, 1.25 Gbps (1.244 Gbps) upstream GPON interfaces • Class B+ ODN (minimum 28 dB link budget) with a 20Km reach (1:32 split) • Extended reach GPON with a 40Km reach (1:8 split) • Up to 64:1 splits per PON • GEM-based PON for efficient packet switched network services • Integrated 10GE and GE aggregation and transport

• Voice service (including voice specials) • Support of SIP based voice from Calix 700 ONTs direct to softswitch networks • GR-303 and TR-08 Mode II TDM voice switch interfaces via Calix C7 TDM Voice Gateway • Ethernet data services, including High-speed Internet access and MEF-compliant Transparent LAN Service (TLS) • T1 leased line and private line • Multi-stage IGMP multicast switching at the E7 GPON OLT and ONT • CATV RF Video: 1550 nm video overlay and 1610 nm RF return

The E7 is temperature hardened and well suited for deployment in remote terminals. All Calix ODC cabinets support deployment of E7 systems, including optional mounting and cable kits to support an integrated third- party EDFA/combiner for overlay RF video.

Calix ONT Interoperability The E7 operates with Calix 700 ONTs, but is designed to operate with third-party ONTs. Calix is actively involved in the Full Service Access Network (FSAN) Group, which is pursuing GPON interoperability among the world’s leading telecommunications services providers, independent test labs, and equipment suppliers. The E7 is compatible with Calix 700 ONTs (700, 700G, and 700GX), including Single Family Unit (SFU), Small Business Unit (SBU), Multi-Dwelling Unit (MDU), and rack-mount models. ONT T1 services are supported using the 766GX ONT. The 740, 740G, and 765G ONTs support T1 services over TDM GPON and are not interoperable with the E7. 700GX ONTs support auto sensing GPON and GE network interfaces, allowing service providers to manage service changes without subscriber onsite technical support.

10/100/1000 Ethernet

Personality Module

TDM or VOIP

2.5 Gbps GPON or AE RF Video or HPNA

Figure 30: One of Many Calix 700 ONT Options

Optical Distribution Network The Optical Distribution Network (ODN) includes all passive equipment, plus the optional active EDFA/combiner for RF video overlay, between the E7 and the subscriber ONT. It includes fiber, splitters, connectors, EDFA/combiners, and splices. The GPON specification allows up to 64 splits per PON. Splitters can be deployed in the central office, the remote terminal, the outside plant, or any combination of the above. E7 Optical Distribution Network:

• Supports single-fiber Class B+ (28 dB link margin) optical distribution network

• Standard Reach OIM o Up to 64 passive optical splits per PON o Up to 20 km reach with 32 way split o Up to 11 km reach with 64 way split

• Extended Reach OIM o Up to 32 passive optical splits per PON o Up to 40 km reach with 8 way split

• GPON with one-way video overlay - three wavelengths per PON: o GPON digital downstream: 1490 nm o GPON digital upstream 1310 nm o Video overlay downstream: 1550 nm

• GPON with two-way video overlay - four wavelengths per PON: o GPON digital downstream: 1490 nm o GPON digital upstream 1310 nm o Video overlay downstream: 1550 nm o Return path upstream: 1610 nm Please contact your account team for assistance and/or access to the Calix PON engineering tool.

ONT ETHERNET SERVICES All services other than RF video overlay are switched within the ONT and transported over the PON as Ethernet services identified by a VLAN ID. ONT-based data services are managed in a similar fashion to the E7 GE and 10GE Ethernet ports. • Ethernet port independence for multiple subscribers • Per subscriber Ethernet mode configuration, 10/100/1000/auto • Per subscriber Ethernet interface upstream/downstream metering up to 400 Mb/s in each direction • MTU of 1550 Bytes; 1536 Bytes on 760 ONT family

VLAN Service Model Within the E7 GPON OLT and ONT, all services exist within a VLAN-tagged service flow. A “native VLAN” does not exist and therefore all services must be classified and placed within an operator defined service VLAN. The ONT Ethernet ports support the same VLAN services as the E7 Ethernet ports, including native IEEE 802.1Q VLAN tagging and IEEE 802.1ad VLAN stacking (Q-in-Q). The following service models are supported. • VLAN per subscriber Ethernet port (1:1) • VLAN per Service (N:1) • Ethernet virtual circuits (VLAN trunks) • Transparent LAN Services

GPON MAC Learning and VLAN Switching Within the E7 GPON OLT and ONT subsystem, MAC learning and switching occurs within a single, outer VLAN ID, referred to as the VLAN C-tag. Packets on the PON always have a C-tag, which consists of a provisioned VLAN ID and p-bit. The ONT is responsible for classifying subscriber traffic and adding/modifying the C-tag. The GPON OLT is responsible for pushing/popping a second VLAN S-tag (if provisioned). The E7 OLT subsystem adds or removes all S-tags as VLAN traffic is handed off into the GPON-4 card’s core Ethernet switch. • For single tagged VLANs the OLT switches on the outer tag. • For double tagged VLANs the OLT switches on the inner tag. The OLT pushes the outer S-tag on packets going upstream and pops the outer S-tag when packets come downstream.

• Note: “single” vs. “double” tagging refers to the network operator tags. Any single or double tagged packet could have additional subscriber tags. For example, a TLAN subscriber may be using double tags, but the C-tag that gets put on is part of a single (or double) operator tag.

C-tag Uniqueness: C-tag uniqueness is required within the GPON OLT. All four GPON OLT ports within the GPON-4 card share a common VLAN ID space and VLAN-ID values across all four OLT ports must be unique. Non-unique subscriber VLAN tags that have had unique provider C-tags added at the ONT, or have had their C-tag changed at the ONT, are supported. • If the operator wants to run 1:1 VLANs, the C-tag must be unique for each subscriber, and cannot use the same C-tag that would be used to identify a service in the N:1 model. • If the operator wants to run N:1 VLANs, the C-tag must be unique for each service, and cannot use the same C-tag as another N:1 or 1:1 VLAN. • In the double tagged case, this means the inner tags must be unique and can cannot be the same as the outer tag of single tagged operator VLANs. • In the single tagged case, this means the outer tag must be unique and cannot be the same as the inner tag of double tagged operator VLANs.

Classification The E7 provides a flexible classification engine for mapping subscriber traffic into a service VLAN. The ONT is responsible for classifying subscriber traffic on the ONT Ethernet interface. A list of match rules (the Match List) may contain both “Tagged” and “Untagged” match rules. • There is a limit of 12 tagged rules per ONT Ethernet port • There is a limit of 16 untagged rules per ONT Ethernet port Tagged match rules match on any combination of the subscriber’s outer tag VLAN-ID and p-bit values. The Tag Protocol Identifier (TPID) is hard-coded to 0x8100 on the PON, but can be changed at the E7 egress Ethernet interface if required. Untagged match rules can match on a portion of the source MAC address as indicated by the Source MAC and Source MAC Mask attributes. If no Source Mac value is provided the rule matches all untagged traffic. Multiple match rules can be applied to a single interface. In a typical example, an untagged traffic match rule would be applied with a specific MAC OUI match criteria, followed by a second untagged match rule with a “match any” criteria. This combination of rules would allow a service provider to use a single Ethernet port on the ONT to connect to a home network with a combination of IPTV and data traffic. The IPTV traffic is identified using the OUI value of the video set top box.

ONT Tag Action The ONT VLAN tag action is similar in concept to a tag action in the E7. The E7 OLT and ONT together can apply the following tag actions on a GPON ONT Ethernet port. • Add a subscriber tag (C-tag) • Change the subscriber tag (C-tag) • Add an outer tag (S-tag) • Add an outer (S-tag) and inner tag (C-tag)

• Add an outer tag (S-tag) and change the inner tag (C-tag)

The E7 GPON-4 OLT subsystem adds or removes all S-tags as VLAN traffic is handed off into the GPON-4 card’s core Ethernet switch. Within the E7 switch (all cards), MAC learning and switching occurs within the outer VLAN ID. Combined with the classification rules outlined above, the ONT supports the following match/tag rules: • Match Untagged / Add C-tag – Untagged can be classified using any portion of the source MAC address (e.g. the OUI). Downstream, this rule removes the tag. • Match Any VLAN-Tagged / Add C-tag – The match rule here ignores VLAN ID and p-bit, i.e., any tagged packet. Downstream, this rule removes the tag. • Match Tagged VLAN ID (any pbit) / Add C-tag – The match rule looks for a specific VLAN ID but ignores p-bit. Downstream, this rule removes the tag. • Match Tagged VLAN ID (any pbit) / Change C-tag – The match rule looks for a specific VLAN ID but ignores p-bit. Downstream, this rule changes the VLAN ID to the original (p-bit not modified). • Match Tagged VLAN ID +Pbit / Add C-tag – The match rule looks for a specific VLAN ID and a specifc p-bit. Downstream, this rule removes the tag. • Match Tagged VLAN ID +Pbit / Change C-tag – The match rule looks for a specific VLAN ID and a specifc p-bit. Downstream, this rule changes the VLAN ID to the original (p-bit not modified). • Match Tagged pbit (any VID) / Add C-tag – The match rule looks for a specifc p-bit value and ignores the VLAN ID. Downstream, this rule removes the tag.

Bandwidth Policing and Shaping A bandwidth profile is used to specify upstream and downstream bandwidth rates that can be applied to individual service members. It is expected that a single bandwidth profile will be applied to many subscribers. Each direction of a traffic shaper can be programmed to provide a specific rate for the shaper. Two values are specified to designate the rate. 1. Committed Information Rate (CIR). The CIR value is the minimum guaranteed rate the ONT will limit flows for the particular direction of the shaper. CIR only applies in the upstream direction. 2. Peak Information Rate (PIR): The PIR value is the maximum bandwidth rate the ONT will allow the service to use if bandwidth is unused or available. PIR should always be greater than or equal to CIR and applies in the upstream and downstream directions. The provisionable limits for each bandwidth rate are as follows: • The minimum metered rate is 64 Kbps • The maximum metered rate is 400 Mbps • Rates can be selected in 64 Kbps increments up to 2 Mbps • Rates can be selected in 1 Mbps above 2 Mbps • Setting the metered rate to 0 Kbps disables the meter

GPON Class of Service (CoS) The E7 GPON supports four Classes of Service over the PON. At the OLT and ONT, individual services with IEEE 802.1p priority bit settings are mapped to the four GPON CoS queues. This mapping applies to all services carried on the GEM-based PON as all services are either native IP services or transported in an IP packet as in the case of Voice or T1 services.

Figure 31: GPON OLT Class of Service Queues

Each ONT service member maps to a particular class of service on the PON based on its IEEE 802.1p priority bit settings. The CIR and PIR values for each service must be consistent with the class of service type as defined in the global GPON CoS table. For example, if a particular service is of type best effort, then the CIR component must be 0. There are three class of service types on the PON, each having specific bandwidth profile characteristics: • Best Effort: CIR = 0, PIR > 0 • Assured Forwarding: CIR > 0, PIR > 0 (and PIR > CIR) • Expedited: CIR > 0, PIR = 0 Bandwidth profiles are referenced by ONT service member. The tag action associated with the service member identifies the p-bit which in turn identifies the class of service (according to the class of service configuration on the PON). The default mapping of GPON CoS to 802.1p bits and CoS type is:

GPON CoS Scheduling Class Type Default P-Bits Use Cos 4 Expedited Forwarding 5, 6, 7 Network control, voice, T1 Cos 3 Assured Forwarding 4 Video Cos 2 Assured Forwarding 3 App signaling, TLAN Cos 1 Best Effort Forwarding 0, 1, 2 Other

Table 7: Default GPON CoS mapping to IEEE 802.1p bits

GPON CoS Planning Guidelines • GPON CoS is a system-level configuration and applies to all PON ports in the E7 system • As a higher number p-bit denotes higher priority, so does a higher GPON CoS value • All p-bits 0..7 must be represent in the mapping table • A p-bit can only belong to a single GPON class of service • P-bits in lower classes of service must be lower than p-bits in higher classes of service • A lower GPON CoS must have lower or equal Scheduling Class Type when compared to a higher CoS. The ordering of Scheduling Class Type values is Best Effort < Assured < Expedited.

Note: Changes to the GPON CoS global configuration that invalidate existing service provisioning will require ONT services on the PON to be deleted and recreated using 802.1p priority values consistent with the new CoS global configuration. As long as CoS mapping changes do not invalidate existing service provisioning no PON reset is required.

OLT PON Queue Management: Bandwidth limits for all the ONT services of a given CoS on a given OLT port are aggregated and applied at the OLT port level. • The sum of all CIR components cannot oversubscribe the PON, thus all services with CIR components will have that much guaranteed bandwidth. • As long as PIR is not oversubscribed all services will be given PIR. • If PIR is over-subscribed, the available bandwidth is distributed evenly across the services

IGMP IGMP V2 snooping is performed at both the ONT and the OLT. This capability insures that only multicast channels that are joined by a particular STB appear on the subscriber network. Channels not joined are dropped at the GPON port of the ONT. Channels not joined by any ONT on the PON are dropped at the GPON OLT port. The ONT processes IGMP messages from the subscriber and is responsible for tagging the packet with the appropriate VLAN. The E7 GPON-4 card is responsible for snooping IGMP at the GE, 10GE, and GPON OLT interfaces and for establishing the multicast flows to and from the network. • The E7 GPON-4 card supports up to 4 video VLANs across the PON.

Multicast Service Profile ONT multicast service profiles enable provisioning of the IPTV service limits that can be applied to subscriber services. Profiles are named and contain the following attributes: • Stream Count – Number of streams allowed at the subscriber port. • Query Interval – This value tells the ONT how long between group specific queries and allows the ONT to quit the channel if it doesn’t receive IGMP reports for a channel within this provisioned time. • Multicast to unicast – Controls whether the ONT will convert multicast to unicast before sending out the subscriber Ethernet port. Unmanaged Layer-2 switches will propagate multicast to all ports on the subscriber’s network. By default, the ONT sends multicast as Layer-2 multicast packets. However, the

ONT supports the capability to send multicast as Layer-2 unicast packets, replicating the unicast to only those STBs which have joined a particular channel on the subscriber’s network.

IP Hosts IP Host ports on the ONT are pseudo-devices which allow bridge port paths for IP Host Termination at the ONT. These types of bridges are used to define data paths for SIP and T1 over Ethernet (PWE3). Each IP Host interface may be configured with a unique IP address, mask, and gateway address. Optionally, the SIP IP Host may obtain the IP address configuration information via DHCP client protocol. Each IP Host has a dedicated MAC address.

ONT PRE-PROVISIONING AND ACTIVATION

ONT Pre-provisioning Due to the unpredictable nature of in-field ONT deployments, the creation and provisioning of ONTs in the E7 system and the actual installation of the ONT are often disjoint activities. The E7 supports multiple methods to pre-provision ONTS to improve efficiency of this operation. Typically, the service operator wishes to provision an ONT in advance of installation and/or discovery of the physical ONT. This enables provisioning of services before the actual equipment is available. The ONT is named with a Global Logical Identifier assigned by the operator. All provisioning related to the ONT (including ports, services, etc.) are indexed using this global identifier as a base. The ONT is created with a reference to an ONT profile. The ONT profile is named with the model number of the ONT (e.g.”711GX”) and describes the numbers and types of ports associated with that ONT. When an ONT is provisioned, the system automatically creates provisioning for the appropriate ONT ports. When pre- provisioning an ONT the operator may provide a means of linking the provisioning to the physical ONT when it is discovered. There are two modes for ways the operator can do this, using the Serial Number and using the Registration ID. The Registration ID method is preferred as it does not require a specific ONT unit be pre- assigned to a subscriber – an ONT is simply pulled from the technician’s truck and installed on the premises as needed. Pre-provisioning by registration ID. The operator creates the ONT record specifying the ONT Global Logical ID, the ONT Profile, and a Registration ID. This Registration ID is a operator supplied value that is independent of the specific ONT equipment, and must be unique among all provisioned ONT records in the system. The Registration ID is to be entered at the ONT during installation, allowing matching of the ONT to the provisioning via the Remote ONT Activation (RONTA) mechanism. This mechanism requires that the provisioned record contains a Registration ID and no Serial Number. Once the matching ONT arrives, the Serial Number will be updated in the provisioned record. The Registration ID will also be retained. Pre-provisioning by serial number. The operator creates the ONT record specifying the ONT Global Logical ID, the ONT Profile, and a Serial Number. The Global Logical ID and Serial Number must each be unique among all provisioned ONT records in the system. The serial number is the value associated with the specific ONT equipment and that will be reported by the ONT when it arrives. This value is determined by the ONT manufacturer, and is found on the ONT and its packaging. Pre-provisioning without serial number or registration ID. The operator creates the ONT record specifying the ONT Global Logical ID and the ONT Profile, but no Serial Number or Registration ID. This allows the operator to begin provisioning ONT services for the unit. This record may not be matched to a physical ONT until the operator updates it to include either a Serial Number or Registration ID. This may be useful to allow pre- provisioning of services, and then later matching of the provisioning to an unassigned ONT.

In all of these cases the ONT and appropriate services are created in the E7 database. This allows the operator to begin service provisioning for this ONT.

ONT Arrival and Activation When an ONT is discovered by the E7 GPON system, the system attempts to match it to an existing “provisioned but not-linked” ONT record in the database. This match may be done by Serial Number or by Registration ID as described above. Once the match is found, the system updates the provisioned record to note that the ONT has been linked to provisioning, and to record its physical location (Shelf, Slot, OLT Port). If the match was made using the Registration ID, the Serial Number reported by the ONT is also updated in the provisioned ONT record. All provisioning associated with the ONT (including ports, services, etc) is sent to the ONT and service is brought up as provisioned. If the ONT software revision does not match the software version incorporated into the E7 system software, the ONT software is remotely upgrades prior to service activation.

VIDEO SERVICES

IPTV services, whether as a broadcast solution or as incremental video on demand (VOD), are gaining widespread acceptance with service providers, yet they have the most demanding deployment requirements in terms of quality because subscriber expectations for video are very high. With hardware-based wire speed forwarding and multicast replication, the E7 delivers efficient, scalable, high-quality IPTV distribution on both GPON and Ethernet interfaces

IGMP SNOOPING OPERATION The E7 supports industry standard IGMP snooping of multicast video leave and join requests sent between the set-top box and the video distribution network. IGMP snooping enables the E7 to determine which ports should be a member of a multicast group. It can provide a local response if the channel already exists on the E7 or it will forward those requests upstream to a multicast router or another IGMP snooping Ethernet switch. When multicast channels are received from the network, the E7 will forward the channels to all ports that are currently members of the requested IGMP membership group. The ability to perform snooping can be enabled on a per VLAN basis. If multicast traffic is received on a VLAN where IGMP snooping is not enabled, the traffic will be handled as broadcast traffic and sent to all ports on the VLAN.

Figure 32 shows a logical IGMP flow across an E7 network.

Figure 32: IGMP Join Request and Multicast Video Flow on EAPS Network

Key E7 IGMP concepts: • IGMP control messages are allowed to traverse all ports regardless if they are blocked by the ring protection protocol to enable population of the bridging tables for all connected E7 platforms.

• Multicast traffic will not be allowed to traverse ports blocked by the ring protection protocol. • The E7 monitors all interfaces associated with the IGMP multicast VLAN for the presence of OSPF messages to automatically detect the multicast router. • The E7 will drop joins and leaves received on router (uplink) side ports. Ports connected to a router are assumed to be facing towards the network and should not receive joins and leaves. • IGMP will be enabled for all ports designated as ring ports.

The E5-312 currently supports RFC 3376 IGMP v3 and IGMP v2. Today, most IPTV set top boxes support IGMP v2 but a few are starting to support IGMP v3. IGMP v3 adds the ability for endpoints to specify the source of the multicast group they wish to join. This adds additional security against denial of service attacks and removes the need for a centralized router to handle IGMP requests. In either scenario, the E7 supports both IGMP v2 and v3 to ensure maximum flexibility. When the E7 detects a device running IGMP v2, it will fall back from IGMP v3 to IGMP v2 operation in accordance with the IGMP v3 standard.

IGMP SCALABILITY The E7 is a Layer 2 device and supports Layer 3 IGMP snooping and line-rate multicast with non-blocking capacity up to the limit of the physical interface. With proper QOS configured to favor Video over other data flows, channel blocking will occur with an E7 only if the number of channels being delivered is greater than the capacity of the fiber. The E7 is optimized for IGMP performance (a considerable portion of the CPU power is reserved for IGMP processing). The E7 cards have been tested to handle a sustained rate of 250 channel changes per second. The system will protect itself from an excessive rate of channel changes by dropping the channel change request traffic. The E7’s IGMP processing delay varies based on the IGMP load on the network. With a typical well designed network, the contribution of each E7 on the overall IGMP response time is a few milliseconds. The maximum channel change latency introduced by the E7 has been measured to be on the order of 10ms to 20ms under load. However, within a network deployment, the E7 is not the only component in the network that impacts the channel change time as perceived by a subscriber. The subscriber set top box and the multicast router are more than likely the largest contributors to the overall IGMP processing time. As a transport and aggregation component of the network, the E7 is highly optimized to provide the least possible impact on the channel change behavior. The E7 facilitates fast convergence of video services after a topology change in the layer 2 network such that the service can be restored to the effected parts of the network sooner than the General Query time configured on the network. Furthermore, the E7 reduces the overall network load on multicast services by utilizing IGMP report suppression to reduce the number of message sent upstream to the multicast router. The E7 has the option to flood IGMP & multicast, or IGMP Snoop (with or without Report Suppression). IGMP proxy is not supported or required.

E7 MANAGEMENT

The E7 has a rich set of management features encompassing configuration, performance monitoring and diagnostics, fault and security management. A management link to each ONT is used to configure services and perform OAM&P operations on the ONTs. The E7 maintains a persistent provisioning database for the E7 and all subtended ONTs.

SYSTEM CONFIGURATION

Commissioning The E7 can be managed through three primary interfaces, all supporting CMS, E7 Web GUI and Calix command line interfaces (CLI): • In-band on any of the Ethernet network interface ports. • 10/100 Base-T Ethernet port on the front of the unit. • 10/100 Base-T Ethernet port on the rear of the unit.

In-band management traffic is sent to a statically assigned E7 IP address only accessible over a pre-provisioned management VLAN to ensure secure network communications. The management VLAN can be provisioned across multiple interfaces allowing the unit to be managed from multiple locations within the network. When the E7 is initially configured, the management VLAN will need to be provisioned. In order for the management interface to be operational the following attributes must be provisioned: • IP Address – Shelf management IP address • IP Mask – IP network mask for the IP address • Gateway – Gateway IP address for network management • VLAN – Management VLAN ID

If dynamic IP address assignment is preferred for local network management, the E7 supports a DHCP server on the front and back Ethernet interfaces on the unit. This allows a PC connected locally to the Ethernet craft port to receive an IP address without requiring the PC’s network interface card to be reprovisioned in order to establish management communications. By default the DCHP Server on the E7 is disabled on the rear Ethernet port, and enabled on the front craft Ethernet interface to provide a dynamic IP addresses to a maximum of 3 clients, this prevents the E7 from becoming a bogus DHCP Server if the craft port is connected to a Local Area network. If the management VLAN is configured, NAT will be used to keep the PC’s IP address local to the craft Ethernet port.

Note: In platform release 1.0 it is possible that a PC connected to the E7 craft interface may be assigned the same IP address as a PC on another E7 craft interface. Therefore, it is suggested that a management VLAN be configured and NAT enabled.

Shelf Controller Configuration Either E7 line card in a dual-card system configuration can be the active shelf controller. The shelf controller manages the FTA and all management interfaces within the system. The first E7 card installed and booted up in the system becomes the shelf controller unless otherwise configured by the user. Each card can be individually configured to allow it to be a shelf controller candidate. By default, both cards are configured for Controller Candidate equals YES. This means there is no preference in the system for which card will become the shelf controller. If a user configures one of the cards to Controller Candidate equals NO, that card will (a) pass shelf control to the other card (revertive) in the event of a power cycle or replacement of the original shelf controller, and (b) block user forced controller switches and upgrades that require control to transfer to the non-preferred card. To preserve services and maintain connectivity to the system, the E7 system software will override the controller candidate selection in the event of a failure in the preferred shelf controller card.

Software Upgrade The E7 supports software (firmware) upgrade via FTP and Secure File Transfer Protocol (SFTP), and allows for the local storage of two software images on the E7 platform. The E7 can be upgraded by issuing the appropriate CLI commands, accessing the upgrade interface in the web GUI, or through CMS. Regardless of the method used to upgrade the platform, the download of the firmware and initialization of the upgrade process are initiated by two separate processes. This allows the software image (firmware) to be downloaded for pre-upgrade staging at any time, while allowing the software upgrade to occur during a maintenance window. The E7 also supports the ability to revert back to a previous software image if a problem is encountered during the upgrade process. Although it is recommended that the E7 software be upgraded in two steps, a command option is available to consolidate both download of files and reset of the system using a single command. In the case of GPON ONTs, the E7 also serves as the agent to upgrade ONT software. In particular, newly attached ONTs will be automatically software upgraded to the current ONT software load incorporated into the E7’s software release.

CONFIGURATION MANAGEMENT Service profiles and templates simplify and speed service activation within the E7. Profiles are defines system wide and all services using the profile are identical always. Templates are a selection of parameters that can be applied to an object (e.g. an interface) and then modified as needed. Examples of profiles and templates within the E7 include: • System level port/interface defaults • ONT – Port type and counts • Class of Service and Policy Maps • Data Services • Video Service profiles • Voice (SIP, H.248, MGCP, etc.) • T1

The E7 logs all configuration changes.

PERFORMANCE MONITORING Historical performance monitoring (PM) information is maintained within the E7 and can be retrieved for traffic management and network engineering analysis. The E7 captures various systems and service statistics and maintains this information in 15 minute bins for 1 day and 24 hour bins for 1 week. Statistics and performance monitoring data is collected for the GE and 10GE interfaces as well as ONT interfaces. Data collection includes: • Received/Transmitted Octets • Received/Transmitted Unicast Packets • Received/Transmitted VOIP Packets • Received/Transmitted Multicast Packets • Received/Transmitted Broadcast Packets • Discarded received Frames • Interface Input Errors

The E7 also supports per port and per system IGMP counters, including: • IGMP V2 Joins Sent/Received • IGMP V3 Reports Sent/Received • IGMP V3 Include/allow Received • IGMP V3 Exclude Block Received • Leaves Sent • Leaves Received • Group Specific Query count • Invalid messages count

In addition to the Ethernet traffic PM data, the E7 system supports ONT T1 PM, ONT PWE3 service PM, ONT PON PM, and additional ONT Ethernet PM statistics and error attributes.

REALTIME PM AND DIAGNOSTICS To supplement historical performance monitoring data, the E7 provides several on-demand tools for real time performance monitoring and diagnostics. Additionally, the E7 continuously monitors many physical and system level attributes, including: • All physical interfaces are monitored for power, temperature, Loss of Signal (LOS), etc. • Continuity check messages automatically monitor faults within an ERPS/EAPS rings

MAC Table Display The E7 MAC learning table can be displayed to enable debugging of layer 2 learning and forwarding values. If desired, the MAC table can be displayed for a specific physical link or VLAN instead of the entire platform.

VLAN Monitoring The E7 can monitor up to 30 VLANs and collect the following items: • Packets transmitted per VLAN per port • Bytes transmitted per VLAN per port • Bandwidth utilized per VLAN per port

Threshold Crossing Alerts The E7 also has the ability to issue an alert based on user defined criteria such as a high rate of discarded frames or frame errors within a specified period of time.

Port Mirroring The E7 can be configured to mirror traffic from one Ethernet port to another Ethernet port. This function is useful in a number of circumstances including debug scenarios and network troubleshooting. In order to enable port mirroring, a port must first be designated as the destination for the mirrored traffic. Once this designation has been made, Ethernet ports can be added to the mirror for observation. A port configured as the destination for mirrored traffic should be reserved for this purpose and should not be used for regular network traffic. In platform release 1.0 only a single source and destination port can be identified at a time.

NOTE: In E7 system software release 1.0, a port mirror can only be established between GE or 10GE ports within a single E7 card.

FAULT MANAGEMENT The E7 reports all events through the Web GUI, CLI, and CMS user interfaces and maintains local logs of system activity. Up to 500 activity records are stored on the local platform for each log type. CMS is utilized to access older records, effectively enabling historical record archiving and searching capabilities. Notification logs are stored for each of the following log types: • Alarms • Alerts • Database Changes • Security Events • Threshold Crossings

All local notification logs can be manually cleared.

SECURITY MANAGEMENT The E7 utilizes a hardened operating system and Calix aggressively follows the security related patches and guidelines to ensure that the E7 remains the most secure solution possible. Calix also customizes the E7 operating system to ensure that unnecessary services that could be a security risk to the system are not included or enabled. Login attempts are logged locally on the system and also sent to CMS or other network management system via SNMP traps. All active sessions continue to be monitored and activities logged. The E7 includes three levels of user access control, secured by individual ID and password credentials. • Admin – read, write, admin privileges • Provisioner – read and write privileges • Reader – read only privileges

The Management access to the E7 is secured through the use of SSH, HTTPS/SSL, SFTP/SCP, and SNMPv3. The E7 protects itself against network based attacks by providing host protection features: slow path packets are rate limited aggressively on a protocol basis to ensure that the system cannot be overwhelmed.

SNMP An SNMP agent on the E7 can be used to provide performance and fault monitoring information to third party SNMP-based monitoring tools. The E7 SNMP agent supports Community-based SNMPv2c (RFCs 1901-1908) and User-based SNMPv3 (RFCs 3411-3418). The agent will respond to requests using either of these protocols. The original SNMPv1 (RFC 1157) and SNMPv2 classic (a.k.a. "SNMPv2 historic" - RFCs 1441-1452) are not supported. The E7 is also compliant to RFC 2578. While the E7 SNMP agent supports a wide range of RFCs, Calix has focused our Engineering and Design Verification Test (DVT) teams’ efforts on a specific subset of these SNMP recommendations. The following information summarizes the supported SNMP feature set in E7 system software R1.0. Description of files: • CALIX-PRODUCT-MIB.my - defines the E7 registration point • CALIX-SMI.my - defines the Calix (“calixNetworks”) registration point • E7-Fault-MIB.txt - E7 current alarm table and alarm counts • E7-Notifications-MIB.txt - E7 notifications • E7-TC.my - E7 support definitions In addition to the current alarm table and alarm counts, the E7 supports the following standard MIBs: • SNMPv2 (mib-2) - system group • SNMP-FRAMEWORK - snmpEngine group • SNMP-MPD - snmpMPDStats • SNMP-USER-BASED-SM - usmStats and usmUserTable • HOST-RESOURCES - hrMemorySize, hrStorageTable, and hrProcessorTable • IF - ifTable and ifXTable

For the IF-MIB, the following numbering scheme is used: • 101-112 - SFP 1GigE ports, card 1 • 201-212 - SFP 1GigE ports, card 2 • 10101-10102 - XFP 10GigE ports, card 1 • 10201-10202 - XFP 10GigE ports, card 2 • 10103-10104 - SFP+ 10GigE ports, card 1 • 10203-10204 - SFP+ 10GigE ports, card 2

E7 APPLICATIONS

The E7 supports a variety of product and network configurations. • Deployments from central office (CO) and remote terminal (RT) locations • 10GE Transport and aggregation using point-to-point and ring topologies • Protected uplink interfaces • GPON FTTX • Point-to-Point Ethernet FTTX

These core building blocks can be used to build many diverse business-specific network configurations, such as residential and business GPON FTTP, residential and business point-to-point Ethernet FTTP, mobile broadband transport, multipoint MEF-compliant transparent LAN service, and aggregation of provider owned GE access devices. The Ethernet transport and aggregation configurations are common to the E7 and E4-400 products and the two systems can be interchanged without restriction in application that are not specific to the two-card E7 configuration.

CO AND RT DEPLOYMENTS The E7 system is designed for both high density CO deployments as well as challenging remote cabinet configurations. Central Office FTTP Configurations The majority of GPON and point-to-point Ethernet FTTP applications are from the central office. GPON has a reach of 20 km (approx. 60 Kft) with standard OIM optics, 40 km with an Extended Reach OIM; far longer than the 18 Kft reach of a typical copper loop. Ethernet single fiber optics support ranges up to 60km. As a result, a larger percentage of end users can be served directly from the central office with a fiber PON than with home run copper. Central office configurations will typically require multiple E7 systems linked by a 10GE copper SFP+ cable to create a single Ethernet bridging and multicast domain. ERPS / Enhanced EAPS is used to link the multiple E7 systems, while RSTP and/or Link Aggregation are used on the uplink to the network router(s).

Figure 33: GPON and Ethernet Deployment from a Central Office

Voice traffic can be switched to a co-located or remote Calix C7 VOIP TDM gateway with a GR-303 switch interface, or sent to a third party SIP softswitch. When deploying GPON, multiple E7 systems should be linked together with a Calix BITS Timing Cable to create a traceable network reference clock for the GPON and T1 services. A single redundant BITS connection in the CO provides an inter-CO network timing reference that can be used to enable differential timing for T1 pseudowire clock recovery. Remote Terminal FTTP Configurations The E7 is well suited to meet the requirements for RT deployments. • Environmentally hardened, redundant architecture • 1RU design is small yet can expand with an extra slot for very low first install cost • Additional chassis are added with subscriber growth yielding a near linear cost curve • Mix and match line cards in a common chassis meet needs of diverse remote access network • Line cards are managed as a single chassis for operational efficiency • Subscribers are aggregated and network resources efficiently shared across protected trunk facilities • Resilient, hot-swappable fan tray requires less power • Integrated aggregation and transport reduces the number of system that need to fit in the RT • The E7 can be easily integrated into most existing outdoor cabinets to cost effectively extend the reach of the FTTX service network.

Remote terminal configurations typically involve multiple cabinets (ODC-100 to ODC-3000) connected by a ring (or star) to the central office. An E7 in the central office terminates the ring and provides the network uplinks.

Figure 34: GPON Deployment from a Remote Terminal

Transport between E7 systems will typically be a 10GE ring topology, but other topologies such as point-to- point, linear chain, or point-to-multipoint star may be used. A remotely deployed E7 is able to support both IPTV and overlay RF video. Many Calix ODC cabinets accommodate EDFAs for video overlay.

ODC-100 Cabinet Solutions The Calix ODC-100 is designed to accommodate multiple E7 systems and is ideal for deep fiber FTTP deployment. GPON splitters and fiber termination can optionally be integrated to provide a complete GPON or optical Ethernet cabinet configuration. Small Cabinet for Space Restricted and Lower Subscriber Locations • Up to 4 ft in height • Optional PON Splitters • AC or Remote Powered Options • Integrated Optional EDFA Copper and/or Fiber Subscriber Access • Supports E5-100’s for ADSL2+/VDSL2 subscriber Access • Supports E7 w/ GPON or Active Ethernet and 10GigE Transport Mounting Options • Pad, Pole, Vault and Cross Connect Mountable Figure 35: ODC-100 Configuration

Larger Cabinet Solutions The Calix ODC-1000, ODC-2000, and ODC-3000 cabinets are designed to serve larger subscriber densities and accommodate a wide variety of additional networking equipment in addition to the E7. ODC-1000E and ODC-2000E configurations are designed for high density E7 configurations.

Figure 36: ODC-1000E and 1000 with C7 Configurations

Consult a Calix account representative for more information on cabinet configurations.

Point to Point Ethernet Services on Every E7 Card The E7 GPON-4 and 10GE-4 cards have been designed to include a large number GE ports that can be used for aggregating other Ethernet equipment or for delivering point-to-point business and residential services alongside the primary E7 applications. Operators can opportunistically leverage their initial E7 investment to and deploy dedicated Ethernet services to local businesses, organizations, and residences requiring high bandwidth network services. Deployment of customer-based Ethernet services is accomplished with Calix 700GX ONTs or provider owned switches or routers.

Figure 37: E7 Enables Point-to-point Ethernet Service Delivery

ETHERNET TRANSPORT AND AGGREGATION In addition to GPON FTTP services, the E7 provides flexible, high-capacity Ethernet transport and network aggregation options to service providers. Also, Metro Ethernet Forum (MEF) compliant services (MEF 9 and 14) are supported for business service delivery.

Ethernet Point-to-Point Aggregation At the most basic level, the E7 can be use to aggregate Ethernet services from the Calix C7, E5, and F5 platforms into a single network uplink. This traditional “star” configuration can be deployed within the same E7 system used for GPON services using the GPON-4 card’s integrated GE and 10GE interfaces. The E7 supports protected connectivity to remote devices using Link Aggregation and RSTP. RSTP protected interfaces must be on different cards within the E7 shelf to provide equipment protection. The ports configured as uplinks on the E7 can be connected to any standards-based Ethernet switches or routers using the same RSTP and/or link aggregation protocols.

Figure 38: Redundant Calix E7 Point-to-Point Ethernet Aggregation

Ethernet Ring Topologies Ring network topologies are often used to ensure carrier grade service delivery because rings enable dynamically protected network service delivery to multiple locations. The E7 supports ITU G.8032 ERPS / Enhanced EAPS for maximum performance and for large Ethernet rings. This ITU standard is recently ratified and multi-vendor interoperability is expected in the coming years. The ERPS / Enhanced EAPS ring can operate across a single GE interface or 10GE interface, depending on network requirements. E7 system software R1.0 supports a single ERPS / Enhanced EAPS ring per E7 shelf, regardless of whether the system has one or two cards installed. IEEE 802.1w Rapid Spanning Tree Protocol (RSTP) is supported on most switches and routers and was primarily designed to prevent network loops and reroute traffic in traditional Ethernet enterprise networks. RSTP can also provide service restoration when two or more links share common end-points in a point-to-point or ring topology. When utilized for small-scale ring topologies or point-to-point configurations, RSTP will converge and provide service restoration within several hundred milliseconds, depending on the number of Ethernet switches in the network. RSTP is therefore not recommended for large-scale ring topologies. The E7 with redundant cards is ideally suited to support subtended rings from the main transport ring. Each ring is terminated on separate line cards in a single E7 shelf and the E7 backplane becomes the alternate

protection path between rings. ERPS / Enhanced EAPS is use for the main transport ring, typically a 10GE ring, and multiple small RSTP rings can be subtended.

Figure 39: E7s Support Multiple Ring Configurations

Ethernet Aggregation Topology The E7 provides the ability to aggregate standards-based Ethernet traffic from the full range of Calix broadband solutions, Ethernet switches and routers. Uplink and downlink aggregation can be provided through the use of one or more GE or 10GE ports. The protocols that the E7 supports for aggregation include ERPS/Enhanced EAPS and single link or redundant links running RSTP or link aggregation.

Figure 40: E7 Ethernet Aggregation

Figure 40 depicts the E7 deployed in a remote location aggregating C7, E5-100, and F5 broadband access platforms. In this configuration, the E7 is aggregating the Ethernet traffic from the remote devices and switching it onto the transport ring for delivery to the network uplink. This diagram shows a redundant two-card E7 system providing aggregation redundancy and network uplink protection.

Figure 41 shows the physical GE connections that enable link protection (RSTP) from the E5-100 Ethernet service platform to the redundant E7 system.

Figure 41: Ethernet Aggregation with RSTP Downlink Protection

Protected Uplink with Virtual Router Redundancy (VRRP) The E7 can be used with IETF RFT 3768 Virtual Router Redundancy Protocol (VRRP) to increase the availability of the default gateway. Service providers will use VRRP to provide router backup should one of routers fail. The E7 does not play a role in this protocol and simply responds to the routers’ ARP and gratuitous ARP messages to determine if a router protection switch has occurred. RSTP is not run on the links between the E7 and the routers because the routers are preventing loops by controlling the advertising of IP and MAC addresses into the E7 Layer 2 network. The redundant E7 connections to the VRRP-enabled routers can be from the same E7 shelf on separate line cards, or from two different E7 systems within the Layer 2 broadcast domain. This latter configuration, with two E7 systems on the same 10GE ring but at different locations, is shown in Figure 42.

Figure 42: E7 Network Connected to Routers Running Virtual Router Redundancy Protocol (VRRP)

Diverse Fiber Route and RSTP Node Protection While the E7, with 2 card slots, makes it easy to deploy protected network elements at a specify location, the need often arises to have diverse fiber routes from a remote network element back to the aggregation locations. This simple network configuration is shown in Figure 43.

Figure 43: Ethernet Aggregation with Diverse Fiber Routes Two E7 systems can be provisioned to provide route diverse network redundancy using RSTP. RSTP domains can be utilized for aggregation of Ethernet devices or to create RSTP protected uplink from multiple locations or E7 shelves and is used with ERPS / Enhanced EAPS rings to create isolated RSTP domains that are connected by the redundant transport ring. RSTP domain isolation localizes topology awareness, prevents traffic from transit across the transport ring when not required, and reduces failover times by reducing the number of RSTP convergence points. With E7 node protection, the ERPS / Enhanced EAPS ring and protection switches are managed independently of the RSTP domain. As a result, topology changes or failures within the RSTP domain are not propagated to ERPS ring, and vice versa. Further, RTSP topology changes in one protected node pair are not propagated to other protected nodes. Multiple node protection RSTP domains are depicted in Figure 44.

Figure 44 Multiple RSTP Node Protection Domains for Uplinks / Aggregation

Two E7 systems forming a protection domain are typically located adjacent on the ERPS / Enhanced EAPS ring, but they are not required to be physically co-located or immediately adjacent. Figure 45 illustrates the flexibility of RSTP node protection and how it can be applied within the transport ring.

RSTP Node RSTP Node RSTP Node RSTP Node Protection Pair Protection Pair Protection Pair Protection Pair

10GE Ring ERPS / Enhanced EAPS

Figure 45: Logical RSTP Node Protection Domains within an ERPS / Enhanced EAPS Ring

The E7 RSTP node protection feature is identical to that of the E5-400. E7 and E5-400 units can be combined to form a protected node configuration.

RSTP Node Protection planning guidelines: • An E7 system can only belong to one RSTP node protection domain

• A protection domain consists of a pair of E7 system on the ERPS / Enhanced EAPS ring and can be utilized for uplink and/or downlink interfaces. • Multiple node protection RSTP domains can be defined within an ERPS / Enhanced EAPS ring. This allows the ERPS / Enhanced EAPS ring to have multiple network aggregation points • When configuring a protection domain with a pair of E7 systems, one E7 is provisioned as the primary RSTP node the other is provisioned as the secondary node. • There is no correlation between the ERPS / Enhanced EAPS ring master node and the assignment of node protection primary and secondary nodes. The ERPS / Enhanced EAPS ring master node can be one of the E7 protected nodes or can be another node on the ring. • For each protected uplink / downlink interface, if the link to the primary node is operational, that link carries traffic and the corresponding link to the secondary node is blocked. If a link fails on the primary node, traffic flows on the secondary link. When the primary link has been restored, the traffic automatically reverts back to the normal operational mode. • When an E7 node is provisioned to be in a node protection domain (primary or secondary), then all of that E7’s interfaces provisioned to have RSTP enabled will become part of the node protection RSTP domain.

T1 PSEUDOWIRE APPLICATIONS The E7 and Calix 766 ONT provide T1 services over GPON and point-to-point GE using pseudowire emulation technology. IETF RFC 3985 and RFC 4197 define the pseudowire emulation edge to edge (PWE3) architecture and provide a standards-based approach to T1 service delivery over a packet switched network (PSN). The PWE3 T1-to-packet interworking function (IWF) is accomplished by dedicated hardware circuitry within the 766GX ONT, that provides high performance with minimal processing delay. A configurable jitter buffer and T1 encapsulation frame size within the ASIC are used to compensate for latency and PDV within the network.

Figure 46: Mobile Backhaul and Business Services using T1 Pseudowire

Traditional T1 services at the GPON 766 ONT are converted to packet flows for transmission over the PON or GE optical network back the E7, where they are transported to a remote pseudowire device that translates the packet stream back to the original T1 format. The Calix 766 ONT uses Structure-Agnostic Time Division Multiplexing (TDM) over Packet (SAToP), RFC 4553, for encapsulation and transport of T1 services over the packet switched network. The ONT uses IP as the SAToP packet format, with UDP for the pseudowire multiplexing layer (i.e. the frame construction is Ethernet / IP / UDP / TDM payload). Therefore, each T1 circuit is addressed with a static IP address and port ID (UDP) that is matched to a far end to complete the end-to-end T1 circuit. Calix E7 T1 PWE3 solutions address the following challenges in emulating a T1 service over a packet network: • Packetization and Encapsulation of T1 Traffic - The T1service has to be packetized and encapsulated before being sent across the PSN. This encapsulation process places a pseudowire control word in front of the T1 payload data. • Attenuate Packet Delay Variation (PDV) - Packet networks create latency and more important PDV, also known as jitter. The T1 service cannot function with the jitter inherent in packet networks and so the pseudo wire emulation must be able to smooth out the jitter of the packet network. This is done by using a jitter buffer, which stores packets on the receive side and transmits them smoothly to the T1 link. • Compensate for Frame Loss and Out-of-Sequence Packets - Packet networks by their nature experience loss of frames and mis-ordering of frames (as a result of congestion, routing paths, etc). The pseudowire emulation mechanism must detect and mask these phenomena from the T1 service, as much as possible. This is facilitated by the use of the RTP protocol. • Recover Clock and Synchronization - Traditional T1 devices require a synchronized clock to function, but the packet switched network by nature is not synchronous. The pseudowire emulation mechanism must regenerate the original T1 timing accurately across the packet network.

Calix 766GX ONT The E7 delivers T1 services when paired with the Calix 766GX ONT, which supports eight (8) T1 service ports. The multipurpose 766GX ONT can be used for mobile backhaul, business services, or MDU applications. • Designed for both Premises and CO deployments o Hardened wall and rack mount options o CO aggregation (1:N) of T1 circuits from multiple premises ONT end points • Industry standards based T1 pseudowire o Pseudowire architecture IETF RFC 3985 and 4197 o Structure-Agnostic Time Division Multiplexing (TDM) over Packet (SAToP) IETF RFC 4553 o Adaptive and differential timing modes o External clock reference input • Optical Distribution Network options o GPON ITU G.984 compliant, up to 40 km reach o 1 Gbps symmetric Active Ethernet, up to 60 km reach • Network compatible with RAD PWE3 gateways o IPmux-24/216 and Gmux-2000 solutions o Adaptive timing only

Figure 47: Calix 766GX ONT with Integrated T1 Pseudowire Technology

T1 Synchronization As telecom operators seek to lower costs by switching from legacy SONET networks to Ethernet networks, Ethernet-based TDM solutions, such as PWE3, are gaining increasing significance. Ethernet networks, however, handle system clocking different from SONET networks. SONET networks require that each node recovers its timing off of the previous node. Thus, any node in the system can trace its timing back to the Central Office Terminal (COT) and the network clock is considered “traceable.” In the case of T1 service, having a traceable network clock means that the T1 service clock at any given node is the same as any other node.

Service Network Clock Service Clock Clock

T1 T1 COT Traceable RT Central Office T1 Remote Terminal T1 Terminal Network T1 T1

Figure 48 – Network / Service Clock Relationship on a Traceable Network

Ethernet networks do not have the same strict clocking requirements as SONET networks. In an Ethernet network each node may have its own timing source, thus the network clock may not be traceable. Given this lack of traceability, the T1 service clock is no longer guaranteed the same at different nodes. The receiver’s service clock now needs to be recovered using timing information stored inside the Ethernet frame. As the network expands, the packets may be re-timed at each node. Larger networks may also have more traffic congestion which causes additional routing delay. These delays complicate the clock recovery algorithm and at some point, simple recovery is no longer possible and a timing protocol or traceable clock must be extended to each end point of the T1 pseudowire.

Three mechanisms are available with the E7 and 766GX ONT for T1 clock recover. • Adaptive Clock Recovery – This method uses the average arrival time of the packets to derive the transmitter’s clock. This is possible since the source device is producing T1 encapsulated frames at a constant rate. By averaging the arrival times, the PDV can be negated and the original sender’s clock recovered. • Differential Clock Recovery – This method employs a residual timestamp at the sender based on the difference between the shared reference (traceable) time and the T1 line. Thus, the receiver utilizes the timestamps in the received frames against its own time reference to correctly adjust its own T1 line timing. When a shared clock is available between the two PWE3 devices (766GX ONT or other interoperable device), this mechanism will produce lower jitter and delay. • Line/Loopback – This is not really a clock “recovery” mechanism. In this configuration received T1 service clock is “looped back” to provide the transmit T1 service clock. Typically, loopback timing is used in a central office or network side of the T1 service and either adaptive or differential timing is used at the remote (subscriber) side of the pseudowire circuit. As a result of the loopback, this mechanism completely eliminates all wander and jitter in the T1 service transmitted back toward the network T1 source, but may require larger jitter buffers, and hence delay, to accommodate this.

NOTE: The entire 766 ONT must be configured for either adaptive or differential timing, which results in all eight T1 ports on the 766 ONT having the same timing algorithm applied. A T1 pseudowire configured for differential timing that does not receive RTP header time stamps from the far end or that does not have a traceable clock will fall back to adaptive timing for service clock reconstruction.

Traceable clock source The 766 ONT receives its traceable reference clock for T1 differential timing recovery from the synchronous GPON optical network connected to the E7 GPON-4 card. The E7 may be using it local Stratum 3 clock or an external BITS input as its timing reference. Up to ten E7 shelves can be connected together to share a local BITS input. A BITS Chaining Cable available from Calix includes connectors to attach the redundant BITS output pins from one E7 shelf to the redundant BITS input pins on a second E7 shelf.

T1 Pseudowire Aggregation The E7 and 700 ONTs can provide leased line (CO to Customer Premises) and private line (Customer Premises to Customer Premises) T1 services over a variety of network configurations. Calix 700 ONTs with integrated T1 pseudowire technology can easily be deployed to small businesses, cell phone towers, and other locations requiring traditional T1 service. For network applications requiring small numbers of T1 services (less than 30), additional Calix 766GX ONTs will typically be used in the central office to terminate the T1 circuit and interconnect to an existing SONET network or digital cross connect system. To facilitate this application, the Calix 766GX ONTs can act as a small T1 gateway, aggregating T1 circuits from multiple remote ONTs into a single CO-based ONT, as shown in Figure 49.

Figure 49: Calix 766 ONT used for T1 Service Delivery and CO Aggregation

In larger networks, the T1 pseudowire circuits may be aggregated by the E7 to a high density pseudowire gateway with channelized OC-12, OC-3, or DS3 interfaces. Calix Compatibility Lab (CCL) partners provide solutions for this high density application. Please contact your Calix account team for more information regarding current CCL partner solutions.

T1 Pseudowire Applications Unlike most applications transported over a packet switched network, packetized TDM services like T1 can be greatly impacted by the timing characteristics of the network. At each end of the T1 pseudowire, the T1 interworking function (IWF) must assemble the packetized T1 payload into its original format, including its original service clock. The ability to reconstruct the service clock is determined by whether a traceable network reference clock exists at both ends of the T1 pseudowire. Figure 50 shows the relationship between network reference clock and the use of differential and adaptive timing methods for service clock recovery. While both adaptive and differential timing algorithms can maintain end-to-end T1 timing stability that conforms to ITU G.823 synchronization requirements, differential timing provides better wander performance for applications that may require greater long term timing stability or that are subject to high PDV.

Figure 50: Differential and Adaptive Timing Domains for T1 Pseudowire Transport

While the T1 encapsulation and data streams are independent in transmit and receive directions, the two ends are coupled with respect to synchronization and T1 clock recovery. For example, if the remote subscriber end of the T1 pseudowire is configured for differential timing, then the network end must send RTP time stamps in the T1 frames and both ends must have a traceable clock reference. Table 8 shows the allowable and recommended configurations for T1 pseudowire service delivery using the E7 GPON-4 card and 766GX ONT.

E7 GPON 766GX ONT to E7 GPON 766GX ONT

Deployment Scenario Timing Modes (location of E7 GPON-4 OLT) (Network Side - Subscriber Side) Pseudowire Network Pseudowire Loopback - Loopback – Differential- Adaptive-Adaptive Side Subscriber Side Adaptive Differential (1) Differential (2) location with co-located (same Recommended Recommended Recommended Not recommended traceable clock (e.g. OLT or connected by (3rd) (1st) (2nd) (3) CO, SONET-fed RT) BITS chaining cable)

location with another location with Recommended Recommended Recommended Not recommended traceable clock (e.g. traceable clock (e.g. (3rd) (1st) (2nd) (3) CO, SONET-fed RT) CO, SONET-fed RT)

location with location without Recommended not supported not supported Not recommended traceable clock (e.g. traceable clock (e.g. (1st) (3) CO, SONET-fed RT) 10GE-fed RT, MDU)

location without co-located (same Recommended Recommended Recommended Not recommended traceable clock (e.g. OLT or connected by (3rd) (1st) (2nd) (3) 10GE-fed RT, MDU) BITS chaining cable)

location without another location Recommended not supported not supported Not recommended traceable clock (e.g. without traceable (1st) (3) 10GE-fed RT, MDU) clock (e.g. 10GE-fed RT, MDU)

GE 766GX ONT or 3rd Party Aggregator (network) to E7 GPON 766GX ONT (subscriber)

Deployment Scenario Timing Modes (location of E7 GPON-4 OLT) (Network Side - Subscriber Side) Pseudowire Network Pseudowire Loopback - Loopback - Differential- Adaptive-Adaptive Side Subscriber Side Adaptive Differential Differential location with co-located (same Recommended not supported not supported Not recommended traceable clock (e.g. OLT or connected by (1st) (3) CO, SONET-fed RT) BITS chaining cable)

location with another location with Recommended not supported not supported Not recommended traceable clock (e.g. traceable clock (e.g. (1st) (3) CO, SONET-fed RT) CO, SONET-fed RT)

location with location without Recommended not supported not supported Not recommended traceable clock (e.g. traceable clock (e.g. (1st) (3) CO, SONET-fed RT) 10GE-fed RT, MDU)

location without co-located (same Recommended not supported not supported Not recommended traceable clock (e.g. OLT or connected by (1st) (3) 10GE-fed RT, MDU) BITS chaining cable)

location without another location Recommended not supported not supported Not recommended traceable clock (e.g. without traceable (1st) (3) 10GE-fed RT, MDU) clock (e.g. 10GE-fed RT, MDU)

(1) Loopback-differential mode provides the lowest wander at network/CO end (2) Differential-differential mode provides the lowest latency (3) Adaptive clock recovery is not recommended for T1 upstream (toward the T1 network timing source) over GPON due to excessive wander

Table 8: T1 service recommended configurations

Figure 51 shows a model for building synchronous networks using the E7 and Calix 766GX ONT. All E7 units are synchronized using the E7’s BITS IN/OUT pins on the back of the unit to create a traceable network clock reference between the E7 systems and the ONTs. The E7 systems can be tied to a local BITS clock or allowed to free run on the E7’s STRATUM 3 local oscillator. The remote subscriber ONT should be configured for differential timing for best performance. The CO network end can use loopback (lowest jitter) or differential (lowest delay) clock recovery.

Figure 51: T1 Pseudowire using Differential Timing; ONT with GPON Line Timing

Figure 52 shows an E7 GPON network where a traceable network reference clock is not available and adaptive timing is used T1 service clock at the remote subscriber end. The CO network must use loopback timing to eliminate timing wander in the upstream T1 path. Because the E7 is a hardware-based switch with low latency, significant PDV will not build up between the T1 endpoints.

Figure 52: T1 Pseudowire using Adaptive Timing over Asynchronous Network

Figure 53 shows an E7 Ethernet network with the remote terminal and CO systems sharing a traceable network clock reference. A traceable network reference clock may be provided by any one of several methods, including: • BITS input • SONET timing distribution • GPS clock input • Timing over Packet – IEEE 1588 Version 2 (v2) Precision Clock Synchronization Protocol

Note: The E7 GPON-4 card is hardware ready for IEEE 1588v2.

In the network below, differential timing can be used to reconstruct the T1 service clocks with high immunity to packet delay variation imparted by the packet switched network over which the pseudowire is transported. The remote subscriber ONT should be configured for differential timing for best performance. The CO network end can use loopback (lowest jitter) or differential (lowest delay) clock recovery.

Figure 53: T1 Pseudowire using Differential Timing over Synchronized Network

T1Pseudowire planning guidelines: • Within a Layer 2 network, T1 services from an ONT are typically placed in a single VLAN for transport between the remote pseudowire devices and/or the IP gateway. Multiple remote locations may be split into separate VLANS for ease of network planning and operation. • Transport of T1 services over a packet network requires a CoS priority and sufficient bandwidth allocation within the packet network to reduce latency and packet loss. T1 service VLANS should be given a high IEEE 802.1p priority “p-bit” value (e.g. 7). • Each Calix ONT is provisioned with a static IP address for the SAToP interworking function (an IP host). Each T1 pseudowire end point thus has a unique source and destination IP address corresponding to the ONT with a UDP port ID for the individual T1 circuit on the ONT. • T1 pseudowires can be transported through an IP network in addition to the E7 Ethernet network. If two T1 pseudowire ONT end points are within the same IP subnet and Layer 2 broadcast domain (VLAN), they will discover each other using Address Resolution Protocol (ARP). The T1 service will not transit through the IP default gateway and no manual configuration is required for each end point to discover their respective destinations. • If two T1 pseudowire circuits have IP addresses in different IP subnets, the data path between the end points will transit through a router external to the E7 network even if there is a direct Layer 2 path between the two ONTs that would stay within the E7 network.

Note: In E7 Release 1.0, T1 services may not be directly connected from one E7 ONT to another E7 ONT on the same GPON OLT port. ONT T1 services on different GPON OLT ports may connect directly if they are in the same IP subnet and broadcast domain.

• To connect a T1 pseudowire circuit from one ONT to anther ONT on the same E7 OLT port, the VLAN broadcast domain for the T1 pseudowire must have a path to the default gateway router. The default router must be configured to maintain the high CoS priority for the T1 pseudowire.

NETWORK COMPATIBILITY The E7 is being tested for interoperability with many standards-based, 3rd party devices including Ethernet switches and routers. Please contact your account team for a list of certified products.

A COMPLEMENT TO THE CALIX ACCESS PORTFOLIO The E7 is a new addition to the Calix E-Series product family and is fully managed by the Calix Management System (CMS). When combined with the other solutions in the Calix access portfolio – the Calix C-Series, F- Series, and P-Series platforms – the E7 plays a key role in providing a comprehensive unified access infrastructure that delivers a single managed network for advanced services while providing operational and capital efficiency

Figure 54: E7 and the Calix Unified Access Infrastructure

E7 SPECIFICATIONS

SYSTEM SPECIFICATIONS

BACKPLANE BANDWIDTH ANALOG/METALLIC INTERFACES 100 Gbps between slots Two standard 25-pair RJ-21 connectors per slot

SHELF SLOTS MANAGEMENT INTERFACES 2 universal line card slots Ethernet 10/100 (RJ-45 connector on back of Calix E7) 1 FTA slot Ethernet 10/100 (RJ-45 connector on Calix E7 Fan Tray) RS-232 (RJ-11 connector on Calix E7 Fan Tray) SHELF DIMENSIONS (W x H x D) In-band Management VLAN 17.6 x 1.75 x 12.2 inches 44.7 x 4.45 x 31.0 cm ALARM I/O INTERFACES Height is 1 RU Wire wrap pins on the back of the E7 shelf Environmental alarm inputs: 7 – user definable WEIGHT (LB/KG) Alarm outputs: 1 – user definable E7 Shelf 5.88 / 2.67 E7 FTA 1.52 / 0.69 BITS TIMING I/O INTERFACES 10GE-4 1.86 / 0.84 Wire wrap pins on the back of the E7 shelf GPON-4 2.02 / 0.92 BITS clock (sink and source) GE-12 1.84 / 0.83 SYNCHRONIZATION OPERATING ENVIRONMENT Synchronization is enabled by the E7 line cards as Temperature: -40 to +65° C (-40° F to +149° F) required. Humidity: 10 to 95% (non-condensing) External reference timing Operating altitude: 10,000 ft (3,049 m) Built-in Stratum-3 clock Hardware-ready to support IEEE 1588v2 and STORAGE ENVIRONMENT Synchronous Ethernet Temperature: -40 to +85° C (-40° F to +185° F) Humidity: 5 to 95% COMPLIANCE NEBS Level 3 compliance (GR-63-CORE, GR-1089- FIBER INTERFACES CORE, GR-3028) All optical ports use pluggable optics (SFP, XFP, SFP+) UL 60950 LC or SC connectors on modules FCC Part 15 Class A

E7 PORT CAPACITY The following table shows the aggregate number of ports available on the front of the line card in dual card E7 systems with default “First Ports” uses as backplane ports. Port capacity with optional backplane port configurations are shown in parenthesis.

E7 Shelf E7 Shelf GPON GE SFP 10GE XFP 10GE SFP+ Slot 1 Slot 2 Ports Ports Ports Ports 10GE-4 Blank 0 12 2 2 GPON-4 Blank 482 2 GE-12 Blank 0 12 0 2 10GE-4 10GE-4 0 24 2 (or 4) 4 (or 2) GPON-4 GPON-4 8 16 2 (or 4) 4 (or 2) GE-12 GE-12 0 24 0 4 (or 2) 10GE-4 GPON-4 4 20 2 (or 4) 4 (or 2) 10GE-4 GE-12 0 24 1 (or 2) 4 (or 2) GPON-4 GE-12 4 20 1 (or 2) 4 (or 2)

POWER AND HEAT DISSIPATION Integrated power management on Calix E7 line cards

• Redundant (A and B) DC battery feeds • Independently monitored battery voltage

• Input Range: –42.5 VDC to –72 VDC

• Fuse: 7.5 Amps (A and B) – assumes standard rerating of fuses to no more than 80 percent of fuse value

E7 System Power Draw Heat Dissipation Component (Watts) (Watts) 10GE-4 55 55 GPON-4 75 75 GE-12 50 50 FTA (CO nominal) 22 6 FTA (RT max) 65 18

ETHERNET SWITCHING CAPACITY

Description Capacity Detail Switching Capacity Wire speed full duplex Dedicated, non-blocking switch port to all forwarding across all ports GPON, GE, and 10GE interfaces PON Bandwidth 2.488 Gbps down 1.244 Gbps up ONT Capacity 64 per PON Each PON has a non-blocking 2.5 Gbps 256 per system symmetric connection to the E7 switch core MAC Address Table Capacity 32,000 MAC Addresses E7 line cards share common table VLAN Support 4094 VLANs VLANs 1002, 1003, 1004, and 1005 are reserved for system use but can be changed to another range. Default Internal VLAN (GE and 10GE 1 This VLAN is utilized to switch all untagged ports only) traffic through the system. This VLAN cannot be deleted, but can be changed. Not supported on ONT Ethernet ports. MTU Packet Size (GE and 10GE ports) 9000 Bytes E7 supports the ability to set the MTU size on an interface to a maximum of 9000 bytes, not including Ethernet header and two VLAN tags for Q-in-Q. MTU Packet Size (GPON services) 1550 Bytes 700 ONTs Total frame size 1536 Bytes 760 ONTs Broadcast Video Channel Support 800 Maximum IGMP group size Number of Nodes Allowed in ERPS 32 E7 cards per ring This number counts all E7 cards located in the Ring ring whether in a dual or single card E7 shelf. This number does not include devices or E7 subtended from the ring. Number of ERPS Enhanced EAPS 1 Platform release 1.0 limitation. Planned for Rings per system expansion in a future platform release. Bandwidth Policing (GE and 10GE 1 Mbps up to line rate ports) Bandwidth Shaping (ONT Ethernet 64 kbps to 2 Mbps in 64kbps ports) increments; 1 Mbps increments from 2 Mbps up to 400 Mbps Number of Egress Priority Queues (GE 8 per port Based on pbit value with pbit=7 highest priority and 10GE ports) Number of Egress Priority Queues 4 per PON P-bit values are mapped into four GPON CoS (GPON) queues Queue Scheduling Algorithm Strict priority with maximum Tail drop is used when dropping packets from and minimum guaranteed queue bandwidth per class

Link Aggregation Groups (GE Ports) Up to 6 GE groups All ports in the group must be GE ports Ports Allowed in GE LAG Groups Up to 8 GE ports can be All ports in the group must be GE ports provisioned per group Link Aggregation Groups (10GE Ports) Up to 2 link aggregation All ports in the group must be 10GE ports groups for 10GE ports Ports Allowed in 10GE LAG Groups Up to 4 -10GE ports can be All ports in the group must be 10GE ports provisioned in a LAG group

PROTOCOLS AND STANDARDS SUPPORTED

Standard Title TR-059 DSL Evolution - Architecture Requirements for the Support of QOS-Enabled IP Services TR-101 Migration to Ethernet-Based DSL Aggregation IEEE 802.1ad Bridged VLANs - 802.1Q Amendment 4, Provider Bridges (QinQ or Stacked VLANs) IEEE 802.1ag Connectivity Fault Management IEEE 802.1D MAC Bridges IEEE 802.1p Priority IEEE 802.1Q Virtual Bridged Local Area Networks (VLAN) IEEE 802.1w Rapid Spanning Tree Protocol (RSTP) IEEE 802.3 LAN/MAN Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, Ethernet IEEE 802.3ad Link Aggregation IEEE 802.3ae 10GBASE-xR/W, 10Gb/s Ethernet over fiber IEEE 802.3ah Ethernet in the First Mile IEEE 1588 v2 Precision Clock Synchronization Protocol IETF RFC 2132 DHCP Options and BOOTP Vendor Extensions IETF RFC 2666 Definitions of Object Identifiers for Identifying Ethernet Chip Sets IETF RFC 2827 Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing IETF RFC 3046 DHCP Relay Agent Information Option IETF RFC 3261 Session Initiation Protocol (SIP) IETF RFC 3376 Internet Group Management Protocol, Version 3 (IGMP v3) IETF RFC 3393 IP Packet Delay Variation Metric for IP Performance Metrics (IPPM) IETF RFC 3619 Ethernet Automatic Protection Switching (EAPS) Version 1

IETF RFC 3637 Definition of Managed Objects for the Ethernet WAN Interface Sublayer IETF RFC 3942 Reclassifying Dynamic Host Configuration Protocol version 4 (DHCPv4) Options, IETF RFC 3985 Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture IETF RFC 4197 Requirements for Edge-to-Edge Emulation of Time Division Multiplexed (TDM) Circuits over Packet Switching Networks IETF RFC 4243 Vendor-Specific Information Sub-option for the DHCP Relay Agent Option IETF RFC 4251 Secure Shell (SSH) Protocol Architecture. IETF RFC 4541 IGMP Snooping IETF RFC 4553 Structure-Agnostic Time Division Multiplexing (TDM) over Packet (SAToP) IETF RFC 4836 Definitions of Managed Objects for IEEE 802.3 Medium Attachment Units (MAUs) ITU-T G.8021 Characteristics of Ethernet transport network equipment functional blocks ITU-T G.984 Gigabit-capable Passive Optical Networks (GPON) ITU-T Y.1730 Requirements for OAM functions in Ethernet-based networks and Ethernet services ITU-T Y.1731 OAM Functions and Mechanisms for Ethernet-based Networks MEF 9 Abstract Test Suite for Ethernet Services at the UNI MEF 14 Abstract Test Suite for Traffic Management Phase 1

SNMP

Standard Title IETF RFC 1901-1908 Community-based SNMPv2c IETF RFC 3411-3418 User-based SNMPv3 IETF RFC 2578 Structure of Management Information Version 2 (SMIv2)

MIB Name Description E7-Fault-MIB E7 current alarm table and alarm counts SNMPv2 (mib-2) system group SNMP-FRAMEWORK snmpEngine group SNMP-MPD snmpMPDStats SNMP-USER-BASED-SM usmStats and usmUserTable HOST-RESOURCES hrMemorySize, hrStorageTable, and hrProcessorTable IF ifTable and ifXTable

ORDERING INFORMATION

CALIX E7

000-00372 ...... E7 Shelf with Fan Tray Assembly and Installation Kit

CALIX E7 LINE CARDS

100-01771 ...... E7 10GE-4 card (12xGE SFP, 2x10GE XFP, 2x10GE SFP+) 100-01772 ...... E7 GE-12 card (12xGE SFP, 2x10GE SFP+) 100-01773 ...... E7 GPON-4 card (4xGPON OIM, 8xGE SFP, 2x10GE XFP, 2x10GE SFP+)

CALIX E7 FAN TRAY ASSEMBLY

100-01451 ...... E7 Fan Tray Assembly 000-00228 ...... E7 Fan Tray Assembly Filter, Package of 10 units

CALIX OPTICAL AND COPPER PLUGGABLE MODULES

Calix offers a full suite of optical and copper modules for E7 line cards. SFP ...... 1 GE optical and copper Small Form-factor Pluggable (SFP) modules SFP+ ...... 10GE optical and copper 10 Gigabit Small Form-factor Pluggable (SFP+) modules GE SFP modules may also be used in SFP+ ports at a 1Gpbs rate XFP ...... 10GE optical 10 Gigabit Small Form-factor Pluggable (XFP) modules GPON OIM ...... 2.5Gbps GPON (Class B+ ODN with minimum 28dB link budget, up to 1:64 splits) ER-GPON OIM ...... 2.5Gbps Extended Reach GPON (up to 40 km with 1:8 split)

GE-12 PLUGGABLE SFP MODULES

The Calix E7 GE-12 card supports only the following pluggable modules in the 12 SFP ports. 100-01660 ...... 1GE SFP, MMF, 500m, I-Temp, keyed 100-01661 ...... 1GE SFP, RJ45 Cu, 100m, I-Temp, keyed 100-01669 ...... 1GE SFP, SMF, BIDI, Tx 1490nm, 20Km, I-Temp, keyed 100-01671 ...... 1GE SFP, SMF, BIDI, Tx 1490nm, 40Km, I-Temp, keyed 100-01673 ...... 1GE SFP, SMF, BIDI, Tx 1490nm, 60Km, I-Temp, keyed

CALIX SOURCED MODULES

High-speed optic module operational tolerances and performance vary significantly and can dramatically affect network operations. To maintain predictable performance and product reliability, the Calix E7 only works with Calix GPON OIM, 10GE XFP and 10GE SFP+ optical modules. Because the Multisource Agreement (MSA) does not fully specify the design of SFP modules supporting 10/100/1000BaseT copper interfaces, these modules must also be ordered exclusively from Calix to ensure correct operation. Calix does not guarantee full compliance to product specifications for units using non-Calix SFPs and does not provide customer service support for optical network issues when non-Calix SFP modules are used. Calix recommends use of Calix supplied SFPs for all service deployments.