Explaining Structured Errors in Gigabit Ethernet
Total Page:16
File Type:pdf, Size:1020Kb
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228773987 Explaining Structured Errors in Gigabit Ethernet ARTICLE · APRIL 2005 READS 13 4 AUTHORS, INCLUDING: Andrew W. Moore University of Cambridge 105 PUBLICATIONS 2,202 CITATIONS SEE PROFILE All in-text references underlined in blue are linked to publications on ResearchGate, Available from: Andrew W. Moore letting you access and read them immediately. Retrieved on: 22 January 2016 Explaining Structured Errors in Gigabit Ethernet Andrew W. Moore, Laura B. James, Richard Plumb and Madeleine Glick IRC-TR-05-032 March 2005 INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel products are not intended for use in medical, life saving, life sustaining applications. Intel may make changes to specifications and product descriptions at any time, without notice. Copyright © Intel Corporation 2003 * Other names and brands may be claimed as the property of others. 1 Explaining Structured Errors in Gigabit Ethernet ¡ ¡ ¢ Andrew W. Moore , Laura B. James , Richard Plumb and Madeleine Glick University of Cambridge, Computer Laboratory [email protected] ¡ University of Cambridge, Department of Engineering, Centre for Photonic Systems £ lbj20,rgp1000 ¤ @eng.cam.ac.uk ¢ Intel Research, Cambridge [email protected] Abstract— A physical layer coding scheme is designed to I. INTRODUCTION make optimal use of the available physical link, providing Many modern networks are constructed as a series of functionality to higher components in the network stack. layers. The use of layered design allows for the modular This paper presents results of an exploration of the errors construction of protocols, each providing a different observed when an optical Gigabit Ethernet link is subject service, with all the inherent advantages of a module- to attenuation. In the unexpected results, highlighted by some data payloads suffering from far higher probability based design. Network design decisions are often based of error than others, an analysis of the coding scheme on assumptions about the nature of the underlying layers. combined with an analysis of the physical light path leads For example, design of an error-detecting algorithm, such us to conclude that the cause is an interaction between the as a packet checksum, will be based upon a number of physical layer and the 8B/10B block coding scheme. We premises about the nature of the data over which it is to consider the physics of a common type of laser used in work and assumptions about the fundamental properties optical communications, and how the frequency charac- of the underlying communications channel over which it teristics of data may affect performance. A conjecture is is to provide protection. made that there is a need to build converged systems, with Yet the nature of the modular, layered design of the combinations of physical, data-link, and network layers optimised to interact correctly. In the mean time, what will network stacks has caused this approach to work against become increasingly necessary is both an identification of the architects, implementers and users. There exists a the potential for failure and the need to plan around it. tension between the desire to place functionality in the most appropriate sub-system, ideally optimised for each incarnation of the system, and the practicalities of mod- ular design intended to allow independent developers to Topic Keywords: Optical Networks, Network construct components that will inter-operate with each Architectures, System design. other through well-defined interfaces. Past experience has lead to assumptions being made in the construction not explicitly taken into account by existing layers (such or operation of one layer's design that can lead to as networks, transport-systems or applications). In fact, incorrect behaviour when combined with another layer. quite the opposite situation occurs where existing layers There are numerous examples describing the problems are assumed to operate normally. caused when layers do not behave as the architects Outline of certain system parts expected. One might be the Section II describes our motivations for this work previous vendor recommendation (and common practice) including new optical networking technologies which for disabling UDP checksums, because the underlying change previous assumptions about the physical layer Ethernet FRC was considered sufficiently strong, led and the implications of the limits on the quantity of to data corruptions for UDP-based NFS and network power useable in optical networks. name (DNS) services [1]. Another example is the re- use of the 7-bit scrambler for data payloads [2], [3]. We present a study of the 8B/10B block-coding system The 7-bit scrambling of certain data payloads (inputs) of Gigabit Ethernet [6], the interaction between an (N,K) results in data that is (mis)identified by the SONET [4] block code, an optical physical layer, and data trans- system as belonging to the control channel rather than the ported using that block code in Section III. In Section IV information channel. Also, Tennenhouse [5] noted how we document our findings on the reasons behind the layered multiplexing had a significant negative impact interactions observed. upon an application's jitter. Further to our experiments with Gigabit Ethernet, in Section V we illustrate how the issues we identify It is our conjecture that this na¨ıve layering, while often have ramifications for systems with increasing physical considered a laudable property in computer communica- complexity. Alongside conclusions, Section VI states a tions networks, can lead to irreconcilable faults due to number of the directions that further work may take. differences in such fundamental measures as the number and nature of errors in a channel, a misunderstanding of II. MOTIVATIONS the underlying properties or needs of an overlaid layer. A. Optical Networks We concentrate upon the na¨ıve layering present when Current work in all areas of networking has led assumptions are made about the properties of protocol to increasingly complex architectures: our interest is layers below or above the one being designed. This is focused upon the field of optical networking, but this further exacerbated by layers evolving as new technolo- is also true in the wireless domain. Our exploration gies drive speed, availability, etc. Often such evolution is of the robustness of network systems is motivated by the increased demands of these new optical systems. Additionally, we are increasingly impacted by operator Wavelength Division Multiplexing (WDM), capable for practice. For example, researchers have observed that example of 160 wavelengths each operating at 10 Gbps up to 60% of faults in an ISP-grade network are due per channel, able to operate at over 5,000 km without to optical events [13]: defined as ones where it was regeneration [7], is a core technology in the current com- assumed errors results directly from operational faults munications network. However the timescale of change of in-service equipment. While the majority of these will for such long haul networks is very long: the same be catastrophic events (e.g., cable breaks), a discussion scale as deployment, involving from months to years of with the authors allows us to speculate that a non-trivial planning, installation and commissioning. To take advan- percentage of these events will be due to the issues of tage of capacity developments at the shorter timescales layer-interaction discussed in this paper. relevant to the local area network, as well as system B. The Power Problem and storage area networks, packet switching and burst switching techniques have seen significant investigation If all other variables are held constant an increase (e.g. [8], [9]). Optical systems such as [10] and [11] have in data rate will require a proportional increase in lead us to recognise that the need for higher data-rates transmitter power. A certain number of photons per bit and designs with larger numbers of optical components must be received to guarantee any given bit error rate are forcing us towards what traditionally have been (BER), even if no thermal noise is present. The arrival technical limits. of photons at the receiver is a Poisson process. If 20 photons must arrive per bit to ensure a BER of ¥§¦©¨ Further to the optical-switching domain, there have for a given receiver, doubling the bit rate means that been changes in the construction and needs of fibre- the time in which the 20 photons must arrive is halved. based computer networks. In deployments containing The number of photons sent per unit time must thus be longer runs of fibre using large numbers of splitters for doubled (doubling the transmission power) to maintain measurement and monitoring and active optical devices, the BER. the overall system loss may be greater than in today's The ramifications of this are that a future system point-to-point links and the receivers may have to cope operating at higher rates will either require twice the with much-lower optical powers. Increased fibre lengths power (a 3dB increase), be able to operate with 3dB used to deliver Ethernet services, e.g., Ethernet in the less power for the given information rate (equivalent first mile [12], along with a new generation of switched to the channel being noisier by that proportion) or a optical networks, are examples of this trend. compromise between these.