Junos Troubleshooting in the NOC 12.b
Lab Guide
Worldwide Education Services
1133 Innovation Way Sunnyvale, CA 94089 USA 408-745-2000 www.juniper.net
Course Number: EDU-JUN-JTNOC This document is produced by Juniper Networks, Inc. This document or any part thereof may not be reproduced or transmitted in any form under penalty of law, without the prior written permission of Juniper Networks Education Services. Juniper Networks, theJuniper Networks logo. Junos. NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Jnc. in the United States and other countries. All other trademarks, service marks, registered trademarks. or registered service marks are the propertyof their respective owners.
Junos Troubleshooting in the NOC Lab Guide, Revision 12.b Copyright© 2014Juniper Networks, Inc. All rights reserved. Printed in USA. Revision History: Revision 10.a-December 2010 Revision 11.a-June 2011 Revision 12.a-March 2013 Revision 12.b-January 2014 The information in this document is current as of the date listed above. The information in this document has been carefully verified and is believed to be accurate for software Release 12.2R2.5. Juniper Networks assumes no responsibilities for any inaccuracies that may appear in this document. In no event willJuniper Networks be liable for direct. indirect, special, exemplary, incidental, or consequential damages resulting from any defect or omission in this document, even if advised of the possibility of such damages.
Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. YEAR 2000 NOTICE Juniper Networks hardware and software products do not suffer from Year 2000 problems and hence are Year 2000 compliant. The Junos operating system has no known time-related limitations through the year 2038. However. the NTP application is known to have some difficulty in the year 2036. SOFTWARELICENSE The terms and conditions for using Juniper Networks software are described in the software license provided with the software. or to the extent applicable, in an agreement executed between you andJuniper Networks, orJuniper Networks agent. By usingJuniper Networks software, you indicate that you understand and agree to be bound by its license terms and conditions. Generally speaking, the software license restricts the manner in which you are permitted to use theJuniper Networks software, may contain prohibitions against certain uses, and may state conditions under which the license is automatically terminated. You should consult the software license for further details. Contents
Lab 1: The Troubleshooting Process ...... 1-1 Part 1: Troubleshooting Scenario 1 ...... 1·2 Part 2: Troubleshooting Scenario 2 ...... 1-4 Part 3: Troubleshooting Scenario 3 ...... 1-6
Lab 2: Identifying Hardware Components ...... 2-1 Part 1: Logging In Using the CU ...... 2-2 Part 2: Verifying Initial Device Configuration ...... 2-3 Part3: Identifying Hardware and Key Components ...... 2-7 Part 4: Utilizing Online Resources...... 2-9
Lab 3: Using Monitoring Tools and Establishing a Baseline ...... 3-1 Part 1: Viewing Alarms, Environmental Conditions, and Hardware Inventory ...... 3-2 Part 2: Establishing a Baseline ...... 3-9 Part 3: Locating Online Resources ...... 3-17
Lab 4: Monitoring Hardware and Environmental Conditions ...... 4-1 Part 1: Monitoring Memory and Storage ...... 4-2 Part 2: Viewing Boot and System Logs ...... 4-7 Part 3: Monitoring the Chassis and Environment ...... 4-13
Lab 5: Control Plane Monitoring and Troubleshooting ...... 5-1 Part 1: Examining User Processes and Daemons ...... 5-2 Part 2: Generating Core Files ...... 5-10 Part 3: Configuring Routing Protocols and Routing Tables ...... 5-13 Part 4: Configuring Bridging ...... 5-27
Lab 6: Monitoring and Troubleshooting Ethernet Interfaces ...... 6-1 Part 1: Performing Interface Troubleshooting ...... 6-2 Part 2: Performing Loopback Testing ...... 6-13 Part 3: Configuring Ethernet OAM ...... 6-19
Lab 7: Isolating and Troubleshooting PFE Issues ...... 7-1 Part 1: Examining the Forwarding Table ...... 7-2 Part 2: Configuring Load Balancing ...... 7-7 Part 3: Troubleshooting Firewall Filters ...... 7-11
Lab 8: Troubleshooting Routing Protocols ...... 8-1 Part 1: Modifying the Existing Configurations ...... 8-2 Part 2: Troubleshooting OSPF Adjacencies ...... •••••...... 8-3 Part 3: Troubleshooting BGP Sessions ...... 8-19 Part 4: Troubleshooting Routing Loops ...... •...... 8-34
Lab 9: Monitoring the Network ...... 9-1 Part 1: Modifying the Existing Configurations ...... 9-2 Part 2: Configuring and Monitoring SNMP ...... •...... ••.....•...... · 9-3 Part 3: Configuring and Monitoring RMON ...... 9·11 Part 4: Configuring and Monitoring In line JFlow ...... 9-19 Part 5: Configuring and Monitoring Inline Port Mirroring ...... •...... 9-23
www.juniper.net Contents • iii iv • Contents www.juniper.net Course Overview
This three-day course is designed to provide introductory troubleshooting skills for engineers in a network operations center (NOC) environment. Key topics within this course include troubleshooting methodology, troubleshooting tools, hardware monitoring and troubleshooting, interface monitoring and troubleshooting, troubleshooting the data plane and control plane on devices running the Junos operating system, staging and acceptance methodology, troubleshooting routing protocols, monitoring the network, and working with JTAC. This course is based on Ju nos operating system Release 12.2R2.5. Objectives
After successfully completing this course, you should be able to: Reduce the time it takes to identify and isolate the root cause of an issue impacting your network. Gain familiarity with Ju nos products as they pertain to troubleshooting. Become familiar with online resources valuable to Junos troubleshooting. Gain familiarity with Junos tools used in troubleshooting.
Identify and isolate hardware issues. Troubleshoot problems with the control plane. Troubleshoot problems with interfaces and other data plane components. Describe the staging and acceptance methodology. Troubleshoot routing protocols. Describe how to monitor your network with SNMP, RMON, JFlow, and port mirroring. Become familiar with JTAC procedures. Intended Audience
The course content is aimed at operators of devices running the Junos OS in a NOC environment. These operators include network engineers, administrators, support personnel, and reseller support personnel. Course Level
Junos Troubleshooting in the NOC is an introductory-level course. Prerequisites
Students should have basic networking knowledge and an understanding of the Open Systems Interconnection (OSI) reference model and the TCP/IP protocol suite. Students should also attend the Introduction to the Junos Operating System (IJOS) course and the Junos Routing Essentials (JRE) course, or have equivalent experience prior to attending this class.
www.juniper.net Course Overview • v Course Agenda
Day1 Chapter 1: Course Introduction Chapter 2: Troubleshooting as a Process The Troubleshooting Process Lab Chapter 3: Junos Product Families
Identifying Hardware Components Lab Chapter 4: Troubleshooting Toolkit Monitoring Tools and Establishing a Baseline Lab
Day2 Chapter 5: Hardware and Environmental Conditions Monitoring Hardware and Environmental Conditions Lab Chapter 6: Control Plane Control Plane Monitoring and Troubleshooting Lab Chapter 7: Data Plane: Interfaces Monitoring and Troubleshooting Ethernet Interfaces Lab Chapter 8: Data Plane: Other Components Isolating and Troubleshooting PFE Issues Lab Day3 Chapter 9: Staging and Acceptance Testing Chapter 10: Troubleshooting Routing Protocols Troubleshooting Routing Protocols Lab Chapter 11: High Availability Chapter 12: Network Monitoring Monitoring the Network Lab Chapter 13: JTAC Procedures Appendix A: Interface Troubleshooting
vi • Course Agenda www.juniper.net Document Conventions
CLI and GUI Text Frequently throughout this course, we refer to text that appears in a command-line interface (CU) or a graphical user interface (GUI). To make the language of these documents easier to read, we distinguish GUI and CU text from chapter text according to the following table.
Style Description Usage Example
Franklin Gothic Normal text. Most of what you read in the Lab Guide and Student Guide.
Courier New Console text: commit complete Screen captures
Noncommand-related Exiting configuration mode syntax GUI text elements: Select File > Open, and then click Menu names Configuration.conf in the Filename text box. Text field entry
Input Text Versus Output Text You will also frequently see cases where you must enter input text yourself. Often these instances will be shown in the context of where you must enter them. We use bold style to distinguish text that is input versus text that is simply displayed.
Style Description Usage Example
Normal CLI No distinguishing variant. Physical interface:fxpO, Enabled Normal GUI View configuration history by clicking Configuration > Histor�
CLI Input Text that you must enter. lab@San_Jose> show route GUI Input Select File > Save, and type config. ini in the Filename field.
Defined and Undefined Syntax Variables Finally, this course distinguishes between regular text and syntax variables, and it also distinguishes between syntax variables where the value is already assigned (defined variables) and syntax variables where you must assign the value (undefined variables). Note that these styles can be combined with the input style as well.
Style Description Usage Example
CLI Variable Text where variable value is policy my-peers already assigned. GUI Variable Click my-peers in the dialog. CLI Undefined Text where the variable's value Type set policy po.licy-name. is the user's discretion and text where the variable's value as ping 10.0.� GUI Undefined shown in the lab guide might Select File > Save, and type differ from the value the user £il.enamein the Filename field. must input.
www.juniper.net Document Conventions • vii Additional Information
Education Services Offerings You can obtain information on the latest Education Services offerings, course dates, and class locations from the World Wide Web by pointing your Web browser to: http://www.juniper.net;training/education/. About This Publication The Junos Troubleshootingin the NOC Lab Guide was developed and tested using software Release 12.2R2.5. Previous and later versions of software might behave differently so you should always consult the documentation and release notes for the version of code you are running before reporting errors. This document is written and maintained by the Juniper Networks Education Services development team. Please send questions and suggestions for improvement to [email protected]. Technical Publications You can print technical manuals and release notes directly from the Internet in a variety of formats: Go to http://www.juniper.net;techpubs/. Locate the specific software or hardware release and title you need, and choose the format in which you want to view or print the document. Documentation sets and CDs are available through your local Juniper Networks sales office or account representative. Juniper Networks Support For technical support, contact Juniper Networks at http://www.juniper.net;customers/supportj, or at 1-888-314-JTAC (within the United States) or 408-745-2121 (from outside the United States).
viii • Additional Information www.juniper.net Lab The Troubleshooting Process
Overview
In this lab, you will focus on the process of troubleshooting. By completing this lab, you will performthe following tasks: Define success for a given problem. Create a flowchart to help isolate the root cause of a problem. Identify possible ways to achieve success. Evaluate the potential impact associated with implementing a solution in a production environment.
www.juniper.net The Troubleshooting Process • Lab 1-1 Junos Troubleshooting in the NOC Part 1: Troubleshooting Scenario 1
In this lab part, you use the troubleshooting process to create an efficient approach to help you identify the root cause of a specific problem. In this scenario, a random card is drawn from a standard 52 card deck. Your objective is to form an approach that will isolate the selected card. Your instructor or lab partner will provide you with the true or false outcome of your decision points. Step 1.1 Define success: Accurately identify the card drawn from a standard deck of cards.
Step 1.2 Isolate the issue:
Draw a simple flowchart of how you might identify the selected card. Remember each decision point should have a true or false outcome and should eliminate potential possibilities regardless of the outcome. Remember, this is just one possible flowchart. The "face value is odd" decision and the "face value is greater than" decision could have been modified or reordered without greatly impacting the number of steps required to identify a solution.
Lab 1-2 • The Troubleshooting Process www.juniper.net Ju nos Troubleshooting in the NOC
standard 52 card deck
I have a card
All other possibilities eliminated.
www.juniper.net The Troubleshooting Process • Lab 1-3 Junos Troubleshooting in the NOC Part 2: Troubleshooting Scenario 2
In this lab part, you use the troubleshooting process to create an efficient approach to help you identify the root cause of a specific problem. In this scenario, you have nine balls that appear equal in size, shape, and color. However, one of the balls is very slightly overweight-Not perceivable to the human eye or feel. That is OK, a scale is provided for your use. Your objective is to identify the ball that is heavier than the rest. Your instructor or lab partner will provide you with the True/False outcome of your decision points. Step 2.1 Define success: Identify the heavy ball.
Step 2.2 Isolate the issue:
Draw a simple flowchart of how you might identifythe heavy ball. Remember each decision point should have a True or False outcome and should eliminate potential possibilities regardless of the outcome. You can take a couple of different approaches to this problem. One approach guarantees the outcome in two steps, while the other might take three steps, but also has the possibility of finding the problem in one. The second approach is common to many network troubleshooting issues, where a quick test can provide the necessary isolation, or some limited follow-up might be required. When troubleshooting, it is not always easy to say if one approach is better than the other. If the outcome is similar, in a similar number of steps, with similar disruption to the network, then it is an acceptable approach.
Lab 1-4 • The Troubleshooting Process www.juniper.net Junos Troubleshooting in the NOC
Nine balls Nine balls and a scale and a scale
Put 3 balls on each side 4 balls ofthe scale on each side of the scale
The heavy ball has been identified
ApproachA ApproachB
www.juniper.net The Troubleshooting Process • Lab 1-5 Junos Troubleshooting in the NOC Part 3: Troubleshooting Scenario 3
In this lab part, you use the troubleshooting process to create an efficient approach to help identify the root cause of a specific problem. Step 3.1 Define success:
Step 3.2 Isolate the issue: Identify the elements required to achieve success.
Identifypossible methods to help isolate the element preventing success.
What impact could testing for the root cause have in a production environment? How might you mitigate these risks?
Draw a simple flowchart of how you might narrow down the focus. Remember each decision point should have a True or False outcome and should bring you closer to understanding the root cause of a problem.
Lab 1-6 • The Troubleshooting Process www.juniper.net Junos Troubleshooting in the NOC Step 3.3 Identify a solution: Identify possible ways to achieve success. (Can you think of more than one way?)
Can you think of a short-term solution that would return a production network to operating status while waiting to implement a permanent long-term solution?
Step 3.4 Implement the solution: Can you think of any potential impact risks associated with implementing the identified solution in a production environment? How might you mitigate those risks?
0 Tell your instructor that you have completed this lab.
www.juniper.net The Troubleshooting Process • Lab 1-7 Junos Troubleshooting in the NOC
Management Network Diagram
Management Addressing
mxA-1 mxD-1 mxA-2 mxD-2 mxB-1 mxB-2 Server mxC-1 Gateway mxC-2 Tenn Server ____
Note: Your instructor will provideaddress and accessinformation_
The Troubleshooting Process Lab What Is the Card?
Lab 1-8 • The Troubleshooting Process www.juniper.net Junes Troubleshooting in the NOC
The Troubleshooting Process Lab Find the Heavy Ball
/ •••••••••
The Troubleshooting Process Lab Client Unable to Connect to the HTIPServer
-.... � � \ ISPB J
iiD
www.juniper.net The Troubleshooting Process • Lab 1-9 Junos Troubleshooting in the NOC
The Troubleshooting Process Lab Two Directly Connected Devices Are Unable to Communicate
• Was working. No known changes.
Lab 1-10 • The Troubleshooting Process www.juniper.net Lab Identifying Hardware Components
Overview
In this lab, you will log in to your router, identify the installed hardware,.determine which hardware components are field-replaceable units (FRUs), and research how to replace faulty hardware components. By completing this lab, you will perform the following tasks: Verify proper operation of network interfaces. Identifyhardware. Determine what is and what is not an FRU. Research how to replace a faulty hardware component.
www.juniper.net Identifying Hardware Components • Lab 2-1 Junos Troubleshooting in the NOC Part 1: LoggingIn Using the CU
In this lab part, you log in to your router and verifythat interfaces are operational and that the base topology used in subsequent labs is in place. Note The instructor will tell you the nature of your access and will provide you with the necessary details to access your assigned device.
Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address.
Question: What is the management address assigned to your device?
Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor.
Step 1.2 Access the CLI for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program:
Protocol:
Hostname:
Port:
D Show quick connect on startup 0 Save session 0 Open in a tab I Connect I I Cancel
Lab 2-2 • Identifying Hardware Components www.juniper.net Junes Troubleshooting in the NOC Step 1.3 Log in to your device with the username l.ab and the password l.ab123. Note that both the name and password are case-sensitive. mxA-1 (ttyuO) login: l.ab Password:
--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/reset. configcommand. After the configuration has been loaded, commit the changes and exit to operational mode before proceeding to Part 2. lab@mxA-1> configure Entering configuration mode
[edit] lab@mxA-1# load override jtnoc/reset.config load complete
[edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1>
Part 2: Verifying Initial Device Configuration
In this lab part, you verify the initial configuration of your assigned device. Step 2.1 Issue the show interfaces terse I no-more command. lab@mxA-1> show interfaces terse I no-more Interface Admin Link Proto Local Remote lc-0/0/0 up up lc-0/0/0.32769 up up vpls pfe-0/0/0 up up pfe-0/0/0.16383 up up inet inet6 pfh-0/0/0 up up pfh-0/0/0.16383 up up inet xe-0/0/0 up down xe-0/0/1 up down xe-0/0/2 up down xe-0/0/3 up down ge-1/0/0 up up ge-1/0/0.141 up up inet 172.18.1.2/30
www.juniper.net Identifying Hardware Components • Lab 2-3 Junos Troubleshooting in the NOC multiservice ge-1/0/0.32767 up up multiservice ge-1/0/1 up up ge-1/0/1.141 up up inet 172.18.2.2/30 multiservice ge-1/0/1.32767 up up multiservice ge-1/0/2 up up ge-1/0/2.141 up up inet 172.18.5.1/30 multiservice ge-1/0/2.32767 up up multiservice ge-1/0/3 up up ge-1/0/3 .141 up up inet 192. 168 .11.1/24 multiservice ge-1/0/3.32767 up up multiservice ge-1/0/4 up up ge-1/0/5 up up ge-1/0/6 up up ge-1/0/7 up up ge-1/0/8 up up ge-1/0/9 up up ge-1/1/0 up up ge-1/1/1 up down ge-1/1/2 up up ge-1/1/3 up down ge-1/1/4 up up ge-1/1/5 up up ge-1/1/6 up up ge-1/1/7 up up ge-1/1/8 up up ge-1/1/9 up up cbpO up up demuxO up up dsc up up emO up up emO.O up up inet 10.0.0.4/8 128.0.0.1/2 128.0.0.4/2 inet6 fe80: :200:ff:fe00:4/64 fecO: :a:0:0:4/64 tnp Ox4 eml up down fxpO up up fxpO.O up up inet 10.210.15.1/27 gre up up ipip up up irb up up loO up up loO.O up up inet 192.168.31.1 --> 0/0 lo0.16384 up up inet 127.0.0.1 --> 0/0 lo0.16385 up up inet lsi up up meO up up meO.O up up mtun up up pimd up up
Lab 2-4 • Identifying Hardware Components www.juniper.net Junes Troubleshootingin the NOC pime up up pipO up up ppO up up tap up up
Question: What are the Admin and Link states for the interfaces listed in the lab diagram for your device?
Answer: Interfaces shown as configured and functioning in the diagram for lab 2 should show Admin and Link status of up as shown in the previous output. If other configured interfaces show a down status, notifyyour instructor.
Question: What are the IP addresses configured on the interfaces listed in the lab diagram for your device?
Answer: The IP addresses should match what is displayed in the lab diagram for lab 2 for your assigned device.
Step 2.2 Next, ping the virtual router attached to your device to ensure connectivity exists. lab@mxA-1> ping 1oca1-VR-Address count 5 PING 192.168.11.2 (192.168.11.2): 56 data bytes 64 bytes from 192.168.11.2: icmp_seq= O ttl= 64 time=0.530 ms 64 bytes from 192.168.11.2: icmp_seq= l ttl= 64 time=0.823 ms 64 bytes from 192.168.11.2: icmp_seq= 2 ttl=64 time=3.424 ms 64 bytes from 192.168.11.2: icmp seq=3 ttl= 64 time= 0.484 ms 64 bytes from 192.168.11.2: icmp_seq=4 ttl= 64 time= 0.483 ms
--- 192.168.11.2 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.483/l.149/3.424/1.145 ms
www.juniper.net Identifying Hardware Components • Lab 2-5 Junos Troubleshooting in the NOC Question: Are you able to ping the virtual router attached to your device?
Answer: As indicated by the output, you should be able to ping the attached virtual router. Notify your instructor if you are unable to successfully ping the virtual router.
Step 2.3 Next, ping the address associated with the remote team's ge-1/0/2 interface. lab@mxA-1> ping rem.ote-team-�e-1/0/2-address count 5 PING 172.18.5.2 (172.18.5.2): 56 data bytes 64 bytes from 172.18.5.2: icmp- seq=O ttl=64 time=l.048 ms 64 bytes from 172.18.5.2: icmp_seq=l ttl=64 time= l2.862 ms 64 bytes from 172.18.5.2: icmp_seq=2 ttl=64 time=0.438 ms 64 bytes from 172.18.5.2: icmp_seq=3 ttl=64 time=0.478 ms 64 bytes from 172.18.5.2: icmp- seq=4 ttl=64 time=0.495 ms
--- 172.18.5.2 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.438/3.064/12.862/4.904 ms
Question: Can you ping the remote team's router?
Answer: As indicated by the output, you should be able to ping the remote team's router. Notifyyour instructor if you are unable to successfully ping the remote team's router.
Lab 2-6 • Identifying Hardware Components www.juniper.net Junos Troubleshooting in the NOC Part 3: Identifying Hardware and Key Components
In this lab part, you identify hardware and key components in your router. Step 3.1
Issue the show chassis hardware command. lab@mxA-1> show chassis hardware I no-more Hardware inventory: Item Version Part number Serial number Description Chassis D4889 MX80 Midplane REV 06 7ll-031594 YK8973 MX80 PEM O Rev 03 740-028288 UG00866 AC Power Entry Module Routing Engine BUILT IN BUILT IN Routing Engine TFEB O BUILT IN BUILT IN Forwarding Engine Processor QXM 0 REV 05 7ll-028408 YK6600 MPC QXM FPC O BUILT IN BUILT IN MPC BUILTIN MIC 0 BUILT IN BUILT IN 4x lOGE XFP PIC 0 BUILT IN BUILT IN 4x lOGE XFP FPC 1 BUILT IN BUILT IN MPC BUILTIN MIC O REV 22 750-028392 YK7479 3D 20x lGE(LAN) SFP PIC 0 BUILT IN BUILT IN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013lll All6746 SFP-T Xcvr 1 REV 02 740-013lll All4818 SFP-T Xcvr 2 REV 02 740-013lll All6860 SFP-T Xcvr 3 REV 02 740-013lll All6971 SFP-T Xcvr 4 REV 02 740-013lll All6814 SFP-T Xcvr 5 REV 02 740-013lll All6816 SFP-T Xcvr 6 REV 02 740-013lll All6260 SFP-T Xcvr 7 REV 02 740-013lll All5260 SFP-T Xcvr 8 REV 02 740-013lll All6813 SFP-T Xcvr 9 REV 02 740-013lll All6719 SFP-T PIC 1 BUILT IN BUILT IN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013lll All5863 SFP-T Xcvr 1 REV 02 740-013lll All6368 SFP-T Xcvr 2 REV 02 740-0131ll A071203 SFP-T REV 740-0131ll All6370 SFP-T Xcvr 34 02 Xcvr REV 02 740-013lll A12ll93 SFP-T Xcvr 5 REV 02 740-0131ll A12ll88 SFP-T Xcvr 6 REV 02 740-013lll All6729 SFP-T Xcvr 7 REV 02 740-0131ll All6891 SFP-T Xcvr 8 REV 02 740-0131ll All4787 SFP-T Xcvr 9 REV 02 740-0131ll All6696 SFP-T Fan Tray Fan Tray
www.juniper.net Identifying Hardware Components • Lab 2- 7 Junos Troubleshooting in the NOC Question: What is the chassis serial number of your assigned device?
Answer: As shown in the output, the Chassis field contains the chassis serial number. The chassis serial number for the MX Series device in the previous output is 04889.
Question: Can the MIC located in the MPC identified as FPC O be replaced without replacing the chassis? Explain your answer.
Answer: No. MIC 0, located in FPC 0, is a built-in MIC. You cannot replace it unless you replace the chassis.
Question: Can the MIC located in the MPC identified as FPC 1 be replaced without replacing the chassis? Explain your answer.
Answer: Yes. The MIC located in FPC 1, is not built-in to the chassis. You can replace it without replacing the chassis.
Question: If the fan tray in the previous output needs replacement, what information from the output is required?
Answer: Fan trays do not have serial numbers. To replace a fan tray only the part description and chassis type is needed.
Lab 2-8 • Identifying Hardware Components www.juniper.net Junos Troubleshooting in the NOC Part 4: Utilizing Online Resources
In this lab part, you find and use online resources to learn how to identifyand replace FRUs in your router. Note As you work through this lab part, you are encouraged to look beyond the questions asked in the lab to familiarize yourself with the available online resources. You are able to use these resources in future labs and later in your own network environment.
Step 4.1
Issue the show chassis hardware I no-more command. lab@mxA-1> show chassis hardware I no-more Hardware inventory: Item Version Part number Serial number Description Chassis E3342 MX80 Midplane REV 07 711-031594 YW2075 MX80 PEM O Rev 04 740-028288 UH02222 AC Power Entry Module Routing Engine BUILTIN BUILT IN Routing Engine TFEB O BUILTIN BUILT IN Forwarding Engine Processor QXM 0 REV 05 711-028408 YW3088 MPC QXM FPC O BUILT IN BUILT IN MPC BUILTIN MIC 0 BUILT IN BUILTIN 4x lOGE XFP PIC 0 BUILTIN BUILTIN 4x !OGE XFP FPC 1 BUILTIN BUILTIN MPC BUILTIN MIC O REV 24 750-028392 YX0925 3D 20x lGE(LAN) SFP PIC 0 BUILT IN BUILT IN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013111 A495782 SFP-T Xcvr 1 REV 02 740-013111 A494172 SFP-T Xcvr 2 REV 02 740-013111 A452847 SFP-T Xcvr 3 REV 02 740-013111 A495298 SFP-T Xcvr 4 REV 02 740-013111 A438159 SFP-T Xcvr 5 REV 02 740-013111 A494218 SFP-T Xcvr 6 REV 02 740-013111 A464011 SFP-T Xcvr 7 REV 02 740-013111 A495540 SFP-T Xcvr 8 REV 02 740-013111 A434868 SFP-T Xcvr 9 REV 02 740-013111 A433506 SFP-T PIC 1 BUILT IN BUILTIN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013111 A496017 SFP-T Xcvr 1 REV 02 740-013111 A464017 SFP-T Xcvr 2 REV 02 740-013111 A494183 SFP-T Xcvr 3 REV 02 740-013111 A494235 SFP-T Xcvr 4 REV 02 740-013111 A494159 SFP-T Xcvr 5 REV 02 740-013111 A382076 SFP-T Xcvr 6 REV 02 740-013111 A434336 SFP-T Xcvr 7 REV 02 740-013111 A494406 SFP-T Xcvr 8 REV 02 740-013111 A494744 SFP-T Xcvr 9 REV 02 740-013111 A494074 SFP-T Fan Tray Fan Tray
www.juniper.net Identifying Hardware Components • Lab 2-9 Junos Troubleshooting in the NOC Step 4.2 Open a Web browser on your local PC.
From the open Web browser, navigate to http: I /www.juniper.net/ techpubs/, find the MX80 FRU documentation, and answer the following questions.
Technical Documentation
Products by Categor; + /..pp!1caticJnA.rceler:�ilon + Pac.KetTr.1Mport + Content and r,1ect1a + RorJtmg + !d>.:ntity and Pl)lir.:/Control + Security + Junos:Platform + Sw:tchirn;i + MoOdiiY + Time Synchroniz:ation + Net#oH;r.1anagement + Wireless
Products by Family ACX Series r-Ji?LScreen Remote Soft,vare t-JetSueen Se1ies A..'( :::_;�ne;3. t'-JMC-RX ax Se:ri8s C Series NSM C!!-'SE:!i€S �JSf.lXcress �A Sene5 Ody-SS!?)°;'../.:f.9SS Client DXS-:ncs oct,�Si::J As:t'f:SS Client (F!PS Edition: EX Series iCSeri".'S PTXSeries If.JPSene:.; OFX Series IMS i"faA S�rv�r R(JUTe 1nsight tt�n3ger ISG Seri>::s SA.Seri(i:_:: J Series $BR.Seri;,:s. - Sonw:irn JCS1200 8creenOS Juno:;Content Portfolio (M-::dia SOX300 F!o•N SO!lltitJn} Service A.utornaUon Junos OS SRC Series Junos Pur:;e SRX Sene:S. Junos Sr;ope Junv1 SDK SSG S::rH?'> Jun0:3 Snaps:hot;ctministr.Jtor STF�i'.-18,�nes .!unos Sp.)ce T Sen es June.bP�i!?ff: Jur.Js\/ ,.t.ppEngine TC.:1.Seri1-:s U-J:%m::� VG'NSenes: VSE S"::ries UAG Series Junos Pulse VXASene� G.�tf:i"W:::fi Vitrnless LAH Mohiteil�xt Br<.,adband Gi:WNaf v,.,;x ano ·/ht..CSerie$ "i'.lJt;-;;·� 1/•,rx GUS (.MXSe�i�]
Products by Name ABC DE F-G H IJ KLM rJ OP QR S TUVV/XYZ
MX series 30 Universal Edge Routers
Hardware Documenrati-0n Sofr,•.1are Documenration !nst.3!1ing. planning. safetr and Gettng started. maintenance, troubleshooting. tiout:!eshooting, 3nd f83tures.
• MX5.J.1X10. UX40 ..3n6 !-l�_.30 • Juno:; 1�.3 UnNers �I.. Edg� Router u,j • Junos 1:!1 • ii1X2.!0�D .. Unr,1er�;.. :itEdg>: Rout.::ir • Junos 11.4 • :,.ix,-:.so.. muniver�;;. .. :ilEct;>: Hout.::ir • Junos 11.:! • !JX.960 30 Univers.3!Ed,:;-:t Router • Junos 11.1
Lab 2-10 • Identifying Hardware Components www.juniper.net Junos Troubleshooting in the NOC
MX5, MX10, MX40, and MXSO 3D Universal Edge Router
MX Series H3rctw3re & SoftwareDccurnentaticn Home
Overview Components Planning Safety Installation Ma;nre�ubleshoot;ng
MaintainingComponents Replac;ng Components�
Tool:, and Parts ReQu11i.HJ trJ Uctint:;;m \1X5, R.e:;1:1c;ng an t.!Xt:i. h!X10. MJ40. 81h1 IX\80 Air fi1X10. LlX40. ::indMXBO Houters Filter Rr.11Jt1n{::klavitenam:e Pri;tedurns f•)r U:X5. Reoiaarig Jn t1X5, r.1:00, MA.40,ar;d U.�80 Fan MX10. MX40. and MXSO F(outNs Tra,- rAaintaimng the r.1X5. r.1X10. UX.:O, .anCMXBO Air Replacin,;;:3n UX5, UX10, MX40. :andMXBO MIC Filt-:r Ht>plac::19 a Cable on an MX5, fJX:iD. M)(40, and r.-l3lntainingth8 MX5, MX10, MX:!O.anc M>(SO r.1xso t.nc F.rn Tr:Jy F Question: What are some of the FRUs found inside your assigned device? Answer: In the previous output and online documentation, the following components are FRUs: PEM 0, MIC O located in FPC 1, all the SFPs and XFPs, and the fan tray. Note lf time allows, research information for your own network devices and familiarize yourself with the hardware and available FR Us. www.juniper.net Identifying Hardware Components • Lab 2-11 Junos Troubleshooting in the NOC Step 4.3 Return to the open terminal session with your assigned device. From the open terminal session with your assigned device, log out of your assigned device. lab@mxA-1> exit mxA-1 (ttyuO) login: 0 Tell your instructor that you have completed this lab. Management Network Diagram .,.,.,,.fxpO .-®/ (On all studentdevices) mxA-1 .,.,.,,. .,.,.,,. Management Network .,,(.Tenninal :'\��;-® '- Connections mxA-2 Server\ \.'- \ \. \ \.' \ , \. Management Addressing \ \.' � \ \. ' mx0-2� mxD-1 \ \. mxD-2 \ vr-device mxB-2 Server mxC-1 Gateway mxC-2 Term Server ----- Note:Your instructorwill provideaddress and accessinfonnation. Lab 2-12 • Identifying Hardware Components www.juniper.net Junos Troubleshooting in the NOC Identifying Hardware Components Lab: Pod A Network Diagram Internet � � ( ISPA) R1 R1 (:;_, (.2) ',_;;z. 2) 0) 6 0 ;'; 0 "",i, 0 mxA-1 ge-1/0/2.141 (.1) 172.18.5.0/30 (.2) ge-1/0/2.141 mxA-2 loO: 192.168.31.1 loO: 192.168.32. 1 ge- /0/3-141 192.168.12.0/30(.l) 1 (.2) VirtualRouters vr2-2 Identifying Hardware Components Lab: Pod B Network Diagram Internet _/--� � r ISPA ' ( ISPB \ R1 \ R1 (.2) '�;z __,/ (.2) 0 0 ('") m 0''"' 6 �� 0 � � t "",i, 0 0 � rnxB-1 rnxB-2 ge-1/0/2.142 (.1) 172.18.5.0/30 (.2) ge-1/0/2.142 loO: 192.168.33.1 loO: 192.168.34.1 ge-1/0/3-141 (. ge-1/0/3·141 l)192.168.13.Q/30 192.168.14.Q/30(.l) (.2) (.2) vrl-1 VirtualRouters vr2-2 Wod:i;;;i:Education Ser.tlces wwwJunlpe,.net �unm ""' \.£..,iJL..._ � www.juniper.net Identifying Hardware Components • Lab 2-13 Junos Troubleshooting in the NOC Identifying Hardware Components Lab: Pod C Network Diagram Internet R1 r:;�/ R1 (-2) .:z=----____/ f'.9 (2) 0 a, '- 0 '- 0s: "" 0 (.1) ...... ------, mxC-1 mxC-2 ge-1/0/2143 {.1) 172.18.5.0/30 (.2) ge-1/0/2.143 loO: 192.168.35.1 loO: 192.168.36.1 ge 1 0/3 141 1 1' ge-1/0/3-141 - / - '· \92168.15.0/30 192.168.16.0/3o'- (.2) (.2) vrl-1 Virtual Routers vr2-2 Identifying Hardware Components Lab: Pod D Network Diagram Internet R1 R1 (.2) (.2) m '- 0 0 i ""' s: (1) ,...... ______, mx0-1 ge-1/0/2.144 {.1) 172.18.5.0/30 (.2) ge-1/0/2.144 mx0-2 loO: 192.168.37 .1 loO: 192.168.38.1 ge-1/0/3-141 (.li . l-1l ge-1/0/3-141 192168. 17. 0 /30 192.168.18.0/3a --�---(2) . (.2) vrl-1 Virtual Routers vr2-2 Lab 2-14 • Identifying Hardware Components www.juniper.net Lab Using Monitoring Tools and Establishing a Baseline Overview This lab demonstrates the monitoring and troubleshooting tools of the Ju nos operating system. By completing this lab, you will perform the following tasks: Monitor the health of the router and troubleshoot issues. Configure and identify baseline components within the router. Use online resources to troubleshoot issues. www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-1 Junos Troubleshooting in the NOC Part 1: Viewing Alarms, Environmental Conditions, and Hardware Inventory In this lab part, you use commands that will allow you to view alarms, environmental conditions, and hardware Inventory on your router. Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address. Question: What is the management address assigned to your device? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CLI for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Protocol: Telnet r=-·----··--····-········! v! Port: D Show quick connect on startup 52]Save session 52]Open in a tab Connect J [ Cancel Step 1.3 Log in to your device with the username l.ab and a password of l.abl.23. Note that both the name and password are case-sensitive. Lab 3-2 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC mxA-1 (ttyuO) login: lab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/1.ab3-start. con£ig command. After the configuration has been loaded, commit the changes and exit to operational mode before moving on to the next step. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# load override jtnoc/lab3-start.config load complete [edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 1.5 Issue the show interfaces terse ge* I match inet I except down command. lab@mxA-1> show interfaces terse ge* I match inet I except down ge-1/0/0.141 up up inet 172.18 .1. 2/30 ge-1/0/1.141 up up inet 172.18.2.2/30 ge-1/0/2.141 up up inet 172.18.5.1/30 ge-1/0/3 .141 up up inet 192.168.11.1/24 Question: Which interfaces are present in the output? Answer: Interfaces ge-1/0/0, ge-1/0/1, ge-1/0/2 and ge-1/0/3 are present in the output. Step 1.6 Issue the show chassis craft-interface command. lab@mxA-1> show chassis craft-interface Front Panel System LEDs: www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-3 Junes Troubleshooting in the NOC Routing Engine OK * Fail Front Panel Alarm Indicators: Red LED Yellow LED Major relay Minor relay PS Status: PS O 1 Red Green * Fan Tray Status: FT O Red Green * Question: What is the status of the routing engine? Answer: Although your results might vary, the routing engine section displays an asterisk in the OK field. If the asterisk is not in the OK field, please inform your instructor as this can indicate a hardware problem with your device. Question: Are any alarms displayed in this output? Answer: No. The Front Panel Alarm Indicators section does not display asterisks next to any of the alarm indicators are displayed. Step 1.7 Issue the show pfe statistics error command. lab@mxA-1> show pfe statistics error Slot O Lab 3-4 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junes Troubleshooting in the NOC HSL2 Errors: ***** No errors on this PFE ***** Question: Are any errors present on this PFE? Answer: No errors should exist. If errors are present please inform your instructor. Step 1.8 Issue the show log messages I match error command and answer the following questions. Note The messages might vary depending on your device. lab@mxA-1> show log messages I match error Aug 31 10:44:16 cfmd[l212]: junos_dfw_session reply_get:1138 Error "session" is NULL. Aug 31 10:47:08 alarmd[l050]: shutting down craftd connection: craftd ipc pipe read error Aug 31 10:47:12 12cp[l084]: junos_dfw_session connect:984 Error "select" returns timeout event for session connection. Aug 31 10:47:15 dfcd[l083]: junos_dfw_session_connect:1005 Error "recv" call with MSG PEEK flag returns error status (54). Aug 31 10:53:11 craftd[l099]: craftd_user_conn_shutdown: socket 13, errno = 0 Aug 31 10:53:11 cfmd[l087]: junos_dfw_trans_purge:1377 Error "session" is not allocated. Sep 13 17:36:54 /kernel: emO.O iff_ifarequest (DELETE) addr: 128.0.0.6 err 22 Sep 13 17:36:54 /kernel: emO.O iff ifarequest (DELETE) addr: 10.0.0.6 err 22 Sep 13 17:36:58 12cp[ll45]: junos_dfw_session connect:1005 Error "recv" call with MSG PEEK flag returns error status (61). Sep 13 17:36:59 dfcd[ll44]: junos_dfw_session_connect:1005 Error "recv" call with MSG PEEK flag returns error status (61). Sep 15 22:02:38 mxA-1 login: LOGIN PAM AUTHENTICATION ERROR: PAM authentication error for user lab Sep 15 22:08:11 mxA-1 craftd[l048]: craftd_user_conn shutdown: socket 8, errno = 0 Sep 15 22:08:12 mxA-1 cfmd[ll48]: dfwdlib full send:127 Complete send can not be done. Error Broken pipe Sep 15 22:08:12 mxA-1 cfmd[ll48]: junos_dfw_trans send:1104 Error in completing transaction from client 4 with user context O Sep 15 22:16:38 mxA-1 chassisd[lll7]: CHASSISD_I2C READ_ERROR: pca9548 disable all channels: read error for group Oat address Oxa, offset -1 www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-5 Junos Troubleshooting in the NOC Question: How does the match error option change the normal show log messages output? Answer: The match error option filters the output and includes lines that contain error in them only. Note Although in the previous output, messages contain the word "error" in them, these messages are not cause for concern. The previous output is typically seen during the boot process when communication between the routing engine and the router's other components are initially being established. Step 1.9 Issue the show chassis hardware I no-more command. lab@mxA-1> show chassis hardware I no-more Hardware inventory: Item Version Part number Serial number Description Chassis D4889 MXBO Midplane REV 06 711-031594 YK8973 MX80 PEM O Rev 03 740-028288 UG00866 AC Power Entry Module Routing Engine BUILT IN BUILTIN Routing Engine TFEB O BUILT IN BUILTIN Forwarding Engine Processor QXM 0 REV 05 711-028408 YK6600 MPC QXM FPC O BUILT IN BUILT IN MPC BUILTIN MIC 0 BUILT IN BUILTIN 4x lOGE XFP PIC 0 BUILT IN BUILT IN 4x lOGE XFP FPC 1 BUILT IN BUILT IN MPC BUILTIN MIC O REV 22 750-028392 YK7479 3D 20x lGE(LAN) SFP PIC 0 BUILT IN BUILT IN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013111 All6746 SFP-T Xcvr 1 REV 02 740-013111 All4818 SFP-T Xcvr 2 REV 02 740-013111 All68 60 SFP-T Xcvr 3 REV 02 740-013111 All6971 SFP-T Xcvr 4 REV 02 740-013111 All6814 SFP-T Xcvr 5 REV 02 740-013111 All6816 SFP-T Xcvr 6 REV 02 740-013111 All6260 SFP-T Xcvr 7 REV 02 740-013111 All5260 SFP-T Xcvr 8 REV 02 740-013111 All6813 SFP-T Xcvr 9 REV 02 740-013111 All6719 SFP-T PIC 1 BUILT IN BUILT IN lOx lGE(LAN) SFP Xcvr 0 REV 02 740-013111 All5863 SFP-T Xcvr 1 REV 02 740-013111 All6368 SFP-T Xcvr 2 REV 02 740-013111 A071203 SFP-T Lab 3-6 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC Xcvr 3 REV 02 740-013111 All6370 SFP-T Xcvr 4 REV 02 740-013111 Al21193 SFP-T Xcvr 5 REV 02 740-013111 Al21188 SFP-T Xcvr 6 REV 02 740-013111 All6729 SFP-T Xcvr 7 REV 02 740-013111 All6891 SFP-T Xcvr 8 REV 02 740-013111 All4787 SFP-T Xcvr 9 REV 02 740-013111 All6696 SFP-T Fan Tray Fan Tray Question: How many 10-Gigabit Ethernet ports does your assigned device have? Answer: Although your results might vary, the previous output shows that there are four 10-Gigabit Ethernet ports. Alternatively, you can use the show chassis fpc pie-status command to gather this information. Question: How many 1-Gigabit Ethernet ports does your assigned device have? Answer: Although your results might vary, the previous output shows that your device has twenty 1-Gigabit Ethernet ports. Alternatively, you can use the show chassis fpc pie-status command to gather this information. Question: Does your assigned device have redundant power supplies? Answer: The previous output shows that your there is just one power supply, PEM 1. The MX80 is capable of redundant power supplies but only one is necessary to power the chassis and all the other components. Step 1.10 Issue the show chassis fpc detail command. lab@mxA-1> show chassis fpc detail Slot O information: State Online Temperature 32 (C) Total CPU DRAM 1024 MB Total SRAM 403 MB Total SDRAM 1316 MB www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-7 Junos Troubleshooting in the NOC Start time 2013-01-31 01:13:22 UTC Uptime 8 days, 18 hours, 49 minutes, 43 seconds Slot l information: State Online Temperature 32 (C) Total CPU DRAM 1024 MB Total SRAM 403 MB Total SDRAM 1316 MB Start time 2013-01-31 01:13:22 UTC Uptime 8 days, 18 hours, 49 minutes, 43 seconds Question: How much DRAM, SRAM, and SDRAM exist within each FPC? Answer: 1024 MB of DRAM, 403 MB of SRAM, and 1316 MB of SDRAM exist within each FPC. Question: What are the current temperatures of both FPCs? Answer: Although your results might vary, the previous output shows that both FPCs are running at a temperature of 32 degrees Celsius. Step 1.11 Issue the show chassis tfeb command. lab@mxA-1> show chassis tfeb TFEB status: Slot O information: State Online Intake temperature 32 degrees c I 89 degrees F Exhaust temperature 50 degrees c I 122 degrees F CPU utilization 4 percent Interrupt utilization 0 percent Heap utilization 19 percent Buffer utilization 13 percent Total CPU DRAM 1024 MB Start time: 2013-01-31 01:13:22 UTC Uptime: 8 days, 18 hours, 51 minutes, 16 seconds Lab 3-8 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junes Troubleshooting in the NOC Question: How long has it been since the last time the TFEB was rebooted? Answer: Although your results might vary, the previous output shows that the TFEB booted 8 days, 18 hours, 51 minutes, and 16 seconds ago. Question: Does your assigned device have a redundant data plane hardware configuration? Answer: No. There is only one PFE, the TFEB, which means there is no data plane redundancy. Question: How much DRAM exists on the TFEB? Answer: There are 1024 MB of DRAM on the TFEB. Question: What are the current intake and exhaust temperatures? Answer: In the previous output, the intake temperature is 32 degrees Celsius and the exhaust temperature is 50 degrees Celsius. Your results might vary, but if they are significantly different than the previous output, please inform your instructor. Part 2: Establishing a Baseline In this lab part, you configure the components SNMP, NTP, and remote system logging, which are necessary to establish a baseline. Step 2.1 Issue the show configuration I no-more command and answer the following questions: lab@mxA-1> show configuration I no-more ## Last commit: 2013-02-08 18:32:42 UTC by lab version 12.2R2.5; system { www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-9 Junos Troubleshooting in the NOC host-name mxA-1; root-authentication encrypted-password "$1$KI99zGk6$MbYFuBbpLffu9tn2.sI711"; ## SECRET-DATA ssh-dsa "ssh-dss AAAAB3NzaClkc3MAAACBAMQrfP2bZyBXJ6PC7XXZ+MzErI8Jl6jah5L4/ 08BsfP2hC7EvRfNoX7MqbrtCX/9gUH9gChVuBCB+ERULMdgRvM5uGhC/ gs4UX+4dBbfBgKYYwgmisM8EoT25m7qI8ybpl2YZvHNznv08h7kr4kpYuQEpKvgsTdH/ Jle4Uqnjv7DAAAAFQDZaqA6QAgbW30/ zveaLCIDj6p0dwAAAIBliL+krWrXiD8NPpY+w4dWXEqaV3bnobzPC4eyxQKBUCOr80Q5YBlWXVBH x9elwBWZwjOSF4hLKHznExnLerVsMuTMA846RbQmSz62vM6kGM13HFonWeQvWiaOTDr78+rOEgWF 2KHBSixL5llmIDW8Gql9hJfD/Dr/ NKP97w3L0wAAAIEAr3FkWU8XbYytQYEKxsIN9PlUQlERXB3G40YwqF0484SlyKyYCfaz+yNsaAJu 2C8UebDIR3GieyNcOAKf3inCG8jQwjLvZskuZwrvlsz/xtcxSoAh9axJcdUfSJYMW/ g+mD26JK1Cliw5rwp2nH9kUrJxeI7IReDp4egNkM4il5o= [email protected]"; ## SECRET-DATA } login { user lab { uid 2000; class super-user; authentication { encrypted-password "$1$84J5Maes$cni5Hrazbd/IEHr/50oY30"; ## SECRET-DATA } services ftp; ssh; telnet; syslog { user * { any emergency; file messages { any notice; authorization info; file interactive-commands interactive-commands any; interfaces { ge-1/0/0 vlan-tagging; unit 141 { vlan-id 141; family inet { address 172.18.1.2/30; } ge-1/0/1 Lab 3-10 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC vlan-tagging; unit 141 { vlan-id 141; family inet { address 172.18.2.2/30; ge-1/0/2 vlan-tagging; unit 141 { vlan-id 141.; family inet { address 172.18.5.1/30; ge-1/0/3 vlan-tagging; unit 141 { vlan-id 141; family inet { address 192.168.11.1/24; fxpO { description "MGMT INTERFACE - DO NOT DELETE"; unit O { family inet { address 10.210.15.1./27; loO unit O { family inet { address 192.168.31.1/32; routing-options static { route 0.0.0.0/0 qualified-next-hop 172.18.1.1; qualified-next-hop 172.18.2.1 { preference 7; www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-11 Junos Troubleshooting in the NOC Question: Which interfaces are shown in the output? Answer: Interfaces are ge-1/0/0, ge-1/0/1, ge-1/0/2, ge-1/0/3, fxpO, and loO are present in the output. Question: Does your device automatically synchronize with an external NTP server? Answer: No. There is no NTP configuration in the previous output. Question: Does your assigned device send syslog information to a remote syslog server? Answer: No. The previous output shows that only local syslogging is present. Step 2.2 Enter configuration mode and navigate to the [edit system ntp] hierarchy level and configure an NTP server for your device. Refer to the lab diagram for the specific address of the NTP server. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit system ntp [edit system ntp] lab@mxA-1# set server X.X.X.X [edit system ntp] lab@mxA-1# Lab 3-12 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC Step 2.3 Navigate to the [edit system] hierarchy level and configure the time zone in relation to the physical location of your router. Don't forget that you can use the Tab key to auto-complete the country and city names. Activate the configuration by issuing the commitcommand. Note The exact location of your router is not needed. Any city within the same time zone will work. [edit system ntp] lab@mxA-1# up [edit system] lab@mxA-1# set time-zone America/Los_Angeles [edit system] lab@mxA-1# connnit comm.it complete [edit system] lab@mxA-1# Step 2.4 Issue the set date ntp command to synchronize your assigned device with the NTP server. [edit system] lab@mxA-1# run set date ntp 8 Feb 11:47:59 ntpdate[30494]: step time server 10.210.15.24 offset 0.000697 sec Step 2.5 Issue the run show ntp associations command. Note It might take a few minutes to completely synchronize with the NTP server. [edit system] lab@mxA-1# run show ntp associations remote ref.id st t when poll reach delay offset jitter *10.210.15.24 10.210.8.72 4 - 62 64 7 0.166 0.022 0.014 www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-13 Junos Troubleshooting in the NOC Question: What is the NTP synchronization status? Answer: The previous output the shows that there is an asterisk(*) in the far left.This asterisk indicates that the router was able to synchronize with the NTP server. Step 2.6 Issue the run show system uptime command. [edit system] lab@mxA-1# run show system uptime Current time: 2013-02-08 11:53:41 PST System booted: 2013-01-30 16:23:51 PST (lwld 19:29 ago) Protocols started: 2013-02-07 12:14:14 PST (23:39:27 ago) Last configured: 2013-02-08 11:47:31 PST (00:06:10 ago) by lab 11:53AM up 8 days, 19:30, 1 user, load averages: 0.00, 0.00, 0.00 Question: Is the correct time shown? Answer: Although your output might vary, the previous output shows that the correct time. If the correct time is not shown, please inform your instructor. Step 2.7 Navigate to the [edit system syslog] hierarchy level. Configure your device to send any successful or failed login attempts to the 172.18.5.2 address. Then commit your configuration and exit configuration mode. [edit system] lab@mxA-1# edit sys1og [edit system syslog] lab@mxA-1# set host 172.18.5.3 authorization any [edit system syslog] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Lab 3-14 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC Step 2.8 Issue the monitor traffic interface ge-1/0/2. VLAN-IDcommand where VLAN-IDis the VLAN ID that is associated with your device's ge-1/0/2 interface. lab@mxA-1> monitor traffic interface ge-1/0/2.VLAN-ID verbose output suppressed, use Step 2.9 Open a new Telnet session to your assigned device's management address and attempt to log in using an invalid name and password. �g�����"liff#,IJ!ffi\lr,,._, ;;;\'»/,,, "'"" '""*"'*'"�oiit � Protocol: Hostname: Port: D Show quick connect on startup � Save session � Open in a tab Connect ,j I Cancel I mxA-1 (ttypO) login: 123 Password: Login incorrect login: Step 2.10 Return to the original console, Telnet, or SSH. session. From the original session, examine the output from the monitor traffic interface ge-1/0/2. VLAN-IDcommand. You can press Ctrl+C key combination to stop the output. lab@mxA-1> monitor traffic interface ge-1/0/2.VLAN-ID verbose output suppressed, use www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-15 Junos Troubleshooting in the NOC Reverse lookup for 172.18.5.1 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 12:10:43.057934 Out IP truncated-ip - 58 bytes missing! 172.18.5.1.syslog > 172.18.5.3.syslog: SYSLOG auth.notice, length: 80 12:10:43.580435 Out IP truncated-ip - 76 bytes missing! 172.18.5.1.syslog > 172.18.5.3.syslog: SYSLOG auth.error, length: 98 12:10:43.580845 Out IP truncated-ip - 78 bytes missing! 172.18.5.1.syslog > 172.18.5.3.syslog: SYSLOG auth.notice, length: 100 "'C 3 packets received by filter O packets dropped by kernel Question: Did the output contain any syslog messages about the authentication failure? Answer: The authentication failures should generate syslog messages that were sent to the 172.18.5.3 address. If you do not see the syslog messages indicating an authentication failure please inform your instructor. Lab 3-16 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junes Troubleshooting in the NOC Part 3: Locating Online Resources In this lab part, you use a Web browser to locate online resources for troubleshooting purposes. Step 3.1 Open a Web browser on your local PC. From the open Web browser, navigate to http: I /www.juniper.net/ techpubs and click the link for the MX Series documentation. Technical Documentation Products IJ.y Category + A.pplic.ationA,:ce!eraton + Paci:et Transport + Content and Mer.1:a + Routing + Identity and Pc!icf Centro! + Security + Jun,)s. Platform + S\,vitr;hlng + /,lobihty + ·r1rne Synciuomz.ation + r�._;.rv.;ort. f,·lanagemert Products by Famil)' A.ex S•rne:� Application D•::li'y''.'.rv Controller r-JetScrt��n Series � Series BX Series: MSM C Series NSU Csntral Manager CTP Se:nes MSf,IXµr>:ss c.,3er1es Otw;:;.:;;e1 Al'ce�s C!i':'nt DX 8enes OcJ15:;ejAc(.ess Clh:T1t tF!PS E Seri�s EditianJ EXSt:rie:> - ICS,:,nei; PTXSeri>?s ti.JP Series QFXSeri.:JS IMS AAA Server Route 1nslgm Manag•.�r ISG S·�m:s SASe11�s .J S>!ri>.l�: SBR Se11es - S1)ft·N:;.;r:;c .Jcs1200- Scr•.!r�nOS Junos Cont0nt Pmttol10 (Uedia SDX300 �1owSt1lut1on: Ser,iu.i t,.utom2ihon Jur1!'J�) OS �'RCS::-11;_:s Junos Fulse SR-\ S�r!es .Juno1.: Sr0pi:-, SRX Sr::,11,:,s 1.\'tth WHf:ll:)%C2r(.i .Jurw�SOK 3SG Se1ies Junos Snapshot ,-:-1mmistrator STRf:1 Serit:s Junos Spai:e T Series JuncisptH:1.: rc�:::.:ene:; Junosv· APP En,Jine LiJ ::'.enes vG·.\t Series f,l Se11�s ·.f!.�F.: SJ?n�t MAG Seri,:,sJunos Puit>: V'/.J..Se:nes Gat•.?WJ) '!tirel(��s l..!,.l'J fJ Products by name !,BCD EFG H 1.1 KL r.1 l'l OP OR STl.l\!',VAYZ Step 3.2 Click the Junos 12. 3 link located in the Software Documentation column. www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-17 Junos Troubleshooting in the NOC V!X Series 30 Unilrersal Edge Routers �ardware Documentation Soft\'1are Documentation nst3ning, planning, sat,e.tr. and Getting started. maint�nance-. rouJlesnooting troutreshaJoting, and features • rJX5.MX10 . M�40. and M...\80 ?D UnrJer:;:::il Edg;:, Houter • r.1x2,10 30 Universal Ed9e Router . • Juno;:; 11A • rsx.M�O JD Unr,:er�;.alEdy;, Rout�r • Junos 11 "2. • f;iX950 30 Urnvers..atEtl5f? Rout::r • Junos.1'1 i • Junoz 10 -1 �olutions Documentation orJigurat1on. �xamples. administration, • juno::..10 J �nd us>:fltl rnmmancts. • Juno�:; 10 2 • lilX Seri�5 F�outNS Soiuthi:-JS • Ju nos '10 O • Subscriber Manaii-:?mrn! • Juno�:; 9.J • L1,J01!etle(t Broa Step 3.3 The previous link brought you to the software documentation for M Series, MX Series, and T Series routers. Take some time to examine the page and use it to answer the following questions. Question: Where do you find the information to troubleshoot a VPLS configuration? Answer: You can find information on VPLS in the VPNs section. Features • A.ccess Prr"vilege • Net.VVik!Aanagcmoi:'nt • Cla�s of S·H'.,1CO::: • Houtmg Po!IC"f Configuration • CornmonCriteria ano Jtmo:;;--F!PS • F!rewaH F:!t>Jrs and Traffic Poiicers • Ois.tribut•:oD0nial··Of--Si.:1v1ce • Rout:ng Prntoc.o!$ Pr,;tectirrn • Se1v102s !merf3Cl?S: • Hlgl1 ...:..-.,.·a1lbtJilit:;" • Sut:.;.crl!.wrtJ3n3gement • LaJ�' 2 C.:;n'figur,:itJ;.in •VPMs....-- • LJ;QiCal St$l>:rns " f/PLS AppliC':i'hor1s. • Dar r:':ne.Exp!or!ng JP,;fi • Multic.:istPrntOC'(;]S • Day Cn-::. 1,11grating EIGEP to OSFF Question: Where do you find information to troubleshoot CoS? Answer: You can find information on Cos under the Features, Class of Service section. Lab 3-18 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC Features • Di�Jri!.H.,te\J Oeni<'J!¥cf-Ser;:ce Prnt0ctlon • Sereices interf;,;ces • HiQh /,\fCH!ab:!itv • L.�hi:.'f2: Config1..w:.ton •VPNs • MPLS \ppiications • rAultic:::iztFrctocc:s Step 3.4 Using a Web browser, navigate to http: I /www.juniper.net/ customers/ support/ and click the Browse Knowledge Base link. <<< Please select your product first ,or select the By Task tab. Support Tools � T�chnlcal Docmnentatron Entflls=ment DY Sei!a: Nurnber � J ..tJet Discussion Foru:-ns My Open C8Si.E:•_i F�!;-lt�s , Pt\�;Juct Speufa'8T1cns ' Ur:2ns!ng Tcchn!c::.!Bui!slins ' Pru1Jur:t icon:�- 8, \tsio St12nn!s R�gist>:r ::.� > End of Life Proti!et:-1Report TR} 82::i:ch , Teshnlcal 8u:iet1ns H(;-::l;fy fibre Tools .!Tasks. ., Step 3.5 Type the phrase how to troubleshoot BGP documentation in the question box and click Ask. �KB Home Sro-wse r'J!KB-Categcnes ! Search T:ps Step 3.6 Examine the Results page examine the output and answer the following question. www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-19 Junos Troubleshooting in the NOC ·�KB Home Your Question 'ii·c�;;·r�-!�·��;;·1es·iicOt'i:g·p··�;;·;;;;:,:;e;:aiion··· -0rKB 10� ;$FO€faifr'S·�·a'rCi(CSi5"CCiri'iiiif=·-airr·;irfK6f··· ":) lnclu,1; KB .t-.rchivelfar EOL Products) :-· Search only KS t..rchrve � Lcg1n assistance Browse ,AJ! KB Ca1e9ortes l Se3rct1 Tips 1------···- ······-·------··-- ················------·····-·-·- ·············---···-·······-----··-····------· s�arched for: how to troubleshoot bgp documentation Page�: [ Nett RESULTS: l!Jac;pOperational Mode Commands. Techoica( Documentation. Sup.port. Juniper Netv:orks BGP Operational uoce Commands summan:es the ccmmand-line in!e;face tCLH commands you can use f,J monitor and troubleshootthe _Borde,rG�teW¥t:YProtocol ( BGPj. Comman•:ts ar& h�-ted. 111 a:ph3betica1 order Bf Product Rcut"::rProd.ucis .. JCS Gy Pro Cu ct: Router P:-ocuc1::; ·• M Senes -t JU NOS By Pru duct Rcut�r Products ·+ MX Series Br · · Product. Router Procucts -i' \IXSeries Sy Procuct: Ro-uterProducts ·-+ T Senes By Product: Router Prccucrs 4 T Series http:/,\..,;.1,i'w.junipe-r.net.'tf:-chcuts1en_ US1junos: 11.1itopicslreferenr:elgenerali!::QP· i!J£xample:Configur ing BGP.Based H.VPLS Using Oitferent Mesh Groups forE:ach Spoke Router-Technical Oocmnentation. Sutmon - Juniper tJet#orks Examole. Configuring BGP·83sed H·\'PLS Usin9 Different Uesri Groups for Each Spoke Router This exam pl-?.shows now to car.figure H-VPLS uting Cifferent mesh groups lo provide H-VPLS runcticnaiirJam: pro,;id€S tto::psforverif;ing and trOObleshootingil"le (onfiguration. This is ont::f JP� of H-'·./PLS (',;nfigur .:,tion possible in li"i�Juniper� Je-tv-1oris implementation By ?reduct. Junos Platform r.ttpJ/'.\",'l=W.jun1perneD'te!'.:"hpubSlen_US/funos 112t1opicstexamp1e,'hvp1s-bgp-configurinQ.html Simr!ar .Answers Question: Did the search produce any results related to troubleshooting BGP? Answer: As seen in the previous output, the search produced results related to troubleshooting BGP. Your results might vary from the output and other KB articles might be available. Note If time allows, search the KB database for other topics of your interest. Step 3.7 Using the Web browser, navigate to http: I /www.juniper.net/techpubs/ software/nog/ and answer the following question. Lab 3-20 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC TECHt�ICAL DOCUrJENTATfON NEWIORK OPERATIONS GUIDES: DOCUMENTATION Re:eases: �2.,1 11'..'..2 112.1 111.4 11.3 i TL'. 1 ·ru ! ·u ,! ! 10 2 i 10 o i 9.3 l .ArchJVes I How to Use t!1e .Jur,r;s OS rnci.rrnir·1t2Uor: These guides cover information about some of the most common and cnicia! tasKs performed by Netv,ork Operations Center (MOC} personnel to help them monitor anc operate M-series and T-series products more effectivelyand efficiently JUNOS Software Documentation GeiAJob� Reader· Network OperationsGuides Posted PDF SHA-1 Baseline 01)31/13 PDF SHA·1 !nterfac"!s 01/31113 pl)f SHf\--1 MPLS 01i31!13 PDF ��HA· 1 MPLS Fast RNCLatt� 01/31113 FC:F l'Hfc.1 MPLS L.09.s 01/31/13 PDF SH!\·1 Rou1er Hardware 01112112 PDF ff.\·1 Question: Which Network Operations Guides are available? Answer: The Network Operations Guides that are available are Baseline, Interfaces, MPLS, MPLS Fast Reroute, MPLS Logs, and Router Hardware. Step 3.8 Return to the open session with your assigned device. From the open session, log out of your assigned device. lab@mxA-1> exit mxA-1 (ttyuO) login: Tell your instructor that you have completed this lab. www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-21 Junos Troubleshooting in the NOC Management Network Diagram Management Addressing mxA-1 mxD-1 mxA-2 mxD-2 mxB-1 vr-device mxB-2 Server mxC-1 Gateway mxC-2 TennServer Note: Your instructorwill provide addressand accessinformation_ Using Monitoring Tools and Establishing a Baseline Lab: Pod A Network Diagram Internet R1 R1 (2) .2) 0 "' 0 c ;::;- Q) "" s (.1) r'----��_, mxA-1 mxA-2 ge-1/0/2.141 (.1) 172.18.5.Q/30 (.2) ge-1/0/2.141 loO: 192.168.31.1 loO: 192. 168.32.1 ge-1/0/3-141 1) ge-1/0/3-141 ):�:192.168.11.Q/30 192.168.12 0/3al (.2) vrl-1 Virtual Routers vr2-2 :fij"'%����� Worldwide SeniicesEducation www;urnpernet .. w,;.'J>-., �� Lab 3-22 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Junos Troubleshooting in the NOC Using Monitoring Tools and Establishing a Baseline Lab: Pod B Network Diagram R1 R1 (2) (2) 0 0 (") (") a, m o' 0 c:i o' o' ...; ;::;- :;:::;- � c:i c:i ...; ·co"' � c:i t c:i (-1) ,....___ .._...__, mxB-1 mxB-2 ge-1,10/2.142(.1) 172.18.5.0/30 (.2) ge-1,10/2.142 100: 192.168-33.1 loO: 192.168.34.1 Using Monitoring Tools and Establishing a Baseline Lab: Pod C Network Diagram R1 (.2) 0 m "'- 0 S! '"''" � mxG-1 mxG-2 ge-1,10/2.143(.1) 172.18.5.0/30 ( 2) ge-1,10/2.143 loO: 192.168.35.1 loO: 192.168.36.1 ge-1,I0/3-141 . -1,I0/3-141 ( l)192.168.15.0/30 192.168.16.Q/30(.l) ge (.2) (.2) vr1-1 Virtual Routers vr2-2 LJU !NoddWfdeEdiiemloii Siol\.C::: : . w ' - www.j www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3-23 Junos Troubleshooting in the NOC Using Monitoring Tools and Establishing a Baseline Lab: Pod D Network Diagram Internet /�"-...._ ( ISPA \ R1 R1 (.2) � ...... ,_/.� mxD-1 mxD-2 ge-l/0/2.144 (.1) 172.18.5.0/30 (.2) ge-lj0/2.144 loO: 192.168.37.1 loO: 192.168.38.1 ge-lj0/3-141 11 ge-l/0/3-141 g;192.168.17.0/30 192.168.18.0/3d- (.2) vrl-1 VirtualRouters vr2-2 Lab 3-24 • Using Monitoring Tools and Establishing a Baseline www.juniper.net Lab Monitoring Hardware and Environmental Conditions Overview This lab demonstrates how to monitor hardware and environmental conditions. You will use the command line interface (CLI) to test and view the status of your router's storage devices. You will view your router's boot messages, log messages, and chassisd logs to determine which hardware events have occurred over time. You will also use various commands to determine the status of memory and storage, log messages, as well as the chassis temperatures. Note that your lab login grants you super-user permissions and that you might have to become root during this lab. Please be careful, and have fun! By completing this lab, you will perform the following tasks: View and test a router's storage devices. Parse through log messages to view hardware events that have occurred. Configure nondefault alarm settings. www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-1 Junes Troubleshooting in the NOC Part 1: Monitoring Memory and Storage In this lab part, you use various commands to view the status of the RE and its memory and storage devices. Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address. Question: What is the management address assigned to your device? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CU for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Hostname: [. D Show quick connect on startup 0 Save session �Open in a tab I Connect j I Cancel Step 1.3 Log in to your device with the username 1.ab and the password 1.abl23. Note that both the name and password are case-sensitive. Lab 4-2 • Monitoring Hardware and Environmental Conditions www.juniper.net Junos Troubleshooting in the NOC mxA-1 (ttyuO) login: lab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/1.ab4-start.config command. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# load override jtnoc/1ab4-start.config load complete [edit] lab@mxA-1# Step 1.5 Change the root password to 1.ab123 by issuing the set system root-authentication plain-text-password command.Then, commit the changes and exit to operational mode before moving on to the next step. [edit] lab@mxA-1# set system root-authentication plain-text-password New password: Retype new password: [edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode [edit] lab@mxA-1> Step 1.6 Issue the show chassis routing-engine command to view the details of your router's RE. lab@mxA-1> show chassis routing-engine Routing Engine status: Temperature 35 degrees c I 95 degrees F CPU temperature 46 degrees c I 114 degrees F DRAM 2048 MB Memory utilization 28 percent CPU utilization: User 0 percent Background 0 percent Kernel 0 percent www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-3 Junos Troubleshooting in the NOC Interrupt O percent Idle 99 percent Model RE-MX80 Serial ID SIN YK8973 Start time 2013-01-31 01:10:41 UTC Uptime 7 days, 23 hours, 52 minutes, 9 seconds Last reboot reason Router rebooted after a normal shutdown. Load averages: 1 minute 5 minute 15 minute 0.00 0.02 0.03 Question: What is the current temperature of the RE? Answer: The answer might vary by student. In the example output, the current RE temperature is 35 degrees C or 95 degrees F. Question: What is the current temperature of the CPU on the RE? Answer: The answer might vary by student. In the example output, the current RE temperature is 46 degrees C or 114 degrees F. Question: How much DRAM exists on the RE? Answer: The answer might vary by student. In the example output, there is 2048 MB of DRAM installed. Question: What is the RE CPU's non-Idle utilization? Answer: The answer might vary by student. In the example output, the RE CPU is currently 0% utilized. Lab 4-4 • Monitoring Hardware and Environmental Conditions www.juniper.net Junes Troubleshooting in the NOC Question: What is the uptime of the RE? Answer: The answer might vary by student. In the example output, the RE has been powered up for 7 days, 23 hours, 52 minutes, and 9 seconds. Step 1.7 Issue the show system storage command to determine the amount of storage space available on your router. lab@mxA-1> show system storage File system Size Used Avail Capacity Mounted on /dev/daOsla 885M 125M 688M 15% I devfs 1. OK 1. OK OB 100% /dev /dev/mdO 42M 42M OB 100% /packages/mnt/jbase /dev/mdl 184M 184M OB 100% /packages/mnt/ jkernel-ppc-12.2R2.5 /dev/md2 33M 33M OB 100% /packages/mnt/ jpfe-MX80-12.2R2.5 /dev/md3 9. lM 9.lM OB 100% /packages/mnt/ jdocs-12.2R2.5 /dev/md4 55M 55M OB 100% /packages/mnt/ jroute-ppc-12.2R2.5 /dev/md5 12M 12M OB 100% /packages/mnt/ jcrypto-ppc-12.2R2.5 /dev/md6 2.BG 10.0K 2.6G 0% /tmp /dev/md7 2.SG 4.7M 2.6G 0% /mfs /dev/daOsle 98M 268K 90M 0% /config procfs 4. OK 4.0K OB 100% /proc /dev/dalslf 2.SG 1. OG 1.6G 39% /var Question: What are the names of the two internal NANO flash drives that exist on your router? Answer: In the output of the command, two drives are listed, daO and dal. Question: What is the current available storage capacity on daO? Answer: The answer will vary by student. In the example output, the daOsla partition has 668 MB available, whereas the daOs le partition has 90 MB available. www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-5 Junos Troubleshooting in the NOC Question: Which flash drive is used to store user home directories? Answer: User home directories are located at /var/ home. A partition of dal is mounted on /var so it is currently being used to store user home directories. Step 1.8 Enter the shell as the root user by issuing the start shell user root command. Enter the password of lab123when prompted. lab@mxA-1> start shell user root Password: root@mxA-1% Step 1.9 Perform a read-only test to determine the integrity of the daO internal NANO flash drive. Use the dd if=/dev/daO of=/dev/null bs=lm command to perform the test. Note It might take between 5 or 10 minutes for the test to complete. Be patient. root@mxA-1% dd if=/dev/daO of=/dev/null bs=lm 3920+0 records in 3920+0 records out 4110417920 bytes transferred in 367.708375 secs (11178472 bytes/sec) Question: Did the integrity test complete successfully? Answer: Yes. The test should have completed successfully. If the test results show that errors occurred, please notify your instructor because your router might be experiencing a hardware failure. Lab 4-6 • Monitoring Hardware and Environmental Conditions www.juniper.net Junes Troubleshooting in the NOC Question: How many bytes were read from the daO flash drive? Answer: The answer might vary by student. In the example output, 4110417920 bytes were read. Step 1.10 Exit from the shell and return to the CLI. root@mxA-1% exit exit Part 2: Viewing Boot and System Logs In this lab part, you use the boot and system logs to view events that have occurred on your router over time. You must complete this part of the lab by accessing your router's console port. Step 2.1 Direct your router to reboot in 20 minutes using the request system reboot in 20 command. lab@mxA-1> request system reboot in 20 Reboot the system in 20? [yes,no] (no) yes Shutdown at Fri Feb 8 19:38:29 2013. [pid 17194] *** System shutdown message from lab@mxA-1 *** System going down at 19:38 lab@mxA-1.> Question: At what time will the router reboot? Answer: The answer will vary by student. In the example output, the router will reboot at 19:38. Step 2.2 Clear the schedule reboot using the clear system reboot command. lab@mxA-1> clear system reboot reboot requested by lab at Fri Feb 8 19:38:29 2013 www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-7 Junos Troubleshooting in the NOC [process id 17194] Terminating... lab@mxA-1> Step 2.3 Direct your router to reboot immediately using the request system reboot command. From the console, you should be able to watch the entire process. Wait forthe reboot to complete before continuing. lab@mxA-1> request system reboot Reboot the system ? [yes,no] (no) yes Shutdown NOW! [pid 17210] *** FINAL System shutdown message from lab@mxA-1 *** System going down IMMEDIATELY lab@mxA-1> JWaiting (max 300 seconds) for system process 'vnlru' to stop... done Waiting (max 300 seconds) for system process 'vnlru mem' to stop... done Waiting (max 300 seconds) for system process 'bufdaemon' to stop... done Waiting (max 300 seconds) for system process 'syncer' to stop ... Syncing disks, vnodes remaining... 1 pfe_send_failed(index 0, type 17), err=32 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 done syncing disks... All buffers synced. Uptime: 7ldl0h59m9s recorded reboot as normal shutdown Rebooting... Step 2.4 Log in to your router and view the boot messages that occurred during the reboot process by issuing the show system boot-messages I no-more command. mxA-1 (ttyuO) login: 1.ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> show system boot-messages I no-more platform_early_bootinit: MX-PPC Series Early Boot Initialization mxppc_set_re_type: hw.board.type is MX80 mxppc_set_re_type: REtype:78, model:mx80, model:MX80, i2cid:2447 WDOG initialized Copyright (c) 1996-2012, Juniper Networks, Inc. All rights reserved. Copyright (c} 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. JUNOS 12.2R2.5 #0: 2012-11-15 14:13:01 UTC Lab 4-8 • Monitoring Hardware and Environmental Conditions www.juniper.net Junos Troubleshooting in the NOC [email protected]:/volume/build/junos/12.2/release/12.2R2.5/ obj-powerpc/j unos /bsd/kernels/ ,JUNIPER-PFC/kernel WARNING: debug.mpsafenet forced to Oas ipsec requires Giant Timecounter "decrementer" frequency 66666666 Hz quality O cpuO: Freescale e500v2 core revision 3.0 cpuO: HIDO 80004000 www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-9 Junos Troubleshooting in the NOC gentbi2: lOOObaseSX-FDX, lOOObaseT-FDX, auto pcibO: Question: The boot messages record the step-by-step process that the RE goes through to boot. Did any errors occur during the boot process? Answer: No errors should occur. Step 2.5 Use the show system uptime command to determine the router's current time and date. The date, as known by your router, will be used to parse log messages in future steps. lab@mxA-1> show system uptime Current time: 2013-02-08 00:37:58 UTC System booted: 2013-01-31 00:23:51 UTC (lwld 00:14 ago) Lab 4-10 • Monitoring Hardware and Environmental Conditions www.juniper.net Junes Troubleshooting in the NOC Protocols started: 2013-02-07 20:14:14 UTC (04:23:44 ago) Last configured: 2013-02-08 00:24:43 UTC (00:13:15 ago) by lab 12:37AM up 8 days, 14 mins, 1 user, load averages: 0.00, 0.00, 0.00 Question: What is the current time and date? Answer: The answer will vary by student. In the example output, the time and date are currently 00:37:58 UTC on Feb 8, 2013. Step 2.6 View the messages log file using the show log messages to view detailed information about the PFE during the reboot that just occurred. It might be helpful to use the match modifier to ensure that only entries from today's date are shown. For example, if today's date is Feb 8th, issue the command show log messages I match "Feb 8" (you might need to use two spaces between month and day). Make an attempt to find a log that shows when the reboot was initiated. If you cannot immediately find the entry, hit Ctrl+c and go to the next step. lab@mxA-1> show log messages I match "Feb 8" Feb 8 00:01:06 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+54373 Feb 8 00:03:34 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+55229 Feb 8 00:06:02 mx80 rpd[ll30]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+55506 Feb 8 00:08:30 mx80 rpd[1130]: bgp_ listen·- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+61679 Feb 8 00:10:58 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+62191 Feb 8 00:13:26 mx80 rpd[1130]: bgp_ listen _accept: Connection attempt from unconfigured neighbor: 172.22.201.2+53976 Feb 8 00:15:54 mx80 rpd[1130]: bgp_ listen accept: Connection attempt from unconfigured neighbor: 172.22.201.2+53347 Feb 8 00:18:22 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+49385 Feb 8 00:20:50 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+60403 Feb 8 00:23:18 mx80 rpd[1130]: bgp- listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+60057 Feb 8 00:25:46 mx80 rpd[1130]: bgp listen- accept: Connection attempt from unconfigured neighbor: 172.22.201.2+56149 Step 2.7 You might notice that matching on the date might not narrow the search down enough because thousands of entries might happen on any one day. Use the previous command but add a second pipe that matches on reboot. For example, if today's date is Dec 7th, issue the command show log messages I match "Feb 8" I match reboot. www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-11 Junos Troubleshooting in the NOC lab@mxA-1> show 1og messages I match "Feb 8" I match reboot Feb 8 19:18:29 mxA-1 mgd[l6416]: UI_REBOOT_EVENT: System rebooted by 'lab' Feb 8 19:18:29 mxA-1 shutdown: reboot requested by lab at Wed Feb 8 19:38:29 2011 Feb 8 19:19:43 mxA-1 mgd[l6416]: UI_REBOOT_EVENT: System rebooted by 'lab' Feb 8 19:19:48 mxA-1 shutdown: reboot by lab: Feb 8 19:22:40 mxA-1 savecore: Router rebooting after a normal shutdown ... Feb 8 19:22:40 mxA-1 savecore: Router rebooting after a normal shutdown .. . Question: Two reboots were requested. At what time did the first reboot request occur? Answer: The answer will vary by student. In the example, the first request was issued at 19:18:29. Question: Two reboots were requested.At what time did the second reboot request occur? Answer: The answer will vary by student. In the example, the second request was issued at 19:19:43. Step 2.8 Using the show log messages command, determine the exact time that FPC 1 came back on line after the reboot. (Hint: Status of FPCs are tracked by chassisd.) lab@mxA-1> show 1og messages I match "Feb 8" I match chassisd I match fpc Feb 8 19:19:56 mxA-1 chassisd[ll37]: CHASSISD IFDEV DETACH FPC: ifdev detach fpc(O) Feb 8 19:19:56 mxA-1 chassisd[ll37]: CHASSISD IFDEV DETACH FPC: ·ifdev_detach_fpc(l) Feb 8 19:19:57 mxA-1 chassisd[ll37]: CHASSISD IFDEV DETACH FPC: ifdev_detach_fpc(O) Feb 8 19:19:57 mxA-1 chassisd[ll37]: CHASSISD IFDEV DETACH FPC: ifdev_detach_fpc(l) Feb 8 19:22:59 mxA-1 chassisd[ll21]: CHASSISD IFDEV DETACH FPC: ifdev_detach_fpc(O) Feb 8 19:22:59 mxA-1 chassisd[ll21]: CHASSISD IFDEV DETACH FPC: ifdev_detach_fpc(l) Feb 8 19:23:52 mxA-1 chassisd[ll21]: CHASSISD FRU EVENT: set fpc online: restarted FPC O Feb 8 19:23:52 mxA-1 chassisd[ll21]: CHASSISD FRU EVENT: set fpc online: restarted FPC 1 Lab 4-12 • Monitoring Hardware and Environmental Conditions www.juniper.net Junes Troubleshooting in the NOC Question: At what time did FPC 1 come back online after the reboot? Answer: The answer will vary by student. In the example, FPC 1 came back online at 19: 23: 52. Part 3: Monitoring the Chassis and Environment In this lab part, you will use the CU to monitor the status of the chassis and environment of your router. Step 3.1 Issue the show chassis temperature-thresholds command to determine the default threshold settings. lab@mxA-1> show chassis temperature-thresholds Fan speed Yellow alarm Red alarm Fire Shutdown (degrees C) (degrees C) (degrees C) (degrees C) Item Normal High Normal Bad fan Normal Bad fan Normal Chassis default 48 54 65 55 75 75 100 Routing Engine 65 70 85 75 95 80 100 Question: At what temperature will the RE cause the router to increase the speed of the fans in the fan tray? Answer: When the RE reaches a temperature of 70 degrees Celsius, the router should increase the speed of the fans. Step 3.2 Issue the show chassis environment command to determine the current temperature of the routing engine in degrees Celsius. lab@mxA-1> show chassis environment Class Item Status Measurement Temp PEM O OK PEM 1 Absent RE 0 Intake OK 28 degrees c I 82 degrees F RE 0 Front Exhaust OK 32 degrees c I 89 degrees F RE O Rear Exhaust OK 30 degrees c I 86 degrees F Routing Engine OK 35 degrees c I 95 degrees F Routing Engine CPU OK 46 degrees c I 114 degrees F TFEB O QX O TSen OK 32 degrees c I 89 degrees F TFEB 0 QX 0 Chip OK 41 degrees c I 105 degrees F TFEB 0 LU 0 TS en OK 32 degrees c I 89 degrees F www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-13 Junos Troubleshooting in the NOC TFEB 0 LU 0 Chip OK 46 degrees c I 114 degrees F TFEB 0 MQ O TSen OK 32 degrees c I 89 degrees F TFEB 0 MQ O Chip OK 39 degrees c I 102 degrees F TFEB 0 TBB PFE TSen OK 29 degrees c I 84 degrees F TFEB 0 TBB PFE Chip OK 37 degrees c I 98 degrees F TFEB 0 TFEB PCIE TS en OK 32 degrees c I 89 degrees F TFEB 0 TFEB PCIE Chip OK 49 degrees c I 120 degrees F Fans Fan 1 OK Spinning at normal speed Fan 2 OK Spinning at normal speed Fan 3 OK Spinning at normal speed Fan 4 OK Spinning at normal speed Fan 5 OK Spinning at normal speed Question: What is the current temperature of the RE? Answer: The answer will might by student. In the example, the RE is currently 35 degrees Celsius. Question: What is the current speed of the fans? Answer: The answer might vary by student. In the example, the fans are spinning at normal speed. Question: When compared to the temperature thresholds, are the fan speed appropriate? Answer: The fans should be spinning at a speed (normal or high) as per the show chassis temperature-thresholds command. Step 3.3 Determine any alarms exist by issuing the show chassis alarms command. lab@mxA-1> show chassis alarms No alarms currently active Lab 4-14 • Monitoring Hardware and Environmental Conditions www.juniper.net Junos Troubleshooting in the NOC Question: Do any active alarms exist? Answer: The answer might vary by student. In the example, no active alarms exist. Step 3.4 Enter configuration mode and change the default chassis alarm settings such that if any Ethernet ports are in the link-down state, the router will generate a red alarm. Commit your configuration an exit to operational mode. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# set chassis a1arm ethernet 1ink-down red [edit] lab@mxA-1# collllilit and-quit commit complete Exiting configuration mode Step 3.5 Once again, determine if any alarms exist by issuing the show chassis alarms command. lab@mxA-1> show chassis a1arms 6 alarms currently active Alarm time Class Description 2013-02-08 00:49:54 OTC Major ge-1/1/2: Link down 2013-02-08 00:49:54 OTC Major ge-1/1/1: Link down 2013-02-08 00:49:54 OTC Major xe-0/0/3: Link down 2013-02-08 00:49:54 OTC Major xe-0/0/2: Link down 2013-02-08 00:49:54 OTC Major xe-0/0/1: Link down 2013-02-08 00:49:54 OTC Major xe-0/0/0: Link down Question: How many active alarms are reported by your router? Answer: The answer might vary by student. In the example, 6 active alarms exist. Step 3.6 Log out of your assigned device. www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-15 Junes Troubleshooting in the NOC lab@mxA-1> exit mxA-1 (ttyuO) login: Tell your instructor that you have completed this lab. Management Network Diagram Ma nt � � ��2-.D. . ; .J '\.. ,- ", _ _ Student '.---_ _J_ ',--m::a. Workstations Management Addressing mx0-1 mxA-2 mx0-2 mxB-1 vr-device mxB-2 Server mxC-1 Gateway mxC-2 Term Serwr ----- Note:Your instructorwill provideaddress and accessinformatio n_ Lab 4-16 • Monitoring Hardware and Environmental Conditions www.juniper.net Junos Troubleshooting in the NOC Monitoring Hardware and Environmental Conditions Lab: Pod A Network Diagram Internet __.!'. �"� ISPA \ I ISPB "\ R1 j ·V,\. / R1 (.2) v �--_/ (.2) �'-' ----/ ,:9 (.1) ,....___ ..__..__.., mxA-1 mxA-2 ge-1,10/ 2.141 (.1) 172.18.5.0/30 ( 2) ge-1/0/2.141 loO: 192.168.31.1 loO: 192.168.32.1 ge-1,/0/3-141 (. ge-1/0/3-141 (.l)1921 68.110/3 . 0 192.168.12.Q/30 l) (.2) . (.2) vrl-1 VirtualRouters vr2-2 Monitoring Hardware and Environmental Conditions Lab: Pod B Network Diagram Internet -- -� � ISPA \ �ISPB -� ( \._ R1 ,\_ / .,'\ J R1 (.2) "'-.,--::: __,./ ,:9 \.• � _.r (.2) mxB-1 mxB-2 ge-1/0/2.142(.1) 172.18.5.0/30 (.2) ge-1,/0/2.142 loO: 192.168.33.1 loO: 192.168.34.1 .l) ge-1/0/3-141 192.168.14.Q/30( (.2) VirtualRouters vr2-2 www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4-17 Junos Troubleshooting in the NOC Monitoring Hardware and Environmental Conditions Lab: Pod C Network Diagram Internet R1 R1 (.2) r'��-0_,/ f.-1_; 2) ,::: (.1)..... ------, mxC.1 mxC-2 ge-1/0/2.143 (.1) 172.18.5.0/30 (.2) ge-1/0/2.143 loO: 192.168.35.1 loO: 192.168.36.1 ge-1/0/3-141 1 ge-1/0/3-141 g;192.168.15.Q/30 192.168.16.0/3d- ) (.2) vrl-1 VirtualRouters vr2-2 Monitoring Hardware and Environmental Conditions Lab: Pod D Network Diagram _..I - Internet � � ( �A) '>'\ �PB � R1 R1 �_./ (.� 1c·� / (.2) ...... -,, .:]' .2) (1)..... ------, mxD-2 mxD-1 ge-1/0/2.144 (.1) 172.18.5.0/30 (.2) ge-1/0/2.144 loO: 192.168.37 .1 100: 192. 168.38.1 ge-1/0/3 141 ge-1/0/3-141 - (.lj192168170. . I'3 0 192.168.180. /30(.i) (.2} . (.2) vrl-1 VirtualRouters vr2-2 Lab 4-18 • Monitoring Hardware and Environmental Conditions www.juniper.net Lab Control Plane Monitoring and Troubleshooting Overview This lab demonstrates how to work with user processes, system daemons, core dump files, and the routing table. You will also configure and work with OSPF and BGP. You will then configure basic Layer 2 bridging. By completing this lab, you will perform the following tasks: View what users are logged into the router and forcibly remove unwanted or hung users. View and restart system daemons. Generate and retrieve core dump files. Configure and monitor OSPF and BGP. Configure and monitor Layer 2 bridging. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-1 Junos Troubleshooting in the NOC Part 1: ExaminingUser Processes and Daemons In this lab part, you examine users logged in to your assigned device. You will also examine and restart daemons that are currently running on the router. Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address. Question: What is the management address assigned to your device? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CLI for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Protocol: Hostname: Port: Firewall: v �·······-···------·-··········,ne-----·--······--·--·-···-�---····-· · O Show quick connect on startup � Save session � Open in a tab I Connect j [ Cancel Step 1.3 Log in to your device with the username lab and the password lab123. Note that both the name and password are case-sensitive. Lab 5-2 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC mxA-1 (ttyuO) login: lab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/1.abS-start. con£ig command. After the configuration has been loaded, commit the changes and exit to operational mode before continuing to the next step. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# [edit] lab@mxA-1# load override jtnoc/1ab5-start.config load complete [edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode [edit] lab@mxA-1> Step 1.5 Issue the show system users command and answer the following questions: lab@mxA-1> show system users 8:08PM up 47 mins, 1 user, load averages: 0. 00, 0. 03, 0.01 USER TTY FROM LOGIN@ IDLE WHAT - lab pO 10.210.14.30 8:09PM -cli (cli) Question: What users are logged in to your assigned device? Answer: As shown in the previous output, only the user, lab, is logged in to your assigned device. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-3 Junos Troubleshooting in the NOC Question: From which IP address is the user lab logged in to your assigned device? Answer: Although your output might vary, the previous output shows that user lab is Jogged in from the IP address of 10.210.14.30. Step 1.6 Open a new Telnet session to your assigned device. D Show quick connect on startup [21Save session 0 Open in a tab I: Connect I I Concel I Step 1.7 On the new Telnet session, log in as user lab with the password supplied by your instructor. mxA-1 (ttypO) login: 1.ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.8 Return to your original session. On your original session, issue the show system users command. lab@mxA-1> show system users 8:12PM up 51 mins, 2 users, load averages: 0.02, 0.02, 0.00 USER TTY FROM LOGIN@ IDLE WHAT lab pO 10.210.14.30 8:09PM - -cli (cli) lab p2 10.210.14.30 8:12PM - -cli (cli) Lab 5-4 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Question: Which users are present on your assigned device? Answer: Two instances of the lab user should be present. Step 1.9 Issue the request system logout user lab terminal ? command. lab@mxA-1> request system logout user 1ab terminal? Possible completions: Question: What value should you use to complete the command to forcibly remove the second instance of user lab from the router? Answer: Although your answer might vary, the previous output shows that the second user has a TIY value of p2. You can use the value of p2 to forcibly logout the second instance of the lab user. Step 1.10 Forcibly remove the second instance of user lab. lab@mxA-1> request system logout user 1ab terminal p2 logout-user: done lab@mxA-1> Step 1.11 Issue the show system users command and answer the following question: lab@mxA-1> show system users 8:14PM up 53 mins, 1 user, load averages: 0.38, 0.20, 0.08 USER TTY FROM LOGIN@ IDLE WHAT lab pO 10.210.14.30 8:09PM - -cli (cli) Question: How many users are logged in to your assigned device? Answer: Only one instance of the lab user is present on your assigned device. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-5 Junos Troubleshooting in the NOC Step 1.12 Issue the show system processes extensive command. lab@mxA-1> show system processes extensive last pid: 23568; load averages: 0.00, 0. 00, 0.00 up 7+19:47:33 20:10:54 133 processes: 3 running, 103 sleeping, 27 waiting Mem: 362M Active, 58M Inact, SOM Wired, 144M Cache, ll2M Buf, 1345M Free Swap: 2915M Total, 2915M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 11 root 1 171 52 OK 16K RUN 182.9H 98.05% idle 1296 root 11 97 0 16732K 8404K ucond 86:48 0.00% clksyncd 13 root 1 -20 -139 OK 16K RUN 75:40 0.00% swi7: clock 1281 root 1 96 0 llOM 14004K select 18:15 0.00% chassisd 1417 root 1 96 0 57384K 32128K select 15:29 0.00% mib2d 1300 root 1 96 0 11960K 7364K select 11 :23 0.00% license-check 1386 root 1 96 0 19168K 11616K select 6:56 0.00% 12ald 1415 root 1 96 0 32072K 12520K select 6:49 0.00% pfed 1403 root 3 20 0 10144K 3028K sigwai 6:33 0.00% jddosd 1299 root 1 96 0 13768K 3324K select 6:06 0.00% shm-rtsdbd 1278 root 1 96 0 2528K 1240K select 4:57 0.00% bslockd 12 root 1 -40 -159 OK 16K WAIT 4:07 0.00% swi2: netisr 0 1418 root 1 96 0 41004K 35564K select 3:53 0.00% snmpd 1413 root 1 96 0 21628K 11812K select 3:53 0.00% smid 9 1390 root 1 96 0 34652K 12960K select 3:12 0.00 6 cosd 28 root 1 -68 -187 OK 16K WAIT 3:07 0.00% irq36: tsecl 1291 root 1 96 0 8972K 4996K select 1:52 0.00% ppmd 1282 root 1 96 0 11992K 6980K select l:45 0.00% alarmd 9 root 1 171 52 OK 16K pgzero 1:27 0.00% pagezero 23 root 1 -52 -171 OK 16K WAIT 1:09 0.00% irq43: i2c0 i2cl 27 root 1 -68 -187 OK 16K WAIT 1:06 0.00% irq35: tsecl 1385 root 1 4 0 61724K 21776K kqread 1:04 0.00% rpd 15 root 1 -16 0 OK 16K - 1:04 0.00% yarrow 1414 root 1 96 0 6172K 2736K select 1:04 0.00% irsd Question: What daemon manages the chassis components of the router? Answer: The chassisd daemon manages the chassis components of the router. Question: What daemon manages the routing function? Answer: The rpd daemon manages the routing function. Lab 5-6 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Step 1.13 Issue the show route command. lab@mxA-1> show route inet.0: 15 destinations, 16 routes (15 active, 0 holddown, O hidden) + =Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/5] 01:00:37 > to 172.18.1.1 via ge-1/0/0.141 [Static/7J 19:53:30 > to 172.18.2.1 via ge-1/0/1.141 10.210.15.0/27 *[Direct/OJ lwOd 19:47:04 > via fxpO.O 10.210.15.1/32 *[Local/OJ lwOd 19:47:04 Local via fxpO.O 172.18.1.0/30 *[Direct/OJ 19:53:30 > via ge-1/0/0.141 172.18.1.2/32 *[Local/OJ 19:53:30 Local via ge-1/0/0.141 172.18.2.0/30 *[Direct/OJ 19:53:30 > via ge-1/0/1.141 172.18.2.2/32 *[Local/OJ 19:53:30 Local via ge-1/0/1.141 172.18.5.0/30 *[Direct/OJ 01:00:37 > via ge-1/0/2.141 172.18.5.1/32 *[Local/OJ 01:00:37 Local via ge-1/0/2.141 172.18.112.1/32 *[Local/OJ 01:00:37 Reject 172.18.113.0/30 * [Direct/OJ 01:00: 34 > via ge-1/1/3.0 172.18.113.1/32 *[Local/OJ 01:00: 37 Local via ge-1/1/3.0 192 .168.11. 0/24 *[Direct/OJ 16:53:18 > via ge-1/0/3.141 192.168 .11.1/32 *[Local/OJ ld 01:12:31 Local via ge-1/0/3.141 192 .168. 31.1/32 *[Direct/OJ ld 01:12:30 > via loO.O Question: What is the amount of time listed for the oldest route in the routing table? Answer: Although your output might vary, the previous output shows that the oldest route has been in the routing table for 1 week. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-7 Junos Troubleshooting in the NOC Question: If for some reason the rpd process were to restart, what would be the effect on the routing table? Answer: The Ju nos OS removes all routing information from the routing table. Then, the Junos OS adds any current routing information through independent and dynamic routing protocols. Step 1.14 Issue the restart routing command. lab@mxA-1> restart routing Routing protocols process started, pid 37465 lab@mxA-1> Step 1.15 Issue the show route command. lab@mxA-1> show route inet.0: 15 destinations, 16 routes (15 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/SJ 00:00:10 > to 172.18.1.1 via ge-1/0/0.141 [Static/7J 00:00:10 > to 172.18.2.1 via ge-1/0/1.141 10.210.15.0/27 *[Direct/OJ 00:00: 10 > via fxpO.O 10.210.15.1/32 *[Local/OJ 00:00:10 Local via fxpO.O 172.18.1.0/30 *[Direct/OJ 00:00:10 > via ge-1/0/0.141 172.18.1.2/32 *[Local/OJ 00:00:10 Local via ge-1/0/0.141 172.18.2.0/30 *[Direct/OJ 00:00: 10 > via ge-1/0/1.141 172.18. 2.2/32 * [Local/OJ 00: 00:10 Local via ge-1/0/1.141 172.18.5.0/30 * [Direct/OJ 00: 00: 10 > via ge-1/0/2.141 172.18.5.1/32 *[Local/OJ 00:00:10 Local via ge-1/0/2.141 172.18.112.1/32 *[Local/OJ 00:00:10 Reject 172.18.113.0/30 *[Direct/OJ 00:00:10 > via ge-1/1/3.0 Lab 5-8 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC 172.18 . 113.1/32 * [Local/OJ 00:00: 10 Local via ge-1/1/3.0 192.168 .11. 0/24 * [Direct/OJ 00: 00:10 > via ge-1/0/3.141 192. 168. 11.1/32 * [Local/OJ 00:00: 10 Local via ge-1/0/3.141 192.168.31.1/32 *[Direct/OJ 00:00:10 > via loO.O Question: What changes are present in the routing table? Answer: All the routes in the routing table were removed and replaced with new routes, which you can see by viewing the current age of the routes in the routing table. Question: What effect would restarting rpd have on OSPF? Answer: Restarting rpd causes OSPF to completely reconverge. All neighbor adjacencies must re-establish, the router must repopulate the linkstate database, and calculations must run on all possible routes to determine the best path. Question: How do you restart only OSPF without disrupting other routing protocols running on the router? Answer: You can restart OSPF by deactivating the [edit protocols ospfJ hierarchy level, committing the configuration, activating the [edit protocols ospf] hierarchy level, and committing the configuration again. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-9 Junos Troubleshooting in the NOC Part 2: Generating Core Files In this lab part, you generate a core dump file, pull it from the router, and prepare it to be inspected. Step 2.1 Issue the show system core-dumps command and answer the following question: lab@mxA-1> show system core-dumps /var/crash/*core*: No such file or directory /var/tmp/*core*: No such file or directory /var/crash/kernel.*: No such file or directory /tftpboot/corefiles/*core*: No such file or directory Question: Are there any core dump files present on your assigned device? Answer: No core dump files are present on your assigned device. Step 2.2 Note Generating a core dump file can possibly cause adverse effects on your device and it is not recommended as a troubleshooting step unless directed by JTAC. Issue the request system core-dump ? command. Note The core-dump part of the previous command is a hidden command and will not autocomplete. lab@mxA-1> request system core-dump? Possible completions: adaptive-services Adaptive services process audit-process Audit process cfm Ethernet OAM connectivity-fault-management process class-of-service Class-of-service daemon database-replication Database replication process ddos-service DDOS Protection Service daemon dhcp-service Dynamic Host Configuration Protocol process dynamic-flow-capture Dynamic flow capture service event-processing Event processing process Lab 5-10 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC firewall Firewall daemon helper Port-forwarding daemon hostname-caching-process Hostname caching process interface-control Interface daemon jflow Traffic sampling daemon 12-control Layer 2 control protocols 12-learning Layer 2 address flooding/learning process 12tp-service Layer 2 Tunneling Protocol daemon lacp Link Aggregation Control Protocol lfm Ethernet OAM link-fault-management process link-management Link management management Management daemon mib-process SNMP MIB-II daemon mpls-traceroute MPLS Periodic Traceroute process pie-services-logging PIC services logging daemon PPP Point-to-point protocol daemon ppp-service Universal edge Point-to-point protocol daemon pppoe PPP over Ethernet daemon redundancy-services Redundancy device process remote-operations Remote operations daemon rmopd RMOPD daemon routing Routing protocol daemon sampling Traffic sampling daemon send Secure Neighbor Discovery Protocol process service-deployment Service deployment system daemon snmp SNMP daemon snooping Multicast snooping protocol daemon subscriber-management Subscriber management process Question: What option can you use to generate an rpd core dump file? Answer: You can use the routing option to generate an rpd core dump file. Step 2.3 Generate a core dump file using the rpd process. lab@mxA-1> request system core-dump routing Generating core dump for routing process using running method Step 2.4 Issue the show system core-dumpscommand to list the core dump files. lab@mxA-1> show system core-dumps /var/crash/*core*: No such file or directory -rw-rw---- 1 root field 37277696 Feb 7 20:20 /var/tmp/rpd.core.O /var/crash/kernel.*: No such file or directory /tftpboot/corefiles/*core*: No such file or directory total l www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-11 Junos Troubleshooting in the NOC Question: What is the location of the core dump file? Answer: As shown in the previous output, the core dump file is in the /var/tmp/ directory. Step 2.5 Next, retrieve the core dump file using FTP. Using the local Windows FTP client, open an FTP session to your assigned device's management address. OK l I Cancel 11 lj_rowse... Step 2.6 Note Make sure that once the FTP session is open, you set the transfer mode to binary and enable hash marks. Once the FTP session is open, log in using the lab user name and perform an FTP get to transfer the core dump file to your local machine. Connected to 10.210.15.1. 220 mxA-1 FTP server (Version 6.00LS) ready. User (10.210.15.1: (none)): l.ab 331 Password required for lab. Password: 230 User lab logged in. ftp> bin 200 Type set to I. ftp> hash Hash mark printing On ftp: (2048 bytes/hash mark) . ftp> cd /var/tmp/ 250 CWD command successful. ftp> get rpd.core.O 200 PORT command successful. 150 Opening BINARY mode data connection for 'rpd.core.0' (24875008 bytes). ############################################################################## ############################################################################## ############################################################################## Lab 5-12 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC ################################################################## 226 Transfer complete. ftp: 24875008 bytes received in 225.53Seconds 110.30Kbytes/sec. ftp> Part 3: Configuring Routing Protocols and Routing Tables In this lab part, you configure OSPF and BGP. You will also examine the contents of the routing table. Step 3.1 Issue the show route command. lab@mxA-1> show route inet.0: 15 destinations, 16 routes (15 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/SJ 00:09:18 > to 172.18.1.1 via ge-1/0/0.141 [Static/7J 00:09:18 > to 172.18.2.1 via ge-1/0/1.141 10.210.15.0/27 *[Direct/OJ 00:09:18 > via fxpO.O 10.210.15.1/32 *[Local/OJ 00:09:18 Local via fxpO.O 172.18.1.0/30 *[Direct/OJ 00:09:18 > via ge-1/0/0.141 172.18.1.2/32 *[Local/OJ 00:09:18 Local via ge-1/0/0.141 172.18.2.0/30 *[Direct/OJ 00:09:18 > via ge-1/0/1.141 172.18.2.2/32 *[Local/OJ 00:09:18 Local via ge-1/0/1.141 172.18.5.0/30 *[Direct/OJ 00:09:18 > via ge-1/0/2.141 172 .18. 5 .1/32 *[Local/OJ 00:09:18 Local via ge-1/0/2.141 172.18.112.1/32 *[Local/OJ 00:09:18 Reject 172.18.113.0/30 *[Direct/OJ 00:09:18 > via ge-1/1/3.0 172.18.113.1/32 *[Local/OJ 00:09:18 Local via ge-1/1/3.0 192.168 .11. 0/24 *[Direct/OJ 00:09:18 > via ge-1/0/3.141 192 .168 .11.1/32 *[Local/OJ 00:09:18 Local via ge-1/0/3.141 192.168.31.1/32 *[Direct/OJ 00:09:18 > via loO.O www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-13 Junos Troubleshooting in the NOC Question: Which protocols populate the routing table? Answer: Routes from the protocols static, direct, and local are populating the routing table. Question: Which route points to unknown destinations? Answer: The default static route points to unknown destinations. Question: Which routing table is present in the output? Answer: The inet. O routing table is present in the output. Step 3.2 To test connectivity, ping the virtual router attachedto the remote team's device within your pod. lab@mxA-1> ping remote-teams-VR PING 192.168.12.2 (192.168.12.2): 56 data bytes Ac --- 192.168.12.2 ping statistics 3 packets transmitted, 0 packets received, 100% packet loss Question: What is the result of the ping test? Answer: The ping test was not successful. Lab 5-14 • Control Plane Monitoring and Troubleshooting www.juniper.net Junes Troubleshooting in the NOC Step 3.3 Because the ping test failure in the previous step, issue a traceroute to the remote team's virtual router. lab@mxA-1> traceroute remote-teams-VR traceroute to 192.168.12.2 (192.168.12.2), 30 hops max, 40 byte packets 1 * * * 2 * * * * * * 3 * * * 4 * * * 28 * * * 29 * * * 30 Question: What does the output from the trace route reveal? Answer: The output from the trace route shows that the first router, which is controlled by ISP A, does not have the proper routing knowledge. Step 3.4 Enter configuration mode and navigate to the [edit protocols ospf] hierarchy level. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit protocols ospf [edit protocols ospf] lab@mxA-1# Step 3.5 Add interfaces ge-1/0/2. VLAN-ID, and loO to Area O under the OSPF protocol. Note The unit associated with interfaces ge-1/0/2 and ge-1/0/3 is not unit 0. Take extra care when configuring these interfaces under OSPF. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-15 Junos Troubleshooting in the NOC [edit protocols ospf] lab@mxA-1# set area O interface ge-1/0/2.VLAN-ID [edit protocols ospf] lab@mxA-1# set area O interface 1o0 Step 3.6 Add the ge-1/0/3.141 as a passive interface in Area O under the OSPF protocol. Activate the configuration by issuing the commitcommand. [edit protocols ospf] lab@mxA-1# set area O interface ge-1/0/3.141 passive [edit protocols ospf] lab@mxA-1# commit commit complete Before proceeding, ensure that the remote student team within your pod is ready to continue on to the next step. Step 3.7 Issue the run show ospf interface and run show ospf neighbor commands. Note It might take a minute for the OSPF adjacencies to reach the Full state. [edit protocols ospf] lab@mxA-1# run show ospf interface Interface State Area DR ID BDR ID Nbrs ge-1/0/2.141 BDR 0.0.0.0 192.168.32.1 192.168.31.1 1 ge-1/0/3.141 DRother 0.0.0.0 0.0.0.0 0.0.0.0 0 loO.O DR 0.0.0.0 192.168.31.1 0.0.0.0 0 [edit protocols ospf] lab@mxA-1# run show ospf neighbor Address Interface State ID Pri Dead 172.18.5.2 ge-1/0/2.141 Full 192.168.32.1 128 31 Question: How many OSPF adjacencies are in the Full state? Answer: One OSPF adjacency is in the Full state. Lab 5-16 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Question: Why have no OSPF adjacencies formed over the ge-1/0/3.141 interface? Answer: Remember that the ge-1/0/3.14 1 interface has the passive option. With the passive option no OSPF adjacencies can form over the interface. However, the subnet associated with that interface is present in the OSPF link-state database. Step 3.8 Issue the run show route command and answer the following questions: [edit protocols ospfJ lab@mxA-1# run show route inet.O: 18 destinations, 19 routes (18 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/SJ 01:30:38 > to 172.18.1.1 via ge-1/0/0.141 [Static/7J 01:30:38 > to 172.18.2.1 via ge-1/0/1.141 10.210.15.0/27 *[Direct/OJ 01:30:38 > via fxpO.O 10.210.15.1/32 *[Local/OJ 01: 30:38 Local via fxpO.O 172.18.1.0/30 *[Direct/OJ 01:30:38 > via ge-1/0/0.141 172.18.1.2/32 * [Local/OJ 01:30: 38 Local via ge-1/0/0.141 172.18.2.0/30 *[Direct/OJ 01:30:38 > via ge-1/0/1.141 172.18.2.2/32 *[Local/OJ 01:30:38 Local via ge-1/0/1.141 172.18.5.0/30 *[Direct/OJ 01:30:38 > via ge-1/0/2.141 172.18.5.1/32 *[Local/OJ 01:30:38 Local via ge-1/0/2.141 172.18.112.1/32 *[Local/OJ 01:30:38 Reject 172.18.113.0/30 *[Direct/OJ 01:30: 38 > via ge-1/1/3.0 172.18.113.1/32 *[Local/OJ 01:30: 38 Local via ge-1/1/3.0 192. 168 .11. 0/24 *[Direct/OJ 01:30: 38 > via ge-1/0/3.141 192.168 .11.1/32 *[Local/OJ 01:30:38 Local via ge-1/0/3.141 192.168.12.0/24 *[OSPF/lOJ 00:00:52, metric 2 > to 172.18.5.2 via ge-1/0/2.141 www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-17 Junos Troubleshooting in the NOC 192.168.31.1/32 *[Direct/OJ 01:30:38 > via lo0.0 192.168.32.1/32 *[OSPF/10] 00:00:52, metric 1 > to 172.18.5.2 via ge-1/0/2.141 224.0.0.5/32 *[OSPF/10] 00:43:34, metric 1 MultiRecv Question: Which protocols are populating the routing table? Answer: Routes from the protocols OSPF, static, direct, and local are populating the routing table. Question: Is there a route to the remote team's virtual router? Answer: Yes. There is an OSPF route for the remote team's virtual router. Step 3.9 Next, ping the remote team's virtual router. [edit protocols ospf] lab@mxA-1# run ping remote-teams-VR PING 192.168.12.2 (192.168.12.2): 56 data bytes 64 bytes from 192.168.12.2: icmp_seq=O ttl=64 time=0.568 ms 64 bytes from 192.168.12.2: icmp_seq=l ttl=64 time=0.489 ms "C --- 192.168.12.2 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.489/0.528/0.568/0.039 ms Question: What is the result of the ping test? Answer: The ping test is successful. Using the OSPF protocol to exchange routing information, the Junos OS is able to acquire better routing information. Step3.10 Next, perform a traceroute to the remote team's virtual router. Lab 5-18 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC [edit protocols ospf] lab@mxA-1# run traceroute remote-teams-VR traceroute to 192.168.12.2 (192.168.12.2), 30 hops max, 40 byte packets 1 172.18.5.2 (172.18.5.2) 0.687 ms 0.438 ms 0.386 ms 2 192.168.12.2 (192.168.12.2) 0.610 ms 0.596 ms 0.576 ms Question: What is the result of the traceroute test? Answer: The traceroute reaches the remote team's device over the ge-1/0/2 link, and then reaches the remote team's virtual router through their ge-1/0/3 link. Step3.11 Navigate to the [edit routing-options J hierarchy level. [edit protocols ospf] lab@mxA-1# top edit routing-options [edit routing-options] lab@mxA-1# Step3.12 Configure a local autonomous system of 65412. [edit routing-options] lab@mxA-1# set autonomous-system 65412 Step3.13 Navigate to the [edit protocols bgp J hierarchy level. [edit routing-options] lab@mxA-1# top edit protocols bgp [edit protocols bgp] lab@mxA-1# Note You must use the interface IP addresses associated with ISP A and ISP B for the BGP peering sessions. Please refer to the lab diagram for further details. Step3.14 Configure the BGP group ISP_ A by issuing the edit group ISP-A command. [edit protocols bgp] lab@mxA-1# edit group ISP-A www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-19 Junes Troubleshooting in the NOC [edit protocols bgp group ISP-A] lab@mxA-1# Step 3.15 Configure the neighbor IP address of ISP A's directly connected interface by issuing the set neighbor ISP-A-IP command. [edit protocols bgp group ISP-A] lab@mxA-1# set neighbor ISP-A-IP Step 3.16 Configure the peer autonomous system number for ISP A by issuing the set peer-as 56155 command. [edit protocols bgp group ISP-A] lab@mxA-1# set peer-as 56155 Step 3.17 Configure the BGP group ISP-B by issuing the up command and edit group ISP-B command. [edit protocols bgp group ISP-BJ lab@mxA-1# up [edit protocols bgp] lab@mxA-1# edit group ISP-B [edit protocols bgp group ISP BJ lab@mxA-1# Step3.18 Configure the neighbor IP address of ISP B's directly connected interface by issuing the set neighbor ISP-B-IP command. [edit protocols bgp group ISP_B] lab@mxA-1# set neighbor ISP-B-IP Step 3.19 Configure the peer autonomous system number for ISP B by issuing the set peer-as 56144 command. Activate the configuration by issuing the commitcommand. [edit protocols bgp group ISP BJ lab@mxA-1# set peer-as 56144 [edit protocols bgp group ISP-BJ lab@mxA-1# commit commit complete Step3.20 Issue the run show bgp summarycommand. [edit protocols bgp group ISP-BJ lab@mxA-1# run show bgp summary Lab 5-20 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Groups: 2 Peers: 2 Down peers: 2 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 0 0 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State I #Acti.ve/Recei ved/Accepted/Damped ... 172.18.1.1 56155 0 6 0 0 1:49 Active 172.18.2.1 56144 0 6 0 0 1:49 Active lab@mxA-1> Question: Did the BGP sessions reach the Established state? Answer: No. The BGP sessions transitioned into the Active state but are not able to reach the Established state. Step 3.21 Toinvestigate why the BGP sessions cannot reach the Established state, configure traceoptions under the BGP protocol. Navigate to the [edit protocols bgp traceoptions J hierarchy level. Once under the BGP trace options hierarchy level, configure the flag of open and the file to be bgp-trace.log. Commit the configuration when complete. [edit protocols bgp group ISP-BJ lab@mxA-1# up 1 edit traceoptions [edit protocols bgp traceoptions] lab@mxA-1# set flag open [edit protocols bgp traceoptions] lab@mxA-1# set file bgp-trace.log [edit protocols bgp traceoptions] lab@mxA-1# commit commit complete [edit protocols bgp traceoptions] lab@mxA-1# Step 3.22 Issue the run show log bgp-trace.log command. Note You might have to wait a minute for the traceoptions log file to begin filling with information. [edit protocols bgp traceoptions] lab@mxA-1# run show log bgp-trace.log www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-21 Junos Troubleshooting in the NOC Feb 7 21:53:31 trace on: Tracing to "/var/log/bgp-trace.log" started Feb 7 21:54:08.391037 advertising graceful restart receiving-speaker-only capability to neighbor 172.18.1.1 (External AS 56155) Feb 7 21:54:08.391217 bgp send: sending 59 bytes to 172.18.1.1 (External AS 56155) Feb 7 21:54:08.391259 Feb 7 21:54:08.391259 BGP SEND 172.18.1.2+62244 -> 172.18.1.1+179 Feb 7 21:54:08.391298 BGP SEND message type 1 (Open) length 59 Feb 7 21:54:08.400164 Feb 7 21:54:08.400164 BGP RECV 172.18.1.1+179 -> 172 .18 .1.2+62244 Feb 7 21:54:08.400206 BGP RECV message type 1 (Open) length 59 Feb 7 21:54:08.400296 bgp_process_open:2824: NOTIFICATION sent to 172.18.1.1 (External AS 56155): code 2 (Open Message Error) subcode 2 (bad peer AS number), Reason: peer 172.18.1.1 (External AS 56155) claims 65001, 56155 configured Feb 7 21:54:08.400324 bgp_send: sending 21 bytes to 172.18.1.1 (External AS 56155) Feb 7 21:54:08.400346 Feb 7 21:54:08.400346 BGP SEND 172.18.1.2+62244 -> 172.18.1.1+179 Feb 7 21:54:08.400380 BGP SEND message type 3 (Notification) length 21 Feb 7 21:54:12.391034 advertising graceful restart receiving-speaker-only capability to neighbor 172.18.2.1 (External AS 56144) Feb 7 21:54:12.391122 bgp_send: sending 59 bytes to 172.18.2.1 (External AS 56144) Feb 7 21:54:12.391148 Feb 7 21:54:12.391148 BGP SEND 172.18.2.2+54532 -> 172.18.2.1+179 Feb 7 21:54:12.391182 BGP SEND message type 1 (Open) length 59 Feb 7 21:54:12.400187 Feb 7 21:54:12.400187 BGP RECV 172.18.2.1+179 -> 172.18.2.2+54532 Feb 7 21:54:12.400230 BGP RECV message type 1 (Open) length 59 Feb 7 21:54:12.400319 bgp_process_open:2824: NOTIFICATION sent to 172.18.2.1 (External AS 56144): code 2 (Open Message Error) subcode 2 (bad peer AS number), Reason: peer 172.18.2.1 (External AS 56144) claims 65002, 56144 configured Feb 7 21:54:12.400346 bgp_send: sending 21 bytes to 172.18.2.1 (External AS 56144) Feb 7 21:54:12.400367 Feb 7 21:54:12.400367 BGP SEND 172.18.2.2+54532 -> 172.18.2.1+179 Feb 7 21:54:12.400401 BGP SEND message type 3 (Notification) length 21 Question: What is the cause of the BGP peering problems? Answer: An autonomous system mismatch exists between your assigned device and the ISP A and ISP B routers. Lab 5-22 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Question: What must you do to resolve the issue? Answer: You must change the peer-as value to 65001 for ISP A and the peer-as value to 65002 for ISP B. Step3.23 Remove the traceoptions configuration that you recently configured and conuni t the configuration. Note Traceoptions can be processor intensive. We recommend to only use traceoptions for troubleshooting purposes and remove the traceoptions when the troubleshooting is finished. [edit protocols bgp traceoptions] lab@mxA-1# up [edit protocols bgp] lab@mxA-1# delete traceoptions [edit protocols bgp] lab@mxA-1# commit commit complete [edit protocols bgp] lab@mxA-1# Step 3.24 Change the peer-as value for the BGP group ISP-A to 65001 by issuing the set ISP-A peer-as 65001 command. [edit protocols bgp] lab@mxA-1# set group ISP-A peer-as 65001 Step3.25 Change the peer-as value for the BGP group ISP-B to 65002 by issuing the set group ISP-B peer-as 65002 command. Commit the configuration and exit to operational mode. [edit protocols bgp] lab@mxA-1# set group ISP-B peer-as 65002 [edit protocols bgp] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-23 Junos Troubleshooting in the NOC Step 3.26 Issue the show bgp summary command. Note You might have to wait few minutes for the BGP session to reach the Established state. lab@mxA-1> show bgp smmnary Groups: 2 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 24 11 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped... 0 172.18.1.1 65001 4 5 0 11 10/ 12/12/0 0/0/0/0 0 172.18.2.1 65002 3 6 0 7 1/ 12/12/0 0/0/0/0 Question: Did the BGP sessions reach the Established state? Answer: Yes. Even though the output does not show the Established state, it does show that your assigned device is receiving routes from the ISP routers. Step 3.27 Issue the show route I no-more command. lab@mxA-1> show route I no-more inet.0: 29 destinations, 43 routes (29 active, 0 holddown, O hidden) + =Active Route, - = Last Active, * = Both 0.0.0.0/0 *[Static/SJ 02:06:45 > to 172.18.1.1 via ge-1/0/0.141 [Static/7J 02:06:45 > to 172.18.2.1 via ge-1/0/1.141 [BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 10.210.15.0/27 *[Direct/OJ 02:06:45 > via fxpO.O 10.210.15.1/32 *[Local/OJ 02:06:45 Local via fxp0.0 Lab 5-24 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC 172 .18.1. 0/30 *[Direct/OJ> 02:06:45 via ge-1/0/0.141 172.18.1.2/32 *[Local/OJ 02:06:45 Local via ge-1/0/0.141 172.18.2.0/30 *[Direct/OJ> 02:06:45 via ge-1/0/1.141 172.18.2.2/32 *[Local/OJ 02:06:45 Local via ge-1/0/1.141 172.18.5.0/30 *[Direct/OJ 02:06:45 > via ge-1/0/2.141 172.18.5.1/32 *[Local/OJ 02:06:45 Local via ge-1/0/2.141 172.18.112.1/32 *[Local/OJ 02:06:45 Reject 172.18.113.0/30 *[Direct/OJ 02:06:45 > via ge-1/1/3.0 172.18.113.1/32 *[Local/OJ 02:06:45 Local via ge-1/1/3.0 172.31.16.0/24 *[BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 [BGP/170J 00:00:33, localpref 100 AS path: 802 5512 3318 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 172.31.17.0/24 *[BGP/170J 00:00:33, localpref 100 > AS path: 65001 I, validation-state: unverified to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 > AS path: 65002 I, validation-state: unverified to 172.18.2.1 via ge-1/0/1.141 172.31.18.0/24 *[BGP/170J 00:00:33, localpref 100 > AS path: 65001 I, validation-state: unverified to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 172.31.19.0/24 *[BGP/170J 00:00:33, localpref 100 > AS path: 65001 I, validation-state: unverified to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 > AS path: 65002 I, validation-state: unverified to 172.18.2.1 via ge-1/0/1.141 172.31.20.0/24 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 > AS path: 65002 I, validation-state: unverified to 172.18.2.1 via ge-1/0/1.141 172.31.21.0/24 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-25 Junos Troubleshooting in the NOC 192.168.11.0/24 *[Direct/OJ 02:06:45 > via ge-1/0/3.141 192 .168 .11.1/32 *[Local/OJ 02:06:45 Local via ge-1/0/3.141 192.168.12.0/24 *[OSPF/lOJ 00:36:59, metric 2 > to 172.18.5.2 via ge-1/0/2.141 192 .168. 31.1/32 *[Direct/OJ 02:06:45 > via loO.O 192 .168. 32 .1/32 *[OSPF/lOJ 00:36:59, metric 1 > to 172.18.5.2 via ge-1/0/2.141 192.168.80.0/28 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 192.168.81.0/28 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 192.168.82.0/29 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 192.168.83.0/27 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 192.168.84.0/27 *[BGP/170J 00:00:33, localpref 100 AS path: 65001 I, validation-state: unverified > to 172.18.1.1 via ge-1/0/0.141 [BGP/170J 00:04:33, localpref 100 AS path: 65002 I, validation-state: unverified > to 172.18.2.1 via ge-1/0/1.141 224.0.0.5/32 *[OSPF/lOJ 01:19:41, metric 1 MultiRecv Question: Which protocols are populating the routing table? Answer: Routes from protocols BGP, OSPF, static, direct, and local are populating the routing table. Lab 5-26 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Part4: Configuring Bridging In this lab part, you examine the ARP table, configure bridging, and examine the bridge table. Step4.1 Issue the show arp command. lab@mxA-1> show arp MAC Address Address Name Interface Flags 5c:5e:ab:00:6f:ff 10.210.14.2 10.210.14.2 fxpO.O none 5c:5e:ab:00:16:ff 10.210.14.5 10.210.14.5 fxpO.O none 5c:5e:ab:00:59:ff 10.210.14.6 10.210.14.6 fxpO.0 none 00:10:db:ff:21:50 10.210.14.30 10.210.14.30 fxpO.O none 02:00:00:00:00:10 128.0.0.50 128.0.0.50 emO.O none 84:18:88:8e:ca:95 172.18.1.1 172.18.1.1 ge-1/0/0.141 none 84:18:88:8e:ca:96 172.18.2.1 172.18.2.1 ge-1/0/1.141 none 5c:5e:ab:00:6f:62 172.18.5.2 172.18.5.2 ge-1/0/2.141 none 84:18:88:8e:ca:98 192.168.11.2 192.168.11.2 ge-1/0/3.141 none Total entries: 9 Question: What interfaces are present in the ARP table? Answer: Although your output may might, the output shows that interfaces fxpO.O, em0.0, ge-1/0/0.141, ge-1/0/1.141, ge-1/0/2.141, and ge-1/0/3.141 are present in the ARP table. Step 4.2 Use the clear arp interface ge-1/0/3. 141 command to remove the ARP entry associated with the ge-1/0/3.141 interface . .lab@mxA-1> clear arp interface ge-1/0/3.141 192.168.11.2 deleted Step4.3 Issue the show arp command. www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-27 Junos Troubleshooting in the NOC lab@mxA-1> show arp MAC Address Address Name Interface Flags 5c:5e:ab:00:6f:ff 10.210.14.2 10.210.14.2 fxpO.O none 5c:5e:ab:00:16:ff 10.210.14.5 10.210.14.5 fxpO.O none 5c:5e:ab:00:59:ff 10.210.14.6 10.210.14.6 fxpO. 0 none 00:10:db:ff:21:50 10.210.14.30 10.210.14.30 fxpO.O none 02:00:00:00:00:10 128.0.0.50 128.0.0.50 emO.O none 84:18:88:8e:ca:95 172.18.1.1 172.18.1.1 ge-1/0/0.141 none 84:18:88:8e:ca:96 172.18.2.1 172.18.2.1 ge-1/0/1.141 none 5c:5e:ab:00:6f:62 172.18.5.2 172.18.5.2 ge-1/0/2.141 none Total entries: 8 lab@mxA-1> Question: What is the status of the ARP entry associated with the ge-1/0/3.141 interface? Answer: The ARP entry is not present in the ARP table. Question: What must you do to replace the ARP entry that was associated with the ge-1/0/3.141 interface? Answer: The ARP entry is replaced automatically when trafficis sent to or from the virtual router connected to your device. Step4.4 Enter configuration mode and navigate to the [edit interfaces ge-1/0/2] hierarchy level and issue the delete command to remove any configuration at this level. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit interfaces ge-1/0/2 Lab 5-28 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC [edit interfaces ge-1/0/2] lab@mxA-1# delete Delete everything under this level? [yes,no] (no) yes [edit interfaces ge-1/0/2] lab@mxA-1# Step4.5 Configure a physical encapsulation of ethernet-bridge. [edit interfaces ge-1/0/2] lab@mxA-1# set encapsulation ethernet-bridge Step4.6 Configure family bridge under the logical Unit 0. [edit interfaces ge-1/0/2] lab@mxA-1# set unit O family bridge Step4.7 Navigate to the [edit bridge-domain lab] hierarchy level. [edit interfaces ge-1/0/2] lab@mxA-1# top edit bridge-domains iab Step 4.8 Configure a domain-typeof bridge. [edit bridge-domains lab] lab@mxA-1# set domain-type bridge Step 4.9 Configure the ge-1/0/2 interface to participate in this bridge domain. [edit bridge-domains lab] lab@mxA-1# set interface ge-1/0/2 Step4.10 Commit the configuration and exit to operational mode. [edit bridge-domains lab] lab@mxA-1# connnit and-quit commit complete Exiting configuration mode lab@mxA-1> Step4.11 Issue the show bridge domain detail command. lab@mxA-1> show bridge domain detail Routing instance: default-switch Bridge domain: lab State: Active Bridge VLAN ID: NA www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-29 Junos Troubleshooting in the NOC Interfaces: ge-1/0/2.0 Total mac count: 1 Question: How many MAC addresses have been learned in the bridge domain? Answer: Although your output might vary, the previous output shows that only one MAC address has been learned in the bridge domain. Question: Which VLAN IDs are associated with the lab bridge domain? Answer: The Bridge VLAN ID field displays NA which means there are no VLAN IDs associated with the lab bridge domain. Step4.12 Log out of your assigned device. lab@mxA-1> exit mxA-1 (ttyuO) login: 0 Tell your instructor that you have completed this lab. Lab 5-30 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Management Network Diagram Management Network Management Addressing ·"""--- mxA-1 mxD-1 mxA-2 mxD-2 mxB-1 vr-device mxB-2 Server mxC-1 Gateway mxC-2 Term Server Server Note: Your instructorwill provideaddress and accessinformation. Control Plane Monitoring and Troubleshooting Lab: Pod A Network Diagram Rl Rl (.2) (.2) m---- 0 � "°'" � (.1)...... ---�--, (.1) mxA-1 mxA-2 ge-1/0/2.141 (.1) 172.18.5.0/30 (.2) ge-1/0/2.141 loO: 192.168.31.1 loO: 192.168.32.1 www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-31 Junos Troubleshooting in the NOC Control Plane Monitoring and Troubleshooting Lab: Pod B Network Diagram Internet bf'�2- (,� � - ---...... / � / � ISPA ISPB -,) ) ( R1 Y- R1 9 1... ' (.2) -_/ (.2) �� __ __./ ( (.1)---�---, mxB-2 mxB-1 ge-1/0/2.142 .( 1) 172.18.5.0/30 (.2) ge-1/0/2.142 loO: 192.168.33.1 100: 192.168.34.1 ge-1/0/3-141 .1 ge-1/0/3-141 ( \92.168.13.0/30 192.168.14.0/3d l) (.)2 (.2) vrl-1 Virtual Routers vr2-2 Control Plane Monitoring and Troubleshooting Lab: Pod C Network Diagram R1 (.2) 0 en 0 --...o" .... mxC-1 mxC-2 ge-1/0/2.143 (.1) 172.18.5.0/30 (.21 ge-1/0/2.143 loO: 192.168.35.1 loO: 192.168.36.1 ge-1/0/3-141 .l) ge-1/0/3-141 g;192.168.15.0/30 192.168.160/30{ {.2) vrl-1 Virtual Routers vr2-2 Lab 5-32 • Control Plane Monitoring and Troubleshooting www.juniper.net Junos Troubleshooting in the NOC Control Plane Monitoring and Troubleshooting Lab: Pod D Network Diagram Rl Rl (2) (2) 0 a, '-.. 0 '-.. � """' � 8 (.1)..... ---��-. (.1) mxD-1 mxD-2 ge-1)0/2.144(.1) 172.18.5.0/30 (2l ge-1/0/2.144 100: 192.168.37.1 loO: 192.168.38.1 www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5-33 Junos Troubleshooting in the NOC Lab 5-34 • Control Plane Monitoring and Troubleshooting www.juniper.net Lab Monitoring and Troubleshooting Ethernet Interfaces Overview This lab demonstrates how to monitor and troubleshoot Ethernet interfaces. You will examine and troubleshoot an Ethernet interface that is in the physical down state. You will then set that interface in local loopback mode to test the health of the interface. Next, you will configure Ethernet OAM and monitor the operation of it. By completing this lab, you will perform the following tasks: Troubleshoot an Ethernet interface. Configure and run a local loopback test. Configure and monitor Ethernet OAM. www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-1 Junos Troubleshooting in the NOC Part 1: Performing Interface Troubleshooting In this lab part, you will troubleshoot a link-down issue. You will first discover the issue, devise a solution, and implement that solution. Note Depending on the class, the lab equipment used might be remote from your physical location. The instructor will inform you as to the nature of your access and will provide you the details needed to access your assigned device. Step 1.1 Ensure that you know to which student device you have been assigned. Check with your instructor if you are not certain. Consult the Management Network Diagram to determine the management address of your student device. Question: What is the management address assigned to your station? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CU for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Lab 6-2 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC v Protocol: [---Telnet------·-·------.-··-····-·--·------·····------, Hostname: I Port: Firewall: v O Show quick connect on startup 0 Save session 0 Open in a tab Connect j j Cancel Step 1.3 Log in to your device with the username lab and the password lab123. Note that both the name and password are case-sensitive. mxA-1 (ttyuO) login: lab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/lab6-start. con£ig command. Activate the configuration by issuing the commitcommand. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# load override jtnoc/lab6-start.config load complete [edit] lab@mxA-1# conunit commit complete [edit] lab@mxA-1# www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-3 Junos Troubleshooting in the NOC Step 1.5 Issue therun show interfaces terse ge-1/0/9 command. [edit] lab@mxA-1# run show interfaces terse ge-1/0/9 Interface Admin Link Proto Local Remote ge-1/0/9 up down ge-1/0/9.0 up down inet 10.10.10.1/30 multiservice Question: What is the status of the ge-1/0/9 interface? Answer: The ge-1/0/9 interface is in the link down state. Question: Which commands can you use to investigate the problem further? Answer: You can use the show log messages, show log chassisd,show interfaces, show interfaces media,and theshow interfaces extensive commands to troubleshoot this issue. Question: Is the monitor traffic interface command for the ge-1/0/9 interface useful in this situation? Why? Answer: No. The command cannot be run on an interface that is in the link down state. Step i6 Issue the run show log messages I match ge-1/0/9 command. [edit] lab@mxA-1# run show log messages I match ge-1/0/9 Jan 31 01:13:43 mxA-1 chassisd[1281]: CHASSISD_IFDEV_CREATE NOTICE: create_pics: created interface device for ge-1/0/9 Jan 31 01:13:47 mxA-1 tfebO mic_sfp_phy_create: - Create SFP PHY ge-1/0/9 successfully Lab 6-4 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Jan 31 01:13:54 mxA-1 mib2d[l417]: SNMP_TRAP_LINK_DOWN: ifindex 521, ifAdminStatus up(1), if.OperStatus down (2), ifName ge-1/0/9 Feb 5 15:29:42 mxA-1 mib2d[l417]: SNMP_TRAP_LINK_DOWN: ifindex 640, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/0/9.0 Feb 5 15:29:50 mxA-1 mib2d[l417]: SNMP_TRAP_LINK_DOWN: ifindex 521, ifAdminStatus up(l), ifOperStatus down(2), ifName ge-1/0/9 Feb 6 19:46:50 mxA-1 mib2d[l417]: SNMP_TRAP_LINK_DOWN: ifindex 640, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/0/9.0 Feb 7 01:43:29 mxA-1 mib2d[l417]: SNMP_TRAP_LINK_DOWN: ifindex 521, ifAdminStatus up(l), ifOperStatus down(2), ifName ge-1/0/9 Feb 7 01:43:31 mxA-1 alarmd[l282]: Alarm set: ge-1/0/9 color=RED, class=ETHER, reason=ge-1/0/9: Link down Feb 7 01:43:31 mxA-1 craftd[l283]: Major alarm set, ge-1/0/9: Link down Question: Does the previous output contain any information that is helpful in troubleshooting this issue? Answer: Although the previous output does not tell you why the interface is physically down, it does lead you to believe that this is not a hardware issue. Step 1.7 Issue the run show log chassisd I match ge-1/0/9 command. [edit] lab@mxA-1# run show log chassisd I match ge-1/0/9 Jan 10 22:57:05 CHASSISD IFDEV CREATE NOTICE: create_pics: created interface device for ge-1/0/9 Jan 10 22:57:05 ifdev_create entered ge-1/0/9 Jan 10 22:57:05 ge-1/0/9: large delay buffer cleared Jan 31 01:07:25 ifd ge-1/0/9 marked as gone Jan 31 01:13:43 Created pie for ge-1/0/9 Jan 31 01:13:43 CHASSISD IFDEV CREATE NOTICE: create_pics: created interface device for ge-1/0/9 Jan 31 01:13:43 ifdev_create entered ge-1/0/9 Jan 31 01:13:43 ge-1/0/9: large delay buffer cleared www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-5 Junos Troubleshooting in the NOC Question: Does the previous output contain any information that is helpful in troubleshooting this issue? Answer: Although the previous output does not tell you why the interface is physically down, it also leads you to believe that this is not a hardware issue. Step 1.8 Issue the run clear interfaces statistics ge-1/0/9 command, wait a minute, and then issue the run show interfaces ge-1/0/9 extensive command. Note Clearing the interface statistics before gathering statistical information is a recommended troubleshooting step. This step removes any stale information that is not related to the present issue. [edit] lab@mxA-1# run clear interfaces statistics ge-1/0/9 [edit] lab@mxA-1# run show interfaces ge-1/0/9 extensive Physical interface: ge-1/0/9, Enabled, Physical link is Down Interface index: 149, SNMP ifindex: 521, Generation: 152 Link-level type: Ethernet, MTU: 1514, Speed: lOOOmbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags Present Running Down Interface flags: Hardware-Down SNMP-Traps Internal: OxO Link flags None Cos queues 8 supported, 8 maximum usable queues Hold-times Up Oms, Down Oms Current address: 80:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 Last flapped 2013-02-07 01:43:29 UTC (01:19:31 ago) Statistics last cleared: 2013-02-07 03:02:36 UTC (00:00:24 ago) Traffic statistics: Input bytes 0 Obps Output bytes 0 Obps Input packets: 0 Opps Output packets: 0 Opps IPv6 transit statistics: Input bytes 0 Output bytes 0 Lab 6-6 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Input packets: 0 Output packets: 0 Dropped traffic statistics due to STP State: Input bytes 0 Output bytes 0 Input packets: 0 Output packets: 0 Input errors: Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 Output errors: Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0 Egress queues: 8 supported, 4 in use Queue counters: Queued packets Transmitted packets Dropped packets O best-effort O 0 0 1 expedited-fo 0 0 0 2 assured-forw 0 0 0 3 network-cont 0 0 0 Queue number: Mapped forwarding classes 0 best-effort 1 expedited-forwarding 2 assured-forwarding 3 network-control Active alarms LINK Active defects LINK MAC statistics: Receive Transmit Total octets 0 0 Total packets 0 0 Unicast packets 0 0 Broadcast packets 0 0 Multicast packets 0 0 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 0 0 MAC pause frames 0 0 Oversized frames 0 Jabber frames 0 Fragment frames 0 VLAN tagged frames 0 Code violations 0 Total errors 0 0 Filter statistics: Input packet count 0 Input packet rejects 0 Input DA rejects 0 Input SA rejects 0 Output packet count 0 Output packet pad count 0 Output packet error count 0 CAM destination filters: 0, CAM source filters: 0 Autonegotiation information: Negotiation status: Incomplete Packet Forwarding Engine configuration: www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-7 Junos Troubleshooting in the NOC Destination slot: 1 Cos information: Direction : Output CoS transmit queue Bandwidth Buffer Priority Limit % bps % usec O best-effort 95 950000000 95 O low none 3 network-control 5 50000000 5 0 low none Interface transmit statistics: Disabled Logical interface ge-1/0/9.0 (Index 331) (SNMP ifindex 640) (Generation 225) Flags: Device-Down SNMP-Traps OxO Encapsulation: ENET2 Traffic statistics: Input bytes O Output bytes O Input packets: 0 Output packets: 0 Local statistics: Input bytes 0 Output bytes 0 Input packets: 0 Output packets: 0 Transit statistics: Input bytes O O bps Output bytes O O bps Input packets: 0 O pps Output packets: 0 O pps Protocol inet, MTU: 1500, Generation: 330, Route table: 0 Flags: Sendbcast-pkt-to-re Addresses, Flags: Dest-route-down Is-Preferred Is-Primary Destination: 10.10.10.0/30, Local: 10.10.10.1, Broadcast: 10.10.10.3, Generation: 266 Protocol multiservice, MTU: Unlimited, Generation: 331, Route table: O Flags: Is-Primary Policer: Input: default_arp_policer� Question: Are any errors present in this output? Answer: No errors should be present in this output. Step 1.9 Issue the run show interface ge-1/0/9 media command. [edit] lab@mxA-1# run show interfaces ge-1/0/9 media Physical interface: ge-1/0/9, Enabled, Physical link is Down Interface index: 149, SNMP ifindex: 521 Link-level type: Ethernet, MTU: 1514, Speed: lOOOmbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags : Present Running Down Lab 6-8 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Interface flags: Hardware-Down SNMP-Traps Internal: OxO Link flags None Cos queues 8 supported, 8 maximum usable queues Current address: B0:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 Last flapped 2013-02-07 01:43:29 UTC (01:20:50 ago) Input rate O bps (0 pps) Output rate O bps (0 pps) Active alarms LINK Active defects LINK MAC statistics: Input bytes: 0, Input packets: 0, Output bytes: 0, Output packets: 0 Filter statistics: Filtered packets: O, Padded packets: 0, Output packet errors: 0 Autonegotiation information: Negotiation status: Incomplete Interface transmit statistics: Disabled Question: Does the previous output contain any information that is helpful in troubleshooting this issue? Answer: The previous output reveals that the negotiation status for the link is incomplete. This information is visible in the show interfaces ge-1/0/9 extensive command, but it was buried among the large amount of output produced by that command. Question: What might cause autonegotiation to fail and the interface to be declared physically down? Answer: A link speed mismatch can cause autonegotiation to fail and the interface to be declared physically down. Step 1.10 Navigate to the [edit interfaces ge-1/0/9] hierarchy level. [edit] lab@mxA-1# edit interfaces ge-1/0/9 [edit interfaces ge-1/0/9] lab@mxA-1# www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-9 Junos Troubleshooting in the NOC Step 1.11 Change the link speed for interface ge-1/0/9 to 100 Mbps. Activate the configuration by issuing the commit command. [edit interfaces ge-1/0/9] lab@mxA-1# set speed lOOm [edit interfaces ge-1/0/9] lab@mxA-1# connnit commit complete [edit interfaces ge-1/0/9] lab@mxA-1# Step 1.12 Issue the run show interfaces ge-1/0/9 media command. [edit interfaces ge-1/0/9] lab@mxA-1# run show interfaces ge-1/0/9 media Physical interface: ge-1/0/9, Enabled, Physical link is Down Interface index: 149, SNMP ifindex: 521 Link-level type: Ethernet, MTU: 1514, Speed: lOOmbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags Present Running Down Interface flags: Hardware-Down SNMP-Traps Internal: OxO Link flags None CoS queues 8 supported, 8 maximum usable queues Current address: 80:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 Last flapped 2013-02-07 01:43:29 UTC (01:24:56 ago) Input rate O bps (0 pps) Output rate O bps (0 pps) Active alarms LINK Active defects LINK MAC statistics: Input bytes: 0, Input packets: 0, Output bytes: 0, Output packets: 0 Filter statistics: Filtered packets: 0, Padded packets: 0, Output packet errors: 0 Autonegotiation information: Negotiation status: Incomplete Interface transmit statistics: Disabled Question: Did changing the link speed solve the autonegotiation issue? Answer: No. The negotiation status displays Incomplete. Lab 6-10 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Question: Because setting the link speed to 100 Mbps did not solve the problem, what is another possible troubleshooting step? Answer: By default, the link speed is set to 1 Gbps and the only other link speed option is 10 Mbps. Step 1.13 Change the link speed for the ge-1/0/9 interface to 10 Mbps and commit the configuration. [edit interfaces ge-1/0/9] lab@mxA-1# set speed lOm [edit interfaces ge-1/0/9] lab@mxA-1# commit commit complete Step 1.14 Issue the run show interfaces ge-1/0/9 media command. [edit interfaces ge-1/0/9] lab@mxA-1# run show interfaces ge-1/0/9 media Physical interface: ge-1/0/9, Enabled, Physical link is Up Interface index: 149, SNMP ifindex: 521 Link-level type: Ethernet, MTU: 1514, Speed: lOm, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags Present Running Interface flags: SNMP-Traps Internal: OxO Link flags None Cos queues 8 supported, 8 maximum usable queues Current address: 80:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 Last flapped 2013-02-07 03:09:59 UTC (00:00:22 ago) Input rate O bps (0 pps) Output rate O bps (0 pps) Active alarms None Active defects None MAC statistics: Input bytes: 0, Input packets: O, Output bytes: 0, Output packets: 0 Filter statistics: Filtered packets: 0, Padded packets: 0, Output packet errors: 0 Autonegotiation information: Negotiation status: Complete Link partner: Link mode: Full-duplex, Flow control: Symmetric/Asymmetric, Remote fault: OK, Link partner Speed: 10 Mbps Local resolution: Flow control: Symmetric, Remote fault: Link OK Interface transmit statistics: Disabled www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-11 Junos Troubleshooting in the NOC Question: Did changing the link speed to 10 Mbps solve the autonegotiation problem? What is the autonegotiation status? Answer: Yes. The autonegotiation status has reached the Complete state and the link is physically up. Also, take note that through autonegotiation your router was able to detect a received link speed of 10 Mbps. Step 1.15 To ensure that the ge-1/0/9 interface is now working correctly, exit to operational mode and issue the ping 10 .10 .10.2 detail. count 5 command to test connectivity to your local R1 device. [edit interfaces ge-1/0/9] lab@mxA-1# exit configuration-mode Exiting configuration mode lab@mxA-1> ping 10.10.10.2 detail. count 5 PING 10.10.10.2 (10.10.10.2): 56 data bytes 64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp seq=O ttl=64 time= 2.221 ms 64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=l ttl= 64 time= O. 856 ms 64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq =2 tt1=64 time= 0.850 ms 64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp- seq= 3 ttl=64 time= 0.803 ms 64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq= 4 ttl=64 time= 0.865 ms --- 10.10.10.2 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.803/l.119/2.221/0.551 ms lab@mxA-1> Question: What did the ping test reveal? Answer: The ping test shows that Layers 1- 3 are functional across the link. Lab 6-12 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Question: Why is it important to test Layer 3 connectivity on the link? Answer: Even though a link might show that it is in the operational state, an intermediate device might be stopping Layer 3 communication rendering the link unusable. Part 2: Performing Loopback Testing In this lab part, you will configure an interface in local loopback mode to test the integrity of the ge-1/0/9 interface. Step 2.1 Enter configuration mode and navigate to the [edit interfaces ge-1/0/9] hierarchy level. Then, configure the loopback option. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit interfaces ge-1/0/9 [edit interfaces ge-1/0/9] lab@mxA-1# set gigether-options loopback Step 2.2 Issue the run show interfaces ge-1/0/9 I match hardware command to obtain the MAC address associated with the ge-1/0/9 interface. [edit interfaces ge-1/0/9] lab@mxA-1# run show interfaces ge-1/0/9 I match hardware Current address: 80:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 Question: What is the MAC address associated with the ge-1/0/9 interface? Answer: Although your output might vary, the previous output shows that the MAC address for the ge-1/0/9 interface is 80:71:1f:c3:03:69. www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-13 Junos Troubleshooting in the NOC Step 2.3 Configure a static ARP entry for the 10.10.10.2 IP address under the 10.10.10.1/24 prefix, which resolves to the MAC address associated with the ge-1/0/9 interface. [edit interfaces ge-1/0/9) lab@mxA-1# set unit O family inet address 10.10.10.1/30 arp 10.10.10.2 mac XX:XX:XX:XX:XX:XX [edit interfaces ge-1/0/9) lab@mxA-1# show speed lOm; gigether-options loopback; unit O { family inet { address 10.10.10.1/30 arp 10.10.10.2 mac 80:71:lf:c3:03:69; Step 2.4 Commit the configuration and exit to operational mode. Then issue the command clear interfaces statistics ge-1/0/9 to reset all of the statistics on the interface. [edit interfaces ge-1/0/9) lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> clear interfaces statistics ge-1/0/9 lab@mxA-1> Step 2.5 Issue the ping 10 .10 .10. 2 rapid count 10 command to loop traffic inside the ge-1/0/9 interface. Note The expected behavior is to see a time-to-live exceeded message. lab@mxA-1> ping 10.10.10.2 rapid count 10 PING 10.10.10.2 (10.10.10.2): 56 data bytes 36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 4fdd O 0000 01 01 4lb6 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Ost 4 5 00 0054 4fe6 0 0000 01 01 4lad 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst Lab 6-14 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC 4 5 00 0054 4ff0 0 0000 01 01 4la3 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 4ff9 0 0000 01 01 419a 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 5003 0 0000 01 01 4190 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 500c 0 0000 01 01 4187 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 5017 0 0000 01 01 417c 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 5022 0 0000 01 01 4171 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 502f 0 0000 01 01 4164 10.10.10.1 10.10.10.2 .36 bytes from 10.10.10.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 5038 0 0000 01 01 415b 10.10.10.1 10.10.10.2 10.10.10.2 ping statistics --- 10 packets transmitted, 0 packets received, 100% packet loss Question: Why are the Time to live exceeded messages appearing? Answe�TheTime to live exceeded messages are appearing because ping packets are being looped inside the interface hardware until the time-to-live field expires. Step 2.6 Issue the show interface ge-1/0/9 extensive command. lab@mxA-1> show interfaces ge-1/0/9 extensive Physical interface: ge-1/0/9, Enabled, Physical link is Up Interface index: 149, SNMP ifindex: 521, Generation: 152 Link-level type: Ethernet, MTU: 1514, Speed: lOm, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Enabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags Present Running Loop-Detected Interface flags: SNMP-Traps Internal: OxO Link flags None CoS queues 8 supported, 8 maximum usable queues Hold-times Up Oms, Down Oms Current address: 80:71:lf:c3:03:69, Hardware address: 80:71:lf:c3:03:69 www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-15 Junos Troubleshooting in the NOC Last flapped : 2013-02-07 03:09:59 UTC (00:28:06 ago) Statistics last cleared: 2013-02-07 03:36:14 UTC (00:01:51 ago) Traffic statistics: Input bytes 53760 0 bps Output bytes 53960 0 bps Input packets: 640 0 pps Output packets: 640 0 pps IPv6 transit statistics: Input bytes 0 Output bytes 0 Input packets: 0 Output packets: 0 Dropped traffic statistics due to STP State: Input bytes 0 Output bytes 0 Input packets: 0 Output packets: 0 Input errors: Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 Output errors: Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: O, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0 Egress queues: 8 supported, 4 in use Queue counters: Queued packets Transmitted packets Dropped packets O best-effort 640 640 0 1 expedited-fa 0 0 0 2 assured-forw 0 0 0 3 network-cont 0 0 0 Queue number: Mapped forwarding classes 0 best-effort 1 expedited-forwarding 2 assured-forwarding 3 network-control Active alarms None Active defects None MAC statistics: Receive Transmit Total octets 65280 65280 Total packets 640 640 Unicast packets 640 640 Broadcast packets 0 0 Multicast packets 0 0 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 0 0 MAC pause frames 0 0 Oversized frames 0 Jabber frames 0 Fragment frames 0 VLAN tagged frames 0 Code violations 0 Total errors 0 0 Filter statistics: Input packet count 640 Lab 6-16 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Input packet rejects 0 Input DA rejects 0 Input SA rejects 0 Output packet count 640 Output packet pad count 0 Output packet error count 0 CAM destination filters: 0, CAM source filters: 0 Autonegotiation information: Negotiation status: Incomplete Packet Forwarding Engine configuration: Destination slot: 1 Cos information: Direction : Output Cos transmit queue Bandwidth Buffer Priority Limit % bps % usec O best-effort 95 9500000 95 0 low none 3 network-control 5 500000 5 0 low none Interface transmit statistics: Disabled Logical interface ge-1/0/9.0 (Index 331) (SNMP ifindex 640) (Generation 225) Flags: SNMP-Traps OxO Encapsulation: ENET2 Traffic statistics: Input bytes 53760 Output bytes 53900 Input packets: 640 Output packets: 640 Local statistics: Input bytes 0 Output bytes 980 Input packets: 0 Output packets: 10 Transit statistics: Input bytes 53760 O bps Output bytes 52920 O bps Input packets: 640 O pps Output packets: 630 O pps Protocol inet, MTU: 1500, Generation: 330, Route table: 0 Flags: Sendbcast-pkt-to-re Addresses, Flags: Is-Preferred Is-Primary Destination: 10.10.10.0/30, Local: 10.10.10.1, Broadcast: 10.10.10.3, Generation: 272 Protocol multiservice, MTU: Unlimited, Generation: 331, Route table: 0 Flags: Is-Primary Policer: Input: default arp_policer� www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-17 Junos Troubleshooting in the NOC Question: Does this output show that this interface is in local loopback mode? Answer: Yes, the Device flags field shows the flag Loop-Detected, which lets you know that the interface is running in local loopback mode. Question: How many input and output packets are recorded on this interface? Is this what you expected to find? Answer: There are 640 input and output packets are on the interface. Even though only 10 ping packets sent, every one passed through the interface 64 times, which is the default time-to-live value for ping packets in the Junos OS. Question: Are any errors listed in this output? Answer: No. Under the Input errors and Output errors sections, none of the counters have increased. Question: Is this interface hardware in good condition? Why? Answer: Yes. The interface hardware is in good condition. No errors are present after the local loopback testing. Lab 6-18 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junes Troubleshooting in the NOC Part 3: Configuring Ethernet OAM In this lab part, you will configure Ethernet OAM and monitor its operation. Step 3.1 Enter configuration mode and navigate to the [edit protocols oam ethernet link-fault-management] hierarchy level. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit protocols oam ethernet link-fault-management [edit protocols oam ethernet link-fault-management] lab@mxA-1# Step 3.2 Configure the ge-1/0/9 interface to participate in remote loopback mode. [edit protocols oam ethernet link-fault-management] lab@mxA-1# set interface ge-1/0/9 remote-loopback Step 3.3 Configure the ge-1/0/9 interface with the allow-remote-loopback option under the negotiation-options. Activate the configuration and exit to operational mode by issuing the commit and-quit command. [edit protocols oam ethernet link-fault-management] lab@mxA-1# set interface ge-1/0/9 negotiation-options allow-remote-loopback [edit protocols oam ethernet link-fault-management] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-l> Step 3.4 Issue the show oam ethernet link-fault-management command and answer the following questions: lab@mxA-1> show oam ethernet link-fault-management Interface: ge-1/0/9 Status: Running, Discovery state: Send Any Peer address: 80:71:lf:c3:03:69 Flags:Remote-Stable Remote-State-Valid Local-Stable Ox50 Remote loopback status: Enabled on local port, Enabled on peer port Remote entity information: Remote MUX action: forwarding, Remote parser action: loopback Discovery mode: active, Unidirectional mode: unsupported Remote loopback mode: supported, Link events: supported Variable requests: unsupported www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-19 Junos Troubleshooting in the NOC Question: What is the OAM status of the ge-1/0/9 interface? Answer: As shown in the previous output, the Status field shows that OAM is running on the interface. Question: What is the peer address? Where is it being acquired from? Answer: Although your output might vary, the peer address in the previous output is 80:71:1f:c3:03:69. The MAC address is acquired from the ge-1/0/9 interface. Question: Why is the MAC address of the ge-1/0/9 interface being used as the peer address? Answer: The ge-1/0/9 interface is detecting its own MAC address as its peer because it has been placed in local loopback mode in the previous lab part. Step 3.5 Log out of your assigned device. lab@mxA-1> exit mxA-1 (ttyuO) login: Tell your instructor that you have completed this lab. Lab 6-20 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Management Network Diagram Management Network Management Addressing mxA-1 mxlH mxA-2 mx0-2 mxB-1 vr-device mxB-2 ____ /_ Server mxC-1 Gateway mxC-2 Tenn Server Server Note: Your instructorwill provideaddress and access information. Monitoring and Troubleshooting Ethernet Interfaces lab: Pod A Network Diagram R1 (.2) 0 en ---- 0 ;; "";, cj (.1) mxA-2 mxA-1 ge-1/0/2.141 (.1) 172.185.0/30 (.2) ge-1/0/2.141 loO: 192.168.31.1 loO: 192.168.32.1 www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-21 Junos Troubleshooting in the NOC Monitoring and Troubleshooting Ethernet Interfaces Lab: Pod B Network Diagram R1 (.2) m ..___ 0 () ;::;- -ri () co 0 mxB-1 ge-1/0/2.142 (.1) 172.18.5.0/30 (.2) ge-1/0/2.142 mxB-2 loO: 192.168.33.1 loO: 192.168.34.1 ge-1/0/3.l4l l . ge-1/0/3.l4l (. i192.168.13.0/30 192.168.14.0/30' l) (.2) (.2) vrl-1 Virtual Routers vr2-2 Monitoring and Troubleshooting Ethernet Interfaces Lab: Pod C Network Diagram R1 R1 (.2) (.2) 0 m ..___ 0 ..-; � 0 ""° 0 -ri (.1)..... ------, mxC-1 ge-1/0/2.143 (.1) 172.18.5.0/30 ( 2) ge-1/0/2.143 mxC-2 loO: 192.168.35.1 loO: 192.168.36.1 ge-1/0/3-141 (.i; 1) ge-1/0/3-141 ( /92 .l-3o .1 5 . 0;30 . 192.168.16.Qf3o'" .2 {.2) vr1-1 Virtual Routers vr2-2 �.. /'--· � �Af"' f"'s�"A"'""" c,20.1.4 ___ ._...... ,. �Ufll� WO!fd�•\E!fucati�� ""'"'".._""' Lab 6-22 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Junos Troubleshooting in the NOC Monitoring and Troubleshooting Ethernet Interfaces Lab: Pod D NetworkDiag ram R1 R1 (.2) (.2) 0 cr, o' 0 � cj ab (.1) (.1) mxD-1 ge-J/0/2.144 (.1) 172.18.5.0/30 i.2) ge-1/0/2.144 mxD-2 loO: 192.168.37.1 loO: 192.168.38.1 www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6-23 Junes Troubleshooting in the NOC Lab 6-24 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net Lab Isolating and Troubleshooting PFE Issues Overview This lab demonstrates how to examine and work with the forwarding table, configure and examine load balancing, and troubleshoot firewall filter issues. By completing this lab, you will perform the following tasks: Examine and remove entries from the forwarding table. Configure and verify load balancing. Troubleshoot firewall filter issues. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-1 Junos Troubleshooting in the NOC Part 1: Examiningthe Forwarding Table In this lab part, you will examine the forwarding table and its relation to the routing table. You will also clear entries in the forwarding table. Note Depending on the class, the lab equipment used might be remote from your physical location. The instructor will inform you as to the nature of your access and will provide you the details needed to access your assigned device. Step 1.1 Ensure that you know to which student device you have been assigned. Check with your instructor if you are not certain. Consult the Management Network Diagram to determine the management address of your student device. Question: What is the management address assigned to your station? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CLI for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Lab 7 -2 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Protocol: �------et-··------� v Hostname: I Port: D Show quick connect on startup 0 Save session 0 Open in a tab [ Connect [ [ Cancel Step 1.3 Log in to your device with the username lab and the password lab123. Note that both the name and password are case-sensitive. mxA-1 (ttyuO) login: 1ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 1.4 Enter into configuration mode and load the device's reset configuration by issuing the load override jtnoc/lab7-start. config command. After the configuration has been loaded, committhe changes before moving on to the next step. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# 1oad override jtnoc/1ab7-start.config load complete [edit] lab@mxA-1# commit commit complete [edit] lab@mxA-1# www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-3 Junos Troubleshooting in the NOC Step 1.5 Navigate to the [edit routing-options J hierarchy level and configure the static route of 192.168.50.1/32 with a next hop of the IP address of the directly connected interface of your team's VR device. Activate the configuration by issuing the conunit command. [edit] lab@mxA-1# edit routing-options [edit routing-options] lab@mxA-1# set static route 192.168.50.1 next-hop .loca.l-VR-IP [edit routing-options] lab@mxA-1# connnit commit complete [edit routing-options] lab@mxA-1# Step 1.6 Examine the routing table and the forwarding table for the 192.168.50.1 route by issuing the run show route 192 .168.50 .1 and the run show route forwarding-table destination 192 .168. 50 .1 commands. [edit routing-options] lab@mxA-1# run show route 192.168.50.1 inet.0: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 192.168.50.1/32 *[Static/SJ 00:00:30 > to 192.168.11.2 via ge-1/0/3.141 [edit routing-options] lab@mxA-1# run show route forwarding-table destination 192.168.50.1 Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif 192.168.50.1/32 user O 192.168.11.2 ucst 572 3 ge-1/0/3.141 Routing table: master.anon .inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm O rjct 540 l Question: What information for this route matches in both the routing and the forwarding tables? Answer: The next hop IP address and interface match in the output obtained from the routing and forwarding tables. Lab 7 -4 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Step 1.7 Issue the run clear route forwarding-table 192 .168. 50 .1 l.ocal.-VR-IPcommand. Note The route part of the clear route forwarding table command is a hidden command and will not autocomplete. [edit routing-options] lab@mxA-1# run clear route forwarding-table 192.168.50.1 iocai-VR-IP delete 192.168.50.1: gateway 192.168.11.2 Step 1.8 Issue the run show route 192 .168. 50 .1 and the run show route forwarding-table destination 192 .168. 50 .1 commands. [edit routing-options] lab@mxA-1# run show route 192.168.50.1 inet.0: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 192.168.50.1/32 *[Static/5] 00:02:11 > to 192.168.11.2 via ge-1/0/3.141 [edit routing-options] lab@mxA-1# run show route forwarding-table destination 192.168.50.1 Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Net if default user 0 84:18:88:8e:ca:95 ucst 585 14 ge-1/0/0.141 default perm 0 rjct 36 2 Routing table: master.anon .inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm O rjct 540 1 Question: Is there anything that does not match between the routing and forwarding table for this route? Answer: Yes. The next-hop interface is different for the route. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -5 Junos Troubleshooting in the NOC Question: What happens to a packet that is destined for the 192.168.50.1 IP address? Answer: Your answer might vary, however, in the previous output the packet would be routed through the ge-1/0/0.141 interface. Question: What can you do to synchronize the route in the routing and forwarding table? Answer: You must remove the route from the routing table and then reapply it to the routing table. This process also adds the new routing information to the forwarding table. Step 1.9 To remove the 192.168.50.1 route from the routing table deactivate the static route from the configuration, commit the configuration, and activate the static route. Activate the configuration and exit to operational mode by issuing the commit and-quit command. [edit routing-options] lab@mxA-1# deactivate static route 192.168.50.1/32 [edit routing-options] lab@mxA-1# commit commit complete [edit routing-options] lab@mxA-1# activate static route 192.168.50.1/32 [edit routing-options] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 1.10 Issue the show route 192 .168. 50 .1 and the show route forwarding-table destination 192 .168. 50 .1 commands. lab@mxA-1> show route 192.168.50.1 inet.O: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * Both Lab 7 -6 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC 192.168.50.1/32 *[Static/SJ 00:00:09 > to 192.168.11.2 via ge-1/0/3.141 lab@mxA-1> show route forwarding-table destination 192.168.50.1 Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif 192.168. 50 .1/32 user O 192.168.11.2 ucst 572 3 ge-1/0/3.141 Routing table: master.anon .inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm O rjct 540 1 Question: Does the forwarding table have the correct information for the route? Answer: Yes. The forwarding table has the correct information for the route, and traffic will be forwarded out the interface listed in the routing table. Part 2: Configuring Load Balancing In this lab part, you will configure and examine the effects of load balancing. First you will load a configuration on the router that contains a preconfigured BGP group that will be receiving routes from a single AS. Please take a few minutes to examine the lab diagram. Step 2.1 Enter configuration mode and issue the l.oad override j tnoc/ l.ab7-p2-start.con£ig command, then active the configuration by issuing the commitcommand. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# load override jtnoc/iab7-p2-start.config load complete [edit] lab@mxA-1# connnit commit complete [edit] lab@mxA-1# Step 2.2 Issue the run show bgp summary command. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-7 Junes Troubleshooting in the NOC [edit] lab@mxA-1# run show bgp summary Groups: 1 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 24 23 0 0 0 0 Peer AS In Pkt Out Pkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped... 172.18.6.1 65000 5 7 0 0 37 11/ 12/12/0 0/0/0/0 172.18.7.1 65000 4 6 0 0 33 12/ 12/12/0 0/0/0/0 Question: Is your assigned device receiving approximately the same amount of routes from both its BGP peers? Answer: Yes. The number of routes received from both peers is almost equal. Step 2.3 Issue the run show route 172. 31. 20. 0/24 command. [edit] lab@mxA-1# run show route 172.31.20.0/24 inet.0: 24 destinations, 36 routes (24 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 172.31.20.0/24 *[BGP/170] 00:01:21, localpref 100, from 172.18.6.1 AS path: 65000 I to 172.18.6.1 via ge-1/0/0.141 > to 172.18.7.1 via ge-1/0/1.141 [BGP/170] 00:01:17, localpref 100 AS path: 65000 I > to 172.18.7.1 via ge-1/0/1.141 Lab 7 -8 • Isolating and Troubleshooting PFE Issues www.juniper.net Junes Troubleshooting in the NOC Question: Does the previous command alone give enough information to determine if traffic destined for the 172.31.20.0/24 network will be load balanced across the two links? Answer: No. Although the previous command does let you know that the routing plane is set up to load balance to the 172.31.20.0/24 network, you must examine the forwarding plane to determine if load balancing will occur. Step 2.4 Issue the run show route forwarding-table destination 172. 31. 20. 0/24 command. [edit] lab@mxA-1# run show route forwarding-table destination 172.31.20.0/24 Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif 172.31.20.0/24 user O ulst 1048574 12 172.18.8.1 ucst 548 4 ge-1/0/1.141 Routing table: master.anon .inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm O rjct 527 1 Question: Will load balancing occur for the 172.31.20.0/24 prefix? Why? Answer: No. In the previous output, only one next hop and only one interface are listed for the prefix. Question: What must you do to enable load balancing? Answer: You must create a policy that allows load balancing and then you must apply that policy to the forwarding table. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -9 Junos Troubleshooting in the NOC Step 2.5 Navigate to the [edit policy-options] hierarchy level. Once there, configure the policy LOAD-BALANCEthat has an action of load-balance per-packet. [edit] lab@mxA-1# edit policy-options [edit policy-options] lab@mxA-1# set policy-statement LOAD-BALANCE then load-ba1ance per-packet Step 2.6 Navigate to the [edit routing-options J hierarchy level and apply the newly created policy to the forwarding table as an export policy. Activate the configuration and exit to operational mode by issuing the commitand-quit command. [edit policy-options] lab@mxA-1# top edit routing-options [edit routing-options] lab@mxA-1# set forwarding-tab1e exportLOAD-BALANCE [edit routing-options] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 2.7 Issue the show route forwarding-table destination 172. 31. 20. 0/24 command. lab@mxA-1> show route forwarding-tab1e destination 172.31.20.0/24 Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif 172.31.20.0/24 user O ulst 1048574 12 172.18.6.1 ucst 561 4 ge-1/0/0.141 172.18.7.1 ucst 562 5 ge-1/0/1.141 Routing table: master.anon .inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm O rjct 538 1 Question: Can load balancing occur for the 172.31.20.0/24 prefix? Why? Answer: Yes. Two next hops and two interfaces are listed in the output for the 172.31.20.0/24 prefix. Lab 7-10 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Question: If equal load balancing does not occur when traffic begins to flow over the links to the 172.31.20.0/24 network, what could you do to equalize the traffic load across both links? Answer: The hash-key option, located in the forwarding-options hierarchy level, could be changed to include Layer 3 and Layer 4 information in the hashing algorithm. Part 3: Troubleshooting Firewall Filters In this lab part, you will load a configuration that has a preconfigured firewall filter. You will examine the router to determine where the firewall is applied. Then you will examine the protocols on the router to ensure that they are still functioning properly. Step 3.1 Enter configuration mode and issue the load override j tnoc/ 1.ab7-p3-s tart. config command. Activate the configuration by issuing the commitcommand. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# 1oad override jtnoc/1ab7-p3-start.config load complete [edit] lab@mxA-1# commit commit complete [edit] lab@mxA-1# Step 3.2 Issue the show firewall command. [edit] lab@mxA-1# show firewa11 family inet { filter PROTECT-RE term ssh { from { protocol tcp; port ssh; then accept; www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-11 Junes Troubleshooting in the NOC term telnet { from { protocol tcp; port telnet; then accept; term bgp from protocol tcp; port bgp; then accept; term icmp-req { from { protocol icmp; icmp-type echo-request; then accept; term icmp-reply { from { protocol icmp; icmp-type echo-reply; then accept; term tracert { from { protocol udp; ttl l; then { reject; term ospf from { source-address 224.0.0.5/32; protocol ospf; term frag from { is-fragment; protocol [ tcp udp icmp ]; then reject; Lab 7-12 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC term vrrp { from { protocol vrrp; then accept; term discard-all then { log; discard; Question: Does the firewall filter allow Telnet traffic? Answer: Yes. The term telnet permits Telnet traffic. Question: Does the firewall filter allow FTP traffic? Answer: No. Any FTP traffic is discarded by the discard-all term. Step 3.3 Issue the run show interface filters command. [edit] lab@mxA-1# run show interfaces filters Interface Admin Link Proto Input Filter Output Filter lc-0/0/0 up up lc-0/0/0.32769 up up vpls xe-0/0/0 up down xe-0/0/1 up down xe-0/0/2 up down xe-0/0/3 up down ge-1/0/0 up up ge-1/0/0.141 up up inet multiservice ge-1/0/0.32767 up up multiservice ge-1/0/1 up up ge-1/0/1.141 up up inet multiservice ge-1/0/1.32767 up up multiservice ge-1/0/2 up up ge-1/0/2.141 up up inet multiservice www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-13 Junos Troubleshooting in the NOC ge-1/0/2.32767 up up multiservice ge-1/0/3 up down ge-1/0/3.141 up down inet multiservice ge-1/0/3.32767 up down multiservice ge-1/0/4 up up ge-1/0/5 up up ge-1/0/6 up up ge-1/0/7 up up ge-1/0/8 up up ge-1/0/9 up down ge-1/1/0 up down ge-1/1/1 up down ge-1/1/2 up up ge-1/1/3 up up ge-1/1/4 up up ge-1/1/5 up up ge-1/1/6 up up ge-1/1/7 up up ge-1/1/8 up up ge-1/1/9 up down cbpO up up demuxO up up dsc up up emO up up emO.O up up inet inet6 tnp eml up down fxpO up up fxpO.O up up inet gre up up ipip up up irb up up loO up up loO.O up up inet PROTECT-RE lo0.16384 up up inet lo0.16385 up up inet lsi up up meO up up mtun up up pimd up up pime up up pipO up up ppO up up tap up up lab@mxA-1> Lab 7-14 • Isolating and Troubleshooting PFE Issues www.juniper.net Junes Troubleshooting in the NOC Question: Where is the PROTECT-RE firewall filter applied? Answer: The PROTECT-RE firewall filter is applied to the loopback interface. Step 3.4 Issue the run show bgp summaryand the run show ospf neighbor commands to check the dynamic routing protocols running on your router. [edit] lab@mxA-1# run show bgp sunnnary Groups: l Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 24 23 0 0 0 0 Peer AS In Pkt Out Pkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped ... 172.18.6.1 65000 154 163 0 0 1:10:06 11/ 12/12/0 0/0/0/0 172.18.7.1 65000 153 161 0 0 1:10:02 12/ 12/12/0 0/0/0/0 [edit] lab@mxA-1# run show ospf neighbor [edit] lab@mxA-1# Question: Do any problems exist with either of the dynamic routing protocols? Answer: Yes. Although the BGP sessions are up, no OSPF neighbor adjacencies exist. Step 3.5 To make sure that OSPF is configured for the ge-1/0/2 interface, issue the run show ospf interface command. [edit] lab@mxA-1# run show ospf interface Interface State Area DR ID BDR ID Nbrs ge-1/0/2.141 DR 0.0.0.0 192.168.31.1 0. 0. 0. 0 0 ge-1/0/3.141 Down 0. 0. 0. 0 0. 0. 0.0 0.0.0.0 0 loO.O DR 0.0.0.0 192.168.31.1 0.0.0.0 0 www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-15 Junos Troubleshooting in the NOC Question: Is the ge-1/0/2 interface participating in OSPF? Answer: Yes. The ge-1/0/2 interface is participating in OSPF. Step 3.6 To examine the traffic being received on the ge-1/0/2 interface, issue the run monitor traffic interface ge-1/0/2. VLAN-IDcommand. [edit] lab@mxA-1# run monitor traffic interface ge-1/0/2.VLAN-ID verbose output suppressed, use Reverse lookup for 172.18.5.1 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 10:34:20.105955 Out IP truncated-ip - 14 bytes missing! 172.18.5.1 > 224.0.0.5: OSPFv2, Hello, length 44 10:34:29.787916 Out IP truncated-ip - 14 bytes missing! 172.18.5.1 > 224.0.0.5: OSPFv2, Hello, length 44 ... "C 2 packets received by filter O packets dropped by kernel lab@mxA-1> Question: What OSPF activity do you see in the output? Answer: The only OSPF activity is OSPF hello messages being sent out of the ge-1/0/2 interface. Lab 7-16 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Question: What can cause this condition with OSPF? Answer: Either your assigned device is discarding the OSPF hello messages, or the remote team's router is not sending them. Question: Can enabling OSPF traceoptions be useful in troubleshooting this issue? Answer: No. Enabling OSPF traceoptions is not helpful because your router is not receiving OSPF hello messages from the remote team's router, traceoptions would provide no useful data. Question: What might be blocking the OSPF hello messages on your router? Answer: The firewall filter that was recently applied to the loopback interface can block exception traffic, such as OSPF hello messages, that is destined for the routing engine. Step 3.7 Examine the firewall filter again by issuing the show firewall command. [edit] lab@mxA-1# show firewall family inet { filter PROTECT-RE term ssh { from { protocol tcp; port ssh; then accept; term telnet { from { protocol tcp; port telnet; www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7-17 Junos Troubleshooting in the NOC then accept; term bgp from protocol tcp; port bgp; then accept; term icmp-req { from { protocol icmp; icmp-type echo-request; then accept; term icmp-reply { from { protocol icmp; icmp-type echo-reply; then accept; term tracert { from { protocol udp; ttl l; then { reject; term ospf from { source-address { 224.0.0.5/32; protocol ospf; term frag from { is-fragment; protocol [ tcp udp icmp ]; then reject; term discard-all then { discard; Lab 7-18 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Question: Is there at term that handles OSPF traffic? Answer: Yes. The ospfterm is configured in the firewall filter to handle OSPF traffic. Step 3.8 To determine if the OSPF hello messages are being processed by the ospfterm, navigate to the [edit firewall family inet filter PROTECT-RE term ospf] hierarchy level and configure the counter ospf-hel.l.os under that term. Activate the configuration by issuing the commitcommand. [edit] lab@mxA-1# edit firewall family inet filter PROTECT-RE term ospf [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# set then count ospf-heIIos [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# commit commit complete [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# Step 3.9 To examine the recently configured counter, issue the run show firewall filter PROTECT-RE command. [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# run show firewall filter PROTECT-RE Filter: PROTECT-RE Counters: Name Bytes Packets ospf-hellos 0 0 Question: Did the counter increment? Why? Answer: No. The counter did not increment, which means that the ospfterm is not processing any OSPF traffic. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -19 Junos Troubleshooting in the NOC Step3.10 Examine the configuration of the PROTECT-RE firewall filter again. [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# up 2 show filter PROTECT-RE { term ssh { from { protocol tcp; port ssh; then accept; term telnet { from { protocol tcp; port telnet; then accept; term bgp from protocol tcp; port bgp; then accept; term icmp-req { from { protocol icmp; icmp-type echo-request; then accept; term icmp-reply { from { protocol icmp; icmp-type echo-reply; then accept; term tracert { from { protocol udp; ttl 1; then { reject; term ospf from { source-address { 224.0.0.5/32; Lab 7-20 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC protocol ospf; then count ospf-hellos; term frag { from { is-fragment; protocol [ tcp udp icmp ]; then { reject; term vrrp from { protocol vrrp; then accept; term discard-all then { log; discard; Question: If the ospf term is not processing OSPF hello messages, which term is processing these messages? Answer: The term discard-all should be processing the hello messages. Step 3.11 To examine the traffic being processed by the term discard-al.l, configure firewall logging for everything that is processed by this term. Activate the configuration by issuing the commitcommand. [edit firewall family inet filter PROTECT-RE term ospf] lab@mxA-1# up 1 [edit firewall family inet filter PROTECT-RE] lab@mxA-1# set term discard-aii then log [edit firewall family inet filter PROTECT-RE] lab@mxA-1# connnit commit complete [edit firewall family inet filter PROTECT-RE] lab@mxA-1# www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -21 Junos Troubleshooting in the NOC Step3.12 Examine the traffic being collected by the firewall log by issuing the run show firewall log command. Note Because OSPF hello messages are sent every few seconds, you might need to issue the run show firewall log command a few times to see the hello messages. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# run show firewall log Log : Time Filter Action Interface Protocol Src Addr Dest Addr 11:03:49 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:03:39 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:03:30 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11: 03:22 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:03:13 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:03:05 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:02:57 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 11:02: 4 9 pfe D ge-1/0/2.141 OSPF 172.18.5.2 224.0.0.5 Question: Do you see any OSPF messages? What does your answer indicate? Answer: Yes. OSPF hello messages from the destination address of 224.0.0.5 on the ge-1/0/2 interface and from the 172.18.5.1 source address appear, which means that the OSPF hello messages are being discarded by the term discard-all. Lab 7-22 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Step 3.13 To determine why the ospf term is not processing OSPF hello messages, issue the show term osp£command. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# show term osp£ from { source-address { 224.0.0.5/32; protocol ospf; then count ospf-hellos; Question: Is anything missing from then stanza of the ospf term that could cause a problem? Answer: No. Although you might think that an accept action is required in the then stanza, the count action by itself counts and accepts packets that match the term. Step3.14 Examine the match criteria more closely by issuing the show term ospf from command. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# show term osp£ from source-address { 224.0.0.5/32; protocol ospf; Question: What item in the match criteria is causing the OSPF traffic to not be evaluated by this term? Answer: OSPF hello messages are sent to the multicast address 224.0.0.5 and are sourced from the interface address of the local router originating them. You must remove the source-address option and you must add a destination-address value of 224.0.0.5. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -23 Junos Troubleshooting in the NOC Step3.15 Remove the source-address option and add a destination address of 224.0.0.5 in the from stanza of the ospfterm. Activate the configuration by issuing the commitcommand. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# delete term ospf from source-address [edit firewall family inet filter PROTECT-RE] lab@mxA-1# set term ospf from destination-address 224.0.0.5 [edit firewall family inet filter PROTECT-RE] lab@mxA-1# show term ospf from { destination-address 224.0.0.5/32; protocol ospf; then count ospf-hellos; [edit firewall family inet filter PROTECT-RE] lab@mxA-1# conunit commit complete Before proceeding, ensure that the remote student team within your pod is ready to continue on to the next step. Step3.16 Issue the run show ospf neighbor command to determine if the OSPF adjacency reaches the Full state. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# run show ospf neighbor Address Interface State ID Pri Dead 172.18.5.2 ge-1/0/2.141 ExStart 192.168.32.1 128 37 Question: Does the OSPF adjacency reach the Full state? Why? Answer: No.The OSPF adjacency does not reach the Full state. The OSPF adjacency is stuck in the ExStart state, which means a problem with exchanging the database descriptor messages exist. Lab 7-24 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Step 3.17 Issue the run show firewall filter PROTECT-RE command to examine the counter that was originally setup to determine if OSPF traffic is being processed by the ospfterm. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# run show firewall filter PROTECT-RE Filter: PROTECT-RE Counters: Name Bytes Packets ospf-hellos 17104 224 Question: Is the counter incrementing? Why? Answer: Yes. The counter should be incrementing, which means that some OSPF traffic is being processed by the ospf term. Step3.18 Issue the run show firewall log command to examine the firewall log to determine if the traffic is being discarded by the discard-all term. [edit firewall family inet filter PROTECT-RE] lab@mxA-1# run show firewall log Log : Time Filter Action Interface Protocol Src Addr Dest Addr 11: 11: 4 6 pfe D ge-1/0/2.141 OSPF 172.18.5.2 172.18.5.1 11: 11: 42 pfe D ge-1/0/2.141 OSPF 172.18.5.2 172.18.5.1 11: 11: 05 pfe D ge-1/0/2.141 OSPF 172.18.5.2 172.18.5.1 ---(more)---[abort] Question: What does the firewall log reveal? Answer: The firewall log shows OSPF traffic sent from the remote team's router, with a source address of the remote team's ge-1/0/2 interface and a destination address of your router's ge-1/0/2 interface, is being discarded. www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -25 Junos Troubleshooting in the NOC Question: Which action must you take to allow this traffic? Answer: You must configure the ospf term to allow this traffic. Step3.19 Configure the ospfterm to match traffic destined to the 172.18.5.0/30 subnet. Activate the configuration and exit to operational mode by issuing the commit and-quitcommand. [edit firewall family inet filter PROTECT-RE) lab@mxA-1# set term ospf from destination-address 172.18.5.0/30 [edit firewall family inet filter PROTECT-RE) lab@mxA-1# show term ospf from { destination-address 224.0.0.5/32; 172.18.5.0/30; protocol ospf; then count ospf-hellos; [edit firewall family inet filter PROTECT-RE) lab@mxA-1# conunit and-quit commit complete Exiting configuration mode lab@mxA-1> Before proceeding, ensure that the remote student team within your pod is ready to continue on to the nextstep. Step3.20 Issue the show ospf neighbor command to determine if the OSPF adjacency reaches the Full state. lab@mxA-1> show ospf neighbor Address Interface State ID Pri Dead 172.18.5.1 ge-1/0/2.143 Full 192.168.35.1 128 38 Lab 7-26 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Question: Does the OSPF adjacency reach the Full state? Answer: Yes. As shown in the previous output, the OSPF adjacency has now reached the Full state Step 3.21 Log out of your assigned device. lab@mxA-1> exit mxA-1 (ttyuO) login: Tell your instructor that you have completed this lab. Management Network Diagram Management Network Management Addressing mxlH mxD-1 mxA-2 mxD-2 mxB-1 vr-- Server Note: Your instructorwill provideaddress and accessinformation_ www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -27 Junos Troubleshooting in the NOC Isolating and Troubleshooting PFE Issues Lab: Pod A Network Diagram Internet mxA-1 mxA-2 ge-1/0/2.141 (.1) 172.18.5.Q/30 (.2) ge-1/0/2.141 loO: 192.168.31.1 loO: 192.168.32.1 ge-1/0/3-141 . ge-1/0/3-141 g;192.168.11.0/30 192.168.12.Q/30( l) (.2) vrl-1 Virtual Routers vr2-2 Isolating and Troubleshooting PFE Issues Lab: Pod B Network Diagram Internet mxB-2 mxB-1 ge-1/0/2.142 (.1 ) 172.18.5.0/30 (.2) ge-1/0/2.142 loO: 192.168.33.1 loO: 192.168.34.1 ge-1/0/3-141 . ge-1/0/3-141 ( l)192168.13 0/30 . 192.168.14.Q/30( l) (.2) . (.2) vrl-1 Virtual Routers vr2-2 WoM:£&1-;�� -.JUIUJ)C\"net "" �-- Lab 7 -28 • Isolating and Troubleshooting PFE Issues www.juniper.net Junos Troubleshooting in the NOC Isolating and Troubleshooting PFE Issues Lab: Pod C Network Diagram Internet mxC-1 mxC-2 ge-1/0/2.143 (.1) 172.18.5.0/30 (.2) ge-1/0/2.143 loO: 192.168.35.1 loO: 192.168.36.1 ge-1/0/3-141 . . ge-1/0/3-141 (l)91 2.168.15.0/30 91 2.168.6.01 /30( l) (2) (.2) YT1-1 Virtual Routers YT2-2 Isolating and Troubleshooting PFE Issues Lab: Pod D Network Diagram rnxD-1 mxD-2 ge-1/0/2.144 (.1) 172.18.5.0/30 (.2) ge-lj0/2.1 44 loO: 192.168.37.1 loO: 192.168.38.1 ge-1/0/3.141 1 . ge-1/0/3-141 (. \92168.17.Qi30 91 2.168.18/.0 30(l) ()2 (.2) vr1-1 Virtual Routers vr2-2 www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7 -29 Junos Troubleshooting in the NOC Lab 7 -30 • Isolating and Troubleshooting PFE Issues www.juniper.net Lab Troubleshooting Routing Protocols Overview In this lab, you discover and troubleshoot OSPF adjacency and BGP session problems. Then, after you fix the problems associated with the OSPF adjacencies and BGP sessions, a routing loop is introduced into the network. You must troubleshoot and fix the routing loop that becomes present in the network. By completing this lab you will perform the following tasks: Troubleshoot OSPF adjacency problems. Troubleshoot BGP session problems. Troubleshoot routing loop problems. www.juniper.net Troubleshooting Routing Protocols • Lab 8-1 Junes Troubleshooting in the NOC Part 1: Modifyingthe ExistingConfigurations In this lab part, you load and activate a predefined configuration saved on your assigned device. This predefined configuration will introduce a number of issues in your network. You will then troubleshoot the issues to restore network functionality. Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address. Question: What is the management address assigned to your device? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CU for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Protocol: Hostname: Port: D Show quick connect on startup 0 Save session 0 Open in a tab Connect ] [ Cancel Lab 8-2 • Troubleshooting Routing Protocols www.juniper.net Junes Troubleshooting in the NOC Step 1.3 Log in to your device with the username lab and the password lab123. Note that both the name and password are case-sensitive. Enter configuration mode and load the labB-start.config configuration from the /var/home/ lab/j tnoc/ directory. Issue the commit and-quitcommand to activate your changes and exit to operational mode. mxA-1 (ttyuO) login: l.ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# l.oad override jtnoc/1ab8-start.config load complete [edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Part2: Troubleshooting OSPF Adjacencies In this lab part, you troubleshoot OSPF adjacency issues. First you must discover what OSPF adjacency issues exist, then you must fix any problems that prevent OSPF adjacencies from reaching the Full state. Step 2.1 Note To simulate a multi-router topology, every physical MX Series device in the lab has been divided into many logical systems. To view the information for a logical system you must use the logical-system option followed by the logical system's name. For example, if you want to view the OSPF adjacencies for logical system R2, you enter the command show ospf neighbor logical-system R2. Examine the OSPF adjacency information on R2 by issuing the show ospf neighbor logical-system R2command. www.juniper.net Troubleshooting Routing Protocols • Lab 8-3 Junes Troubleshooting in the NOC lab@mxA-1> show ospf neighbor logical-system R2 lab@mxA-1> Question: How many OSPF adjacencies should be present in the output? Answer: From examining the Lab 8 diagram, you can see that R2 should have one OSPF adjacency with R3. Question: Where should you start in troubleshooting the missing OSPF adjacency? Answer: Verifying the Layers 1 through 3 of the OSI model by issuing ping tests is a good place to start. Step 2.2 Attempt to ping R3 from R2 by issuing the ping 10 .1.1. 2 logical-system R2 rapid command. lab@mxA-1> ping 10.l.l.2 logical-system R2 rapid PING 10.1.1.2 (10.1.1.2): 56 data bytes ! ! ! ! ! --- 10.1.1.2 ping statistics 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.439/0.688/1.660/0.486 ms Question: What do the results of the ping test mean? Answer: The successful ping tests show that Layers 1 through 3 are operational. Lab 8-4 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Step 2.3 Examine the state of the OSPF interfaces on R2 by issuing the show ospf interface logical-system R2 command. lab@mxA-1> show ospf interface logical-system R2 Interface State Area DR ID BDR ID Nbrs lo0.2 DRother 0.0.0.0 0.0.0.0 0.0.0.0 0 lt-1/0/0.1 DR 0.0.0.0 10.1.100.2 0.0.0.0 0 Question: What is the current state of the lt-1/0/0.1 interface in OSPF? Answer: The lt-1/0/0.1 currently thinks that it is the DR, which means it is setup as a broadcast interface. Note that the interface is participating in OSPF area 0, as it should be, and it is not detecting any neighbors on the broadcast segment it is associated with. Step 2.4 Troubleshoot the OSPF adjacency problem on R2 further by issuing the monitor traffic interface lt-1/0/0.l detail command lab@mxA-1> monitor traffic interface lt-1/0/0.1 detail Address resolution is ON. Use Reverse lookup for 10.1.1.2 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 01:26:21.955223 In IP (tos OxcO, ttlJunos Troubleshooting in the NOC 1, id 3608, offset 0, flags [none], proto: OSPF (89), length: 76) 10.1.1.2 > 224.0.0.5: OSPFv2, Hello, length 56 [len 44] Router-ID 10.1.100.3, Backbone Area, Authentication Type: simple (1) Simple text password: 12345 Options [External, LLS] Hello Timer 3s, Dead Timer 20s, Mask 255.255.255.248, Priority 128 Designated Router 10.1.1.2 LLS: checksum: Oxfff6, length: 3 Extended Options (1), length: 4 Options: OxOOOOOOOl [LSDB resync] 01:26:23.803879 Out IP (tos OxcO, ttl 1, id 3637, offset 0, flags [none], proto: OSPF (89), length: 76) 10.1.1.1 > 224.0.0.5: OSPFv2, Hello, length 56 [len 44] Router-ID 10.1. 100.2, Backbone Area, Authentication Type: simple (1) Simple text password: 12345 Options [External, LLS] www.juniper.net Troubleshooting Routing Protocols • Lab 8-5 Junos Troubleshooting in the NOC Hello Timer 3s, Dead Timer 20s, Mask 255.255.255.252, Priority 128 Designated Router 10.1.1.1 LLS: checksum: Oxfff6, length: 3 Extended Options (1), length: 4 Options: OxOOOOOOOl [LSDB resync] Question: What are some of the reasons that can cause any OSPF adjacency to not reach the Full state? Answer: Some of the reasons that can cause OSPF adjacencies to fail to reach the Full state are listed here: Password value mismatch Password type mismatch Hello timer mismatch Dead timer mismatch Subnet mask mismatch Interface type mismatch Area number mismatch Area type mismatch MTU mismatch Duplicate Router IDs Question: From examining the previous output, why is the OSPF adjacency failing to reach the Full state? What must you do to fix the problem? Answer: The previous output shows that the subnet mask is mismatched for the link between R2 and R3. The lt-1/0/0.2 interface on R3 is configured with a /29 subnet mask. You must configure the lt-1/0/0.2 interface on R3 with the correct subnet mask of /30 to resolve the problem. Lab 8-6 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Step 2.5 Enter configuration mode and navigate to R3's lt-1/0/0.2 interface hierarchy by issuing theedit logical-systems R3 interfaces lt-1/0/0.2 command. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit logical-systems R3 interfaces lt-1/0/0.2 [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# show encapsulation ethernet; peer-unit l; family inet { mtu 1500; address 10.1.1.2/29; [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# Step 2.6 Replace the /29 subnet mask on the lt-1/0/0.2 interface with a /30 subnet mask. Then, commit the configuration. [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# replace pattern /29 with /30 [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# show encapsulation ethernet; peer-unit 1; family inet { mtu 1500; address 10.1.1.2/30; [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# commit commit complete Step 2.7 Examine the R2 to R3 OSPF adjacency by issuing therun show ospf neighbor logical-system R2 [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# run show ospf neighbor logical-system R2 Address Interface State ID Pri Dead 10.1.1.2 lt-1/0/0.l Full 10.1.100.3 128 19 www.juniper.net Troubleshooting Routing Protocols • Lab 8-7 Junos Troubleshooting in the NOC Question: What is the state of the R2 to R3 OSPF adjacency? Answer: The R2 to R3 OSPF adjacency should reach the Full state. If the OSPF adjacency has not reached the Full state yet, please wait a few minutes. If after a few minutes the OSPF adjacency still does not reach the Full state, please inform your instructor. Step 2.8 Examine the OSPF adjacency states on R3 by issuing the run show ospf neighbor logical-system R3 command. [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# run show ospf neighbor logical-system R3 Address Interface State ID Pri Dead 10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 19 Question: Which OSPF adjacencies are not present in the output? Answer: The R3 to R5 and R3 to R6 OSPF adjacencies are not present in the output. Step 2.9 Test the Layer 3 connectivity by pinging R5 and R6 from R3 by issuing therun ping 10.1.1.10 logical-system R3 rapidandrun ping 10.1.1.6 logical-system R3 rapid commands. [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# run ping 10.1.1.10 logical-system R3 rapid PING 10.1.1.10 (10.1.1.10): 56 data bytes ! ! ! ! ! --- 10.1.1.10 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.441/0.489/0.655/0.083 ms [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# run ping 10.1.1.6 logical-system R3 rapid PING 10.1.1.6 (10.1.1.6): 56 data bytes ! ! ! ! ! --- 10.1.1.6 ping statistics 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.454/0.699/1.661/0.481 ms Lab 8-8 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting inthe NOC Question: What are the results of the ping tests? Answer: The successful ping tests show that Layers 1 through 3 are operational. Step 2.10 Navigate to the [edit logical-systems R3 protocols ospf J hierarchy level. Configure the OSPF traceoptions to flag OSPF hello packets and OSPF errors in detail. Then, configure the Ju nos OS to store the traceoptions in the file osp£-trace. log. Then, committhe configuration. [edit logical-systems R3 interfaces lt-1/0/0 unit 2] lab@mxA-1# up 3 edit protocols ospf [edit logical-systems R3 protocols ospf] lab@mxA-1# set traceoptions flag hello detail [edit logical-systems R3 protocols ospf] lab@mxA-1# set traceoptions flag error detail [edit logical-systems R3 protocols ospf] lab@mxA-1# set traceoptions file osp£-trace.iog [edit logical-systems R3 protocols ospf] lab@mxA-1# show traceoptions { file ospf-trace.log; flag hello detail; flag error detail; export ospf-export; area 0.0.0.0 { interface lt-1/0/0.2 hello-interval 3; dead-interval 20; authentication { simple-password "$9$J2GHqP5Qn9Af5hS"; ## SECRET-DATA interface lt-1/0/0.3 { hello-interval 5; dead-interval 50; authentication { simple-password "$9$NldYgaZUH.PJZ9A"; ## SECRET-DATA interface lt-1/0/0.5 { interface-type p2p; hello-interval 10; www.juniper.net Troubleshooting Routing Protocols • Lab 8-9 Junos Troubleshooting in the NOC dead-interval 40; authentication { simple-password "$9$ZuUk.fTz6Ct5Tcy"; ## SECRET-DATA interface lo0.3 passive; [edit logical-systems R3 protocols ospf] lab@mxA-1# commit commit complete [edit logical-systems R3 protocols ospf] lab@mxA-1 Step 2.11 Examine the traceoptions file by issuing therun show log R3/osp£-trace.iog I match 10.1.1.lOand therun show log R3/ ospf-trace. iog I find 10.1.1. 6 commands. Note Viewing traceoptions log files is a little bit different since you are using logical systems. Notice that the previous command has you place a R3/ in front of the traceoptions log file name that you created a few steps ago. The R3/ designation denotes that the traceoptions file is stored within the R3 logical system. [edit logical-systems R3 protocols ospf] lab@mxA-1# run show log R3/osp£-trace.iog match 10.1.1.10 Jan 14 20:46:07.175178 OSPF packet ignored: authentication failure from 10.1.1.10 Jan 14 20:46:15.623090 OSPF packet ignored: authentication failure from 10.1.1.10 Jan 14 20:46:24.655029 OSPF packet ignored: authentication failure from 10.1.1.10 Jan 14 20:46:33.060970 OSPF packet ignored: authentication failure from 10.1.1.10 Jan 14 20:46:41.358808 OSPF packet ignored: authentication failure from 10.1.1.10 Jan 14 20:46:50.157730 OSPF packet ignored: authentication failure from 10.1.1.10 [edit logical-systems R3 protocols ospf] lab@mxA-1# run show log R3/osp£-trace.iog find 10.1.1.6 Jan 14 21:12:05.188671 OSPF packet ignored: area mismatch (1.0.0.0) from 10.1.1.6 on intf lt-1/0/0.5 area 0.0.0.0 Lab 8-10 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Jan 14 21:12:05.188754 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 IFL 346 area 0.0.0.0) Jan 14 21:12:05.188778 Version 2, length 44, ID 10.1.100.6, area 1.0.0.0 Jan 14 21:12:05.188798 checksum Ox7118, authtype 1 Jan 14 21:12:05.188820 mask 255.255.255.248, hello ivl 10, opts Oxl2, prio 128 Jan 14 21:12:05.188840 dead ivl 40, DR 10.1.1.6, BDR 0.0.0.0 Question: What problems do you see in the traceoptions output? Answer: Although your output might vary slightly from the previous output, you should see an OSPF authentication failure between R3 and R5. Then, you should see an area ID mismatch between R3 and R6. Question: What are the next steps you can take to resolve the problems seen in the traceoptions output? Answer: The next steps you can take, is to set the authentication on the OSPF adjacency between R3 and R5 to the same password. Then, you can change the OSPF area ID on R6 to area Oto solve the problem between R3 and R6. Step2.12 Change the OSPF password that is in use between R3 and R5 by issuing the set area O interface lt-1/0/0.3 authentication simple-password jtnoc command for R3. The issue the top set logical-systems R5 protocols ospf area O interface lt-1/0/0.4 authentication simple-password jtnoc command for R5. [edit logical-systems R3 protocols ospf] lab@mxA-1# set area O interface lt-1/0/0.3 authentication simple-password jtnoc [edit logical-systems R3 protocols ospf] lab@mxA-1# top set logical-systems RS protocols ospf area O interface lt-1/0/0.4 authentication simple-password jtnoc www.juniper.net Troubleshooting Routing Protocols • Lab 8-11 Junos Troubleshooting in the NOC Step 2.13 Change the OSPF area on R6 to OSPF area O by issuing the top rename logical-systems R6 protocols ospf area 1.0.0.0 to area O command. Commit the configuration when complete. [edit logical-systems R3 protocols ospf] lab@mxA-1# top rename logical-systems R6 protocols ospf area 1.0.0.0 to area O [edit logical-systems R3 protocols ospf] lab@mxA-1# top show logical-systems R6 protocols ospf export ospf-export; area 0.0.0.0 { interface lt-1/0/0.6 authentication { simple-password "$9$axZikmfT3/CPfEc"; ## SECRET-DATA interface lt-1/0/0.8 { authentication { simple-password "$9$ZsUk.fTz6Ct5Tcy"; ## SECRET-DATA interface lo0.6 passive; [edit logical-systems R3 protocols ospf] lab@mxA-1# coilllllit commit complete Step 2.14 Issue run show ospf neighbor logical-system R3 command to examine the OSPF adjacency states of R3. [edit logical-systems R3 protocols ospf] lab@mxA-1# run show ospf neighbor logical-system R3 Address Interface State ID Pri Dead 10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 18 Question: What does the output show? Answer: The R3 to R5 and R3 to R6 adjacency states are still absent from the output. Lab 8-12 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Step 2.15 Clear the OSPF traceoptions log by issuing the run clear log R3/ ospf-trace . .logcommand. Issue the run show log R3/ ospf-trace . .log I find "he.I.lo 10. 1. 1. 10" and the run show log R3/ osp£-trace . .log I find "he.I.lo 10. 1.1. 6" commands. [edit logical-systems R3 protocols ospf] lab@mxA-1# run clear log R3/osp£-trace.iog [edit logical-systems R3 protocols ospf] lab@mxA-1# run show log R3/osp£-trace.iog I find "heiio io.i.i.io" Jan 14 21:44:21.215309 OSPF rcvd Hello 10.1.1.10 -> 224.0.0.5 (lt-1/0/0.3 IFL 345 area 0.0.0.0) Jan 14 21:44:21.215386 Version 2, length 44, ID 10.1.100.5, area 0.0.0.0 Jan 14 21:44:21.215409 checksum OxO, authtype 1 Jan 14 21:44:21.215431 mask 255.255.255.252, hello_ivl 10, opts Oxl2, prio 128 Jan 14 21:44:21.215452 dead_ivl 40, DR 10.1. 1.10, BDR 0.0.0.0 Jan 14 21:44:21.215478 OSPF restart signaling: Received hello with LLS data from nbr ip=l0.1.1.10 id=l0.1.100.5. Jan 14 21:44:21.215503 OSPF packet ignored: hello interval mismatch 10 from 10.1.1.10 on intf lt-1/0/0.3 area 0.0.0.0 edit logical-systems R3 protocols ospf] lab@mxA-1# run show log R3/osp£-trace.iog I find "heiio io.i.L6" Jan 14 22:21:15.767039 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 IFL 346 area 0.0.0.0) Jan 14 22:21:15.767088 Version 2, length 44, ID 10.1.100.3, area 0.0.0.0 Jan 14 22:21:15.767109 checksum OxO, authtype 1 Jan 14 22:21:15.767131 mask 255.255.255.252, hello_ivl 10, opts Oxl2, prio 128 Jan 14 22:21:15.767152 dead_ivl 40, DR 10.1.1.6, BDR 0.0.0.0 Jan 14 22:21:15.767177 OSPF restart signaling: Received hello with LLS data from nbr ip=l0.1.1.6 id=l0.1.100.3. Jan 14 22:21:15.767201 OSPF packet ignored: our router ID received from 10.1.1.6 on intf lt-1/0/0.5 area 0.0.0.0 Question: What appears to be the current problems with the OSPF adjacencies? Answer: The problem with the R3 to R5 OSPF adjacency is a hello interval mismatch. The problem with the R3 to R6 OSPF adjacency is a duplicate router ID. www.juniper.net Troubleshooting Routing Protocols • Lab 8-13 Junos Troubleshooting in the NOC Question: What are the hello and dead interval settings that are being received on R3 from R5? Answer: The current hello and dead interval settings that R3 is receiving from R5 are 10 and 40, respectively. Question: What should the router ID of R6 be? Answer: The router ID of R6 should be 10.1.100.6. Step 2.16 Examine the current hello and dead interval settings for R3 by issuing the run show ospf interface lt-1/0/0.3 detail logical-system R3 I match hel.1.ocommand. [edit logical-systems R3 protocols ospf] lab@mxA-1# run show ospf interface lt-1/0/0.3 detail logical-system R3 I match he.I.lo Hello: 5, Dead: 50, ReXrnit: 5, Not Stub Question: How can you change R3's configuration to fix the R3 to R5 OSPF adjacency problem? Answer: Changing the hello interval to 10 seconds and the dead interval to 40 seconds fixes the current R3 to R5 OSPF adjacency problem. Step 2.17 Make sure your current configuration hierarchy position is [edit logical-systems R3 protocols ospf], then change the hello interval to 10 seconds and dead interval to 40 seconds on R3's lt-1/0/0.3 interface. [edit logical-systems R3 protocols ospf] lab@mxA-1# set area O interface lt-1/0/0.3 hello-interval .10 dead-interval 40 Lab 8-14 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Note Alternatively, you can simply delete the hello and dead interval settings for the R3 to R5 OSPF adjacency. The default hello and dead interval for OSPF are 10 and 40 seconds, respectively. Step 2.18 Change the router ID for R6 by issuing the top rename logical-systems R6 interfaces lo0.6 family inet address 10.1.100.3/32 to address 10.1. 100 . 6 command. Then, comrnit the configuration. [edit logical-systems R3 protocols ospf] lab@mxA-1# top rename logical-systems R6 interfaces lo0.6 family inet address 10.1.100.3/32 to address 10.1.100.6 [edit logical-systems R3 protocols ospf] lab@mxA-1# top show logical-systems R6 interfaces lo0.6 family inet { address 10.1.100.6/32; [edit logical-systems R3 protocols ospf] lab@mxA-1# commit commit complete Step 2.19 Examine the OSPF adjacency states on R3 by issuing the run show ospf neighbor logical-system R3 command. [edit logical-systems R3 protocols ospf] lab@mxA-1# run show ospf neighbor logical-system R3 Address Interface State ID Pri Dead 10.1.1.l lt-1/0/0.2 Full 10.1.100.2 128 17 10.1.1.10 lt-1/0/0.3 Full 10.1.100.5 128 35 Question: What are the states of the OSPF adjacencies on R3? Answer: Only the R3 to R6 OSPF adjacency is missing and not in the Full state. Step2.20 Issue the run clear log R3/ospf-trace . .logto clear old log information. Then, examine the traceoptions file by issuing the run show log R3/ospf-trace . .log I find "he.I.lo 10. 1. 1. 6" command. www.juniper.net Troubleshooting Routing Protocols • Lab 8-15 Junes Troubleshooting in the NOC [edit logical-systems R3 protocols ospf] lab@mxA-1# run clear log R3/osp£-trace.iog [edit logical-systems R3 protocols ospf] lab@mxA-1# run show log R3/osp£-trace.iog I find "heiio 10.1.1.6" Jan 14 23:43:30.585875 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 !FL 346 area 0.0.0.0) Jan 14 23:43:30.585911 Version 2, length 48, ID 10.1.100.6, area 0.0.0.0 Jan 14 23:43:30.585932 checksum OxO, authtype 1 Jan 14 23:43:30.585959 mask 255.255.255.252, hello_ivl 10, opts Ox12, prio 128 Jan 14 23:43:30.585982 dead_ivl 40, DR 10.1.1.6, BDR 0.0.0.0 Jan 14 23:43:30.586008 OSPF restart signaling: Received hello with LLS data from nbr ip=l0.1.1.6 id=l0.1.100.6. Jan 14 23:43:30.586034 OSPF packet ignored: configuration mismatch from 10.1.1.6 on intf lt-1/0/0.5 area 0.0.0.0 Question: What can you determine from the traceoptions output? Answer: The traceoptions output tells you that a configuration mismatch is present, but it does not tell you what the configuration mismatch is. Step 2.21 Examine the OSPF configuration messages that R3 and R6 are exchanging by issuing the run monitor traffic interface 1 t-1/0/0. 5 detail command. [edit logical-systems R3 protocols ospf] lab@mxA-1# run monitor traffic interface lt-1/0/0.5 detail Address resolution is ON. Use Reverse lookup for 10.1.1.6 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 23:38:18.302492 In IP (tos OxcO, ttl 1, id 17508, offset 0, flags [none], proto: OSPF (89), length: 80) 10.1.1.6 > 224.0.0.5: OSPFv2, Hello, length 60 [len 48] Router-ID 10.1.100.6, Backbone Area, Authentication Type: simple (1) Simple text password: 12345 Options [External, LLS] Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.252, Priority 128 Designated Router 10.1.1.6 Neighbor List: 10.1.100.3 LLS: checksum: Oxfff6, length: 3 Lab 8-16 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Extended Options (1), length: 4 Options: OxOOOOOOOl [LSDB resync] 23:38:22.682130 Out IP (tos OxcO, ttl 1, id 17602, offset 0, flags [none], proto: OSPF (89), length: 76) 10.1.1.5 > 224.0.0.5: OSPFv2, Hello, length 56 [len 44] Router-ID 10.1.100.3, Backbone Area, Authentication Type: simple (1) Simple text password: 12345 Options [External, LLS] Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.252, Priority 128 LLS: checksum: Oxfff6, length: 3 Extended Options (1), length: 4 Options: OxOOOOOOOl [LSDB resync] "C 2 packets received by filter O packets dropped by kernel Question: Which OSPF configuration parameters between R3 and R6 are mismatched? Answer: Although it might be difficult to determine which configuration parameters are mismatched in the previous output, you might notice that R6 is sending configuration parameters that contain DR information, whereas R3 is not. This bit of information tells you that R6's lt-1/0/0.6 interface is configured in the default broadcast mode, whereas the R3's lt-1/0/0.5 interface is configured in point-to-point mode, or point-to-multipoint mode. These type of configuration parameters are incompatible. Step 2.22 Ensure your current configuration hierarchy position is [edit logical-systems R3 protocols ospf], then place R3's lt-1/0/0.5 interface in the default broadcast mode by issuing the delete area O interface lt-1/0/0. 5 interface-typecommand. Then, committhe configuration. [edit logical-systems R3 protocols ospf] lab@mxA-1# delete area O interface lt-1/0/0.5 interface-type [edit logical-systems R3 protocols ospf] lab@mxA-1# show area O interface lt-1/0/0.5 hello-interval 10; dead-interval 40; authentication { simple-password "$9$ZuUk.fTz6Ct5Tcy"; ## SECRET-DATA www.juniper.net Troubleshooting Routing Protocols • Lab 8-17 Junos Troubleshooting in the NOC [edit logical-systems R3 protocols ospf] lab@mxA-1! commit commit complete Step 2.23 Examine the OSPF adjacency states on R3 by issuing the run show ospf neighbor logical-system R3 command. [edit logical-systems R3 protocols ospf] lab@mxA-1! run show ospf neighbor logical-system R3 Address Interface State ID Pri Dead 10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 19 10.1.1.10 lt-1/0/0.3 Full 10.1.100.5 128 31 10.1.1.6 lt-1/0/0.5 Full 10.1.100.6 128 36 Question: What are the states of the OSPF adjacencies on R3? Answer: All of the necessary OSPF adjacencies are now in the Full state. Step 2.24 Now that the OSPF adjacency problems for R2 and R3 have been resolved. Examine the OSPF adjacency between R5 and R6 by issuing the run show ospf neighbor logical-system RS command. [edit logical-systems R3 protocols ospf] lab@mxA-1! run show ospf neighbor logical-system RS Address Interface State ID Pri Dead 10.1.1.9 lt-1/0/0.4 Full 10.1.100.3 128 36 10.1.1.14 lt-1/0/0.7 Full 10.1.100.6 128 31 Question: What are the states of the OSPF adjacencies on R5? Answer: All of the necessary OSPF adjacencies are now in the Full state. Lab 8-18 • Troubleshooting Routing Protocols www.juniper.net Junes Troubleshooting in the NOC Step 2.25 Make sure your current configuration hierarchy position is [edit logical-systems R3 protocols ospf], then, remove the traceoptions for R3. Next, issue the commit and-quit command to activate the configuration and exit to operational mode. [edit logical-systems R3 protocols ospf] lab@mxA-1# delete traceoptions [edit logical-systems R3 protocols ospf] lab@mxA-1# conunit and-quit commit complete Exiting configuration mode lab@mxA-1> Part 3: Troubleshooting BGP Sessions In this lab part, you troubleshoot and resolve problems with BGP sessions. Step 3.1 In your routing domain, R2, R3, and R4 should have full mesh IBGP sessions between each other. Then, R2 should have an EBGP session with ISP-1, and R3 should have an EBGP session with ISP-2. Examine the state of the BGP sessions on R2 by issuing the show bgp summary logical-system R2command. lab@mxA-1> show bgp summary logical-system R2 Groups: 2 Peers: 3 Down peers: 2 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.O 0 0 0 0 0 0 Peer AS InPkt Out Pkt OutQ Flaps Last Up/Own Statel#Active/Received/Accepted/Damped ... 10.1.100.3 11111 889 888 0 7 1:13:36 0/ 0/0/0 0/0/0/0 10.1.100.4 11111 871 866 0 6 8:00 Active 172.16.17.1 54321 0 3702 0 0 3d 4:24:57 Active www.juniper.net Troubleshooting Routing Protocols • Lab 8-19 Junos Troubleshooting in the NOC Question: What are the states of the BGP sessions on R2? Answer: The BGP session from R2 to R3 is in the Established state. The BGP sessions from R2 to R4 and R2 to ISP-1 are in the Active state. Question: What does that Active state mean for the BGP sessions from R2 to R4 and R2 to ISP-1? Answer: The Active state for these sessions means that R2 is actively attempting to form a BGP session with those BGP neighbors. However, the fact that R2 has not moved from the Active state to the Established state for these sessions shows that there is a problem establishing the BGP sessions. Question: What address is R4 supposed to be using to form an IBGP session with R2? Answer: R4 is supposed to be using the 10.1.100.4 address to form an IBGP session with R2. Step 3.2 To begin troubleshooting the R2 to R4 IBGP session enter configuration mode. Next, navigate to the [edit logical-systems R4 protocols bgp] hierarchy level. Then, configure the BGP traceoptions to flag the open message and place the information from the traceoptions in the bgp-trace. .logfile. Commit the configuration when complete. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit logical-systems R4 protocols bgp Lab 8-20 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC [edit logical-systems R4 protocols bgp] lab@mxA-1# set traceoptions file bgp-trace.log [edit logical-systems R4 protocols bgp] lab@mxA-1# set traceoptions flag open [edit logical-systems R4 protocols bgp] lab@mxA-1# commit commit complete [edit logical-systems R4 protocols bgp] lab@mxA-1# Step 3.3 Wait a few minutes for the traceoptions file to fill with information, then when a few minutes pass, issue the run show log R4/bgp-trace. .logcommand. [edit logical-systems R4 protocols bgp] lab@mxA-1# run show log R4/bgp-trace.iog Jan 15 03:14:01 trace on: Tracing to "/var/log/R4/bgp-trace.log" started Jan 15 03:14:32.750784 advertising graceful restart receiving-speaker-only capability to neighbor 10.1.100.2 (Internal AS 11111) Jan 15 03:14:32.750973 bgp send: sending 59 bytes to 10.1.100.2 (Internal AS 11111) Jan 15 03:14:32.751001 Jan 15 03:14:32.751001 BGP SEND 172.16.1.6+59694 -> 10.1.100.2+179 Jan 15 03:14:32.751036 BGP SEND message type 1 (Open) length 59 Jan 15 03:14:32.751086 BGP SEND version 4 as 11111 holdtime 90 id 172.16.1.6 parmlen 30 Jan 15 03:14:32.751110 BGP SEND MP capability AFI=l, SAFI=] Jan 15 03:14:32.751130 BGP SEND Refresh capability, code=128 Jan 15 03:14:32.751149 BGP SEND Refresh capability, code=2 Jan 15 03:14:32.751170 BGP SEND Restart capability, code=64, time=120, flags= Jan 15 03:14: 32. 751192 BGP SEND 4 Byte AS-Path capability (65), as_num 11111 Jan 15 03:14:32.752083 bgp_recv: read from peer 10.1.100.2 (Internal AS 11111) failed: Connection reset by peer Question: Why is the BGP session between R2 and R4 failing to establish? Answer: Although it might be difficult to determine from the output. You might notice that R4 is sending BGP messages to R2 from the 172.16.1.6 address. R2 is configured to receive BGP messages from R4 on the 10.1.100.4 address. www.juniper.net Troubleshooting Routing Protocols • Lab 8-21 Junos Troubleshooting in the NOC Step 3.4 Ensure that you are at the [edit logical-systems R4 protocols bgp] hierarchy level. Then, examine R4's BGP configuration. [edit logical-systems R4 protocols bgp] lab@mxA-1# show traceoptions { file bgp-trace.log; flag open; group IBGP { type internal; neighbor 10.1.100.2; neighbor 10.1.100.3; Question: Which configuration statement must you add to the IBGP group to tell R4 to source the BGP messages from the 10.1.100.4 address? Answer: You must add the local-address 10.1 . 100. 4 statement to the IBGP group to tell R4 to source BGP messages from the 10.1.100.4 address. Step 3.5 Configure the IBGP group on R4 to use the 10.1.100.4 address as the source address to send BGP messages from. Then, committhe configuration. [edit logical-systems R4 protocols bgp] lab@mxA-1# set group IBGP local-address 10.1.100.4 [edit logical-systems R4 protocols bgp] lab@mxA-1# commit commit complete Lab 8-22 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Step 3.6 Wait a few minutes for the BGP sessions to attempt to establish again, then issue the run show bgp summary logical-system R2command to examine the R2 to R4 BGP session on R2. [edit logical-systems R4 protocols bgp] lab@mxA-1# run show bgp sununary logical-system R2 Groups: 2 Peers: 3 Down peers: 2 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 0 0 0 0 0 0 Peer AS InPkt Out Pkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped ... 10.1.100.3 11111 1034 1035 0 7 2:19:06 0/ 0/0/0 0/0/0/0 10.1.100.4 11111 871 866 0 6 1:13:30 Active 172.16.17.1 54321 0 3756 0 0 3d 5:30:27 Active Question: Which state is the R2 to R4 BGP session in? Answer: The R2 to R4 BGP session has remained in the Active state. Step 3.7 Examine the R4 to R2 BGP session on R4 by issuing the run show bgp summary logical-system R4 command. [edit logical-systems R4 protocols bgp group IBGP] lab@mxA-1# run show bgp sununary logical-system R4 Groups: 1 Peers: 2 Down peers: 2 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 0 0 0 0 0 0 Peer AS InPkt Out Pkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped... 10.1.100.2 11111 0 0 0 0 2:03 Idle 10.1.100.3 11111 0 0 0 0 2:03 Idle www.juniper.net Troubleshooting Routing Protocols • Lab 8-23 Junos Troubleshooting in the NOC Question: What does the BGP sessions being in the Idle state mean? Answer: A BGP session is in the Idle state if the router cannot begin to initialize the BGP session. Typically, this state occurs when BGP cannot route the BGP messages to the peer. Step 3.8 Examine R4's BGP traceoptions file by issuing the run show log R4/bgp-trace.l.og I last 2 command. [edit logical-systems R4 protocols bgp group IBGP] lab@mxA-1# run show log R4/bgp-trace.1og I last 2 Jan 15 04:22:13.831815 bgp_peer_init: BGP peer 10.1.100.2 (Internal AS 11111) local address 10.1.100.4 not found. Leaving peer idled Jan 15 04:22:13.833662 bgp_peer_init: BGP peer 10.1.100.3 (Internal AS 11111) local address 10.1.100.4 not found. Leaving peer idled Question: What can you determine from the traceoptions file? Answer: The traceoptions file shows that the 10.1.100.4 address is not found on R4. Because the address cannot be found, R4 cannot source BGP packets from that address. Step 3.9 Examine the loopback interface on R4 by issuing the run show interfaces terse loO. 4 command. [edit logical-systems R4 protocols bgp group IBGP] lab@mxA-1# run show interfaces terse lo0.4 error: interface lo0.4 not found Lab 8-24 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Question: What can you determine from the interface output? Answer: The lo0.4 interface is not configured. Step 3.10 Navigate to the [edit logical-systems R4 interfaces] hierarchy level. Next, configure the loopback interface with the set loO. 4 family inet address 10.1.100. 4 command. Then, committhe configuration. [edit logical-systems R4 protocols bgp group IBGPJ lab@mxA-1# up 2 edit interfaces [edit logical-systems R4 interfaces] lab@mxA-1# set 1o0.4 fami1y inet address 10.1.100.4 [edit logical-systems R4 interfaces] lab@mxA-1# commit commit complete [edit logical-systems R4 interfaces] lab@mxA-1# Step 3.11 Wait a few minutes for the R2 to R4 BGP session to attempt to establish again. Then, issue the run show bgp summary logical-system R2command to examine the R2 to R4 BGP session. [edit logical-systems R4 interfaces] lab@mxA-1# run show bgp summary 1ogical-system R2 Groups: 2 Peers: 3 Down peers: 1 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 0 0 0 0 0 0 Peer AS InPkt Out Pkt OutQ Flaps Last Up/Dwn Statel#Active/Received/Accepted/Damped ... 0 10.1.100.3 11111 1168 1167 7 3:18:39 0/ 0/0/0 0/0/0/0 6 0 10.1.100.4 11111 7 6 1:58 0/ 0/0/0 0/0/0/0 0 172.16.17.1 54321 0 3804 0 3d 6:30:00 Active www.juniper.net Troubleshooting Routing Protocols • Lab 8-25 Junos Troubleshooting in the NOC Question: What is the state of the R2 to R4 BGP session? Answer: The R2 to R4 BGP session is now in the Established state. Step3.12 Ensure that you are at the [edit logical-systems R4 interfaces] hierarchy level. Next, remove R4's BGP traceoptions by issuing the up 1 delete protocols bgp traceoptions command. Then, committhe configuration. [edit logical-systems R4 interfaces] lab@mxA-1# up 1 delete protocols bgp traceoptions [edit logical-systems R4 interfaces] lab@mxA-1# commit commit complete Step3.13 Examine the BGP session between R2 and ISP-1 by issuing the run show bgp neighbor 172 .16 .17 .1 logical-system R2command. [edit logical-systems R4 interfaces] lab@mxA-1# run show bgp neighbor 172.16.17.1 logical-system R2 Peer: 172.16.17.1 AS 54321 Local: 172.16.17.2 AS 11111 Type: External State: Active Flags: Question: What is the current state of the BGP session? Answer: The BGP session is in the Active state. Lab 8-26 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Question: What does the error message in the output tell you? Answer: The error message of Open Message Error lets you know that there is a error contained in the BGP open message. Step3.14 Navigate to the [edit logical-systems R2 protocols bgp group isp-1] hierarchy level. Next, configure traceoptions and flag the BGP open messages in detail. Store the information in the bgp-isp1-trace. .logfile. Finally, committhe configuration. [edit logical-systems R4 interfaces] l.ab@mxA-lll up 2 [edit logical-systems] l.ab@mxA-lll edit R2 protocols bgp group isp-1 [edit logical-systems R2 protocols bgp group isp-1] l.ab@mxA-lll set traceoptions flag open detail [edit logical-systems R2 protocols bgp group isp-1] l.ab@mxA-lll set traceoptions file bgp-ispl-trace.log [edit logical-systems R2 protocols bgp group isp-1] l.ab@mxA-lll connnit commit complete [edit logical-systems R2 protocols bgp group isp-1] l.ab@mxA-lll Step3.15 Wait a few minutes for the traceoptions file to fill with information, then issue the run show log R2/bgp-ispl-trace. .logcommand. [edit logical-systems R2 protocols bgp] l.ab@mxA-lll run show log R2/bgp-ispl-trace.log Jan 15 19:20:20 trace on: Tracing to "/var/l.og/R2/bgp-ispl-trace.l.og" started Jan 15 19:20:30.399094 advertising graceful restart receiving-speaker-only capability to neighbor 172.16.17.1 (External. AS 54321) Jan 15 19:20:30.399189 bgp send: sending 59 bytes to 172.16.17.1 (External. AS 54321) Jan 15 19:20:30.399214 Jan 15 19:20:30.399214 BGP SEND 172.16.17.2+49890 -> 172.16.17.1+179 Jan 15 19:20:30.399249 BGP SEND message type 1 (Open) length 59 Jan 15 19:20:30.399310 BGP SEND version 4 as 11111 holdtime 90 id 10.1.100.2 parmlen 30 Jan 15 19:20:30.399334 BGP SEND MP capability AFI=l, SAFI=l www.juniper.net Troubleshooting Routing Protocols • Lab 8-27 Junos Troubleshooting in the NOC Jan 15 19:20:30.399353 BGP SEND Refresh capability, code=l28 Jan 15 19:20:30.399372 BGP SEND Refresh capability, code=2 Jan 15 19:20:30.399394 BGP SEND Restart capability, code=64, time=120, flags= Jan 15 19:20:30.399416 BGP SEND 4 Byte AS-Path capability (65), as num 11111 Jan 15 19:20:30.400172 Jan 15 19:20:30.400172 BGP RECV 172.16.17.1+179 -> 172.16.17.2+49890 Jan 15 19:20:30.400214 BGP RECV message type 1 (Open) length 59 Jan 15 19:20:30.400238 BGP RECV version 4 as 12345 holdtime 90 id 10.10.10.100 parmlen 30 Jan 15 19:20:30.400258 BGP RECV MP capability AFI=l, SAFI=l Jan 15 19:20:30.400278 BGP RECV Refresh capability, code=128 Jan 15 19:20:30.400297 BGP RECV Refresh capability, code=2 Jan 15 19:20:30.400318 BGP RECV Restart capability, code=64, time=120, flags= Jan 15 19:20:30.400340 BGP RECV 4 Byte AS-Path capability (65), as_num 12345 Jan 15 19:20:30.400429 bgp_process_open:2824: NOTIFICATION sent to 172.16.17.1 (External AS 54321): code 2 {Open Message Error) subcode 2 (bad peer AS number), Reason: peer 172.16.17.1 (External AS 54321) claims 12345, 54321 configured Jan 15 19:20:30.400739 bgp send: sending 21 bytes to 172.16.17.1 (External AS 54321) Jan 15 19:20:30.400769 Jan 15 19:20:30.400769 BGP SEND 172.16.17.2+49890 -> 172.16.17.1+179 Jan 15 19:20:30.400803 BGP SEND message type 3 (Notification) length 21 Jan 15 19:20:30.400824 BGP SEND Notification code 2 (Open Message Error) subcode 2 (bad peer AS number) Question: What is the problem with the R2to ISP-1 BGP session? Answer: R2 current has the wrong peer AS number configured. The traceoptions log show that R2 has AS 54321 configured, but ISP-2is claiming that it is AS 12345. Step 3.16 Ensure that you are at the[edit logical-systems R2 protocols bgp group isp-1 J hierarchy level. Next, remove the recently configured traceoptions. Then, change the peer-as value to 12345 for the isp-1 BGP group. Next, committhe configuration. [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# delete traceoptions [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# set peer-as 12345 Lab 8-28 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# commit commit complete Step 3.17 Wait a few minutes for the R2 to ISP-1 BGP session to attempt to establish again, then issue the run show bgp neighbor 172 .16 .17 .1 logical-system R2command. [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# run show bgp neighbor 172.16.17.1 logical-system R2 Peer: 172.16.17.1+179 AS 12345 Local: 172.16.17.2+51971 AS 11111 Type: External State: Established Flags: www.juniper.net Troubleshooting Routing Protocols • Lab 8-29 Junos Troubleshooting in the NOC Question: What is the state of the R2 to ISP-1 BGP session? Answer: The R2 to ISP-1 BGP session is in the Established state. Step3.18 Examine the R3 to ISP-2 BGP session by issuing the run show bgp neighbor 10. 0.1.1 l.ogical-system R3command. [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# run show bgp neighbor 10.0.1.1 logical-system R3 Peer: 10.0.1.1 AS 54321 Local: 10.0.1.2 AS 11111 Type: External State: Active Flags: Question: What is the current state of the R3 to ISP-2 BGP session? Answer: The previous output shows that the R3 to ISP-2 BGP session is in the Active state. Depending on when you issue the previous command, the BGP session might be in the Connect state. Question: Which options are present for the R3 to ISP-2 BGP session? Answer: The options present for the R3 to ISP-2 BGP session are Preference, PeerAS, and Refresh. Lab 8-30 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Step 3.19 Navigate to the [edit logical-systems R3 protocols bgp group isp-2 J hierarchy level.Next, configure traceoptions and flag the BGP open messages in detail. Store the information in the bgp-isp2-trace. .logfile. Finally, commit the configuration. [edit logical-systems R2 protocols bgp group isp-1] lab@mxA-1# top edit logical-systems R3 protocols bgp group isp-2 [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# set traceoptions flag open detail [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# set traceoptions file bgp-isp2-trace.1.og [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# couunit commit complete [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# Step3.20 Wait a few minutes for the traceoptions file to fill with information, then issue the run show log R3/bgp-isp2-trace. .logcommand. [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# run show log R3/bgp-isp2-trace.1.og Jan 15 19:49:42 trace_on: Tracing to "/var/log/R3/bgp-isp2-trace.log" started Jan 15 19:50:37.306509 bgp_connect complete: error connecting to 10.0.1.1 (External AS 54321): Socket is not connected Jan 15 19:53:05.308024 bgp_connect complete: error connecting to 10.0.1.1 (External AS 54321): Socket is not connected Question: What can you determine from the traceoptions log? Answer: The traceoptions log is not very helpful. It only contains information about an error when connecting to ISP-2, but it does not specify what the error is. Step 3.21 Ensure that you are at the [edit logical-systems R3 protocols bgp group isp-2 J hierarchy level. Then, remove the recently configured traceoptions and coIIIIllit the configuration. [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# delete traceoptions www.juniper.net Troubleshooting Routing Protocols • Lab 8-31 Junos Troubleshooting in the NOC [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# commit commit complete Step3.22 Examine the BGP messages that R3 is sending and receiving by issuing the run monitor traffic interface ge-1/0/1.VLAN-ID detail command. (VLAN-IDis the VLAN ID associated with the ge-1/0/1 interface.) [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# run monitor traffic interface ge-1/0/1.VLAN-ID detail Address resolution is ON. Use Reverse lookup for 10.0.1.2 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 20:07:00.316625 Out IP (tos OxcO, ttl 1, id 39056, offset 0, flags [none], proto: TCP (6), length: 48) 10.0.1.2.50666 > 10.0.1.1.bgp: S 2826701364:2826701364(0) win 16384 Question: What can you determine from the output? Answer: Although it might be difficult to determine the problem. If you look closely at the BGP messages that R3 is receiving from ISP-2, you might notice that MD5 authentication is in use. Whereas the BGP messages that R3 is sending to ISP-2 are not using any form of authentication. Lab 8-32 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Question: Can you determine the password that ISP-2 is using for the R3 to ISP-2 BGP session? Why? Answer: No. The password is encrypted using MD5. In a real network situation like this, you would have to contact the administrators of ISP-2 to acquire the . password. Step 3.23 Ensure you are at the [edit logical-systems R3 protocols bgp group isp-2] hierarchy level. Then, issue the set authentication-key jtnoc12345 command. Next, issue the conunit and-quit command. [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# set authentication-key jtnoc12345 [edit logical-systems R3 protocols bgp group isp-2] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 3.24 Wait a few minutes for the R3 to ISP-2 BGP session to attempt to establish again, then issue the show bgp neighbor 10. 0 .1.1 logical-system R3 command. lab@mxA-1> show bgp neighbor 10.0.1.1 logical-system R3 Peer: 10.0.l.1+55243 AS 54321 Local: 10.0.1.2+179 AS 11111 Type: External State: Established Flags: www.juniper.net Troubleshooting Routing Protocols • Lab 8-33 Junos Troubleshooting in the NOC NLRI that restart is negotiated for: inet-unicast NLRI of received end-of-rib markers: inet-unicast NLRI of all end-of-rib markers sent: inet-unicast Peer supports 4 byte AS extension (peer-as 54321) Peer does not support Addpath Table inet.O Bit: 10001 RIB State: BGP restart is complete Send state: in sync Active prefixes: 0 Received prefixes: 5 Accepted prefixes: 5 Suppressed due to damping: 0 Advertised prefixes: 2 Last traffic (seconds): Received 16 Sent 1 Checked 12 Input messages: Total 10 Updates 2 Refreshes O Octets 273 Output messages: Total 11 Updates 1 Refreshes O Octets 314 Output Queue[OJ: 0 Question: What is the state of the R3 to ISP-2 BGP session? Answer: The R3 to ISP-2 BGP session should be in Established state. Part4: Troubleshooting Routing Loops In this lab part, you discover and troubleshoot a routing loop between the Host device and the Server device. Step4.1 Preform a traceroute test from the Host device to the Server device by issuing the traceroute 10 .10 .10 .100 logical-system Host command. lab@mxA-1> traceroute 10.10.10.100 logical-system Host traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets 1 10.12.12.1 (10.12.12.1) 0.563 ms 0.362 ms 0.332 ms 2 10.1.1.13 (10.1.1.13) 0.386 ms 2.834 ms 3.134 ms 3 172.16.1.13 (172.16.1.13) 3.207 ms 0.964 ms 0.384 ms 4 172.16.1.5 (172.16.1.5) 0.379 ms 0.385 ms 0.377 ms 5 10.1.1.2 (10.1.1. 2) 0.382 ms 0.491 ms 0.389 ms 6 10.1.1.10 (10.1.1.10) 0.417 ms 0.407 ms 0.400 ms 7 172.16.1.13 (172.16.1.13) 0.403 ms 0.410 ms 0.404 ms 8 172.16.1.5 (172.16.1.5) 0.403 ms 0.411 ms 0.393 ms 9 10.1.1.2 (10.1.1.2) 0.403 ms 0.407 ms 0.412 ms 10 10.1.1.10 (10.1.1.10) 0.425 ms 0.475 ms 0.422 ms 11 172.16.1.13 (172.16.1.13) 0.417 ms 0.436 ms 0.420 ms 12 172.16.1.5 (172.16.1.5) 0.414 ms 0.415 ms 0.411 ms 13 10.1.1.2 (10.1.1.2) 0.427 ms 0.440 ms 0.426 ms 14 10.1.1.10 (10.1.1.10) 0.472 ms 0.459 ms 0.439 ms 15 172.16.1.13 (172.16.1.13) 0.445 ms 0.453 ms 0.449 ms Lab 8-34 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC 16 172.16.1.5 (172.16.1.5) 0.537 ms 0.448 ms 0.435 ms 17 10.1.1.2 (10.1.1.2) 0.442 ms 0.451 ms 0.442 ms 18 10.1.1.10 (10.1.1.10) 1.017 ms 0.491 ms 0.463 ms 19 172.16.1.13 (172.16.1.13) 0.466 ms 0.470 ms 0.469 ms 20 172.16.1.5 (172.16.1.5) 0.459 ms 0.470 ms 0.457 ms 21 10.1.1.2 (10.1.1. 2) 0.473 ms 0.474 ms 0.460 ms 22 10.1.1.10 (10.1.1.10) 0.486 ms 0.488 ms 0.472 ms 23 172.16.1.13 (172.16.1.13) 0.481 ms 0.487 ms 0.480 ms 24 172.16.1.5 (172.16.1.5) 0.607 ms 0.498 ms 0.487 ms 25 10.1.1.2 (10.1.1.2) 0.482 ms 0.499 ms 0.474 ms 26 10.1.1.10 (10.1.1.10) 0.516 ms 0.518 ms 0.516 ms 27 172.16.1.13 (172.16.1.13) 0.590 ms 0.511 ms 0.515 ms 28 172.16.1.5 (172.16.1.5) 0.505 ms 0.510 ms 0.503 ms 29 10.1.1.2 (10.1.1.2) 0.513 ms 0.519 ms 0.497 ms 30 10.1.1.10 (10.1.1.10) 0.540 ms 0.526 ms 0.512 ms Question: Which path are the traceroute packets taking? Answer: The traceroute path begins with R6, then R5, then R4, then R2, then R3, and back to R5. Once it reaches R5 the second time the packets continue to loop through R4, R2, R4, and R5. Question: On which routers should you begin your troubleshooting for the routing loop? Answer: R2 and R5 are both points of route redistribution and they are both in the path of the routing loop. Begin by examining the routing tables of R2 and R5. Step 4.2 Examine the routing tables of R2 and R5 by issuing the show route 10.10.10.100 logical-system R2and show route 10.10.10.100 logical-system R5com mands. lab@mxA-1> show route 10.10_ .10.100 logical-system R2 inet.0: 30 destinations, 45 routes (30 active, 0 holddown, 1 hidden) + =Active Route, - = Last Active, * Both www.juniper.net Troubleshooting Routing Protocols • Lab 8-35 Junos Troubleshooting in the NOC 10.10.0.0/16 *[OSPF/150] 03:29:13, metric 3, tag O > to 10.1.1.2 via lt-1/0/0.1 [BGP/165] 03:29:19, localpref 100 AS path: 12345 I, validation-state: unverified > to 172.16.17.1 via ge-1/0/0.121 lab@mxA-1> show route 10.10.10.100 logical-system R5 inet.0: 29 destinations, 46 routes (29 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 10.10.0.0/16 *[RIP/100] 03:29:34, metric 3, tag O > to 172.16.1.13 via lt-1/0/0.13 Question: What can you determine from the outputs? Answer: R2 has the 10.10.0.0/16prefix in it's routing table from OSPF and BGP, but the OSPF version of the route is active. R5 has the 10.10.0.0/16 in it's routing table from RIP. Question: Where could R2 be receiving the OSPF version of the route? Answer: The 10.10.0.0/16 route is being received through BGP on R2 from ISP-1. It is reasonable to assume that R3 might be receiving the same route from ISP-2 through BGP and might be redistributing it into OSPF. Step 4.3 Examine the routing table of R3 by issuing the show route 10.10. 10. 100 logical-system R3 command. lab@mxA-1> show route 10.10.10.100 logical-system R3 inet.0: 30 destinations, 38 routes (30 active, 0 holddown, 1 hidden) + =Active Route, - = Last Active, * = Both 10.10.0.0/16 *[OSPF/150] 03:45:30, metric 3, tag O > to 10.1.1.10 via lt-1/0/0.3 Lab 8-36 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC [BGP/170] 02:10:46, localpref 100 AS path: 54321 I, validation-state: unverified > to 10.0.1.1 via ge-1/0/1.121 Question: Where is R3 learning the BGP version of the 10.10.0.0/16 route? Answer: R3 is learning the BGP version of the route from ISP-2. Step 4.4 In an effort to determine which router R3 is learning the OSPF version of the prefix from, examine the OSPF LSDB on R3 for the LSA that represents the 10.10.0.0/16 prefix. You can examine the LSA by issuing the show ospf database lsa-id 10 .10. 0. 0 logical-system R3 command. lab@mxA-1> show ospf database lsa-id 10.10.0.0 logical-system R3 OSPF AS SCOPE link state database Type ID Adv Rtr Seq Age Opt Cksum Len Extern 10.10.0. 0 10.1.100.5 Ox80000006 2361 Ox22 Oxda47 36 Question: Which router is R3 learning the 10.10.0.0/16 prefix from? Answer: Examining the Adv Rtr field shows that R3 is learning the prefix from 10.1.100.5, which is R5. Step 4.5 Examine R5 for the 10.10.0.0/16prefix by issuing the show route 10 .10.10 . 100 logical-system R5 command. lab@mxA-1> show route 10.10.10. 100 logical-system R5 inet.0: 29 destinations, 46 routes (29 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 10.10. 0.0/16 *[RIP/100] 04:2 3:46, metric 3, tag O > to 172.16.1.13 via lt- 1/0/0.13 www.juniper.net Troubleshooting Routing Protocols • Lab 8-37 Junos Troubleshooting in the NOC Question: Which router is R5 learning the 10.10.0.0/16 prefix from? Answer: R5 is learning the prefix through RIP from R4. Step4.6 Examine R4 for the 10.10.0.0/16 prefix by issuing the show route 10 .10.10 .100 logical-system R4 command. lab@mxA-1> show route 10.10.10.100 logical-system R4 inet.0: 28 destinations, 28 routes (28 active, 0 holddown, 0 hidden) + =Active Route, - = Last Active, * = Both 10.10.0.0/16 *[RIP/100] 04:29:03, metric 2, tag O > to 172.16.1.5 via lt-1/0/0.12 Question: Which router is R4 learning the 10.10.0.0/16 prefix from? Answer: R4 is learning the prefix through RIP from R2. Step4.7 Examine the OSPF LSDB on R2 for the LSA that represents the 10.10.0.0/16 prefix by issuing the show ospf database lsa-id 10 .10. 0. 0 logical-system R2command. lab@mxA-1> show ospf database lsa-id 10.10.0.0 logical-system R2 OSPF AS SCOPE link state database Type ID Adv Rtr Seq Age Opt Cksum Len Extern 10.10.0.0 10.1.100.5 Ox80000006 2296 Ox22 Oxda47 36 Question: Which router is R2 learning the 10.10.0.0/16 prefix from? Answer: Examining the Adv Rtr field shows that R2 is learning the prefix from R5. Lab 8-38 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Question: The BGP version of the 10.10.0.0/16 route is inactive on R2 and R3, and because the prefix is inactive it is ineligible to be redistributed into OSPF. How could the BGP version of the routes be inactive, but the other versions of the route be active in RIP and OSPF? Answer: The route is being redistributed from BGP into OSPF. Next, it is being redistributed from OSPF into RIP. Then, the prefix is being redistributed from RIP back into OSPF. R2 and R3 now believe that they have two versions of the route that are independent of each other. Question: What can you do to resolve the routing loop problem? Answer: There are a few steps that you must take to fix the routing loop problem. However, the first important step to take is to ensure that the version of the prefix that first enters your network is the most preferred version of the prefix. In this scenario, the first version of the route to enter your network is the BGP version of the route on R2 and R3. Step 4.8 Enter configuration mode and navigate to the [edit logical-systems R2 protocols J hierarchy level. Configure BGP to have a protocol preference value of 99. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit logical-systems R2 protocols [edit logical-systems R2 protocols] l.ab@mxA-1# set bgp preference 99 [edit logical-systems R2 protocols] lab@mxA-1# www.juniper.net Troubleshooting Routing Protocols • Lab 8-39 Junos Troubleshooting in the NOC Step 4.9 Navigate to the [edit logical-systems R3 protocols J hierarchy level and configure BGP to have a protocol preference value of 149. Then, issue the commit and-quit command to activate the configuration and exit to operational mode. [edit logical-systems R2 protocols] lab@mxA-1# top edit logical-systems R3 protocols [edit logical-systems R3 protocols] lab@mxA-1# set bgp preference 149 [edit logical-systems R3 protocols] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Question: Why was the preference value for BGP set to 99 on R2 and 149 on R3? Answer: R2 is running RIP, which has preference values of 100, and OSPF, which has a preference value of 150 for external routes. To avoid any preference problems between RIP, OSPF, and BGP on R2, a BGP preference value 99 is advisable. R3 is only running BGP and OSPF which requires you to set the BGP preference value to 149 or below. Step4.10 Preform a traceroute test from the Host device to the Server device by issuing the traceroute 10 .10 .10 .100 logical-system Host command. lab@mxA-1> traceroute 10.10.10.100 logical-system Host traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets 1 10.12.12.1 (10.12.12.1) 0.501 ms 0.387 ms 0.338 ms 2 10.1.1.5 (10.1.1.5) 0.364 ms 0.380 ms 0.368 ms 3 10.10.10.100 (10.10.10.100) 0.573 ms 0.572 ms 0.555 ms Question: What path did the traceroute take? Answer: The traceroute went to R6, to R3, and then arrived at the Server. Lab 8-40 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Question: What other actions can you take to ensure that the routing loop problem is resolved? Answer: Traceroute tests from other locations, such as R4, can help provide further confirmation on the fix. Step4.11 Preform a traceroute test from R4 to the Server device by issuing the traceroute 10.10 .10. 100 logical-system R4 command. lab@mxA-1> traceroute 10.10.10.100 logical-system R4 traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets 1 172.16.1.5 (172.16.1.5) 0.495 ms 0.377 ms 0.333 ms 2 10.10.10.100 (10.10.10.100) 0.597 ms 0.566 ms 0.544 ms Step4.12 Preform a traceroute test from R1 to the Server device by issuing the traceroute 10.10 .10. 100 logical-system Rl command. lab@mxA-1> traceroute 10.10.10.100 logical-system R1 traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets 172.16.1.1 (172.16.1.1) 0.556 ms 0.458 ms 0.331 ms 2 10.10.10.100 (10.10.10.100) 0.585 ms 0.551 ms 0.536 ms Step4.13 Preform a traceroute test from R5 to the Server device by issuing the traceroute 10 .10 .10. 100 logical-system R5 command. lab@mxA-1> traceroute 10.10.10.100 logical-system R5 traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets 1 172.16.1.13 (172.16.1.13) 0.469 ms 0.477 ms 0.359 ms 2 172.16.1.5 (172.16.1.5) 0.383 ms 0.379 ms 0.371 ms 3 10.10.10.100 (10.10.10.100) 0.590 ms 0.568 ms 0.557 ms Question: Do the traceroute tests show that the routing loop problem is resolved? Answer: Yes, the routing loop is fixed. Step4.14 Log out of your assigned device using the exit command. www.juniper.net Troubleshooting Routing Protocols • Lab 8-41 Junos Troubleshooting in the NOC lab@srxA-1> exit srxA-1 (ttyuO) login: 0 Tell your instructor that you have completed this lab. Management Network Diagram Management Addressing mxA-1 mxD-1 mxA-2 mxD-2 mxB-1 vr-device mxB-2 Server mxG-1 Gateway mxG-2 Term Server Note: Yourinstructor will provideaddress and accessinformation. Lab 8-42 • Troubleshooting Routing Protocols www.juniper.net Junos Troubleshooting in the NOC Troubleshooting Routing Protocols Lab: Mx.A.. 1 Network Diagram llt-1/0/0.10 f I R1 '(.2) f ..,._,..=--� I :'; '" I 0 I ;:;- I' ,;, f I 0 I RIP I "'"- I ,-l I l � I ('� I f � I l Li) I ,-l l 0 I I ? I l �8 ! I lt-1,10/0.14 tt-1,10/0.13 \ 13) 172.16.1.12/30 (.14 - ---==----'----! ------ Troubleshooting Routing Protocols Lab: MxA.,,2 Network Diagram ------lt-1/0/0.10 R1 ---�'".2i 172.16.1.0/30 �:a:=T= .-{ 0 ? t RIP www.juniper.net Troubleshooting Routing Protocols • Lab 8-43 Junes Troubleshooting in the NOC Troubleshooting Routing Protocols Lab: MxB-1 Network Diagram___ _, ------lt·l/0/0.10 10.1.100.2 I R1 ·2) 172.16.1-0/30 I --� I '='� = 10.1.100.3 I .... "' I 0 R4 10.1.100.4 I I > R5 10.1.100.5 I � , R6 10.1.100.6 I 0 I RIP Host I I s I ,.j I Lab 8-44 • Troubleshooting Routing Protocols www.juniper_net Junos Troubleshooting in the NOC Troubleshooting Routing Protocols Lab: Mxc.. 1 Network Diagram --r---i -..... ,__ _ Internet ••server 10.10.10.100 ��------l1t·1/0/o.10 R1 10.1.100.2 I ...:::_J.2} 10.1.1.0/30 l (Cs� 10.1.100.3 :2 R4 10.1.100.4 Id OSPF R5 10.1.100.5 l: s- I R6 10.1.100.6 I RIP I Host J I r r I I I I I I � I d I c," � � 6'.-i I :4! \ I \ R4 www.juniper.net Troubleshooting Routing Protocols • Lab 8-45 Junos Troubleshooting in the NOC ------lt·l/0/0.10 1 f R . 2) 172.16.1.0/30 ._=-___. . : � s I o : 3 � I: I I RIP I I I I I I I I I I � ; d � 3o \ r"'-' _:::'.�·_ _., I \ R4 Troubleshooting Routing Protocols lab: MxD-2 Network Diagram ------lt·l/0/0.10 Lab 8-46 • Troubleshooting Routing Protocols www.juniper.net Lab Monitoring the Network Overview In this lab, you configure and monitor SNMP, RMON, in line JFlow, and in line port mirroring. Throughout the lab you will have opportunities to troubleshoot various aspects of the lab. By completing this lab you will perform the following tasks: Configure and Monitor SNMP. Configure and Monitor RMON. Configure and Monitor in line JFlow. Configure and Monitor inline port mirroring. www.juniper.net Monitoring the Network • Lab 9-1 Junos Troubleshooting in the NOC Part 1: Modifyingthe ExistingConfigurations In this lab part, you load and activate a predefined configuration saved on your assigned device. This predefined configuration contains minimal configuration for the main routing system and configuration for various logical systems. Step 1.1 Ensure that you know to which device you have been assigned. Check with your instructor if you are not certain. Consult the management network diagram to determine your device's management address. Question: What is the management address assigned to your device? Answer: The answer varies and depends on your assigned device. If you are unsure of your assignment, ask your instructor. Step 1.2 Access the CLI for your device using either the console, Telnet, or SSH as directed by your instructor. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: O Show quick connect on startup 0 Save session G!JOpen in a tab Connect j [ Cancel Lab 9-2 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Step 1.3 Log in to your device with the username l.ab and the password 1.ab123. Note that both the name and password are case-sensitive. Enter configuration mode and load the 1.ab9-start. config configuration from the /var /home/lab/j tnoc/ directory. Issue the commit and-quitcommand to activate your changes and exit to operational mode. mxA-1 (ttyuO) login: 1.ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# 1.oad override jtnoc/1ab9-start.con£ig load complete [edit] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Part 2: Configuring and Monitoring SNMP In this lab part, you configure and monitor SNMP. Note To simulate the SNMP server, JFlow Collector/ Analyzer, and Host devices, your physical MX Series device in the lab has been divided into three logical systems, plus the main routing system. To perform an operation for a logical system you must use the logical-system option followed by the logical system's name. For example, if you want to ping the SNMP server from the Host logical system, you can issue the ping 10 .11.11.100 logical-system Host command. Step 2.1 Examine the MIB database for MIB names associated with the system MIB name by performing a MIB walk. Perform the MIB walk by issuing the show snmp mib walk system command. www.juniper.net Monitoring the Network • Lab 9-3 Junos Troubleshooting in the NOC lab@mxA-1> show snmp mib wa1k system sysDescr.0 = Juniper Networks, Inc. mxBO internet router, kernel JUNOS 12.2R2.5, Build date: 2012-11-15 17:22:13 UTC Copyright (c) 1996-2012 Juniper Networks, Inc. sysObjectID.0 jnxProductNameMXBO sysUpTime.O 154627810 sysContact.O sysName.O mxA-1 sysLocation.0 sysServices.0 6 Question: Which MIB names are present under the system MIB name? Answer: The sysDescr. 0,sysObjectID. 0, sysUpTime.O,sysName.O,and sysServices. O MIB names are present. Step 2.2 Examine the last 10 MIB names under the j nxBoxAnatomy MIB name by issuing the show snmp mib walk jnxBoxAnatomy I last 10 command. lab@mxA-1> show snmp mib wa1k jnxBoxAnatomy I 1ast 10 jnxFruPsdAssignment.8.2.3.0 0 jnxFruPsdAssignment.8.2.4.0 = 0 jnxFruPsdAssignment.9.1..0.0 = 0 jnxFruPsdAssignment.20.1.1.0 0 jnxFruPsdAssignment.20.1.2.0 0 jnxFruPsdAssignment.20.2.1.0 0 jnxFruPsdAssignment.20.2.2.0 0 jnxBoxKernelMemoryUsedPercent.0 = 1 jnxBoxSystemDomainType.O = 1 jnxBoxPersonality.O = jnxProductNameMXBO Question: What are the last two MIB names in the output? Answer: The last two MIB names in the output are jnxBoxSystemDomainType.Oand jnxBoxPersonality.0. Lab 9-4 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Question: Can you determine the OID of the j nxBoxPersonality. 0 MIB name from the previous output? Why? Answer: No. The OID is not present in the output. Step 2.3 Examine the XML output of the j nxBoxPersonali ty. O MIB name by issuing the show snmp mib get jnxBoxPersonali ty. 0 I display xmlcommand. lab@rnxA-1> show snmp mib get jnxBoxPersonality.O I display xml Question: What is the OID of the j nxBoxPersonali ty. 0 MIB name? Answer: The OID of the j nxBoxPersonality. 0 MIB name is 1.3. 6.1.4 .1.2636.3.1.18. o. Step 2.4 Enter configuration mode and navigate to the [edit snmp J hierarchy level. lab@rnxA-1> configure Entering configuration mode [edit] lab@rnxA-1# edit snmp [edit snrnpJ lab@rnxA-1# www.juniper.net Monitoring the Network • Lab 9-5 Junos Troubleshooting in the NOC Step 2.5 Configure the publ.iccommunity to have read-only access to the MIB. [edit snmp] lab@mxA-1# set collllllunity pub1ic authorization read-on1y Step 2.6 Configure the public community to only accept SNMP requests from the SNMP server (10.11.11.100). [edit snmp] lab@mxA-1# set collllllunity pub1ic c1ients 10.11.11.100 [edit snmp] lab@mxA-1# set collllllunity pub1ic c1ients 0/0 restrict Step 2.7 Configure the trap group v2-traps to send SNMPv2c traps to the SNMP server (10.11.11.100). [edit snmp] lab@mxA-1# set trap-group v2-traps targets 10.11.11.100 Question: Is it necessary to configure the SNMP version of the trap group in this situation? Why? Answer: It is not necessary to configure the SNMP version for the trap group because the Ju nos OS sends SNMPv1 and SNMPv2c traps by default. Step 2.8 Configure the Ju nos OS to send traps to the SNMP server for chassis, startup, and link events. Activate the configuration and exit to operational mode by issuing the commit and-quit command. [edit snmp] lab@mxA-1# set trap-group v2-traps categories chassis [edit snmp] lab@mxA-1# set trap-group v2-traps categories startup [edit snmp] lab@mxA-1# set trap-group v2-traps categories 1ink [edit snmp] lab@mxA-1# commit and-quit commit complete Lab 9-6 • Monitoring the Network www.juniper.net Junes Troubleshooting in the NOC Exiting configuration mode lab@mxA-1> Step2.9 Open a second Telnet, or SSH, session to your assigned device. You can use this new session to monitor the interface of the SNMP server logical system. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: Port: @:-1 Firewall: v D Show quick connect on startup 0 Save session 0 Open in a tab I Connect j [ Cancel Step2.10 Log in to your device with the username 1.ab using a password of 1.ab123. Note that both the name and password are case-sensitive. mxA-1 (ttyuO) login: lab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step2.11 Monitor the SNMP server logical system's interface by issuing the monitor traffic interface ge-1/1/8 detail command. lab@mxA-1> monitor traffic interface ge-1/1/8 detail Address resolution is ON. Use www.juniper.net Monitoring the Network • Lab 9-7 Junos Troubleshooting in the NOC Step 2.12 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, issue the request snmp spoof-trap linkDowncommand. lab@mxA-1> request snmp spoof-trap linkDown Spoof-trap request result: trap sent successfully Step 2.13 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, examine the output and answer the following questions. Note You might notice an ICMP message that the SNMP server is sending back to your MX Series device that the SNMP trap port is not reachable. The SNMP server is just a logical system on your MX Series device that does not have the SNMP trap port open. This unreachable SNMP trap port message is expected for this lab. lab@mxA-1> monitor traffic interface ge-1/1/8 detail Address resolution is ON. Use Reverse lookup for 10.11.11.1 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 22:38:10.438325 In IP (tos OxO, ttl 64, id 45809, offset 0, flags [none], proto: UDP (17), length: 145) 10.11.11.1.63930 > 10.11.11.100.snmptrap: 130173102101SNMPvl 104108C=v2-traps la4164Trap(100) 10610cE:2636.l.1.1.2.57 14010410.210.15.1102101 linkDownl02101 143104155414603130142 13011610610einterfaces.ifTable.ifEntry.ifindex.1073741824=1021041073741824 13011310610einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=1021013 13011310610einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=1021017 22:38:10.438363 Out IP (tos OxO, ttl 255, id 45815, offset 0, flags [DF], proto: ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port snmptrap unreachable, length 36 IP (tos OxO, ttl 64, id 45809, offset 0, flags [none], proto: UDP (17), length: 145) [ I ip] Lab 9-8 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC 22:38:10.438444 In IP (tos OxO, ttl 64, id 45812, offset 0, flags [none], proto: UDP (17), length: 169) 10.11.11.1.63930 > 10.11.11.100.snmptrap: 13018118al02101SNMPv2c 104108C=v2-traps la717bV2Trap(123) 10210410210110210113016d 130110106108system.sysUpTime.0=143104155414603 i3011710610aS:l.l.4.l.0=106109S:1.1.5.3 13011610610einterfaces.ifTable.ifEntry.iflndex.1073741824=1021041073741824 13011310610einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=1021013 13011310610einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=1021017 22:38:10.438462 Out IP (tos OxO, ttl 255, id 45818, offset 0, flags [DF], proto: ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port snmptrap unreachable, length 36 Question: Which version of SNMP traps are present? Answer: The output shows two link down traps being sent; an SNMPvl and an SNMPv2c trap. Question: What is the SNMP community that is present in the traps? Answer: The output shows that the SNMP community v2-traps is present in the trap messages. Question: Why are there not any interfaces present in the trap message? Answer: The trap message is simply a spoofed link down trap message. There is no interface specified in the spoofed trap. Question: How can you add interface information to the spoofed trap? Answer: You can use the variable-bindings option with the spoof-trap command to add interface information to the spoofed trap. www.juniper.net Monitoring the Network • Lab 9-9 Junos Troubleshooting in the NOC Step 2.14 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, add the ge-1/0/8 interface to the spoofed trap by issuing the request snmp spoof-trap linkDown variable-bindings "ifindex [ge-1/0/8.0] = ge-1/0/8.0" command. lab@mxA-1> request snmp spoof-trap linkDown variable-bindings "i£Index [ge-1/0/8.0] = ge-1/0/8.0" Spoof-trap request result: trap sent successfully Step 2.15 View the SNMP interface index for the ge-1/0/8.0 interface by issuing the show interfaces ge-1/0/8. 0 I match snmpcommand. lab@mxA-1> show interfaces ge-1/0/8.0 I match snmp Logical interface ge-1/0/8.0 (Index 332) (SNMP if Index 640) Flags: SNMP-Traps OxO Encapsulation: ENET2 Question: What is the SNMP interface index for the ge-1/0/8.0 interface? Answer: Although your results might vary, the previous output shows that the SNMP interface index for the ge-1/0/8.0 interface is 640. Step 2.16 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, examine the output for the SNMP interface index of interface ge-1/0/8.0 that you acquired in the previous step. 02:05:32.966074 In IP (tos OxO, ttl 64, id 18533, offset 0, flags [none], proto: UDP (17), length: 140) 10.11.11.1.63930 > 10.11.11.100.snmptrap: 13016el02101SNMPvl 104108C=v2-traps la415fTrap(95) 10610cE:2636.l.l.l.2.57 14010410.210.15.1102101 linkDownl02101 14310415665885713013d 13011110610binterfaces.ifTable.ifEntry.ifindex.640= 102102640 13011310610einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=1021013 13011310610einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=1021017 02:05:32.966111 Out IP (tos OxO, ttl 255, id 18539, offset 0, flags [DFJ, proto: ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port snmptrap unreachable, length 36 IP (tos OxO, ttl 64, id 18533, offset 0, flags [none], proto: UDP (17), length: 140) [ I ip] Lab 9-10 • Monitoring the Network www.juniper.net Junes Troubleshooting in the NOC 02:05:32.966259 In IP (tos OxO, ttl 64, id 18536, offset 0, flags [none], proto: UDP (17), length: 164) 10.11.11.1.63930 > 10.11.11.100.snmptrap: 130181185102101SNMPv2c 104108C=v2-traps la7176V2Trap(118) 102104102101102101130168 130110106108system.sysUpTime.0=143104156658857 13011710610aS:l.l.4.l.0=106109S:l.1.5.3 13011110610binterfaces.ifTable.ifEntry.ifindex.640=102102640 13011310610einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=1021013 13011310610einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=1021017 02:05:32.966279 Out IP (tos OxO, ttl 255, id 18542, offset 0, flags [OF], proto: ICMP (1), length: 56) 10 .11.11.100 > 10 .11.11.1: ICMP 10 .11.11.100 udp port snmptrap unreachable, length 36 IP (tos OxO, ttl 64, id 18536, offset 0, flags [none], proto: UDP (17), length: 164) [ I ip] Question: Which SNMP interface index is present in the output? Answer: Although your results might vary, the previous output displays the SNMP interface index of 640 in the trap message. Part 3: Configuring and Monitoring RMON In this lab part, you configure and monitor RMON. Note Remember that to simulate the SNMP server, JFlow Collector/ Analyzer, and Host devices, your physical MX Series device in the lab has been divided into three logical systems, plus the main routing system. To perform an operation for a logical system you must use the logical-system option followed by the logical system's name. For example, if you want to ping the SNMP server from the Host logical system, you can issue the ping 10 .11.11.100 logical-system Host command. Step 3.1 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, enter configuration mode and navigate to the [edit snmp rmon J configuration hierarchy level. lab@mxA-1> configure Entering configuration mode www.juniper.net Monitoring the Network • Lab 9-11 Junos Troubleshooting in the NOC [edit] lab@mxA-1# edit snmp rm.on [edit snmp rmon] lab@mxA-1# Step 3.2 Examine the SNMP interface index for the ge-1/0/8.0 interface by issuing the run show interfaces ge-1/0/8. 0 I match snmpcommand. [edit snmp rmon] lab@mxA-1# run show interfaces ge-1/0/8.0 I match snrrrp Logical interface ge-1/0/8.0 (Index 338) (SNMP ifindex 642) Flags: SNMP-Traps OxO Encapsulation: ENET2 Question: Which SNMP interface index is present for the ge-1/0/8.0 interface? Answer: Although your output might vary, the previous output shows that the ge-1/0/8.0 interface contains the SNMP interface index of 642. Step 3.3 Set an RMON alarm , which has an index value of 1, to use the variable ifHCOutlSecRate -K, where Kis the SNMP interface index value that you found in step 3.2. [edit snmp rmon] lab@mxA-1# set alarm 1 variable ifHCOutlSecRate.� Step 3.4 Set the sample-type for alarm 1 to the absolute-value option. Then, configure alarm 1 to sample the traffic rate once every 5 seconds. [edit snmp rmon] lab@mxA-1# set alarm 1 sample-type absolute-value [edit snmp rmon] lab@mxA-1# set alarm 1 interval 5 Step 3.5 Configure alarm 1 with a rising-threshold value of 9000000 and a failing-threshold value of 2000000. [edit snmp rmon] lab@mxA-1# set alarm 1 rising-threshold 9000000 [edit snmp rmon] lab@mxA-1# set alarm 1 falling-threshold 2000000 Lab 9-12 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Step 3.6 Configure event 1 to record a log message locally and send a trap message to the SNMP server when the rising threshold is reached for alarm 1. [edit snmp rmon] lab@mxA-1# set event 1 type 1og-and-trap [edit snmp rmon] lab@mxA-1# set a1arm 1 rising-event-index 1 Step 3.7 Configure event 2 to only record a log message locally when the falling threshold is reached for alarm 1. Activate the configuration by issuing the commit command. [edit snmp rmon] lab@mxA-1# set event 2 type 1og [edit snmp rmon] lab@mxA-1# set a1arm 1 fa11ing-event-index 2 [edit snmp rmon] lab@mxA-1# show alarm 1 { interval 5; variable ifHCOutlSecRate.642; sample-type absolute-value; rising-threshold 9000000; falling-threshold 2000000; rising-event-index l; falling-event-index 2; event 1 { type log-and-trap; event 2 { type log; [edit snmp rmon] lab@mxA-1# commit commit complete Step 3.8 Examine the current number of sent SNMP traps by issuing the run show snmp statistics I find output command. [edit snmp rmon] lab@mxA-1# run show snmp statistics I find output Output: Packets: 321861, Too bigs: 0, No such names: 0, Bad values: 0, General errors: 0, Get requests: 0, Get nexts: 0, Set requests: 0, Get responses: 321839, Traps: 22 www.juniper.net Monitoring the Network • Lab 9-13 Junos Troubleshooting in the NOC Question: How many traps have been sent? Answer: Although your output might vary, the previous output shows that 22 traps have been sent. Step 3.9 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, terminate the interface monitoring output by issuing the Ctrl + c key combination. Then, ping the SNMP server from the Host device by issuing the ping 10. 11. 11. 100 size 1400 rapid count 1000000 logical-system Host command. IP (tos OxO, ttl 64, id 31451, offset 0, flags [none], proto: UDP (17), length: 140) [ I ipl 02:25:49.505889 In IP (tos OxO, ttl 64, id 31454, offset 0, flags [none], proto: UDP (17), length: 164) 10.11.11.1.63930 > 10.11.11.100.snmptrap: 130181185102101SNMPv2c 104108C=v2-traps I a7 I 76V2 Trap (118 ) I O2 I O 4 I O2 I O 1 I O 2 I O 1 I 3 0 I 6 8 130110106108system.sysUpTime.0=143104165420511 13011710610aS:l.1.4.1.0=106109S:l.1.5.3 13011110610binterfaces.ifTable.ifEntry.ifindex.642=102102642 13011310610einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=1021013 13011310610einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=1021017 02:25:49.505909 Out IP (tos OxO, ttl 255, id 31460, offset 0, flags [DF], proto: ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port snmptrap unreachable, length 36 IP (tos OxO, ttl 64, id 31454, offset 0, flags [none], proto: UDP (17), length: 164) [lip] "C 8 packets received by filter O packets dropped by kernel lab@mxA-1> ping 10.11.11.100 size 1400 rapid count 1000000 logical-system Host PING 10.11. 11.100 (10.11.11.100): 1400 data bytes ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Step 3.10 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, issue the run show srunp rmon alarms command. [edit snmp rmon] lab@mxA-1# run show snmp rmon alarms Lab 9-14 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Alarm Index Variable description Value State 1 monitor ifHCOutlSecRate.642 10921728 rising threshold Question: Which RMON alarm is present? Answer: The rising threshold alarm is present. Step3.11 Examine the RMON events by issuing the run show snmp rmon events command. [edit snmp rmon] lab@mxA-1# run show snmp rmon events Event Index Type Last Event 1 log and trap 2013-01-29 20:03:55 UTC Question: What does the output suggest? Answer: The output suggests that an event with the index of 1 was triggered which resulted in a log being generated and a trap message being sent. Step 3.12 Examine the logs for the RMON event by issuing the run show log messages I match SNMPD_RMON_EVENTLOG command. [edit snmp rmon] l.ab@mxA-1# run show log messages I match SNMPD RMON EVENTLOG Jan 29 19:52:06 mxA-1 snmpd[1447]: SNMPD RMON EVENTLOG: Event 1 triggered by Alarm 1, rising threshold (9000000) crossed, (variable: ifHCOutlSecRate.642, value: 11117112) Jan 29 19:52:16 mxA-1 snmpd[1447]: SNMPD_RMON_EVENTLOG: Event 2 triggered by Alarm 1, falling threshold (2000000) crossed, (variable: ifHCOutlSecRate.642, value: 0) Jan 29 20:03:56 mxA-1 snmpd[1447]: SNMPD_RMON_EVENTLOG: Event 1 triggered by Alarm 1, rising threshold (9000000) crossed, (variable: ifHCOutlSecRate.642, value: 10921728) www.juniper.net Monitoring the Network • Lab 9-15 Junes Troubleshooting in the NOC Question: Is the log message for the RMON event present? Answer: Yes. The log message for the RMON event is present. Step3.13 Examine the current number of SNMP traps sent by issuing the run show snmp statistics I find output command. [edit snmp rmon] lab@mxA-1# run show snmp statistics I find output Output: Packets: 322176, Too bigs: 0, No such names: 0, Bad values: 0, General errors: 0, Get requests: 0, Get nexts: 0, Set requests: 0, Get responses: 322154, Traps: 22 Question: What is the current value of sent SNMP traps? Did the Junos OS send an SNMP trap in response to the RMON event? Answer: Although your output might vary, the previous output shows that 22 SNMP traps have been sent. The SNMP traps value has not incremented from the previous time that you examined the output. This output shows that traps are not being sent in response to the RMON event. Step3.14 Examine the SNMP traps configuration. [edit snmp rmon] lab@mxA-1# up 1 show trap-group v2-traps categories { chassis; link; startup; targets { 10.11.11.100; Lab 9-16 • Monitoring the Network www.juniper.net Junes Troubleshooting in the NOC Question: Why is the Junos OS not sending traps in response to the RMON event? Answer: The Junos OS is not sending traps in response to the RMON event because the RMON alarm category is not defined for the trap group. Step 3.15 Configure the RMON alarm category for the v2-traps trap group. Activate the configuration and exit to operational mode by issuing the commit and-quit command. [edit snmp rmon] lab@mxA-1# up 1 set trap-group v2-traps categories rmon-alarm [edit snmp rmon] lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 3.16 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, terminate the previous ping operation by issuing the Ctrl + c key combination. Then, ping the SNMP server from the Host device by issuing the ping 10.11.11.100 size 1400 rapid count 1000000 logical-system Host command. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! , .,.. c --- 10.11.11.100 ping statistics --- 357077 packets transmitted, 357077 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.799/0.911/39.663/0.690 ms lab@mxA-1> ping 10.11.11.100 size 1400 rapid count 1000000 logical-system Host PING 10.11.11.100 (10.11.11.100): 1400 data bytes ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Step 3.17 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, issue the show snmp statistics I find output command. www.juniper.net Monitoring the Network • Lab 9-17 Junos Troubleshooting in the NOC lab@mxA-1> show snmp statistics find output Output: Packets: 322383, Too bigs: 0, No such names: 0, Bad values: 0, General errors: 0, Get requests: 0, Get nexts: 0, Set requests: O, Get responses: 322359, Traps: 24 Question: What is the current value of sent SNMP traps? Did the Junos OS send an SNMP trap in response to the RMON event? Answer: Although your output might vary, the previous output shows that the Junos OS has sent 24 SNMP traps. This output indicates that the Junos OS did send an SNMP trap in response to the RMON event. Step 3.18 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, terminate the output by issuing the Ctrl + c key combination. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! "C --- 10.11.11.100 ping statistics --- 357077 packets transmitted, 357077 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.799/0.911/39.663/0.690 ms lab@mxA-1> Return to the original terminal session established with your assigned MX Series device. From the original terminal session, issue the show snmp rmon alarms command. lab@mxA-1> show snmp rmon a1arms Alarm Index Variable description Value State 1 monitor ifHCOutlSecRate.642 O falling threshold Lab 9-18 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Question: Which RMON alarm is currently active? Answer: The falling threshold RMON alarm is currently active. Part 4: Configuring and Monitoring In line JFlow In this lab part, you configure and monitor in line JFlow. Note Remember that to simulate the SNMP server, JFlow Collector/Analyzer, and Host devices, your physical MX Series device in the lab has been divided into three logical systems, plus the main routing system. To perform an operation for a logical system you must use the logical-system option followed by the logical system's name. For example, if you want to ping the SNMP server from the Host logical system, you can issue the ping 10 .11.11.100 logical-system Host command. Step 4.1 Enter configuration mode and navigate to the [edit services flow-monitoring version-ipfix] hierarchy level. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit services flow-monitoring version-ipfix [edit services flow-monitoring version-ipfix] lab@mxA-1# Step 4.2 Configure the my-inline-jflow-ttemplate to use the 1Pv4 template configuration. [edit services flow-monitoring version-ipfix] lab@mxA-1# set template my-in1ine-jf1ow-t ipv4-template Step 4.3 Navigate to the [edit forwarding-options sampling instance my-inline-j flow-i] hierarchy level. Configure the my-inline-jflow-i sampling instance to sample every packet of a flow. www.juniper.net Monitoring the Network • Lab 9-19 Junos Troubleshooting in the NOC [edit services flow-monitoring version-ipfix] lab@mxA-1# top edit forwarding-options sampling instance my-inline-jflow-i [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# set input rate 1 [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# Step 4.4 Configure the my-inline-j flow-i sampling instance's output parameters to send 1Pv4 traffic to the JFlow server (10.10.10.100) using port 2055. Next the output parameters must use the IPFIX template m.y-inl.ine-jfl.ow-t. Then, configure the output parameters to use the source address of 10.10.10.1 for the in line JFlow process. [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# set family inet output flow-server 10.10.10.100 port 2055 [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# set family inet output flow-server 10.10.10.100 version-ipfix template my-inline-jflow-t [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# set family inet output inline-jflow source-address 10.10.10.1 [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# show ## ## Warning: Instance not defined under chassis hierarchy ## input { rate 1; family inet output { flow-server 10.10.10.100 port 2055; version-ipfix { template { my-inline-jflow-t; inline-j flow source-address 10.10.10.1; Step 4.5 Navigate to the [edit chassis] hierarchy level. Configure the TFEB to use the sampling instance m.y-inl.ine-jfl.ow-i. Lab 9-20 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC [edit forwarding-options sampling instance my-inline-jflow-i] lab@mxA-1# top edit chassis [edit chassis] lab@mxA-1# set tfeb slot O sampling-instance my-iniine-jfiow-i [edit chassis] lab@mxA-1# Step 4.6 Navigate to the [edit firewall family inet filter inline-j flow-filter] hierarchy level. Configure a firewall filter term that samples traffic that is going to the SNMP server (10.11.11.100). [edit chassis] lab@mxA-1# top edit firewall family inet filter iniine-jfiow-fiiter [edit firewall family inet filter inline-jflow-filter] lab@mxA-1# set term 1 from destination-address 10.11.11.100 [edit firewall family inet filter inline-jflow-filter] lab@mxA-1# set term 1 then sample [edit firewall family inet filter inline-jflow-filter] lab@mxA-1# set term 1 then accept [edit firewall family inet filter inline-jflow-filter] lab@mxA-1# Step 4.7 Navigate tothe [edit interfaces ge-1/0/7 unit OJ hierarchy level.Apply the inl.ine-j£1.ow-£il.terto the ge-1/0/7 interface as an input firewall filter. Activate the configuration and exit to operational mode by issuing the commit and quitcommand. [edit firewall family inet filter inline-jflow-filter] lab@mxA-1# top edit interfaces ge-1/0/7.0 [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# set family inet filter input iniine-jfiow-fiiter [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# commit and-quit commit complete Exiting configuration mode lab@mxA-1> Step 4.8 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, open an FTP session with the SNMP server from the Host device by issuing the ftp 10. 11. 11. 100 logical-system Host command. Login using the username l.ab and password l.abl.23. www.juniper.net Monitoring the Network • Lab 9-21 Junos Troubleshooting in the NOC lab@mxA-1> ftp 10.11 ,.11.100 logical-system Host Connected to 10.11.11.100. 220 mxA-1 FTP server (Version 6.00LS) ready. Name (10.11.11.100:lab): Lab 331 Password required for lab. Password: 230 User lab logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> Step 4.9 Return to the first terminal session established with your assigned MX Series device. From the first terminal session, issue the show services accounting flow inline-jflow command. Note If you do not see any active flows in the output, restart the FTP session in Step 4.8. lab@mxA-1> show services accounting flow inline-jflow Flow information TFEB Slot: 0 Flow Packets: 6141, Flow Bytes: 770453 Active Flows: 2, Total Flows: 37 Flows Exported: 27, Flow Packets Exported: 27 Flows Inactive Timed Out: 17, Flows Active Timed Out: 19 Question: How many active flows is in line JFlow monitoring? Answer: Although your output might vary, the previous output shows that the inline JFlow is monitoring two active flow. Question: How many flows has in line JFlow exported? Answer: Although your answer might vary, the previous output shows that 27 flows have been exported by inline JFlow. Lab 9-22 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Part 5: Configuring and Monitoring In line Port Mirroring In this lab part, you configure and monitor in line port mirroring. Note Remember that to simulate the SNMP server, JFlow Collector/Analyzer, and Host devices, your physical MX Series device in the lab has been divided into three logical systems, plus the main routing system. To perform an operation for a logical system you must use the logical-system option followed by the logical system's name. For example, if you want to ping the SNMP server from the Host logical system, you can issue the ping 10. 11.11.100 logical-system Host command. Step 5.1 Enter configuration mode and navigate to the [edit forwarding-options port-mirroring instance] hierarchy level. lab@mxA-1> configure Entering configuration mode [edit] lab@mxA-1# edit forwarding-options port-mirroring instance [edit forwarding-options port-mirroring instance] lab@mxA-1# Step 5.2 Configure a port mirroring instance called parent-1 that has input parameters that port mirrors every packet. [edit forwarding-options port-mirroring instance] lab@mxA-1# set parent-1 input rate 1 Step 5.3 Configure a port mirroring instance called chil.d-1that inherits its input parameters from the parent-1 port mirroring instance. Then, configure the 1Pv4 output parameters of the chil.d-1 port mirroring instance to use the ge-1/0/9 interface to send port mirrored packets to the 10 .10 .10 .100 address. [edit forwarding-options port-mirroring instance] lab@mxA-1# set child-1 input-parameters-instance parent-1 [edit forwarding-options port-mirroring instance] lab@mxA-1# set child-1 family inet output interface ge-1/0/9 next-hop 10.10.10.100 www.juniper.net Monitoring the Network • Lab 9-23 Junes Troubleshooting in the NOC [edit forwarding-options port-mirroring instance] lab@mxA-1# show parent-1 { input { rate 1; child-1 input-parameters-instance parent-1; family inet { output { interface ge-1/0/9.0 { next-hop 10.10.10.100; Step 5.4 Navigate to the [edit chassis] hierarchy level. Bindtheparent-1port mirroring instance to FPC 1. [edit forwarding-options port-mirroring instance] lab@mxA-1# top edit chassis [edit chassis] lab@mxA-1# set fpc 1 port-mirror-instance parent-1 [edit chassis] lab@mxA-1# Step 5.5 Navigate to the [edit firewall family inet filter inline-port-mirror] hierarchylevel. Configure a firewall filter term that port mirrors traffic that is going to the SNMP server (10.11.11.100) to the chil.d-1port mirroring instance. [edit chassis] lab@mxA-1# top edit firewall family inet filter inline-port-mirror [edit firewall family inet filter inline-port-mirror] lab@mxA-1# set term 1 from destination-address 10.11.11.100 [edit firewall family inet filter inline-port-mirror] lab@mxA-1# set term 1 then port-mirror-instance child-1 [edit firewall family inet filter inline-port-mirror] lab@mxA-1# show term 1 { from { destination-address { 10.11.11.100/32; then port-mirror-instance child-1; Lab 9-24 • Monitoring the Network www.juniper.net Junes Troubleshooting in the NOC [edit firewall family inet filter inline-port-mirrorJ lab@mxA-1# Step 5.6 Navigate to the [edit interfaces ge-1/0/7 unit OJ hierarchy level. Remove the previous firewall filter that was used to sample packets from the interface. Then, apply the in.line-port-.mirrorfirewallfilter as an input filter to the interface. Finally, activate the configuration by issuing the commitcommand. [edit firewall family inet filter inline-port-mirrorJ lab@mxA-1# top edit interfaces ge-1/0/7.0 [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# delete family inet filter [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# set family inet filter input in1ine-port-mirror [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# commit commit complete [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# Step 5.7 Open a third Telnet, or SSH, session to your assigned device. You can use this new session to monitor the interface of the JFlow-Collector_Analyzer logical system. Refer to the management network diagram for the IP address associated with your station. The following example uses Telnet to access an MX Series device using the SecureCRT program: www.juniper.net Monitoring the Network • Lab 9-25 Junes Troubleshooting in the NOC Protocol: Hostname: i Port: Firewall: •!None-- �--········�-·�--- v·· I D Show quick connect on startup 0 Save session 0 Open in a tab I Connect I I Cancel Step 5.8 Log in to your device with the username labusing a password of labl23. Note that both the name and password are case-sensitive. mxA-1 (ttyuO) login: 1.ab Password: --- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC lab@mxA-1> Step 5.9 From the third terminal session, ping the SNMP server from the Host device by issuing the ping 10 .11.11.100 logical-system Host command. lab@mxA-1> ping 10.11.11.100 1.ogical.-system Host PING 10.11.11.100 (10.11.11.100): 56 data bytes 64 bytes from 10.11.11.100: icmp_seq=O ttl=63 time=0.672 ms 64 bytes from 10.11.11.100: icmp_seq=l ttl=63 time=0.630 ms 64 bytes from 10.11.11.100: icmp_seq=2 ttl=63 time=0.618 ms Step 5.10 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, exit the FTP session and monitor the JFlow Collector_Analyzer's interface for the sampled packets by issuing the monitor traffic interface ge-1/1/9 command. Lab 9-26 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC ftp> bye 221 Goodbye. lab@mxA-1> monitor traffic interface ge-1/1/9 Address resolution is ON. Use Question: What does the lack of information in the output reveal? Answer: The lack of information in the output reveals that the port mirroring is not working correctly. Step 5.11 Return to the first terminal session established with your assigned MX Series device. From the first terminal session, examine the state of the child-1 port mirroring instance by issuing the run show forwarding-options port-mirroring chil.d-1command. [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# run show forwarding-options port-mirroring child-1 Instance Name: child-1 Instance Id: 3 Input parameters: Rate 1 Run-length 0 Maximum-packet-length 0 Output parameters: Family State Destination Next-hop inet down ge-1/0/9.0 10.10.10.100 Question: What is the state of the port mirroring instance? Answer: The port mirroring instance is in the down state. www.juniper.net Monitoring the Network • Lab 9-27 Junos Troubleshooting in the NOC Step 5.12 Examine the ARP table for an ARP entry associated with the interface that points towards the JFlow Collector/Analyzer by issuing the run show arp interface ge-1/0/9 command. [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# run show arp interface ge-1/0/9 [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# Question: What does the lack of information in the output reveal? Answer: The lack of information in the output reveals that your assigned device is not learning the MAC address of the interface associated with the JFlow Collector/Analyzer. Question: What must you do to statically add an entry in the ARP table for the interface associated with the JFlow Collector/Analyzer? Answer: You must first learn the MAC address of the interface that is associated with the JFlow Collector/Analyzer. Then, you can statically add an ARP entry to your assigned device's ge-1/0/9 interface for the interface associated with the JFlow Collector/Analyzer. Step 5.13 To find the MAC address of the interface associated with the JFlow Collector/ Analyzer (JFlow-Collector_Analyzer logical system) issue the run show interfaces ge-1/1/9 I match hardwarecommand. [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# run show interfaces ge-1/1/9 I match hardware Current address: 80:71:lf:c3:03:81, Hardware address: 80:71:lf:c3:03:81 Lab 9-28 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Question: What is the MAC address of the ge-1/1/9 interface? Answer: Although your answer might vary, the previous output displays a MAC address of 80:71:1f:c3:03:81 for the ge-1/1/9 interface. Step 5.14 Navigate to the [edit interfaces ge-1/0/7 unit OJ hierarchy level. Configure a static ARP entry for the JFlow Collector/Analyzer (10.10.10.100) using the MAC address that you acquired in Step 5.13. Then, activate the configuration by issuing the commitcommand. [edit interfaces ge-1/0/7 unit OJ lab@mxA-1# up 2 edit ge-1/0/9.0 [edit interfaces ge-1/0/9 unit OJ lab@mxA-1# set family inet address 10.10.10.1/24 arp 10.10.10.100 mac ge-1/1/9-MAC [edit interfaces ge-1/0/9 unit OJ lab@mxA-1# commit commit complete [edit interfaces ge-1/0/9 unit OJ lab@mxA-1# Step 5.15 Examine the state of the child-1 port forwarding instance by issuing the run show forwarding-options port-mirroring chil.d-1 command. [edit interfaces ge-1/0/9 unit OJ lab@mxA-1# run show forwarding-options port-mirroring chiid-1 Instance Name: child-1 Instance Id: 3 Input parameters: Rate 1 Run-length 0 Maximum-packet-length 0 Output parameters: Family State Destination Next-hop inet up ge-1/0/9.0 10.10.10.100 www.juniper.net Monitoring the Network • Lab 9-29 Junos Troubleshooting in the NOC Question: What is the state of the child-1 port mirroring instance? Answer: The previous output shows the child-1 port mirroring instance should be in the up state. Step 5.16 Return to the second terminal session established with your assigned MX Series device. From the second terminal session, examine the output in the terminal window. If you canceled the interface monitoring output, issue the monitor traffic interface ge-1/1/9 command again. Type the Ctrl + c key combination when you are finished examining the output. Then, issue the exit command to close the terminal session. lab@mxA-1> monitor traffic interface ge-1/1/9 verbose output suppressed, use Reverse lookup for 10.12.12.100 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use 01:16:46.553813 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346, seq 3294, length 64 01:16:47.555273 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346, seq 3295, length 64 01:16:48.556873 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346, seq 3296, length 64 "C 3 packets received by filter O packets dropped by kernel lab@mxA-1> exit Lab 9-30 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Question: What do you see in the output? Answer: The JFlow Collector/Analyzer is receiving port mirrored packets that contain the source address of 10.12.12.10 0 (the Host logical system) and the destination address of 10.1 1.11.10 (the SNMP Server logical system). Step 5.17 Return to the third terminal session established with your assigned MX Series device. From the third terminal session, end the ping operation by typing the Ctrl + c key combination. Then, issue the exit command to close the terminal session. 64 bytes from 10.11.11.100: icmp_seq=3723 ttl=63 time=0.672 ms 64 bytes from 10.11.11.100: icmp_seq=3724 ttl=63 time=0.686 ms 64 bytes from 10.11.11.100: icmp_seq=3725 ttl=63 time=3.643 ms 64 bytes from 10.11.11.100: icmp_seq=3726 ttl=63 time=0.682 ms 64 bytes from 10.11.11. 100: icmp_seq=3727 ttl=63 time=S.014 ms 64 bytes from 10.11.11.100: icmp_seq=3728 ttl=63 time=2.118 ms 64 bytes from 10.11.11.100: icmp_seq=3729 ttl=63 time=3.589 ms 64 bytes from 10.11.11.100: icmp_seq=3730 ttl=63 time=0.748 ms 64 bytes from 10.11.11.100: icmp seq=3731 ttl=63 time=4.118 ms "C --- 10.11.11.100 ping statistics --- 3732 packets transmitted, 3732 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.583/1.122/25.294/1.902 ms lab@mxA-1> exit Step 5.18 Return to the original terminal session established with your assigned MX Series device. From the original terminal session, navigate to the [edit J hierarchy level and issue the load override jtnoc/reset. configcommand to load the reset configuration file. Activate the configuration change and return to operational mode by using the commit and-quitcommand. [edit interfaces ge-1/0/9 unit OJ lab@mxA-1# top [edit] lab@mxA-1# load override jtnoc/reset.config load complete www.juniper.net Monitoring the Network • Lab 9-31 Junos Troubleshooting in the NOC [edit] lab@mxA-1# commit and-quit configuration check succeeds commit complete Exiting configuration mode lab@mxA-1> Step 5.19 Log out of your assigned MX Series device. lab@mxA-1> exit mxA-1 (ttyuO) login: Tell your instructor that you have completed this lab. Management Network Diagram Management Network Management Addressing mx0-1 mx0-2 vr-device Server Gateway mxC-2 TermServer _____ Note: Your instructorwill provide addressand accessinformation. Lab 9-32 • Monitoring the Network www.juniper.net Junos Troubleshooting in the NOC Monitoring the Network Lab: Pod A Network iagram SNMP SNMP Server -r:J Server - • b;;J 10.11.11.100 10.11.11.100 • q, � � �-c} � ..) ",..,. CY "v� Jflow " !"f Jffow .:J � Collector/Analyzer Collector/Analyzer 10.10.10.100 10.10.10.100 mxA-1 (1) 10.10.10.0/24 iiD 10.10.10.0/24 (.1) mxA-2 • ge-1/0/9 ge-1/1/9 ..,· o• ··· ge-1/1/9 ge-1/0/9 ge-1/0/7 (.1) (.1) ge-1/0/7 10.12.12.0/24 10.12.12.0/24 ge-1/1/7 Host e 10.12.12.100 0 El 00\ik:"""= �LJ WorldwideEducation Services y�J;,;;. -� ..,...... ,.,....net Monitoring the Network Lab: Pod B Network Diagram SNMP SNMP Server Server -o -o 10.11.11.100 • . 10.11.11.100 . • m, q, � � ,_,a\.,.::> ", ge-1/0/7 (.1) (1) ge-1/0/7 10.12.12 0/24 10.12.12.0/24 ge-l/1/7 ge-1/1/7 •. D Host Host • D g . 10.12.12.100 10.12.12.100 g . www.juniper.net Monitoring the Network • Lab 9-33 Junes Troubleshooting in the NOC Monitoring the Network Lab: Pod C Network Diagram SNMP SNMP Server Server - 10.11.11.100 10.11.11.100 • Jflow Jflow Collector/Analyzer Collector/Analyzer 10.10.10.100 10.10.10.100 mxC-1 (.1) 10.10.10.0124 - n iiD. 10.10.10.0/24 (.1, mxC-2 ge-1/0/9 ge-1/1/9 bJ ge-1/1 9 ge-1/0/9 / ge-1/0/7 (.1) (.1) ge-1/0/7 10.12.12.Q/24 10.12.12.Q/24 ge-1/1/7 G Host Host • n 10.12.12.100 10.12.12.100 q 6J Monitoring the Network Lab: Pod D Network Diagram SNMP SNMP Server Server 10.11.11.100 -• o, 10.11.11.100 . Jflow Jflow Collector/Analyzer Collector/Analyzer 10.10.10.100 10.10.10.100 mxD-1 (.1) 10.10.10.0/24 .- •- [l 10.10.10.0/24 (.1) mxD-2 ge 1/0/9 ge-1/1/9 o bdge-1/1/9 ge-1/0/9 - ge-1/0/7 (.1) (.1) ge-1/0/7 10.12.12.Q/24 10.12.12.0/24 ge 1 ge-1/1/7 - / : bJ Host Host £'l·- � n 10.12.12.100 10.12.12.100 Lab 9-34 • Monitoring the Network www.juniper.net