Advanced Networking with NetworkManager in RHEL8 for Servers Stop touching ifcfg-eth0! Marc Skinner rincipa! So!ution Architect Agenda ● What / How ● $ooling ● Getting information ● Simple modifications ● (p # Down control ● $eaming ● *ridges ● +LANs ● Infini-and . , o,* ● Examples 2 "hat is NetworkManager0 ● A networking service manager – Manage1 configure a!! things networking ● Introduced in RHEL6 3 ho-hum ● 4ptiona! in RHEL5 … getting better ● Defau!t in RHEL8 … I love it 3 "hat are the bene'ts of NetworkManager? ● rovides an A I through )-B(S – Mu!tiple application integration – rovides consistenc6 for graphica! desktop environments ● 7ront end to configuration fi!es 4 How to use NetworkManager0 ● Insta!! ● Ena-!e ● Start ● Configure – Command line – (,1 both te/t and graphica! 5 NetworkManager $oo!ing ● nmcli – Command-!ine ● nmtui – $e/t (ser Interface ● nm-connection-editor / control-center – %raphica! (ser Interface ● Cockpit – We- administrative porta! ● Ansi-!e Role 6 NetworkManager Co&mand-!ine ● nmcli and nmtui – nmcli is included in “NetworkManager: – nmtui is included in 9NetworkManager-tui: # yum -y install NetworkManager NetworkManager-tui # systemctl enable NetworkManager # systemclt start NetworkManager ● 4ptiona! . add bash completion if not insta!!ed! # yum -y install bash-completion 7 %etting Information ● Is the device connected0 ; nmcli device status ● Connect a device ; nmcli device enp<s0 connect 8 %etting Information ● Get connection information 9 %etting ,nfor&ation ● Show a!! configura-!e options =20? -6 defau!t ● (se -f for '!tering b6 fie!d ; nmcli -f “fie!d: con show enp1s0 'e!d = connection1 ipv41 bridge1 etc 10 11 12 %etting Information ● Show device detai!s 13 Adding a device ● Add a new device B<C t6pesD 14 Simp!e Modifications ● Rename an interface connection name # nmcli con modify Wired\ connection\ 3 con-name ens3 ● Modif6 IPvA address # nmcli con modify ens3 ipv4 addresses !"# !$% 4& %!'#4 ● Modif6 IPvA gatewa6 # nmcli con modify ens3 ipv4 gateway !"# !$% 4& !& ● Modif6 IPvA method # nmcli con modify ens3 ipv4 method static 15 Simp!e Modifications ● Modif6 IPvA DNS # nmcli con modify ens3 ipv4 dns !"# !$% 4& !()!"# !$% 4& !$ ● Modif6 IPvA DNS-options # nmcli con modify ens3 ipv4 dns-options rotate)timeout*! ● Modif6 IPvA DNS-search # nmcli con modify ens3 ipv4 dns-search +rhlab skinnerlabs com)i skinnerlabs com+ ● Modif6 M$( # nmcli con modify ens3 %&#-3-ethernet mtu "&&& 16 Up / Down Contro! ● *ring connection down # nmcli con down enp3s& ● *ring connection up # nmcli con up enp3s& ● Re!oad a!! connections # nmcli con reload 17 $ea&ing ● $eaming is the replacement for *onding – ; -ond2tea& ● $eaming a!!ows for N?= N,8s to be configured as a logica! device with speci'c -ene'ts based on the runner se!ected – -roadcast (a!! portsD – roundrobin (a!! ports in turnD – active-ackup (one port unti! fai!ureD – !oad-a!ance (a!! ports with a hashD – random (a!! ports randoml6D – !acp B80>E<ad LA8 – reFuires LA8 switchD 18 $ea&ing ● Create a LA8 $EAM ca!!ed tea&= using two NICs # nmcli con add type team ifname team1 con-name team1 # nmcli con modify team1 team config ,-+runner+* -+name+* +lacp+) +active+* true) +fast.rate+* true) +t/.hash+* 0+ipv4+)+tcp+)+udp+12) +link.watch+* -+name+* +ethtool+2) +t/.balancer+* - +name+* +basic+22, # nmcli con add type ethernet con-name team1-enp10s0 ifname enp10s0 master team1 # nmcli con add type ethernet con-name team1-enp9s0 ifname enp9s0 master team1 19 $ea&ing ● +iew teaming port status # teamnl team! ports #* enp!&s&* up !&&&Mbit 34 3* enp"s&* up !&&&Mbit 34 20 $ea&ing ● +iew tea&ing status#con'guration # teamdctl team1 state view setup: runner: lacp ports: enp10s0 link watches* enp9s0 link watches: link summary: up link summary: up instance[link_watch_01* instance[link_watch.&1* name: ethtool name: ethtool link: up link: up down count: 0 down count: 0 runner* runner: aggregator 54: 2, Selected selected: yes aggregator ID: 2, state: current Selected runner: active: yes selected: yes fast rate: yes state: current 21 $ea&ing ● +iew teaming configuration. lots of data # teamdctl team! state dump # teamdctl team! config dump 22 Bridges ● *ridge mode turns a NIC into a la6er > switch ● Can ena-!e#disa-!e ST BSpanning $ree rotocolD ● Needed for mu!tiple virtua! machines ● No NA$ ● 4ptiona! *ridge tooling – RHEL 6#5 # yum install bridge-utils – RHEL 8 # ip bridge 23 Bridges ● Create bridge named G+M with , addressed assigned # nmcli con add type bridge ifname kvm con-name kvm ipv4 address !"# !$% 33 !#'#4 ipv4 method static ipv4 gateway !"# !$% 33 # ipv4 dns +!"# !$% 33 44)!"# !$% 33 (&)!"# !$% 33 !(+ ipv4 dns- options +rotate)timeout*!+ ipv4 dns-search +ib skinnerlabs com)i skinnerlabs com+ # nmcli con add type bridge-slave ifname enp3s0 master kvm 24 Bridges ● Create bridge named $EST with N4 addressed assigned # nmcli con modify enp4s&f& ipv4 method disabled ipv$ method ignore # nmcli con add type bridge ifname test con-name test # nmcli con add type bridge-slave ifname enp4s&f& master test ● 7urn off 678 9 on by default # nmcli con modify test bridge stp no 25 Bridges ● Show bridge con'guration # nmcli -f bridge con show test bridge mac-address* -- bridge.stp: no bridge priority* 3#:$% bridge forward-delay* !( bridge hello-time* # bridge ma/-age* #& bridge ageing-time* 3&& bridge group-forward-mask* & bridge multicast-snooping* yes bridge vlan-filtering* no bridge vlan-default-pvid* ! bridge vlans* -- 26 Bridges ● Show bridge status – brief # brctl show test bridge name bridge id 678 enabled interfaces test %&&& &&!b#!(!4&!& no enp4s&f& 27 +LANs ● +LAN @ +irtua! Loca! Area Network ● Isolated La6er > ● A!!ows for mu!tip!e isolated networks to share the same ph6sica! mediu& ● +LANs use +LAN ,)s 0-A0HC ● +LAN ,) 1 is defau!t 28 +LANs ● Add +LAN 32 to enp3s0 with IP address # nmcli connection add type vlan con-name enp3s& 32 ifname enp3s& 32 id 32 dev enp3s& ip4 192.168.32.11/24 # ip add ; grep enp3s& 3# #!* enp3s0.32@enp3s0* <=>?@4A@67)MBC75A@67)B8)C?WD>.B8E mtu !(&& Fdisc noFueue state B8 group default Flen !&&& inet 192.168.32.11/24 brd !"# !$% 3# #(( scope global noprefi/route enp3s& 3# ● Add +LAN 60 to e/isting tea&=0 with no IP address # nmcli connection add type vlan con-name team!& 60 ifname team!& 60 id 60 dev team!& ipv4.met od disabled ipv6.met od ignore 29 ,n'niband / I o,* B,nternetProtco! over InfinibandD ● Infini-and is specia!iIed !ow latency1 high performance networking gear ● $6pica!!6 run in native RDMA BRemote Direct Memor6 AccessD mode – Applications must be a-!e to understand R)MA ● Can run in IPoIB mode (more compati-!eD – Applications can use traditiona! , 30 ,n'niband - I o,* ● Create IPoIB on Infiniband NIC with MTU 65520 # nmcli connection delete ib& # nmcli connection add type in!iniband con-name ib& ifname ib& transport-mode connected mtu 6""20 # nmcli connection modify ib& ipv4 addresses !"# !$% !&3 (&'#4 # nmcli connection modify ib& ipv4 method static 31 ,n'niband – Ipo,* ● Validate IP Information # ip addr ; grep ib& %* ib0* <=>?@4A@67)MBC75A@67)B8)C?WD>.B8E mtu 6""20 Fdisc pfifo.fast state B8 group default Flen #($ inet 192.168.103.12/24 brd !"# !$% !&3 #(( scope global noprefi/route ib& 32 ,n'niband – Ipo,* – $oo!ing ● Status tooling # yum install infiniband-diags # ibstatus 5nfiniband device ,mthca&, port ! status* default gid* fe%&*&&&&*&&&&*&&&&*&&&#*c"&#*&&#a*ad3" base lid* &/4 sm lid* &/! state* 4* @A75GD phys state* (* CinkBp rate* #& Hb'sec I4J 44>K link.layer* 5nfini=and 33 EXAMPLES Sing!e N,8 with I Address 35 Sing!e N,8 with I Address ● Add IP#%"#)NS#4ptions to N,8J ens< # nmcli con modify ens3 ipv4.addresses 192.168.33."0/24 # nmcli con modify ens3 ipv4.gate#a$ 192.168.33.1 # nmcli con modify ens3 ipv4.dns 192.168.33.1" # nmcli con modify ens3 ipv4 method static # nmcli con modify ens3 ipv4.dns-options rotate%timeo&t:1 # nmcli con modify ens3 ipv4 dns-search +ib skinnerlabs com)i skinnerlabs com+ # nmcli con modify ens3 ipv$ method ignore 36 Dua! N,8 with LA8 $EAM with I Address 37 Dua! N,8 with LA8 $EAM with I Address ● Disa-!e N,8s so the6 donKt tr6 to get IP information # nmcli con modify enp$s&f& ipv4.met od disabled ipv6.met od ignore # nmcli con modify enp$s&f! ipv4.met od disabled ipv6.met od ignore ● Create $EAM # nmcli con add type team ifname team10 con-name team10 ● Create $EAM runner # nmcli con modify team10 team config ,-+runner+* -+name+* +lacp+) +active+* true) +fast.rate+* true) +t/.hash+* 0+ipv4+)+tcp+)+udp+12) +link.watch+* -+name+* +ethtool+2) +t/.balancer+* - +name+* +basic+22, 38 Dua! N,8 with LA8 $EAM with I Address ● Attach N,Cs to $EAM # nmcli con add type ethernet con-name team!&-enp$s&f& ifname enp$s&f& master team10 # nmcli con add type ethernet con-name team!&-enp$s&f! ifname enp$s&f! master team10 ● Add IP#DNS # nmcli con modify team!& ipv4.addresses 192.168.33."0/24 # nmcli con modify team!& ipv4 dns !"# !$% 33 !( # nmcli con modify team!& ipv4 method static 39 Luad N,8 with LA8 $EAM with three Bridges1 three +LANs1 one with I address 40 Luad N,8 with LA8 $EAM with three Bridges1 three +LANs1 one with I address ● Disa-!e N,8s so the6 donKt tr6 to get IP information # nmcli con modify enp4s&f& ipv4 method disabled ipv$ method ignore # nmcli con modify enp4s&f! ipv4 method disabled ipv$ method ignore # nmcli con modify enp(s&f& ipv4 method disabled ipv$ method ignore # nmcli con modify enp(s&f! ipv4 method disabled ipv$ method ignore 41 Luad N,8 with LA8 $EAM with three Bridges1 three +LANs1 one with I address ● Create Bridge with IP (defau!t +LAND # nmcli con add type bridge ifname