FT2000+/64 LXC 容器虚拟化系统解决方案 （V1.0）

天津飞腾信息技术有限公司

二零一九年八月

更新记录

版本号 发布部门 作者 发布日期 备注

1.0 飞腾软件技术方案部 2 谢明 2019-08-12 初稿

版权所有© 天津飞腾信息技术有限公司 2019。保留一切权利。

注意 天津飞腾信息技术有限公司对其发行的或与合作公司共同发行的包括但不限于产品的全部内容及材料所 拥有版权等知识产权，受法律保护。非经本公司书面许可，任何单位及个人不得擅自摘抄、复制本文档 内容的部分或全部，并不得以任何形式传播。

免责声明 我们仅仅提供技术上的咨询，对利用文档搭建环境所从事的研发活动没有技术支持责任，对相关研发成 果没有连带责任。

目 录

1 编写目的 ...... 2 2 LXC 容器 ...... 2 3 LXC 容器的搭建 ...... 2 3.1 内核编译选项 ...... 2 3.2 联网安装 ...... 7 3.3 创建 lxc 容器 ...... 9 3.4 启动容器 ...... 10 3.5 停止容器 ...... 10 3.6 lxc 常用命令 ...... 10

1

1 编写目的

本文主要记录了 lxc 容器相关概念的介绍，lxc 容器的搭建，lxc 容器所需要的内核选项，相关的配置 等。下面所有验证和实验都是在飞腾 2000+/64 平台上完成的，后续更新的版本中我们会继续完善相关的 功能和测试。

2 LXC 容器

LXC 为 Container 的简写。可以提供轻量级的虚拟化，以便隔离进程和资源，而且不需要提供指令解释机 制以及全虚拟化的其他复杂性。容器有效地将由单个操作系统管理的资源划分到孤立的组中，以更好地在孤立的组 之间平衡有冲突的资源使用需求。下图是 KVM 虚拟机和容器之间的区别：

KVM Container

APP

Guest OS

Hypervisor APP

Host OS Host OS

可以看出来，容器中的的程序是直接运行在 Host OS 上，省去了 Guest OS 和 层，所以效 率大大提高。与传统虚拟化技术相比，它的优势在于： 1）与宿主机使用同一个内核，性能损耗小； 2）不需要指令级模拟； 3）不需要即时(Just-in-time)编译； 4）容器可以在 CPU 核心的本地运行指令，不需要任何专门的解释机制； 5）避免了准虚拟化和系统调用替换中的复杂性； 6）轻量级隔离，在隔离的同时还提供共享机制，以实现容器与宿主机的资源共享。

3 LXC 容器的搭建

3.1 内核编译选项

内核版本 v4.19.1，为了使系统支持 LXC，内核必须选上如下选项。 General setup ---> <*> Kernel .config support [*] Enable access to .config through /proc/config.gz Kernel Performance Events And Counters ─> [*] Kernel performance events and counters [*] Control Group support ---> [*] Memory controller

2

[*] IO controller [*] CPU controller ---> [*] PIDs controller [*] RDMA controller [*] Freezer controller [*] Cpuset controller [*] Include legacy /proc//cpuset file [*] Device controller [*] Simple CPU accounting controller [*] controller [*] Namespaces support ---> [*] UTS namespace [*] IPC namespace [*] User namespace [*] PID Namespaces [*] Network namespace [*] Checkpoint/restore support [*] Networking support ---> --- Networking support Networking options ---> Packet socket Packet: sockets monitoring interface Unix domain sockets UNIX: socket monitoring interface Transformation user configuration interface PF_KEY sockets [*] TCP/IP networking [*] IP: kernel level autoconfiguration [*] IP: DHCP support [*] IP: BOOTP support [*] IP: RARP support -*- IP: TCP syncookie support Virtual (secure) IP: tunneling IP: IPsec transport mode IP: IPsec tunnel mode IP: IPsec BEET mode INET: socket monitoring interface UDP: socket monitoring interface [*] TCP: advanced congestion control ---> Binary Increase Congestion (BIC) control <*> CUBIC TCP TCP Westwood+ H-TCP The IPv6 protocol --->

3

IPv6: IPsec transport mode IPv6: IPsec tunnel mode IPv6: IPsec BEET mode IPv6: IPv6-in-IPv4 tunnel (SIT driver) [*] Network packet filtering framework () ---> [*] Advanced netfilter configuration Bridged IP/ARP packets filtering Core Netfilter Configuration ---> [*] Netfilter ingress support {M} Netfilter NFACCT over NFNETLINK interface {M} Netfilter NFQUEUE over NFNETLINK interface {M} Netfilter LOG over NFNETLINK interface {M} Netfilter OSF over NFNETLINK interface Netfilter connection tracking support -*- Connection mark tracking support [*] Connection tracking zones [*] Supply CT list in (OBSOLETE) [*] Connection tracking events [*] Connection tracking timeout [*] Connection tracking timestamping -*- Connection tracking labels [*] DCCP protocol connection tracking support [*] SCTP protocol connection tracking support [*] UDP-Lite protocol connection tracking support Amanda backup protocol support FTP protocol support H.323 protocol support IRC protocol support NetBIOS name service protocol support SNMP service protocol support PPtP protocol support SANE protocol support SIP protocol support TFTP protocol support Connection tracking interface Connection tracking timeout tuning via Netlink Connection tracking helpers in user-space via Netlink [*] NFQUEUE and NFLOG integration with Connection Tracking Netfilter nf_tables support Netfilter nf_tables set infrastructure [*] Netfilter nf_tables mixed IPv4/IPv6 tables support [*] Netfilter nf_tables netdev tables support Netfilter nf_tables number generator module Netfilter nf_tables conntrack module

4

Netfilter nf_tables counter module Netfilter nf_tables connlimit module Netfilter nf_tables log module Netfilter nf_tables limit module Netfilter nf_tables masquerade support Netfilter nf_tables redirect support Netfilter nf_tables nat module Netfilter nf_tables tunnel module Netfilter nf_tables stateful object reference module Netfilter nf_tables queue module Netfilter nf_tables quota module Netfilter nf_tables reject support Netfilter nf_tables hash module Netfilter nf_tables socket match support Netfilter nf_tables passive OS fingerprint support Netfilter nf_tables tproxy support -M- Netfilter packet duplication support Netfilter nf_tables netdev packet duplication support Netfilter flow table mixed IPv4/IPv6 module Netfilter flow table module {M} Netfilter Xtables support (required for ip_tables) *** Xtables combined modules *** -M- nfmark target and match support -M- ctmark target and match support set target and match support *** Xtables targets *** AUDIT target support CHECKSUM target support "CLASSIFY" target support "CONNMARK" target support "DSCP" and "TOS" target support -M- "HL" hoplimit target support "HMARK" target support IDLETIMER target support LOG target support "MARK" target support -M- "SNAT and DNAT" targets support -M- "NETMAP" target support "NFLOG" target support "NFQUEUE" target Support -M- "RATEEST" target support -M- REDIRECT target support "TEE" - packet cloning to alternate destination "TPROXY" target transparent proxying support

5

"TCPMSS" target support "TCPOPTSTRIP" target support *** Xtables matches *** "addrtype" address type match support "bpf" match support "control group" match support "cluster" match support "comment" match support "connbytes" per-connection counter match support "connlabel" match support "connlimit" match support "connmark" connection mark match support "conntrack" connection tracking match support "cpu" match support "dccp" protocol match support "devgroup" match support "dscp" and "tos" match support -M- "ecn" match support "esp" match support "hashlimit" match support "helper" match support -M- "hl" hoplimit/TTL match support "ipcomp" match support "iprange" address range match support "ipvs" match support "l2tp" match support "length" match support "limit" match support "mac" address match support "mark" match support "multiport" Multiple port match support "nfacct" match support "osf" Passive OS fingerprint match "owner" match support IPsec "policy" match support "physdev" match support "pkttype" packet type match support "quota" match support "rateest" match support "realm" match support "recent" match support "sctp" protocol match support "socket" match support "state" match support

6

"statistic" match support "string" match support "tcpmss" match support "time" match support "u32" match support IP set support --->全选 IP virtual server support --->全选 IP: Netfilter Configuration --->全选 IPv6: Netfilter Configuration --->全选 [*] Ethernet Bridge nf_tables support ---> Netfilter nf_tables bridge reject support Ethernet Bridge tables (ebtables) support --->全选 802.1d Ethernet Bridging 802.1Q/802.1ad VLAN Support NETLINK: socket monitoring interface [*] Network priority cgroup -*- Network classid cgroup Device Drivers ---> [*] Network device support ---> MAC-VLAN support Virtual ethernet pair systems ---> FUSE (Filesystem in Userspace) support

3.2 联网安装 lxc apt-get install lxc apt-get install python3-lxc apt-get install lxc-templates apt-get install librbd1 安装完以上软件后，执行命令 ifconfig -a，能够看到 lxcbr0 网卡信息，如下所示： lxcbr0: flags=4099 mtu 1500 inet 10.0.3.1 netmask 255.255.255.0 broadcast 0.0.0.0 inet6 fe80::216:3eff:fe00:0 prefixlen 64 scopeid 0x20 ether 00:16:3e:00:00:00 txqueuelen 1000 (Ethernet) RX packets 31 bytes 3305 (3.3 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 26 bytes 3254 (3.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 系统是否支持 lxc 输入命令：lxc-checkconfig，看输出是否有 missing，如果没有，说明支持，如果有，需要在内核配置中添 加相关模块。正确配置的输出如下： --- Namespaces --- Namespaces: enabled

7

Utsname namespace: enabled Ipc namespace: enabled Pid namespace: enabled User namespace: enabled Network namespace: enabled

--- Control groups --- : enabled

Cgroup v1 mount points: /sys/fs/cgroup/ /sys/fs/cgroup/cpuset /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/perf_event /sys/fs/cgroup/pids /sys/fs/cgroup/freezer /sys/fs/cgroup/devices /sys/fs/cgroup/blkio /sys/fs/cgroup/rdma /sys/fs/cgroup/memory

Cgroup v2 mount points: /sys/fs/cgroup/unified

Cgroup v1 clone_children flag: enabled Cgroup device: enabled Cgroup sched: enabled Cgroup cpu account: enabled Cgroup memory controller: enabled Cgroup cpuset: enabled

--- Misc --- Veth pair device: enabled, loaded Macvlan: enabled, not loaded Vlan: enabled, not loaded Bridges: enabled, loaded Advanced netfilter: enabled, not loaded CONFIG_NF_NAT_IPV4: enabled, loaded CONFIG_NF_NAT_IPV6: enabled, not loaded CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, loaded CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded

8

FUSE (for use with lxcfs): enabled, loaded

--- Checkpoint/Restore --- checkpoint restore: enabled CONFIG_FHANDLE: enabled CONFIG_EVENTFD: enabled CONFIG_EPOLL: enabled CONFIG_UNIX_DIAG: enabled CONFIG_INET_DIAG: enabled CONFIG_PACKET_DIAG: enabled CONFIG_NETLINK_DIAG: enabled File capabilities:

Note : Before booting a new kernel, you can check its configuration usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

3.3 创建 lxc 容器 lxc-create -n lxc-ubuntu0 -t ubuntu，默认情况下，这个命令会创建一个最小的 Ubuntu 环境，版本号与你 的宿主机一致，我的是 ubuntu18.04，64 位，容器的存放路径为/var/lib/lxc。 创建需要联网下载需要的软件，所以必须保证主机能上网，创建容器的时间与容器类型和网速有关， ubuntu 容器在 4G 网络一般 15 分钟左右可以完成，创建容器时输出如下，只截取开头一部分。 root@localhost:/var/cache/lxc# lxc-create -n lxc-ubuntu0 -t ubuntu Checking cache download in /var/cache/lxc//rootfs-arm64 ... Installing packages in template: apt-transport-https,ssh,vim,language-pack-en Downloading ubuntu bionic minimal ... I: Retrieving InRelease I: Checking Release signature I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32) I: Retrieving Packages I: Validating Packages I: Retrieving Packages I: Validating Packages I: Resolving dependencies of required packages... I: Resolving dependencies of base packages... I: Found additional base dependencies: language-pack-en-base libcomerr2 libedit2 libgpm2 libgssapi-krb5-2 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libpython3.6 libssl1.0.0 libwrap0 openssh-client openssh-server openssh-sftp-server vim-runtime I: Checking component main on http://ports.ubuntu.com/ubuntu-ports... I: Checking component universe on http://ports.ubuntu.com/ubuntu-ports... I: Retrieving adduser 3.116ubuntu1 I: Validating adduser 3.116ubuntu1 I: Retrieving apt 1.6.1

9

I: Validating apt 1.6.1 I: Retrieving apt-utils 1.6.1 I: Validating apt-utils 1.6.1 I: Retrieving base-files 10.1ubuntu2 I: Validating base-files 10.1ubuntu2 I: Retrieving base-passwd 3.5.44 I: Validating base-passwd 3.5.44 I: Retrieving bash 4.4.18-2ubuntu1 I: Validating bash 4.4.18-2ubuntu1 I: Retrieving bsdutils 1:2.31.1-0.4ubuntu3 I: Validating bsdutils 1:2.31.1-0.4ubuntu3 I: Retrieving busybox-initramfs 1:1.27.2-2ubuntu3 I: Validating busybox-initramfs 1:1.27.2-2ubuntu3

3.4 启动容器

执行命令lxc-start -n lxc-ubuntu0 启动创建好的容器，执行命令lxc-console -n lxc-ubuntu0 进入容器界面， 登录用户名和密码都是ubuntu，进入后，执行sudo passwd root修改root密码。

3.5 停止容器

在主机界面执行命令lxc-stop -n lxc-ubuntu0 停止正在运行的容器，或者在容器内执行sudo poweroff。

3.6 lxc 常用命令

lxc-create -n -t 创建容器； lxc-ls --fancy 列出容器详细信息； lxc-start -n -d 后台运行容器； lxc-console -n 打开容器控制台； lxc-stop -n 停止容器； lxc-destroy -n 删除容器； lxc-copy -n -n

10