Compilation and Installation Guide (Robox Container)

Kunpeng BoostKit for ARM Native

Issue 11 Date 2021-07-05

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2021. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied.

Contents

1 Robox Android Container Security Description...... 1 1.1 Overview...... 1 1.2 Security Description...... 1 1.2.1 Ubuntu Security Update...... 1 1.2.2 Docker Security Hardening...... 2 1.2.3 Android Security Update...... 2 2 Robox Android Container Compilation Guide (Kunpeng 920)...... 3 2.1 Introduction...... 3 2.2 Environment Requirements...... 4 2.3 Configuring the Compilation Environment...... 4 2.3.1 Configuring the System...... 5 2.3.2 Installing Dependency Libraries and Basic Components...... 6 2.3.3 Replacing the Docker Storage Driver...... 6 2.3.4 Installing a Remote Desktop...... 7 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors...... 7 2.5 Starting the Android System in Robox Containers...... 14 2.6 Building the strace Binary File...... 15 2.7 Enabling Multiple GPUs...... 17 3 Android Image Compilation Guide (x86-based Environment)...... 19 3.1 Introduction...... 19 3.2 Environment Requirements...... 19 3.3 Preparing the x86 Basic Compilation Environment...... 20 3.4 Downloading the Source Code and Applying Patches...... 21 3.5 Compiling the Android Source Code...... 22 A Change History...... 23

1 Robox Android Container Security Description

1.1 Overview 1.2 Security Description

1.1 Overview The Robox container solution is a virtualization solution that enables the Android OS using the Docker container. This document describes the security specifications of the robox container solution. Read this document carefully.

1.2 Security Description In the Robox container solution, Huawei provides self-developed binary files, targeted open-source patch files, and script files. The targeted open-source patch files and script files are for reference only, and no commercial commitments are made. In addition, customers or independent software vendors (ISVs) are responsible for the development and maintenance of other components involved in the solution, such as the OS on the host side, Docker, and Android OS. To ensure the trusted and secure running of the ARM native solution in commercial scenarios, it is strongly recommended that customers or ISVs install and use the latest security patches and security hardening measures for the open- source software involved in the solution, including but not limited to the following suggestions. 1.2.1 Ubuntu Security Update When using Ubuntu, ensure that the latest security updates are used in a timely manner. Using the latest security updates to repair the system in a timely manner can prevent the OS from being affected by vulnerabilities and attacked by malicious software, and ensure the proper running of the Robox Android container on the system.

You can periodically run the apt-get command in Ubuntu to check whether security updates are available in the OS. If yes, install the updates in a timely manner. For details, see the description on the official website of Ubuntu. In addition to installing the latest security patches in a timely manner, you also need to perform security hardening on the OS of the server, for example, configuring strong passwords and disabling unnecessary service ports. For details, see the description on the official website of Ubuntu. 1.2.2 Docker Security Hardening Docker security hardening measures include but not limited to: hardening the OS of the host, configuring strict access control policies, controlling the Docker container resource quota, prohibiting untrusted images, periodically performing security scanning, and updating patches. Periodically perform security check and hardening for Docker containers to ensure proper running. For details, see the description on the official website of Docker containers. 1.2.3 Android Security Update The Android ecosystem is supported by Google, which not only provides system updates with improved functionality and stability, but also provides users with security updates that ensure device security. Security update patches are mainly provided by the Android Open Source Project (AOSP) and the upstream Linux kernel and system on a chip (SOC) manufacturers to ensure that Android devices are not affected by the latest security vulnerabilities of hardware and software. Google periodically pushes security updates to devices and releases security update notices. Use the source code provided by the AOSP and the patch link provided in the security update notice to perform security update in a timely manner based on the site requirements to ensure the proper running of the ARM native solution. Periodically update and harden the open-source software involved in the solution. For details, see the official documents of the open-source software.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 2 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

2 Robox Android Container Compilation Guide (Kunpeng 920)

2.1 Introduction 2.2 Environment Requirements 2.3 Configuring the Compilation Environment 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors 2.5 Starting the Android System in Robox Containers 2.6 Building the strace Binary File 2.7 Enabling Multiple GPUs

2.1 Introduction This document describes how to build, compile, and use the Robox Android container environment, specifically, how to configure the environment and how to download, build, compile, and use the code. This document is intended for users who need to build, compile, and use the Robox Android container environment. The container is implemented based on the basic framework Anbox. Before starting the container, start the session manager on the host to start services such as X, input, and adb, and then start the container. Figure 2-1 shows the basic architecture.

Figure 2-1 Basic architecture of the container

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 3 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

2.2 Environment Requirements

Hardware Requirements

Item Description

Server One TaiShan 200 server (model 2280) and one x86 server

System SSD (recommended) or HDD drive

x86 More than 100 GB free memory and preinstalled Java 1.8.X environme nt

CA UTION

If the external power cable of a video card is used, the external power cable of the riser card for Kunpeng 920 must be used. Do not use the external power cable of the riser card for Kunpeng 916. Otherwise, the GPU will be damaged because the 12 V power cable and ground cable have different wire sequences.

OS Requirements

Ubuntu 18.04.1 is installed on the TaiShan 200 server (model 2280).

Obtain the OS ISO file from http://old-releases.ubuntu.com/releases/18.04.1/ ubuntu-18.04.1-server-arm64.iso.

Contact Huawei engineers to obtain the OS installation guide.

CA UTION

During software selection in the last step of installing the OS, you need to select only OpenSSH server.

Local Tool Requirements 1. A remote tool is installed. MobaXterm is recommended. 2. The VNC Viewer is installed. URL: https://www.realvnc.com/en/connect/download/viewer/

2.3 Configuring the Compilation Environment

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 4 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

2.3.1 Configuring the System

Prerequisites ● The Ubuntu 18.04.1 OS has been installed. ● The network connection is normal, and the server can communicate with the external network.

● The download source has been configured. If no download source is configured, perform the following operations.

Configuring the Download Source

Step 1 Log in to the server using SSH and access the CLI.

Step 2 Delete the number signs (#) from all # deb-src lines in /etc/apt/sources.list.

Run the following command to change the download source:

sed -i "s/# deb-src/ deb-src/g" /etc/apt/sources.list

The information is as follows:

Step 3 Update the source. apt-get update

----End

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 5 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

2.3.2 Installing Dependency Libraries and Basic Components

Step 1 Install the basic dependency libraries of the system.

NO TICE

If a .deb package fails to be obtained during the installation, manually download and install the package based on the website displayed in the message. Then, reinstall the package that fails to be obtained.

apt install dpkg libncurses5-dev libncursesw5-dev libssl-dev cmake cmake-data debhelper dbus google- mock libboost-dev libboost-filesystem-dev libboost-log-dev libboost-iostreams-dev libboost-program- options-dev libboost-system-dev libboost-test-dev libboost-thread-dev libcap-dev libsystemd-dev libdbus-1- dev libegl1-mesa-dev libgles2-mesa-dev libglib2.0-dev libglm-dev libgtest-dev liblxc1 libproperties-cpp-dev libprotobuf-dev libsdl2-dev libsdl2-image-dev lxc-dev pkg-config protobuf-compiler libboost- filesystem1.62.0 libboost-system1.62.0 docker.io dkms libboost-iostreams1.62.0 apt install build-essential apt install mesa-common-dev

Step 2 Download libprocess-cpp3_3.0.1-0ubuntu5_arm64.deb, libdbus- cpp5_5.0.0+16.10.20160809-0ubuntu2_arm64.deb, and libdbus-cpp- dev_5.0.0+16.10.20160809-0ubuntu2_arm64.deb. Step 3 Install the three dependencies. dpkg -i libprocess-cpp3_3.0.1-0ubuntu5_arm64.deb dpkg -i libdbus-cpp5_5.0.0+16.10.20160809-0ubuntu2_arm64.deb dpkg -i libdbus-cpp-dev_5.0.0+16.10.20160809-0ubuntu2_arm64.deb

----End 2.3.3 Replacing the Docker Storage Driver Open a new window and perform the following operations:

Step 1 Log in to the server using SSH and access the CLI. Step 2 View Docker information. docker info

Step 3 Change the version of the storage driver. ● If the value of storage driver is overlay or overlay2, you do not need to change the value.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 6 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

● If the value of storage driver is aufs, change aufs to overlay. a. Open the /etc/default/docker file. vim /etc/default/docker b. Add the following script: DOCKER_OPTS= -s overlay c. Restart Docker for the modification to take effect. /etc/init.d/docker restart d. Run the following command to view the storage driver version of Docker: docker info | grep Storage

----End 2.3.4 Installing a Remote Desktop Step 1 Install a remote desktop. apt install xfce4 xfce4-* xrdp Step 2 Open the .xsession folder. cd /home/ubuntu vi .xsession NO TE

In the preceding commands, /home/ubuntu is the user folder. Step 3 Add the following content to the .xsession file: xfce4-session Step 4 Save the setting and check that the content is added. cat .xsession xfce4-session Step 5 Restart the xrdp remote desktop. /etc/init.d/xrdp restart Then, you can access the graphical desktop of the server that uses Huawei Kunpeng 920 processors as the root user from the remote desktop of the Windows OS.

----End

2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors

NO TE

Download the Robox source code from https://github.com/kunpengcompute/robox/tree/ master.

Step 1 Decompress the downloaded package robox-master.zip to the /home/ directory. unzip robox-master.zip

----End

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 7 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

Applying the ExaGear Transcoding Patch NO TE

Obtain the ExaGear transcoding patch from the following path: https://github.com/kunpengcompute/robox/tree/master/Exagear

Step 1 Create a directory. mkdir -p /home/compiler mkdir -p /home/exagear Step 2 Download the source code files linux_4.15.0.orig.tar.gz, linux_4.15.0-65.74.diff.gz, and linux_4.15.0-65.74.dsc to the /home/compiler/ directory. To download the kernel source code for a physical machine, visit https:// launchpad.net/ubuntu/+source/linux/4.15.0-65.74. Step 3 Go to the /home/compiler/ directory. cd /home/compiler/ ls linux_4.15.0-65.74.diff.gz linux_4.15.0-65.74.dsc linux_4.15.0.orig.tar.gz Step 4 Create the source code directory linux-4.15.0 in the current directory. dpkg-source -x linux_4.15.0-65.74.dsc Step 5 Upload the downloaded Android and kernel folders to /home/exagear. cd /home/exagear ls android //Transcoding patch of the Android source code kernel //Transcoding patch of the physical machine kernel Step 6 Copy the transcoding patch to the kernel source code directory. cp /home/exagear/kernel/ubuntu-4.15.0-65.74.patch /home/compiler/linux-4.15.0/ cd /home/compiler/linux-4.15.0/ patch -p1 < ubuntu-4.15.0-65.74.patch Step 7 Apply the performance monitoring unit (PMU) patch. Patch location: /home/robox-master/kernel/patch/ 1001_drivers_perf_hisi_update_the_sccl_id_ccl_id_when_MT.patch

cp 1001_drivers_perf_hisi_update_the_sccl_id_ccl_id_when_MT.patch /home/compiler/linux-4.15.0/ cd /home/compiler/linux-4.15.0/ patch -p1 < 1001_drivers_perf_hisi_update_the_sccl_id_ccl_id_when_MT.patch Step 8 Apply the kernel vmalloc performance patch. Patch location: /home/robox-master/kernel/patch/ 1002_mm_vmalloc_improve_vmap_allocation.patch

cp 1002_mm_vmalloc_improve_vmap_allocation.patch /home/compiler/linux-4.15.0/ cd /home/compiler/linux-4.15.0/ patch -p1 < 1002_mm_vmalloc_improve_vmap_allocation.patch

----End

Compiling and Installing the Kernel

Step 1 Create the .config file. make menuconfig //After this command is executed, save the configuration and exit to generate the .config file.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 8 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

Step 2 Ensure that the .config file contains the following configuration items and the same values. If the content of the file is inconsistent with the following, modify it manually. CONFIG_BINFMT_MISC=y CONFIG_EXAGEAR_BT=y CONFIG_CHECKPOINT_RESTORE=y CONFIG_PROC_CHILDREN=y CONFIG_VFAT_FS=y CONFIG_INPUT_UINPUT=y CONFIG_HISI_PMU=y Step 3 Perform compilation and installation. make -j64 //Perform compilation. make modules_install //Install modules. make install //Install the kernel. cd /boot/grub sudo update-grub2 //Update the boot option. Step 4 Reboot the kernel. reboot

----End

Registering Transcoding Rules After the New Kernel Is Used After the physical machine is restarted, enable the transcoding registration function.

NO TICE

Download the ExaGear. This software is restricted for use. Contact the Huawei frontline engineer to obtain the software. URL: https://support.huawei.com/enterprise/en/kunpeng-computing/kunpeng- computing-media-pid-251431619/software/253129399/?idAbsPath=fixnode01| 23710424|251364417|9856629|251431619

Step 1 Check that the binfmt_misc file system is mounted. By default, the file system is mounted. If the file system is not mounted, manually mount it. mount -t binfmt_misc none /proc/sys/fs/binfmt_misc Step 2 Register the ExaGear transcoding rule. Ensure that the directories are the same as the /opt/exagear/ubt_a32a64 directory. mkdir -p /opt/exagear cd /home/exagear/ tar zxvf ExaGear_ARM32-ARM64_V1.5.tar.gz cp /home/exagear/ExaGear_ARM32-ARM64_V1.5/ubt_a32a64 /opt/exagear/ cd /opt/exagear //Directory for storing the binary file for transcoding. The directory is /opt/exagear/ ubt_a32a64 during echo registration. chmod +x ubt_a32a64 //Avoid registration with permission denied.

CA UTION

Each time the server is restarted, manually register the transcoding rule again. echo ":ubt_a32a64:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff \xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\xfe\xff\xff\xff:/opt/exagear/ubt_a32a64:POCF" > / proc/sys/fs/binfmt_misc/register

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 9 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

Step 3 Check whether the ExaGear rule is successfully registered and ensure that the /opt/exagear/ubt_a32a64 directory is consistent. cat /proc/sys/fs/binfmt_misc/ubt_a32a64 enabled interpreter /opt/exagear/ubt_a32a64 flags: POCF offset 0 magic 7f454c4601010100000000000000000002002800 mask ffffffffffffff000000000000000000feffffff Step 4 (Optional) Check whether the ARM32-to-ARM64 program of the physical kernel is running properly. The mpro32 program can be any ARM 32-bit binary executable file. echo 1 > /proc/sys/fs/binfmt_misc/status //Default status of the transcoding function. echo 1 indicates that the transcoding function is enabled, and echo 0 indicates that the transcoding function is disabled. ./mpro32 mprotect successhellow workd Step 5 (Optional) echo 0 indicates that the function is disabled. Test the program. echo 0 > /proc/sys/fs/binfmt_misc/status ./ubt_a32a64 mpro32 mprotect successhellow workd

----End

Compiling and Installing binder.ko and ashmem.ko on the Server That Uses Huawei Kunpeng 920 Processors Step 1 Download the kernel source code for kernel module compilation. apt search linux-source apt install linux-source-4.15.0 Step 2 Copy the source code of ashmem and binder. cd /home/robox-master/kernel/robox-modules cp anbox.conf /etc/modules-load.d/ cp 99-anbox.rules /lib/udev/rules.d/ cp -rT ashmem /usr/src/anbox-ashmem-1 cp -rT binder /usr/src/anbox-binder-1 Step 3 Use the DKMS for compilation and installation. dkms install anbox-ashmem/1 dkms install anbox-binder/1 Step 4 Install the ko module to the kernel. The binder_linux module must contain parameters.

CA UTION

Each time the server is restarted, remove the binder_linux module and reinstall it.

modprobe ashmem_linux modprobe binder_linux num_devices=254 lsmod | grep -e ashmem_linux -e binder_linux chmod 777 /dev/ashmem /dev/binder* Step 5 If the permissions of the ashmem and binder attributes are different from those in the following command output, run the chmod command to add the permissions. ls -alh /dev/binder* /dev/ashmem crwxrwxrwx 1 root root 10, 55 Oct 22 10:47 /dev/ashmem

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 10 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

crwxrwxrwx 1 root root 511, 0 Oct 22 10:47 /dev/binder0 crwxrwxrwx 1 root root 511, 0 Oct 22 10:47 /dev/binder1 ...

----End

Registering an Image with android.img and Storing It in Docker This section provides the Android image package for test only. You can click android.img to obtain it. Use the Android image compiled based on Android Image Compilation Guide (x86-based Environment). The actual name prevails. In this document, the name of the tested Android image package is used as an example. Assume that the Android image has been uploaded to the /home directory on the server that uses Huawei Kunpeng 920 processors.

Step 1 Log in to the server using SSH and access the CLI. Step 2 Mount the Android image to the /mnt directory. cd /home mount /home/android.img /mnt

NO TE

The android.img file is compiled in the x86 environment. Therefore, you should mount android.img in the /mnt directory after the server is restarted. Step 3 Go to the /mnt directory. cd /mnt Step 4 Register an image. tar --numeric-owner -cf- . | docker import - android:robox_with_exagear Step 5 Check the system container. docker images If the following information is displayed, the Android container exists:

REPOSITORY TAG IMAGE ID CREATED SIZE android robox_with_exagear xxxxxxxx x seconds ago xxxMB

----End

Compiling the Robox Source Code on the Server That Uses Huawei Kunpeng 920 Processors

Step 1 Create a compilation directory. cd /home/robox-master mkdir build cd build Step 2 Run the following command in the /home/robox-master/build directory to configure compilation: cmake .. Step 3 Add the following content to the /usr/include/glm/gtx/transform.hpp file: vim /usr/include/glm/gtx/transform.hpp Add the following definition to line 21:

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 11 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

#define GLM_ENABLE_EXPERIMENTAL The following figure shows the modification result.

Step 4 Perform compilation and installation. make -j64 make install

----End

Enabling a GPU in Robox on the Server That Uses Huawei Kunpeng 920 Processors

Step 1 Install the xfce4 desktop and related tools. apt install -y xfce4 mesa-utils x11vnc vainfo Step 2 Modify the xorg.conf configuration file. cd /etc/X11 touch xorg.conf Step 3 Add the following content to the xorg.conf file. Change the bus ID based on the PCI number of the GPU on the server.

CA UTION

The elements in the bus ID are separated by colons (:).

Section "ServerFlags" Option "DontVTSwitch" "on" Option "AutoAddDevices" "off" Option "AutoEnableDevices" "off" Option "AutoAddGPU" "off" Option "AutoBindGPU" "off" EndSection Section "Device" Identifier "AMD" Driver "amdgpu" BusID "pci:01:00:00" EndSection Section "Monitor" Identifier "monitor0" Modeline "1280x720" 74.50 1280 1344 1472 1664 720 723 728 748 -hsync +vsync Option "enable" "true" EndSection Section "Screen" Identifier "screen0" Device "AMD" Monitor "monitor0" DefaultDepth 24 SubSection "Display" Depth 24 Modes "1280x720" EndSubSection EndSection Step 4 Query the PCI number of the GPU. lspci | grep AMD 81:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon Pro WX

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 12 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

xxxx] 81:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX xxx]

CA UTION

1. The bus ID in the lspci command output is a hexadecimal number, but the bus ID in the configuration file is a decimal number. Therefore, the value needs to be converted. For example, if the bus ID is 81:00.0, you need to enter pci: 129:00:00 in the configuration file. 2. The bus ID format in the configuration file is pci:xx:xx:xx, which is separated by colons (:). However, the bus ID in the lspci command output is xx:xx.x. Do not directly copy the bus ID in the lspci command output to overwrite the bus ID in the configuration file. 3. The value of the Modeline field (1280x720) must be greater than or equal to the resolution of the Android system.

Step 5 Compile and install mesa-19.0.8. To reinstall the Mesa, perform the following steps: 1. Download the Mesa code. cd /home/ git clone https://anongit.freedesktop.org/git/mesa/mesa.git 2. Go to the Mesa code directory and switch to the 19.0.8 branch. cd /home/mesa git checkout mesa-19.0.8 3. Download the compilation dependency. apt build-dep mesa apt install libomxil-bellagio-dev libva-dev llvm-7 llvm-7-dev python-mako 4. Run the autogen.sh script to generate Makefile. ./autogen.sh --enable-texture-float --with-gallium-drivers=radeonsi,swrast --with-dri- drivers=radeon,swrast --with-platforms=drm,x11 --enable-glx-tls --enable-shared-glapi --enable-dri3 --enable-lmsensors --enable-gbm --enable-xa --enable-osmesa --enable-vdpau --enable-nine -- enable-omx-bellagio --enable-va --with-llvm-prefix=/usr/lib/llvm-7 --enable-llvm --target=aarch64- linux-gnu CFLAGS="-fsigned-char -O2" CPPFLAGS="-fsigned-char -O2" CXXFLAGS="-fsigned-char - O2" --enable-autotools 5. Perform compilation and installation. make -j32 && make install 6. Change the library link sequence. vim /etc/ld.so.conf 7. Add /usr/local/lib to the beginning of the file. cat /etc/ld.so.conf /usr/local/lib include /etc/ld.so.conf.d/*.conf 8. Run the following command: ldconfig 9. Check whether Xorg can be started properly. Xorg :0 -config /etc/X11/xorg.conf – If Xorg can be started properly, go to the next section. – If Xorg cannot be started, go back to 5.f. That is, delete the /usr/local/lib line from the /etc/ld.so.conf file, run the ldconfig command again, and then start Xorg. If Xorg starts, perform Step 5.6 to Step 5.8. (Do not perform 5.i.)

----End

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 13 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

2.5 Starting the Android System in Robox Containers

Step 1 Copy the startup script robox in the binaryFiles directory in the robox-master source code downloaded in 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors to the /home/robox-master directory.

NO TE

Change the image name in the docker run command in the robox script to the name registered locally. For details about the name, see Registering an image. You can also view the name by running the docker images command. Step 2 On the server that uses Huawei Kunpeng 920 processors, run the modified robox script to start Robox Android containers. 1. Set the environment variable for the containers to run on GPUs. export DISPLAY=:0

CA UTION

1. The value of DISPLAY in the robox script must be the same as that specified in the preceding command. 2. Before starting Robox, check whether the environment variable XDG_RUNTIME_DIR exists. If the environment variable does not exist, add export XDG_RUNTIME_DIR=/run/user/0 to the start position of the robox executable script and ensure that the /run/user/0 directory exists. 3. Start the Robox containers in sequence. That is, start instance 1, instance 2, instance 3, and so on in sequence. Do not start the robox containers in random order. Otherwise, the Robox containers cannot be connected using the host IP addresses.

2. Start Robox containers. – Start the first container instance. ./robox -v start 1 – Start the second container instance. ./robox -v start 2 Step 3 Check the Docker instance process. docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b77d371b402c android:robox_with_exagear "/anbox-init.sh" 13 seconds ago Up 11 seconds 0.0.0.0:5561->5555/tcp instance2 77b2c041315f android:robox_with_exagear "/anbox-init.sh" 2 hours ago Up 2 hours 0.0.0.0:5559->5555/tcp instance1 Step 4 Check whether the two host sessions corresponding to instance 1 and instance 2 are running properly. ps -aux | grep session root 4330 0.0 0.0 9332 6160 ? Ss Oct22 0:01 /usr/bin/dbus-daemon --session -- address=systemd: --nofork --nopidfile -- systemd-activation --syslog-only root 172678 22.1 0.0 6433328 250472 pts/8 Sl 19:51 25:38 anbox session-manager --run-

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 14 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

multiple=instance1 --standalone -- experimental --single-window --gles-driver=translator --window-size=720,1280 root 215155 1.4 0.0 5196228 185688 pts/8 Sl 21:46 0:01 anbox session-manager --run- multiple=instance2 --standalone -- experimental --single-window --gles-driver=translator --window-size=720,1280

Step 5 Log in to the Android container and check whether the Robox instances are started successfully. docker exec -it instance1 sh 77b2c041315f:/ # getprop | grep sys.boot.completed [sys.boot_completed]: [1] //If the value of sys.boot.completed is 1, the Android system is started.

Step 6 Connect to the Android container on Windows.

The following steps use ARDC as an example. You can also use other Android remote connection software. 1. Run the adb.exe program in the \ARDC\utils\ directory in the CLI on Windows. C:\\ARDC\utils> adb connect YOUR_IP:5561 connected to YOUR_IP:5561 2. The IP address and port number to be connected on Windows are the IP address and container port number of the server that uses Huawei Kunpeng 920 processors, for example, port 5561. Run the #docker ps command on the server that uses Huawei Kunpeng 920 processors to obtain the ADB port information of the instance. CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b77d371b402c android:robox_with_exagear "/anbox-init.sh" 13 seconds ago Up 11 seconds 0.0.0.0:5561->5555/tcp instance2 3. Run the ARDC.exe program in the ARDC directory on Windows. Choose View > Mode, and select Screenshot. In the Devices menu, select the device that has been connected to the ADB.

CA UTION

Do not use the Shutdown, Reboot, and Home tags under Key.

To stop Running robox container instances, run the following commands: cd /home/robox-master ./robox -v stop 1 //Stop the first container instance. ./robox -v stop 2 //Stop the second container instance.

----End

2.6 Building the strace Binary File

Step 1 strace is a Linux user space tracker that can be used for diagnosis, debugging, and teaching. If the Android Virtual Device (AVD) is running on the Kunpeng 920, the system call cannot be properly displayed when you use strace to trace the process of a running 32-bit application, as shown in the following figure. If you need to use strace to trace the process, rebuild the strace binary file.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 15 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

Step 2 Obtain the strace binary file.

Compile the strace binary file. For details about the compilation method, see Kunpeng BoostKit for ARM Native FAQs.

Step 3 Replace the strace file.

When an exception occurs in the system call of the strace tracing process, replace the current strace file with the new strace binary file after compilation.

The procedure is as follows:

1. Upload the strace binary file to the directory of the running server. For example, the /home directory. cd /home 2. Run the adb command to copy the strace binary file to the current AVD to replace the strace file in /system/bin/. The following uses emulator-5554 as an example. adb connect 0.0.0.0:5559 adb -s 0.0.0.0:5559 shell remount adb -s 0.0.0.0:5559 push strace /system/bin/

Step 4 Verify the result.

Install a 32-bit APK in the AVD whose strace binary has been replaced. The following describes how to install TikTok in emulator-5554. adb -s 0.0.0.0:5559 install --abi armeabi-v7a com.ss.android.ugc.aweme.apk

Run TikTok. Run the adb command to view the progress number.

adb -s 0.0.0.0:5559 shell ps -ef

Run the strace command to trace the running status.

strace -p 12646

The following figure shows the running status. The name of the system call function can be displayed normally.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 16 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

----End

2.7 Enabling Multiple GPUs This section describes how to enable multiple GPUs on a server and bind a Robox instance to a fixed GPU when starting the Robox.

Binding Xorg to a GPU Each GPU corresponds to a .conf file. For example, if two GPUs are installed on a server, create the xorg0.conf and xorg1.conf files in the /etc/X11 directory. A .conf file is associated with a GPU via the PCI bus ID.

NO TE

The xorg0.conf and xorg1.conf files can be stored based on the site requirements. You need to specify the correct directory when starting Xorg. This document uses the /etc/X11 directory as an example. xorg0.conf:

Section "ServerFlags" Option "DontVTSwitch" "on" Option "AutoAddDevices" "off" Option "AutoEnableDevices" "off" Option "AutoAddGPU" "off" Option "AutoBindGPU" "off" EndSection Section "Device" Identifier "AMD" Driver "amdgpu" BusID "pci:1:00:00" EndSection Section "Monitor" Identifier "monitor0" Option "enable" "true" #Modeline "1680x1050_60.00" 146.25 1680 1784 1960 2240 1050 1053 1059 1089 -hsync +vsync #Modeline "1024x768_60.00" 63.50 1024 1072 1176 1328 768 771 775 798 -hsync +vsync EndSection Section "Screen" Identifier "screen0" Device "AMD" Monitor "monitor0" DefaultDepth 24 SubSection "Display" Depth 24 #Modes "1680x1050_60.00" #Modes "1024x768_60.00" EndSubSection EndSection xorg1.conf:

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 17 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 2 Robox Android Container Compilation Guide Container) (Kunpeng 920)

Section "ServerFlags" Option "DontVTSwitch" "on" Option "AutoAddDevices" "off" Option "AutoEnableDevices" "off" Option "AutoAddGPU" "off" Option "AutoBindGPU" "off" EndSection Section "Device" Identifier "AMD" Driver "amdgpu" BusID "pci:2:00:00" EndSection Section "Monitor" Identifier "monitor0" Option "enable" "true" #Modeline "1680x1050_60.00" 146.25 1680 1784 1960 2240 1050 1053 1059 1089 -hsync +vsync #Modeline "1024x768_60.00" 63.50 1024 1072 1176 1328 768 771 775 798 -hsync +vsync EndSection Section "Screen" Identifier "screen0" Device "AMD" Monitor "monitor0" DefaultDepth 24 SubSection "Display" Depth 24 #Modes "1680x1050_60.00" #Modes "1024x768_60.00" EndSubSection EndSection To obtain the bus ID in the configuration file, run the following commands. xxxx indicates the GPU model.

lspci |grep xxxx 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon Pro WX xxxx] 02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon Pro WX xxxx]

CA UTION

1. The bus ID in the lspci command output is a hexadecimal number, but the bus ID in the configuration file is a decimal number. Therefore, the value needs to be converted. 2. The bus ID format in the configuration file is pci:xx:xx:xx, which is separated by colons (:). However, the bus ID in the lspci command output is xx:xx.x. Do not directly copy the bus ID in the lspci command output to overwrite the bus ID in the configuration file.

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 18 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 3 Android Image Compilation Guide (x86-based Container) Environment)

3 Android Image Compilation Guide (x86- based Environment)

3.1 Introduction 3.2 Environment Requirements 3.3 Preparing the x86 Basic Compilation Environment 3.4 Downloading the Source Code and Applying Patches 3.5 Compiling the Android Source Code

3.1 Introduction

The Android image package is required for compiling and building the robox source code. Huawei provides a compiled Android image package for test only. You can directly use the image package or manually compile and build an Android image by referring to this document.

The name of the Android image package provided by Huawei may contain information such as the timestamp. Use the actual name during the operation.

All the paths used in this document are example only. Replace the paths with actual paths during the operation.

3.2 Environment Requirements

Hardware Requirements

Table 3-1 lists the hardware requirements.

Table 3-1 Hardware requirements

Item Description

Server One x86 server with Java 1.8.X installed

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 19 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 3 Android Image Compilation Guide (x86-based Container) Environment)

Item Description

Drive space Remaining drive capacity > 100 GB

OS Requirements

Table 3-2 lists the OS requirements.

Table 3-2 OS requirements

Item Description

OS Ubuntu 18.04.x or Ubuntu 16.04.x This document uses Ubuntu 18.04.1 as an example.

Drive space Remaining drive capacity > 100 GB

Tools

You need to install a remote tool such as MobaXterm on the local maintenance terminal. MobaXterm is recommended.

3.3 Preparing the x86 Basic Compilation Environment

For details about how to prepare the compilation environment, see https:// source.android.com/setup/build/initializing.

Step 1 Download the OS image.

URL: http://old-releases.ubuntu.com/releases/18.04.1/ubuntu-18.04-server- amd64.iso

Step 2 Uncomment deb-src in /etc/apt/source.list. sed -i "s/# deb-src/ deb-src/g" /etc/apt/sources.list

Step 3 Install compilation dependencies. apt update apt install -y openjdk-8-jdk apt install -y libx11-dev libreadline6-dev libgl1-mesa-dev g++-multilib apt install -y git flex bison gperf build-essential libncurses5-dev apt install -y tofrodos python-markdown libxml2-utils xsltproc zlib1g-dev apt install -y dpkg-dev libsdl1.2-dev apt install -y git-core gnupg flex bison gperf build-essential apt install -y zip curl zlib1g-dev gcc-multilib g++-multilib apt install -y libc6-dev apt install -y lib32ncurses5-dev x11proto-core-dev libx11-dev apt install -y libgl1-mesa-dev libxml2-utils xsltproc unzip m4 apt install -y lib32z-dev ccache apt install -y bc python flex bison gperf libsdl-dev build-essential zip curl

----End

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 20 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 3 Android Image Compilation Guide (x86-based Container) Environment)

3.4 Downloading the Source Code and Applying Patches Step 1 Go to the home directory and download the Android source code. NO TE

Download the Robox source code from https://github.com/kunpengcompute/robox/tree/ master. Upload the Robox source code to the /home directory. cd /home/ unzip robox-master.zip If a download failure message is displayed indicating a verification to be performed, run the following command to skip the verification: export GIT_SSL_NO_VERIFY=1 Step 2 Download the Android source code. 1. Download the Repo tool and ensure that it is executable. mkdir ~/bin PATH=~/bin:$PATH curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo chmod a+x ~/bin/repo 2. Download the Android source code. mkdir -p /home/android cd /home/android/ repo init -u https://github.com/anbox/platform_manifests.git -b anbox repo sync -j64 //Synchronize the code. The total size of the directory into which the code is downloaded is about 100 GB. 3. Use the snapshot XML file in the repository to roll back the repository code. The snapshot is in the source package binaryFiles/snapshot20191206.xml downloaded in Step 1. cp /home/robox-master/binaryFiles/snapshot20191206.xml /home/android/.repo/manifests/ repo init -m snapshot20191206.xml repo sync -d -j64 //Roll back the code. 4. Delete the anbox directory from the /home/android/vendor/ directory. rm -rf /home/android/vendor/anbox 5. Copy the Robox code downloaded in Step 1 to the specified directory. cp -r /home/robox-master /home/android/vendor/ cd /home/android/vendor/ mv robox-master anbox Step 3 Download the ExaGear transcoding patches.

NO TICE

The ExaGear patches include the Android source patches and physical machine kernel patches. Here, the Android source patches are used. You can follow the following link to obtain the patches: https://github.com/kunpengcompute/robox/tree/master/Exagear

1. Upload the android folder to the /home/ directory. mkdir -p /home/exagear cd /home/exagear

Issue 11 (2021-07-05) Copyright © Huawei Technologies Co., Ltd. 21 Kunpeng BoostKit for ARM Native Compilation and Installation Guide (Robox 3 Android Image Compilation Guide (x86-based Container) Environment)

ls android //Install the transcoding patch of the Android source code. 2. Copy the transcoding patches to the Android source code directory. cp -r /home/exagear/android/android-7.1.1_r13.patch /home/android/ 3. Copy the vendor directory in the transcoding package to the Android source code directory and merge with the original vendor directory.

NO TE

The vendor/anbox/ directory exists in the /home/android/ directory. After the vendor directory in the transcoded source code is copied to the /home/android/ directory, the two vendor directories are merged instead of the original one being overwritten. cp -r /home/exagear/android/vendor /home/android/ ls /home/android/vendor/ anbox huawei Step 4 Copy the Android patch to the /home/android directory. cp -r /home/robox-master/patch/android-7.1.1_r13-V1.0/* /home/android Step 5 Apply the transcoding patches and Android patch. cp -r /home/robox-master/patch/patch.sh /home/android cd /home/android sh patch.sh

----End

3.5 Compiling the Android Source Code

Step 1 Compile the Android source code. cd /home/android/ source build/envsetup.sh //Configure the compilation environment. lunch anbox_arm64-userdebug //Select the compilation environment. export JACK_EXTRA_CURL_OPTIONS=-k export LC_ALL=C make -j48 //Compile the source code. The required ramdisk.img and system.img Android image files are generated in the /home/android/out/target/product/arm64/ directory. Step 2 Synthesize the Docker container image file android.img required by Robox. cd /home/android/vendor/anbox/ scripts/create-package.sh /home/android/out/target/product/arm64/ramdisk.img /home/android/out/target/ product/arm64/system.img The android.img image is synthesized in the current directory, which is /home/ android/vendor/anbox/. Step 3 Upload the android.img file to the /home directory on the server that uses the Huawei Kunpeng 920 processor for building the Robox container solution.

----End

A Change History

Date Description

2021-07-05 This issue is the eleventh official release. Updated 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors.

2021-05-27 This issue is the tenth official release. Changed "robox" to "Robox" (except for those in file names).

2021-03-31 This issue is the ninth official release. Changed the solution name from "Kunpeng ARM Native Solution" to "Kunpeng BoostKit for ARM Native".

2021-01-26 This issue is the eighth official release. Modified the document based on review comments.

2020-12-30 This issue is the seventh official release. ● Changed the solution name to "Kunpeng ARM native solution" and the document name to Kunpeng ARM Native Solution Compilation and Installation Guide (robox Container). ● Added the robox Android Container Security Description. ● Changed the document name robox Android Container Porting Guide (Kunpeng 920) to robox Android Container Compilation Guide (Kunpeng 920). ● Changed the document name Android Image Compilation and Building Guide (x86-based Environment) to Android Image Compilation Guide (x86- based Environment).

Date Description

2020-12-14 This issue is the sixth official release. ● robox Android Container Porting Guide (Kunpeng 916): removed. ● robox Android Container Porting Guide (Kunpeng 920): modified the compilation environment configuration and kernel compilation operations, updated the download link of ExaGear patches, and optimized robox code. ● Android Image Compilation and Building Guide (x86- based Environment): updated the download link of ExaGear patches and optimized robox code.

2020-11-16 This issue is the fifth official release. Modified 2.2 Environment Requirements.

2020-10-15 This issue is the fourth official release. ● Android Image Compilation and Building Guide (x86- based Environment): modified 3.2 Environment Requirements. ● robox Android Container Porting Guide (Kunpeng 920): modified 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors.

2020-09-21 This issue is the third official release. Changed the solution name to "cloud native solution."

2020-05-23 This issue is the second official release. ● Modified the robox Android Container Porting Guide (Kunpeng 920) as follows: Added the path and description for obtaining the exagear-a32a64-docker.tar.gz package in 2.4 Compiling the Kernel Source Code on the Server That Uses Huawei Kunpeng 920 Processors. ● Modified the Android Image Compilation and Building Guide (x86-based Environment) as follows: – Added 3.1 Introduction. – Added the path and description for obtaining the exagear-a32a64-docker.tar.gz package in 3.4 Downloading the Source Code and Applying Patches.

2020-03-20 This issue is the first official release.