Porting Guides

Kunpeng Database Solution

Porting Guides

Issue 04 Date 2021-03-30

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2021. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied.

Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base Bantian, Longgang Shenzhen 518129 People's Republic of China

Website: https://www.huawei.com Email: [email protected]

Contents

1 MySQL 5.7.27 Porting Guide (CentOS 7.6)...... 1 1.1 Introduction...... 1 1.2 Environment Requirements...... 2 1.3 Compilation and Installation Process...... 3 1.4 Configuring the Compilation Environment...... 3 1.4.1 Configuring the Local Yum Source...... 3 1.4.2 Upgrading CMake...... 5 1.4.3 Upgrading GCC...... 7 1.5 Compiling and Installing MySQL...... 8 1.6 Troubleshooting...... 10 1.6.1 CMake Target Version Does Not Take Effect After the Upgrade...... 10 1.6.2 CMake Compilation and Installation Failed...... 10 1.6.3 "error:'prctl' was not declared in this scope" Reported...... 11 2 (Recommended) MySQL 8.0.x Porting Guide (CentOS 7.6)...... 13 2.1 Introduction...... 13 2.2 Environment Requirements...... 14 2.3 Compilation and Installation Process...... 15 2.4 Configuring the Compilation Environment...... 15 2.4.1 Configuring the Local Yum Source...... 15 2.4.2 Upgrading CMake...... 17 2.4.3 Upgrading GCC...... 19 2.5 Compiling and Installing MySQL...... 21 2.6 Troubleshooting...... 22 2.6.1 CMake Target Version Does Not Take Effect After the Upgrade...... 23 2.6.2 CMake Compilation and Installation Failed...... 23 3 MySQL 8.0.x Porting Guide (openEuler 20.03)...... 25 3.1 Introduction...... 25 3.2 Environment Requirements...... 26 3.3 Compilation and Installation Process...... 27 3.4 Configuring the Compilation Environment...... 27 3.4.1 Installing Dependencies...... 27 3.4.2 Upgrading CMake...... 29

3.4.3 Upgrading GCC...... 31 3.5 Configuring the Operating Environment...... 32 3.5.1 Disabling the Firewall...... 32 3.5.2 Disabling SELinux...... 33 3.5.3 Creating a User Group and a User...... 33 3.5.4 Creating a Data Drive...... 34 3.5.5 Creating a Data Directory...... 37 3.6 Compiling and Installing MySQL...... 37 3.7 Running MySQL...... 40 3.8 Uninstalling MySQL...... 45 4 MariaDB 10.3.8 Porting Guide (CentOS 7.6)...... 46 4.1 Introduction...... 46 4.2 Environment Requirements...... 48 4.3 Configuring the Compilation Environment...... 49 4.4 Compiling and Installing MariaDB...... 50 5 MongoDB 4.0.12 Porting Guide (CentOS 7.6)...... 53 5.1 Introduction...... 53 5.2 Environment Requirements...... 54 5.3 Compilation and Installation Process...... 55 5.4 Configuring the Compilation Environment...... 55 5.4.1 Configuring the Local Yum Source...... 56 5.4.2 Upgrading CMake...... 57 5.4.3 Upgrading GCC...... 59 5.4.4 Installing Cython...... 61 5.4.5 Installing PyYAML...... 62 5.4.6 Installing Typing...... 64 5.4.7 Installing Cheetah3...... 65 5.5 Compiling and Installing MongoDB...... 66 5.6 Compiling and Installing MongoDB Tools...... 67 5.7 Troubleshooting...... 71 5.7.1 CMake Target Version Does Not Take Effect After the Upgrade...... 71 5.7.2 No space left on device...... 72 5.7.3 "processor does not support `crc32cb w2,w2,w3'" Reported...... 72 5.7.4 "No module named pkg_resources" Reported...... 73 6 Percona 8.0.15 Porting Guide (CentOS 7.6)...... 74 6.1 Introduction...... 74 6.2 Environment Requirements...... 75 6.3 Compilation and Installation Process...... 76 6.4 Configuring the Compilation Environment...... 76 6.4.1 Configuring the Local Yum Source...... 76 6.4.2 Upgrading CMake...... 78

6.4.3 Upgrading GCC...... 80 6.5 Compiling and Installing Percona...... 81 6.6 Troubleshooting...... 84 6.6.1 CMake Target Version Does Not Take Effect After the Upgrade...... 85 7 PostgreSQL 11.3 Porting Guide (CentOS 7.6)...... 86 7.1 Introduction...... 86 7.2 Environment Requirements...... 89 7.3 Compilation and Installation Process...... 90 7.4 Configuring the Compilation Environment...... 91 7.4.1 Configuring the Local Yum Source...... 91 7.4.2 Upgrading CMake...... 93 7.4.3 Upgrading GCC...... 94 7.5 Compiling and Installing PostgreSQL...... 96 7.6 Troubleshooting...... 99 7.6.1 CMake Target Version Does Not Take Effect After the Upgrade...... 99 8 Greenplum 6.0.0 Porting Guide (CentOS 7.6)...... 100 8.1 Introduction...... 100 8.2 Environment Requirements...... 101 8.3 Compilation and Installation Process...... 103 8.4 Configuring the Compilation Environment...... 104 8.4.1 Configuring the Local Yum Source...... 104 8.4.2 Installing Python Dependencies...... 106 8.4.3 Upgrading CMake...... 120 8.4.4 Installing zstd-dev...... 122 8.4.5 Installing Xerces...... 123 8.4.6 Installing re2c...... 125 8.4.7 Installing Ninja...... 128 8.4.8 Installing GPORCA...... 129 8.5 Compiling and Installing Greenplum...... 132 8.6 Troubleshooting...... 134 8.6.1 Greenplum Compilation Failed Due to Incorrect ORCA Version...... 134 9 Kunpeng BoostKit for Database Porting Guide...... 136 9.1 Introduction...... 136 9.2 Common Atomic Operations (C Language)...... 136 9.2.1 C11 Memory Model...... 137 9.2.2 Recommended Memory Barrier...... 137 9.2.3 Atomic Operations...... 138 9.2.4 Spin Operation...... 138 9.3 PostgreSQL...... 138 9.3.1 Atomic Operations...... 138 9.3.1.1 Memory Barrier...... 138

9.3.1.2 Spin Function...... 139 9.3.1.3 TAS & Clear...... 139 9.3.1.4 CAS...... 139 9.3.1.5 Fetch_add...... 140 9.3.2 CRC Checksum Algorithm...... 141 9.4 MySQL...... 141 9.4.1 Spin Function...... 141 9.4.2 CRC Checksum Algorithm...... 141 9.4.3 Cache Line Alignment Optimization...... 141 9.4.4 MySQL Read/Write Deadlock Due to ARM Weak Memory Sequence...... 143 A Change History...... 148

1 MySQL 5.7.27 Porting Guide (CentOS 7.6)

1.1 Introduction 1.2 Environment Requirements 1.3 Compilation and Installation Process 1.4 Configuring the Compilation Environment 1.5 Compiling and Installing MySQL 1.6 Troubleshooting

1.1 Introduction

MySQL Overview

MySQL is a relational database management system (RDBMS) developed by MySQL AB in Sweden. It is one of the most popular RDBMSs, especially for web applications.

Relational databases deliver high efficiency and flexibility because data is stored in different tables instead of in a large data warehouse. MySQL is optimal for small- and medium-sized websites thanks to its small size, fast speed, low cost, and especially the open-source code.

MySQL uses the Structured Query Language (SQL), the most common standard language for accessing databases.

MySQL uses dual-licensing distribution and is available in two editions: community edition and commercial edition.

For more information, visit the official MySQL website.

Programming language: C/C++

Brief description: RDBMS

Recommended Version MySQL 5.7.27

1.2 Environment Requirements

Hardware Requirements Table 1-1 lists the hardware requirements.

Table 1-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

Drive If a performance test needs to be performed, at least two drives are required: one system drive and one data drive. If no performance test needs to be performed, the data directory can be created on the system drive. Configure the number of drives based on actual requirements.

OS Requirements Table 1-2 lists the OS requirements.

Table 1-2 OS requirements Item Version

CentOS 7.6 for ARM

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Requirements The following software packages are required for configuring the MySQL compilation environment: ● OS image Obtain the dependencies from the OS image. ncurses ncurses-devel libaio-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel zlib-devel net-tools cmake bison openssl openssl- devel

● CMake and GCC source code packages Download the CMake and GCC source code and upgrade CMake and GCC. URL of the CMake 3.5.2 source code: https://cmake.org/files/v3.5/ cmake-3.5.2.tar.gz URL of the GCC 7.3.0 source code: https:// mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz

1.3 Compilation and Installation Process

Figure 1-1 shows the compilation and installation process of the MySQL source code.

Figure 1-1 MySQL compilation and installation process

1.4 Configuring the Compilation Environment

1.4.1 Configuring the Local Yum Source

Mounting the OS Image

Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

#mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart.

Method 2: Use the KVM to mount the OS image, and run the following command to mount the image to the /mnt/ directory.

#mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source

NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. #mv /etc/yum.repos.d /etc/yum.repos.d.bak

Step 2 Create the /etc/yum.repos.d directory for the new Yum source. #mkdir /etc/yum.repos.d

Step 3 Go to the directory created. #cd /etc/yum.repos.d

Step 4 Create a local.repo file. 1. Create a local.repo file. #vi local.repo 2. Add the following information to the local.repo file. [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 Example:

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file.

Step 5 Make the local source take effect. #yum clean all #yum makecache

----End

Installing OS Dependencies Step 1 Install dependencies. # yum -y install ncurses ncurses-devel libaio-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc- devel zlib-devel net-tools cmake openssl openssl-devel

----End 1.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2. Step 1 Download the CMake 3.5.2 package. URL: https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz Step 2 Upload the package to the /home directory on the server and decompress it. #cd /home #tar -zxvf cmake-3.5.2.tar.gz

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2

Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

In the command, -j 96 can make full use of the multi-core feature of the CPUs to accelerate the compilation. The number following -j indicates the number of CPU cores. You can run the cat /proc/cpuinfo | grep processor | wc -l command to query the number of CPU cores. The value specified must be less than or equal to the number of CPU cores.

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End 1.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 1 Download GCC 7.3.0.

URL: https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz

Step 2 Upload gcc-7.3.0.tar.gz to the /home directory and decompress it. #cd /home #tar -xvf gcc-7.3.0.tar.gz

Step 3 Install GCC.

Go to the GCC directory decompressed. #cd /home/gcc-7.3.0 1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

– --infodir=PATH: specifies the directory in which the GCC log information is stored. The default directory is /usr/share/info. – --enable-bootstrap: enables bootstrap installation. NO TE

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command. 2. Compile the GCC source code. #make -j 96 NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

1.5 Compiling and Installing MySQL Step 1 Obtain the MySQL source code. 1. Download the MySQL source code package (including Boost Headers). #cd /home #wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-boost-5.7.27.tar.gz --no-check- certificate You can also download the MySQL source code using a browser and upload it to the /home directory on the server. URL: https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql- boost-5.7.27.tar.gz

2. Decompress the package to generate the mysql-5.7.27 directory. #tar -zxvf mysql-boost-5.7.27.tar.gz

Step 2 Go to the /home/mysql-5.7.27 source code directory and create a compilation directory. #cd /home/mysql-5.7.27 #mkdir build

Step 3 Go to the compilation directory and configure MySQL. #cd build #cmake .. -DBUILD_CONFIG=mysql_release -DCMAKE_INSTALL_PREFIX=/usr/local/mysql - DMYSQL_DATADIR=/data/mysql/data -DWITH_BOOST=/home/mysql-5.7.27/boost/boost_1_59_0

Table 1-3 describes the key parameters. Set the directories to match your installation.

Table 1-3 Parameter description

Parameter Description

DBUILD_CONFIG Specifies the compile parameters to be used. The value mysql_release indicates that the compile parameters in the official MySQL release are used.

DCMAKE_INSTALL_ Specifies the software installation directory. In this PREFIX example, the installation directory is /usr/local/mysql. The installation directory in this document is for reference only. Replace it with the actual directory to match your installation.

DMYSQL_DATADIR Specifies the directory for storing data files during database creation. In this example, the data directory is / data/mysql/data.

DWITH_BOOST Specifies the directory of boost_1_59_0 after the MySQL source code package is decompressed. For example, if the package is decompressed in the /home directory, the directory is /home/mysql-5.7.27/boost/boost_1_59_0.

Step 4 Compile MySQL. # make -j 96

Step 5 Install MySQL. # make -j 96 install

NO TE

Step 6 Check the installation directory. #ls /usr/local/mysql/

Step 7 Check the database version.

#/usr/local/mysql/bin/mysql --version

----End

1.6 Troubleshooting

1.6.1 CMake Target Version Does Not Take Effect After the Upgrade

Symptom

The target version does not take effect after CMake is upgraded (1.4.2 Upgrading CMake).

Possible Causes

The original CMake version is not uninstalled.

Procedure

Step 1 Remove the original CMake. #yum remove cmake -y

Step 2 Create a soft link for CMake of the target version. #ln -s /usr/local/bin/cmake /usr/bin

Step 3 Check the CMake version. #cmake --version

----End 1.6.2 CMake Compilation and Installation Failed

Symptom

The error message shown in Figure 1-2 is displayed after the following commands are executed:

# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql - DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 - DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 - DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_MYISAM_STORAGE_ENGINE=1 - DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 -DDEFAULT_CHARSET=utf8mb4 - DDEFAULT_COLLATION=utf8mb4_general_ci -DWITH_EMBEDDED_SERVER=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/home/mysql-5.7.27/boost/boost_1_59_0

Figure 1-2 CMake error message

Procedure

Step 1 Go to the /home/mysql-5.7.27 directory and delete the CMakeCache.txt file. #cd /home/mysql-5.7.27 #rm CMakeCache.txt

Step 2 Run the following commands again: #cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql - DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 - DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 - DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_MYISAM_STORAGE_ENGINE=1 - DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 -DDEFAULT_CHARSET=utf8mb4 - DDEFAULT_COLLATION=utf8mb4_general_ci -DWITH_EMBEDDED_SERVER=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/home/mysql-5.7.27/boost/boost_1_59_0

----End 1.6.3 "error:'prctl' was not declared in this scope" Reported

Symptom

The following error is reported when the make command is executed on MySQL 5.7.27:

Procedure

Step 1 Copy the prctl.h file to the /home/mysql-5.7.27-1/include directory. #cp /usr/include/sys/prctl.h /home/mysql-5.7.27-1/include

Step 2 Add the following information to the mysqld.cc file. #vi /home/mysql-5.7.27-1/sql/mysqld.cc

Step 3 Run the installation command: #make -j 96

NO TE

----End

2 (Recommended) MySQL 8.0.x Porting Guide (CentOS 7.6)

2.1 Introduction 2.2 Environment Requirements 2.3 Compilation and Installation Process 2.4 Configuring the Compilation Environment 2.5 Compiling and Installing MySQL 2.6 Troubleshooting

2.1 Introduction

MySQL Overview

MySQL is a relational database management system (RDBMS) developed by MySQL AB in Sweden. It is one of the most popular RDBMSs, especially for web applications.

MySQL uses the Structured Query Language (SQL), the most common standard language for accessing databases.

MySQL uses dual-licensing distribution and is available in two editions: community edition and commercial edition.

For more information, visit the official MySQL website.

Programming language: C/C++

Brief description: RDBMS

Recommended Version MySQL 8.0.17 and later versions

NO TE

You need to upgrade MySQL 8.0.16 by installing the patch on the official website.

2.2 Environment Requirements

Hardware Requirements Table 2-1 lists the hardware requirements.

Table 2-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

Drive If a performance test needs to be performed, at lease two drives are required: one system drive and one data drive. If no performance test needs to be performed, the data directory can be created on the system drive. Configure the number of drives based on actual requirements.

OS Requirements Table 2-2 lists the OS requirements.

Table 2-2 OS requirements Item Version

CentOS 7.6 for ARM or later. This document uses 7.6 as an example.

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Packages The following software packages are required for configuring the MySQL compilation environment: ● OS image Obtain the dependencies from the OS image.

bison ncurses ncurses-devel libaio-devel openssl openssl-devel gmp gmp- devel mpfr mpfr-devel libmpc libmpc-devel ● CMake and GCC source code packages Download the CMake and GCC source code and upgrade CMake and GCC. URL of the CMake 3.5.2 source code: https://cmake.org/files/v3.5/ cmake-3.5.2.tar.gz URL of the GCC 7.3.0 source code: https:// mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz

2.3 Compilation and Installation Process

Figure 2-1 shows the compilation and installation process of the MySQL source code.

Figure 2-1 MySQL compilation and installation process

2.4 Configuring the Compilation Environment

2.4.1 Configuring the Local Yum Source

Mounting the OS Image

Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart. Method 2: Use the KVM to mount the OS image, and run the following command to mount the image to the /mnt/ directory.

mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. mv /etc/yum.repos.d /etc/yum.repos.d.bak Step 2 Create the /etc/yum.repos.d directory for the new Yum source. mkdir /etc/yum.repos.d Step 3 Go to the directory created. cd /etc/yum.repos.d Step 4 Create a local.repo file. 1. Create a local.repo file. vi local.repo 2. Add the following information to the local.repo file: [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 Example:

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file. Step 5 Make the local source take effect. yum clean all yum makecache

----End

Installing OS Dependencies

Step 1 Install dependencies. yum -y install bison ncurses ncurses-devel libaio-devel openssl openssl-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel wget tar GCC GCC-c++ git rpcgen CMake

----End 2.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2.

Step 1 Download the CMake 3.5.2 package. URL: https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz Step 2 Upload the package to the /home directory on the server and decompress it. #cd /home #tar -zxvf cmake-3.5.2.tar.gz

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2

Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End 2.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 1 Download GCC 7.3.0. URL: https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz Step 2 Upload gcc-7.3.0.tar.gz to the /home directory and decompress it. #cd /home #tar -xvf gcc-7.3.0.tar.gz

Step 3 Install GCC. Go to the GCC directory decompressed. #cd /home/gcc-7.3.0

1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

– --perfix=PATH: specifies the destination directory in which GCC is installed. The default directory is /usr. – --mandir=PATH: specifies the directory in which GCC software files are stored. The default directory is /usr/share/man. – --infodir=PATH: specifies the directory in which the GCC log information is stored. The default directory is /usr/share/info. – --enable-bootstrap: enables bootstrap installation. NO TE

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command. 2. Compile the GCC source code. #make -j 96

NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

2.5 Compiling and Installing MySQL

NO TE

If the installation fails, run the following command to clear the environment, and then repeat the operations in this section: rm -rf /home/mysql-8.0.17

Step 1 Obtain the source code. 1. Download the MySQL source code package (including Boost Headers). URL: https://downloads.mysql.com/archives/community/

2. Upload mysql-boost-8.0.17.tar.gz to the /home directory on the server and decompress it. cd /home tar -zxvf mysql-boost-8.0.17.tar.gz

Step 2 Go to the /home/mysql-8.0.17 source code directory and create the build directory. cd /home/mysql-8.0.17 mkdir build Step 3 Go to the build directory and configure MySQL. cd build cmake .. -DBUILD_CONFIG=mysql_release -DCMAKE_INSTALL_PREFIX=/usr/local/mysql - DMYSQL_DATADIR=/data/mysql/data -DWITH_BOOST=/home/mysql-8.0.17/boost/boost_1_69_0

Table 2-3 describes the key parameters. Set the directories to match your installation.

Table 2-3 Parameter description

Parameter Description

DBUILD_CONFIG Specifies the compile parameters to be used. The value mysql_release indicates that the compile parameters in the official MySQL release are used.

DCMAKE_INSTALL_ Specifies the software installation directory. In this PREFIX example, the installation directory is /usr/local/mysql. The installation directories in the documents are for reference only. Replace them with the actual directories to match your installation.

DMYSQL_DATADIR Specifies the directory for storing data files during database creation. In this example, the data directory is / data/mysql/data.

DWITH_BOOST Specifies the directory of boost_1_69_0 after the MySQL source code package is decompressed. For example, if the package is decompressed in the /home directory, the directory is /home/mysql-8.0.17/boost/boost_1_69_0.

Step 4 Compile MySQL. make -j 96

Step 5 Install MySQL. make -j 96 install

NO TE

Step 6 Check the installation directory. ls /usr/local/mysql/

Step 7 Check the database version. /usr/local/mysql/bin/mysql --version

----End

2.6 Troubleshooting

2.6.1 CMake Target Version Does Not Take Effect After the Upgrade

Symptom The target version does not take effect after CMake is upgraded (2.4.2 Upgrading CMake).

Possible Causes The original CMake version is not uninstalled.

Procedure

Step 1 Remove the original CMake. #yum remove cmake -y Step 2 Create a soft link for CMake of the target version. #ln -s /usr/local/bin/cmake /usr/bin Step 3 Check the CMake version. #cmake --version

----End 2.6.2 CMake Compilation and Installation Failed

Symptom The error message shown in Figure 2-2 is displayed after the following commands are executed:

cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql/data - DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 - DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 - DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 - DWITH_MYISAM_STORAGE_ENGINE=1 -DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 - DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_general_ci - DWITH_EMBEDDED_SERVER=1 -DCMAKE_C_COMPILER=/usr/bin/GCC -DDOWNLOAD_BOOST=1 - DWITH_BOOST=/home/mysql-8.0.17/boost/boost_1_69_0 -DFORCE_INSOURCE_BUILD=1

Figure 2-2 CMake compilation error information

Procedure

Step 1 Go to the /home/mysql-8.0.17 directory and delete the CMakeCache.txt file. cd /home/mysql-8.0.17 rm cmakeCache.txt Step 2 Run the following commands: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql/data - DSYSCONFDIR=/etc -DWITH_INNOBASE_STORAGE_ENGINE=1 - DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 - DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 - DWITH_MYISAM_STORAGE_ENGINE=1 -DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 - DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_general_ci - DWITH_EMBEDDED_SERVER=1 -DCMAKE_C_COMPILER=/usr/bin/GCC -DDOWNLOAD_BOOST=1 - DWITH_BOOST=/home/mysql-8.0.17/boost/boost_1_69_0 -DFORCE_INSOURCE_BUILD=1

----End

3 MySQL 8.0.x Porting Guide (openEuler 20.03)

3.1 Introduction 3.2 Environment Requirements 3.3 Compilation and Installation Process 3.4 Configuring the Compilation Environment 3.5 Configuring the Operating Environment 3.6 Compiling and Installing MySQL 3.7 Running MySQL 3.8 Uninstalling MySQL

3.1 Introduction

MySQL Overview MySQL is a relational database management system developed by MySQL AB in Sweden. Currently, it belongs to Oracle. It is one of the most popular RDBMSs in the industry, especially for web applications. Relational databases deliver high efficiency and flexibility because data is stored in different tables instead of in a large data warehouse. MySQL uses the Structured Query Language (SQL), the most common standard language for accessing databases. MySQL uses dual-licensing distribution and is available in two editions: Community Edition and Commercial Edition. MySQL is optimal for small or medium-sized websites because of its small size, fast speed, low cost, and especially the open source code. For more information, visit the official MySQL website. Programming language: C/C++ Brief description: RDBMS

Recommended Version MySQL 8.0.17 and later versions

NO TE

You need to upgrade MySQL 8.0.16 by installing the patch on the official website.

3.2 Environment Requirements Table 3-1 lists the hardware requirements.

Table 3-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

Table 3-2 lists the OS requirements.

Table 3-2 OS requirements Item Version Remarks

openEuler 20.03 To query the version, run cat /etc/ openEuler-release.

Kernel 4.19.90 To query the version, run uname -r.

NO TE

When installing an OS, select the Server installation mode and Network Servers, Performance Tools, Basic Web Server, Development Tools, and System Tools to minimize manual operations. Table 3-3 lists the software requirements.

Table 3-3 Software version requirements

Software Version Remarks

CMake 3.12.1 To query the version, run cmake --version. (preinstalled with the OS)

GCC 7.3.0 To query the version, run gcc -v. (preinstalled with the OS)

MySQL 8.0.x To query the version, run /usr/local/mysql/bin/ mysql --version.

3.3 Compilation and Installation Process

Figure 3-1 shows the process for installing MySQL from the source code.

Figure 3-1 MySQL compilation and installation process

3.4 Configuring the Compilation Environment

3.4.1 Installing Dependencies

Mounting the OS Image

Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

mount /home/openEuler-20.03-LTS-everything-aarch64-dvd.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart of the OS.

Method 2: Use the KVM to mount the OS image, and then run the following command to mount the image to the /mnt/ directory. If the system displays a message indicating that no image is available after you run the command, reconnect to the image and then run the following command again:

mount /dev/sr0 /mnt/

Configuring the Local Yum Source

Step 1 Back up the original Yum source configuration file of the system. mv /etc/yum.repos.d /etc/yum.repos.d.bak

Step 2 Create the /etc/yum.repos.d directory for the new Yum source. mkdir /etc/yum.repos.d

Step 3 Go to the directory created. cd /etc/yum.repos.d

Step 4 Create the local.repo file. 1. Create the local.repo file. vi local.repo 2. Add the following information to the local.repo file. [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 3. Save and exit the local.repo file.

Step 5 Make the local source take effect. yum clean all yum makecache

----End

Installing Dependencies

Step 1 Install dependencies. yum -y install bison ncurses ncurses-devel libaio-devel openssl openssl-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel wget tar gcc gcc-c++ git rpcgen CMake

NO TE

If the OS image mounted (in Mounting the OS Image) is not openEuler-20.03-LTS- everything-aarch64-dvd.iso, some RPM packages may missing after you install dependencies using the yum command. If this occurs, download the missing RPM packages from the mirror site and run the rpm -ivh *.rpm command to install them. Download the RPM packages from the mirror site: https://mirrors.huaweicloud.com/openeuler/openEuler-20.03-LTS/everything/aarch64/ Packages/

----End 3.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2.

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2 Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End

3.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 3 Install GCC. Go to the GCC directory decompressed. #cd /home/gcc-7.3.0 1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

NO TE

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command.

2. Compile the GCC source code. #make -j 96

NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

3.5 Configuring the Operating Environment

3.5.1 Disabling the Firewall

NO TE

It is recommended that the firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements.

Step 1 Stop the firewall. systemctl stop firewalld.service

Step 2 Disable the firewall. systemctl disable firewalld.service

NO TE

Once disabled, the firewall will not start upon OS reboot.

Step 3 Check the firewall status. systemctl status firewalld.service

----End 3.5.2 Disabling SELinux

Step 1 Set SELinux to permissive mode to temporarily disable SELinux. setenforce 0

Step 2 Check that SELinux is in permissive mode. getenforce

NO TE

To temporarily enable SELinux, run the following command to set SELinux to enforcing mode: setenforce 1 To permanently disable SELinux, perform the following steps: 1. Run the vim /etc/sysconfig/selinux command to open the SELinux file and change SELINUX=enforcing to SELINUX=disabled. vim /etc/sysconfig/selinux

2. Save the file and restart the server. 3. Check the SELinux status. If SELinux status is disabled, SELinux is disabled. /usr/sbin/sestatus -v

----End 3.5.3 Creating a User Group and a User

NO TE

For security purposes, create a dedicated user and user group for the database process on the server OS. The group/user created here is an OS group/user, not a database group/user. The user will be added in the my.cnf file in Step 4.

Step 1 Create a user group named mysql. groupadd mysql

Step 2 Create a user named mysql. useradd -g mysql mysql

Step 3 Set the password for the mysql user. passwd mysql

Enter the password twice for confirmation.

----End 3.5.4 Creating a Data Drive If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1: Using fdisk to Create a Data Drive or Method 2: Using LVM to Create a Data Drive. If no performance test is required, go to 3.5.5 Creating a Data Directory.

Method 1: Using fdisk to Create a Data Drive

Step 1 Create a partition, for example, /dev/nvme0n1. fdisk /dev/nvme0n1 Step 2 Type p (print the partition table) and press Enter to display the current partition. Step 3 Type n (add a new partition) and press Enter. Step 4 Type p and press Enter to create a primary partition. Step 5 Type 1 and press Enter to set the partition number to 1. Step 6 Press Enter to retain the default setting for the first sector. Step 7 Press Enter to retain the default setting for the last sector.

Step 8 Type p and press Enter, and check that the partition information is correct. Step 9 Type w and press Enter to save the settings. Step 10 Create a file system, for example, xfs. ls /dev/nvme* mkfs.xfs -f /dev/nvme0n1p1 Step 11 Mount the partition to /data of the OS. mkdir /data mount /dev/nvme0n1p1 /data Step 12 Enable the data drive to be automatically mounted after the OS is restarted. Open the /etc/fstab file, and add the last line in the following figure to the file. In the last line, /dev/sdb1 is only an example.

----End

Method 2: Using LVM to Create a Data Drive

Install the LVM2 package (contained in the image) and create a data drive as follows:

Step 1 Install LVM2. yum -y install lvm2

Step 2 Create a physical volume, for example, sdb. pvcreate /dev/sdb

Step 3 Create a physical volume group, for example, datavg. vgcreate datavg /dev/sdb

Step 4 Create a logical volume, for example, datalv of 600 GB. lvcreate -L 600G -n datalv datavg

Step 5 Create a file system. mkfs.xfs /dev/datavg/datalv

Step 6 Create a data directory and mount it. mkdir /data mount /dev/datavg/datalv /data

Step 7 Enable the data drive to be automatically mounted after the OS is restarted. Run the vi /etc/fstab command to open the /etc/fstab file and add the last line in the following figure to the file.

In the last line, /dev/datavg/datalv is only an example.

----End

3.5.5 Creating a Data Directory

Step 1 Create the directory /data and related directories for processes. mkdir /data mkdir -p /data/mysql cd /data/mysql mkdir data tmp run log Step 2 Change the user group and user permission on the /data directory to mysql:mysql. chown -R mysql:mysql /data ll /

----End

3.6 Compiling and Installing MySQL

NO TE

If the installation fails, run the following command to clear the environment, and then repeat the operations in this section: rm -rf /home/mysql-8.0.17

Step 1 Obtain the MySQL source code. 1. Download the MySQL source code package (including Boost Headers). URL: https://downloads.mysql.com/archives/community/

2. Upload mysql-boost-8.0.17.tar.gz to the /home directory on the server and decompress it. cd /home tar -zxvf mysql-boost-8.0.17.tar.gz

Table 3-4 describes the key parameters. Set the directories to match your installation.

Table 3-4 Parameter description Parameter Description

DBUILD_CONFIG Specifies the build parameters to be used. The value mysql_release indicates that the build parameters in the official MySQL release are used.

DCMAKE_INSTALL_ Specifies the software installation directory. In this PREFIX example, the installation directory is /usr/local/mysql.

DMYSQL_DATADIR Specifies the directory for storing data files during database creation. In this example, the data directory is / data/mysql/data.

Parameter Description

Step 4 Compile MySQL. make -j 96 Step 5 Install MySQL. make -j 96 install

NO TE

Step 7 Change the user group and user permission on the installation directory /usr/ local/mysql/ to mysql:mysql. chown -R mysql:mysql /usr/local/mysql/ ll /usr/local/mysql/

Step 8 Check the database version. /usr/local/mysql/bin/mysql --version

----End

3.7 Running MySQL

Step 1 Modify the configuration file. 1. Delete the old files and create a my.cnf file. rm -f /etc/my.cnf echo -e "[mysqld_safe]\nlog-error=/data/mysql/log/mysql.log\npid-file=/data/mysql/run/ mysqld.pid\n[mysqldump]\nquick\n[mysql]\nno-auto-rehash\n[client]\ndefault-character- set=utf8\n[mysqld]\nbasedir=/usr/local/mysql\nsocket=/data/mysql/run/mysql.sock\ntmpdir=/ data/mysql/tmp\ndatadir=/data/mysql/data \ndefault_authentication_plugin=mysql_native_password\nport=3306\nuser=mysql" > /etc/ my.cnf

CA UTION

In the my.cnf file, the file paths (including the software installation path basedir and data path datadir) are examples. user=mysql specifies the OS user. It is the user created in 3.5.3 Creating a User Group and a User.

2. Check that the my.cnf file is correctly modified. cat /etc/my.cnf

3. Change the user group and user permission on the /etc/my.cnf configuration file to mysql:mysql. chown mysql:mysql /etc/my.cnf ll /etc/my.cnf

Step 2 Add the MySQL service to the service list and grant permissions. chmod 777 /usr/local/mysql/support-files/mysql.server cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql chkconfig mysql on Change the user group and user permission on /etc/init.d/mysql to mysql:mysql.

chown -R mysql:mysql /etc/init.d/mysql ll /etc/init.d/mysql

Step 3 Set environment variables. 1. Change the user group and user permission on the environment variable file /etc/profile to mysql:mysql. chown mysql:mysql /etc/profile ll /etc/profile

2. Switch to the mysql user. su - mysql whoami 3. Add the path of the MySQL binary file to the environment variable. export PATH=$PATH:/usr/local/mysql/bin >> /etc/profile

CA UTION

In the command, /usr/local/mysql/bin in the PATH parameter is the absolute path of the bin file in the MySQL installation directory. Change the path as required to match your installation.

4. Make the environment variable settings take effect. source /etc/profile 5. Check whether the environment variables take effect. env

Step 4 Initialize the database. mysqld --defaults-file=/etc/my.cnf --initialize Or

/usr/local/mysql/bin/mysqld --defaults-file=/etc/my.cnf --initialize

NO TE

● In the command output, the second line from the bottom contains the initial password, which will be used for accessing the database. ● If the initialization fails, and the message "--initialize specified but the data directory has files in it." is displayed, delete the data and initialize the database again. ls /data/mysql/data rm -rf /data/mysql/data/* After the initialization is complete, check whether the user group and user permission on the data file /data/mysql/data in the data directory is mysql:mysql (because the OS user configured in the /etc/my.cnf file is user=mysql).

ll /data/mysql/data

Step 5 Start the database.

CA UTION

If you run the service mysql start command as the root user (su - root) to start the database service for the first time, the system displays a message indicating that the database fails to start because the mysql.log file is missing, as shown in the following figure.

Perform the following operations to rectify the fault: 1. Run the su - mysql command to switch to the mysql user and start the database service. The mysql.log file is generated in the /data/mysql/log directory. 2. Run the service mysql stop command to stop the database service. If you start the database service as the root user, no error is reported.

service mysql start

mysqld --defaults-file=/etc/my.cnf &

/usr/local/mysql/bin/mysqld --defaults-file=/etc/my.cnf &

Check the database process.

ps -ef | grep mysql

Check the database listening port.

netstat -anpt netstat -anpt | grep mysql netstat -anpt | grep 3306

Step 6 Log in to the database.

NO TE

Enter the initial password generated in Step 4. mysql -uroot -p -S /data/mysql/run/mysql.sock

/usr/local/mysql/bin/mysql -uroot -p -S /data/mysql/run/mysql.sock

Step 7 Configure the database accounts. 1. After logging in to the database, change the password of the root user for logging in to the database. alter user 'root'@'localhost' identified by "123456"; 2. Create a root user for all the other hosts in the domain. create user 'root'@'%' identified by '123456'; 3. Grant the root users the same access permissions as the local root user. grant all privileges on *.* to 'root'@'%'; flush privileges;

Step 8 Exit the database. Run the \q or exit command to exit the database.

exit

Step 9 Use the new password to log in to the database. mysql -uroot -p -S /data/mysql/run/mysql.sock

/usr/local/mysql/bin/mysql -uroot -p -S /data/mysql/run/mysql.sock

Exit the database.

exit Step 10 Stop the database. service mysql stop Or

mysqladmin -uroot -p123456 shutdown -S /data/mysql/run/mysql.sock Or

/usr/local/mysql/bin/mysqladmin -uroot -p123456 shutdown -S /data/mysql/run/mysql.sock Check the database process.

ps -ef | grep mysql

----End

3.8 Uninstalling MySQL

Step 1 Stop the database process. ps -ef | grep mysql kill -9 Process ID Step 2 Delete the corresponding directories. ls /usr/local/mysql ls /data/mysql/ rm -rf /usr/local/mysql rm -rf /data/mysql

----End

4 MariaDB 10.3.8 Porting Guide (CentOS 7.6)

4.1 Introduction 4.2 Environment Requirements 4.3 Configuring the Compilation Environment 4.4 Compiling and Installing MariaDB

4.1 Introduction

MariaDB Overview MariaDB is a community-developed, commercially supported fork of MySQL, intended to remain free and open-source software under the GNU General Public License (GPL). MariaDB maintains high compatibility with MySQL, ensuring a drop-in replacement capability with library binary parity and exact matching with MySQL APIs and commands. MariaDB also provides many better new features. Figure 4-1 shows the MariaDB architecture.

Figure 4-1 MariaDB architecture

When MariaDB receives an SQL statement, the detailed implementation process is as follows:

1. When a client connects to MariaDB, an authentication is performed based on the client's hostname, user name, and password. Authentication can optionally be delegated to a plugin. 2. If the login is successful, the client sends an SQL query to the server. The parser analyzes the SQL statements. 3. The server checks whether the client has the permissions required for the requested resources. 4. If the requested data is stored in the query cache, the result is immediately returned to the client. 5. The optimizer locates the fastest execution strategy or query plan. That is, the optimizer decides the tables to be read, indexes to be accessed, and temporary internal tables to be used. A good strategy can minimize access to drives and queuing operations. 6. The storage engines read and write data and index files, and the caches speed up the read and write operations. The transactions and foreign keys are implemented at the storage engine layer.

The storage engine at the physical layer manages and controls data, including data files, data, indexes, and caches, to make data management and read more efficient. Each table has a .frm file that contains table definitions.

The storage engines manage and store data in different ways, and support different features and performance. For example:

● MyISAM is suitable for read-intensive operations. It does not support transactions and supports full-text indexes. ● InnoDB supports transactions, row locks, and foreign keys.

● MEMORY stores data in memory. ● CSV stores data in CSV format. Programming language: C++ Brief description: open-source fork of MySQL

Recommended Version MariaDB 10.3.8

NO TE

This document applies to MariaDB 10.3.8. It also provides reference for compilation of other MariaDB versions.

4.2 Environment Requirements

Hardware Requirements Table 4-1 lists the hardware requirements.

Table 4-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

OS Requirements Table 4-2 lists the OS requirements.

Table 4-2 OS requirements Item Version

CentOS 7.6 for Arm or later. This document uses 7.6 as an example.

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Requirements The MariaDB source code package is required for MariaDB compilation. Download it in advance. URL: https://downloads.mariadb.org/mariadb/10.3.8/

4.3 Configuring the Compilation Environment Configure and install the dependency libraries required for compilation of MariaDB. The following MariaDB compilation dependency libraries need to be installed: Local Yum source installation: readline-devel zlib-devel openssl* cmake git gcc* perl* bison can be directly obtained from the ISO image.

Mounting the OS Image Mount the OS image using either of the following methods: Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source

NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

vi local.repo 2. Add the following information to the local.repo file: [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 Example:

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file. Step 5 Make the local source take effect. yum clean all yum makecache

----End

Installing OS Dependencies Step 1 Install dependencies. yum -y install readline-devel zlib-devel openssl* cmake git gcc* perl* bison

----End

4.4 Compiling and Installing MariaDB Step 1 Download the MariaDB source code package (for example, MariaDB 10.3.8) and upload it to a directory (for example, /root) on the server.

URL: https://downloads.mariadb.org/mariadb/10.3.8/

Step 2 Go to the /root directory and decompress the MariaDB source code package. cd /root tar zxvf mariadb-10.3.8.tar.gz Step 3 Go to the MariaDB source code directory decompressed. cd mariadb-10.3.8 Step 4 Perform compilation. cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb -DMYSQL_DATADIR=/data/mariadb/data - DSYSCONFDIR=/etc -DWITHOUT_TOKUDB=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 - DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 - DWITH_READLINE=1 -DWITH_SSL=system -DWITH_ZLIB=system -DWITH_LOBWRAP=0 - DMYSQL_UNIX_ADDR=/data/mariadb/run/mysql.sock -DDEFAULT_CHARSET=utf8 - DDEFAULT_COLLATION=utf8_general_ci

Table 4-3 describes the key directories involved. Modify them to match your installation.

Table 4-3 Key directories involved in compilation and installation Directory Description

- Specifies the software installation directory. DCMAKE_INSTALL_PR EFIX

-DMYSQL_DATADIR Specifies the data file directory.

-DSYSCONFDIR Specifies the configuration file directory.

-DMYSQL_UNIX_ADDR Specifies the sock file directory.

Step 5 Perform installation. make -j 32 && make install

NO TE

In the command, -j 32 can make full use of the multi-core feature of the CPUs to accelerate the compilation. The number following -j indicates the number of CPU cores. You can run the cat /proc/cpuinfo | grep processor | wc -l command to query the number of CPU cores. The value specified must be less than or equal to the number of CPU cores.

----End

5 MongoDB 4.0.12 Porting Guide (CentOS 7.6)

5.1 Introduction 5.2 Environment Requirements 5.3 Compilation and Installation Process 5.4 Configuring the Compilation Environment 5.5 Compiling and Installing MongoDB 5.6 Compiling and Installing MongoDB Tools 5.7 Troubleshooting

5.1 Introduction

MongoDB Overview MongoDB is a database based on distributed file storage. It is written by C++ language and designed to provide scalable high-performance data storage solutions for WEB applications. MongoDB is a product between the relational database and the non-relational database, is the most abundant in the non- relational database, most like the relational database. Programming language: C++ Brief description: distributed file storage database

NO TE

This document applies only to MongoDB 4.0.12.

5.2 Environment Requirements

Hardware Requirements Table 5-1 lists the hardware requirements.

Table 5-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

Drive If a performance test needs to be performed, at least two drives are required: one system drive and one data drive. Use an SSD or NVMe SSD as the data drive for higher performance. If no performance test needs to be performed, the data directory can be created on the system drive. Configure the number of drives based on actual requirements.

OS Requirements Table 5-2 lists the OS requirements.

Table 5-2 OS requirements Item Version Remarks

CentOS 7.6 for Arm cat /etc/redhat-release

NO TE

1. When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations. 2. During MongoDB 4.0.12 installation, the size of the /root partition is set to 50 GB or greater. You can run the df -h /root or lsblk command to check the size of the /root partition.

Software Version Requirements Table 5-3 lists the software requirements.

Table 5-3 Software Version Requirements Software Version

Dependency Using the Local Yum Source

CMake https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz

GCC https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/ gcc-7.3.0.tar.gz

Cython https://github.com/cython/cython.git

PyYAML https://github.com/yaml/pyyaml.git

typing https://github.com/python/typing.git

Cheetah3 https://github.com/CheetahTemplate3/cheetah3.git

MongoDB https://github.com/mongodb/mongo/archive/ r4.0.12.tar.gz

5.3 Compilation and Installation Process Figure 5-1 shows the compilation and installation process of the MongoDB source code.

Figure 5-1 MongoDB compilation and installation process

5.4 Configuring the Compilation Environment

5.4.1 Configuring the Local Yum Source

Mounting the OS Image

Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart.

Method 2: Use the KVM to mount the OS image, and run the following command to mount the image to the /mnt/ directory.

mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source

NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. mv /etc/yum.repos.d /etc/yum.repos.d.bak

Step 2 Create the /etc/yum.repos.d directory for the new Yum source. mkdir /etc/yum.repos.d

Step 3 Go to the directory created. cd /etc/yum.repos.d

Step 4 Create a local.repo file. 1. Create a local.repo file. vi local.repo 2. Add the following information to the local.repo file: [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 Example:

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file.

Step 5 Make the local source take effect. yum clean all yum makecache

----End

Installing OS Dependencies

Step 1 Install dependencies. #yum -y install libyaml libyaml-devel python-setuptools libcurl-devel python-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel libpcap-dev*

NO TE

1. libpcap-dev* is the dependency required for compiling and installing MongoDB tools. 2. Check the libpcap-dev version. rpm -qa | grep libpcap-dev

----End 5.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2.

Step 1 Download the CMake 3.5.2 package.

URL: https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz

Step 2 Upload the package to the /home directory on the server and decompress it. #cd /home #tar -zxvf cmake-3.5.2.tar.gz

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2

Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End 5.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 3 Install GCC. Go to the GCC directory decompressed. #cd /home/gcc-7.3.0

1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command. 2. Compile the GCC source code. #make -j 96

NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

5.4.4 Installing Cython

Method 1: Downloading and Installing Cython Online

Step 1 Go to the home directory. #cd /home/ Step 2 Clone Cython in the GitHub code repository. #git config --global http.sslVerify false #git clone https://github.com/cython/cython.git Step 3 Install Cython. #cd /home/cython #python setup.py install

----End

Method 2: Downloading and Installing Cython Offline

Step 1 Click https://github.com/cython/cython.git to visit the web page, as shown in the following figure. Click Download ZIP to download the installation package and upload it to the /home directory on the server.

Step 2 Go to the /home directory and decompress the package. #cd /home #unzip cython-master.zip

Step 3 Go to the decompressed file directory and install Cython. #cd /home/cython-master #python setup.py install

----End 5.4.5 Installing PyYAML

Method 1: Downloading and Installing PyYAML Online

Step 1 Go to the home directory. #cd /home/ Step 2 Clone PyYAML in the GitHub code repository. #git clone -b 3.11 https://github.com/yaml/pyyaml.git Step 3 Install PyYAML. #cd /home/pyyaml #python setup.py install

----End

Method 2: Downloading and Installing PyYAML Offline

Step 1 Click https://github.com/yaml/pyyaml.git to visit the web page, as shown in the following figure. Click Download ZIP to download the installation package and upload it to the /home directory on the server.

Step 2 Go to the /home directory and decompress the package. #cd /home #unzip pyyaml-master.zip

Step 3 Go to the decompressed file directory and install PyYAML. #cd pyyaml-master #python setup.py install

----End

5.4.6 Installing Typing

Method 1: Downloading and Installing Typing Online

Step 1 Go to the home directory. #cd /home/ Step 2 Clone Typing in the GitHub code repository. #git clone -b 3.6.1 https://github.com/python/typing.git Step 3 Install Typing. #cd typing #python setup.py install

----End

Method 2: Downloading and Installing Typing Offline

Step 1 Click https://github.com/python/typing.git to visit the web page, as shown in the following figure. Click Download ZIP to download the installation package and upload it to the /home directory on the server.

Step 2 Go to the /home directory and decompress the package. #cd /home #unzip typing-master.zip

Step 3 Go to the decompressed file directory and install Typing. #cd typing-master #python setup.py install

----End 5.4.7 Installing Cheetah3

Method 1: Downloading and Installing Cheetah3 Online Step 1 Go to the home directory. #cd /home/ Step 2 Clone Cheetah3 in the GitHub code repository. #git clone -b 3.0.0 https://github.com/CheetahTemplate3/cheetah3.git Step 3 Install Cheetah3. #cd cheetah3 #python setup.py install ----End

Method 2: Downloading and Installing Cheetah3 Offline Step 1 Click https://github.com/CheetahTemplate3/cheetah3.git to visit the web page, as shown in the following figure. Click Download ZIP to download the installation package and upload it to the /home directory on the server.

Step 2 Go to the /home directory and decompress the package. #cd /home #unzip cheetah3-master.zip

Step 3 Go to the decompressed file directory and install Cheetah3. #cd cheetah3-master #python setup.py install

----End

5.5 Compiling and Installing MongoDB Step 1 Download the MongoDB source code. #cd /home #wget https://github.com/mongodb/mongo/archive/r4.0.12.tar.gz --no-check-certificate You can also download the source code using a browser and upload it to the / home directory on the server. URL: https://github.com/mongodb/mongo/archive/r4.0.12.tar.gz Step 2 Decompress the package. #tar zxvf mongo-r4.0.12.tar.gz

NO TE

Run the wget command to download the package and run the rename command. mv r4.0.12.tar.gz mongo-r4.0.12.tar.gz Step 3 Go to the /home/mongo-r4.0.12 source code directory. #cd /home/mongo-r4.0.12 Step 4 Perform compilation. (The compilation takes about 10 to 15 minutes.) #python2 buildscripts/scons.py MONGO_VERSION=4.0.12 all CFLAGS="-march=armv8-a+crc - mtune=generic" -j 64 --disable-warnings-as-errors

NO TE

In the command, -j 64 can make full use of the multi-core feature of the CPUs to accelerate the compilation. The number following -j indicates the number of CPU cores. You can run the cat /proc/ cpuinfo | grep processor | wc -l command to query the number of CPU cores. The value specified must be less than or equal to the number of CPU cores.

Step 5 Perform installation. #mkdir /usr/local/mongo #python2 buildscripts/scons.py MONGO_VERSION=4.0.12 --prefix=/usr/local/mongo --disable- warnings-as-errors CFLAGS="-march=armv8-a+crc" install -j 64

After the installation is complete, the bin directory is automatically generated in /usr/local/mongo. Check the content in the bin directory.

#ll /usr/local/mongo/bin

Step 6 Delete the debugging information. #cd /usr/local/mongo/bin #strip mongos #strip mongod #strip mongo

----End

5.6 Compiling and Installing MongoDB Tools

Step 1 Download the source code package of the Go compiler. #cd /home #wget https://golang.org/dl/go1.13.5.linux-arm64.tar.gz --no-check-certificate

You can also download the source code using a browser and upload it to the / home directory on the server. URL: https://golang.org/dl/ Step 2 Decompress the Go source code package to the /usr/local directory.

#tar -C /usr/local -xzf go1.13.5.linux-arm64.tar.gz #ls /usr/local/go Step 3 Set environment variables of the Go compiler. #export PATH=$PATH:/usr/local/go/bin Run the go env command to obtain the GOROOT configuration.

#export GOROOT="/usr/local/go" Check whether the environment variables of the Go compiler are set successfully.

#env | grep GOROOT #env | grep PATH

Step 4 Check the Go compiler version. #whereis go #/usr/local/go/bin/go version

Step 5 Download the MongoDB tools source code. #cd /root #wget https://github.com/mongodb/mongo-tools/archive/r4.0.6.tar.gz --no-check-certificate You can also download the source code using a browser and upload it to the /root directory on the server. URL: https://github.com/mongodb/mongo-tools/archive/r4.0.6.tar.gz Step 6 Decompress the MongoDB tools source code package. #tar zxvf r4.0.6.tar.gz #ls mongo-tools-r4.0.6 Step 7 Set environment variables of MongoDB tools. #cd /root/mongo-tools-r4.0.6 #chmod +x set_goenv.sh #source set_goenv.sh Step 8 Change the compilation directory of MongoDB tools. The mongo-tools-r4.0.6 directory is generated after the decompression. You need to perform the following operations to change the directory structure. Otherwise, an error will be reported during compilation. Run the following command in the same directory as mongo-tools-r4.0.6 (/root in this document):

#cd /root #mkdir -p mongodb-tools-4.0.6/src/github.com/mongodb #mv mongo-tools-r4.0.6 mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools Step 9 Download and decompress the gopacket package to replace the gopacket package of MongoDB tools.

1. Download the gopacket package.

#cd /root #wget https://github.com/google/gopacket/archive/master.zip --no-check-certificate You can also download the source code using a browser and upload it to the /root directory on the server. URL: https://github.com/google/gopacket/archive/master.zip 2. Decompress the gopacket package.

#unzip master.zip #ll gopacket-master 3. Replace the gopacket package of MongoDB tools.

#ll /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/vendor/github.com/google/ gopacket #rm -rf /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/vendor/github.com/google/ gopacket #mv gopacket-master /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/vendor/ github.com/google/gopacket #ll /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/vendor/github.com/google/ gopacket Step 10 Create a GCC soft link. 1. Check the GCC path and version.

#whereis gcc #/usr/bin/gcc --version

2. Create a GCC soft link.

#mkdir -p /opt/mongodbtoolchain/v2/bin #ln -s /usr/bin/gcc /opt/mongodbtoolchain/v2/bin/aarch64-mongodb-linux-gcc #ll /opt/mongodbtoolchain/v2/bin/aarch64-mongodb-linux-gcc

NO TE

1. Establish a soft link. ln -s Source file Soft link ln -s /usr/bin/gcc /opt/mongodbtoolchain/v2/bin/aarch64-mongodb-linux-gcc 2. View the soft link. ll soft link Soft link -> Source file ll /opt/mongodbtoolchain/v2/bin/aarch64-mongodb-linux-gcc 3. Delete the soft link. rm -rf Soft link rm -rf /opt/mongodbtoolchain/v2/bin/aarch64-mongodb-linux-gcc

Step 11 Compile and install MongoDB tools. #cd /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools #./build.sh

View the generated binary executable file.

#ls /root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/bin

Step 12 Check whether the MongoDB tools commands are normal. (The following uses mongorestore and mongoreplay as examples.) #/root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/bin/mongorestore --version #/root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/bin/mongoreplay --version #/root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/bin/mongorestore --help #/root/mongodb-tools-4.0.6/src/github.com/mongodb/mongo-tools/bin/mongoreplay --help

----End

5.7 Troubleshooting

5.7.1 CMake Target Version Does Not Take Effect After the Upgrade

Symptom The target version does not take effect after CMake is upgraded (5.4.2 Upgrading CMake).

Possible Causes The original CMake version is not uninstalled.

Procedure

Step 1 Remove the original CMake. #yum remove cmake -y Step 2 Create a soft link for CMake of the target version. #ln -s /usr/local/bin/cmake /usr/bin Step 3 Check the CMake version. #cmake --version

----End

5.7.2 No space left on device

Symptom

The error message "src/third_party/mozjs-45/extract/js/src/jit/MCallOptimize.cpp: 3611:1: fatal error: error writing to /tmp/ccGS0uQN.s: No space left on device" is displayed during compilation.

Procedure

A large number of files are generated during compilation. Therefore, you need to place the compilation files in a directory with sufficient space. For example, place the files in the home directory and delete the packages in the root directory. 5.7.3 "processor does not support `crc32cb w2,w2,w3'" Reported

Symptom

The error message "/tmp/ccNLTPMq.s:35: Error: selected processor does not support `crc32cb w2,w2,w3'" is displayed during compilation.

Procedure

Step 1 Add CFLAGS="-march=armv8-a+crc -mtune=generic" during compilation. #python2 buildscripts/scons.py MONGO_VERSION=4.0.12 all CFLAGS="-march=armv8-a+crc - mtune=generic" -j 64 --disable-warnings-as-errors

----End 5.7.4 "No module named pkg_resources" Reported

Symptom

Procedure

Step 1 Download and install setuptools. #cd /home #wget --no-check-certificate https://bootstrap.pypa.io/ez_setup.py #sudo python ez_setup.py -insecure Step 2 Download the pip installation package and add a soft link. #cd /home #wget https://pypi.python.org/packages/11/b6/ abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz #tar -xf pip-9.0.1.tar.gz #cd pip-9.0.1 #sudo python setup.py install #ln -s /usr/local/python27/bin/pip /usr/bin/pip

----End

6 Percona 8.0.15 Porting Guide (CentOS 7.6)

6.1 Introduction 6.2 Environment Requirements 6.3 Compilation and Installation Process 6.4 Configuring the Compilation Environment 6.5 Compiling and Installing Percona 6.6 Troubleshooting

6.1 Introduction

Percona Server Overview Percona Server is a distribution of the MySQL relational database management system (RDBMS) created by Percona. It is an independent database product fully compatible with MySQL and can replace the storage engine with XtraDB without modifying any code. According to the statement from Percona, Percona Server is the closest version of the official MySQL Enterprise distribution. However, to ensure its control on the product functionality, Percona manages the code and does not accept contributions from external developers. Percona provides a high-performance XtraDB engine, a high-availability PXC solution, and a DBA management toolkit such as Percona Toolkit. For more information, visit the official Percona website. Programming language: C/C++ Brief description: RDBMS

Recommended Version Percona 8.0.15

6.2 Environment Requirements

Hardware Requirements Table 6-1 lists the hardware requirements.

Table 6-1 Hardware requirements Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

OS Requirements Table 6-2 lists the OS requirements.

Table 6-2 OS requirements Item Version

CentOS 7.6 for Arm or later. This document uses 7.6 as an example.

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Requirements Before the compilation, obtain the OS image, and CMake, GCC, and Percona source code packages. URL of the CMake 3.5.2 source code: https://cmake.org/files/v3.5/ cmake-3.5.2.tar.gz URL of the GCC 7.3.0 source code: https:// mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz URL of the Percona source code: https://www.percona.com/downloads/Percona- Server-LATEST/

The following two types of Percona dependencies need to be installed:

1. Local Yum Source ncurses ncurses-devel libaio-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel libcurl-devel readline-devel zlib-devel openssl* cmake git gcc* perl* bison net-tools You can obtain these dependencies from the ISO image. 2. Source Code Dependencies To upgrade CMake and GCC to a later version, download and compile the CMake and GCC source code and then install the target versions.

6.3 Compilation and Installation Process

Figure 6-1 shows the compilation and installation process of the Percona source code.

Figure 6-1 Percona compilation and installation process

6.4 Configuring the Compilation Environment

6.4.1 Configuring the Local Yum Source

Mounting the OS Image

Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

#mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

#mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. #mv /etc/yum.repos.d /etc/yum.repos.d.bak Step 2 Create the /etc/yum.repos.d directory for the new Yum source. #mkdir /etc/yum.repos.d Step 3 Go to the directory created. #cd /etc/yum.repos.d Step 4 Create a local.repo file. 1. Create a local.repo file. #vi local.repo 2. Add the following information to the local.repo file. [local] name=local.repo baseurl=file:///mnt enabled=1 gpgcheck=0 Example:

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file. Step 5 Make the local source take effect. #yum clean all #yum makecache

----End

Installing OS Dependencies

Step 1 Install dependencies. #yum -y install ncurses ncurses-devel libaio-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc- devel libcurl-devel readline-devel zlib-devel openssl* cmake git gcc* perl* bison net-tools

----End 6.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2.

Step 1 Download the CMake 3.5.2 package.

URL: https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz

Step 2 Upload the package to the /home directory on the server and decompress it. #cd /home #tar -zxvf cmake-3.5.2.tar.gz

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2

Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End

6.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 3 Install GCC. Go to the GCC directory decompressed. #cd /home/gcc-7.3.0 1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

NO TE

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command.

2. Compile the GCC source code. #make -j 96

NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

6.5 Compiling and Installing Percona

Step 1 Download the Percona source code package percona-server-8.0.15-6.tar.gz (Percona 8.0.15 is used as an example) and upload it to the /home directory on the server. URL: https://www.percona.com/downloads/Percona-Server-LATEST/

Step 2 Download the Boost library package and upload it to the /home directory on the server. URL: https://nchc.dl.sourceforge.net/project/boost/boost/1.68.0/ boost_1_68_0.tar.gz Step 3 Go to the /home directory and decompress the Boost library package and Percona source code package. #cd /home #tar xvf boost_1_68_0.tar.gz #tar zxvf percona-server-8.0.15-6.tar.gz Step 4 Modify the source code to fix the deadlock bug existing in the source code downloaded. 1. Go to the Percona source code directory decompressed. #cd /home/percona-server-8.0.15-6 2. Modify the sync0rw.cc file. #vi storage/innobase/sync/sync0rw.cc Replace lines 512 to 514 in the /storage/innobase/sync/sync0rw.cc file with the following content: bool recursive; os_thread_id_t writer_thread; if (!pass) { recursive = lock->recursive; os_rmb; writer_thread = lock->writer_thread; } The following two figures show the file before and after the modification.

Figure 6-2 Before the modification

Figure 6-3 After the modification

CA UTION

The format of the code added to the file must be the same as that shown in Figure 6-3.

Step 5 Go to the Percona source code directory decompressed. #cd /home/percona-server-8.0.15-6

Step 6 Compile the source code. #cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release - DFEATURE_SET=community -DWITH_BOOST=/home/boost_1_68_0 -DCMAKE_INSTALL_PREFIX=/usr/ local/percona -DMYSQL_DATADIR=/data/percona -DSYSCONFDIR=/etc -DWITHOUT_TOKUDB=1 - DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 - DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_SSL=system -DWITH_ZLIB=system - DWITH_LOBWRAP=0 -DMYSQL_UNIX_ADDR=/data/percona/run/mysql.sock - DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DFORCE_INSOURCE_BUILD=1

Table 6-3 describes the major directories involved. Modify the directories to match your installation.

Table 6-3 Major directories involved in compilation and installation

Parameter Description

DCMAKE_INSTAL Specifies the software installation directory. In this example, L_PREFIX it is /usr/local/percona.

DMYSQL_DATADI Specifies the directory for storing data files during database R creation. In this example, it is /data/percona.

DCMAKE_C_COM Specifies the directory for storing GCC. If no directory is PILER specified when GCC is installed, the default directory /usr/ local/bin is used.

Parameter Description

DWITH_BOOST Specifies the directory of boost_1_68_0 after the Percona source code package is decompressed. For example, if the source code package is decompressed in the /home directory, the directory is /home/boost_1_68_0.

Step 7 Install Percona. #make -j 96

NO TE

#make install

----End

6.6 Troubleshooting

6.6.1 CMake Target Version Does Not Take Effect After the Upgrade

Symptom The target version does not take effect after CMake is upgraded (6.4.2 Upgrading CMake).

Possible Causes The original CMake version is not uninstalled.

Procedure

Step 1 Remove the original CMake. #yum remove cmake -y Step 2 Create a soft link for CMake of the target version. #ln -s /usr/local/bin/cmake /usr/bin Step 3 Check the CMake version. #cmake --version

----End

7 PostgreSQL 11.3 Porting Guide (CentOS 7.6)

7.1 Introduction 7.2 Environment Requirements 7.3 Compilation and Installation Process 7.4 Configuring the Compilation Environment 7.5 Compiling and Installing PostgreSQL 7.6 Troubleshooting

7.1 Introduction

Overview Figure 7-1 shows the PostgreSQL architecture. Table 7-1 describes the major processes.

Figure 7-1 PostgreSQL architecture

Table 7-1 Major processes in PostgreSQL Ty Name Description pe

Ma Postmaster As the first process to be started, Postmaster controls all in the backend processes and is responsible for accepting pro and closing the database connections. ces s

Res Postgres This process (also called postmaster) manages backend ide (resident resident processes. By default, this process listens on nt process) Unix domain sockets and TCP/IP port 5432 and waits for pro the frontend to process the connections. You can change ces the listening port number in the postgresql.conf file of s PostgreSQL.

Ty Name Description pe

Su Postgres This subprocess determines whether to allow a bpr (subprocess) connection according to the security policy defined by oce the pg_hba.conf file. According to the security policy, ss this subprocess rejects certain IP addresses and networks, allows only certain users to connect to databases, or allows connections only to certain databases. Postgres receives the query from the frontend, retrieves data in the target database, and returns the result. It also updates the database. The updated data is recorded in transaction logs (also called WAL logs in PostgreSQL). When the system is powered off, or the server breaks down or is restarted, the logs can be used for data restoration. In addition, the logs can also be used for data restoration in other scenarios. In PostgreSQL 9.0 or later, WAL logs can be transferred to other PostgreSQL systems to implement real-time database replication.

Au SysLogger The main process starts the Syslogger auxiliary process xili (system log only when logging_collection in the Postgres.conf file ary process) is set to on. pro ces BgWriter This process writes dirty pages from the shared memory s (background to drives to improve the performance of data insertion, write process) update, and deletion.

WalWriter This process writes modification operations into drives (write-ahead before data is modified so that the data does not need log process) to be persisted into files in subsequent real-time data updates.

PgArch WAL logs are recycled. The PgArch process is responsible (archiving for backing up WAL logs before they are overwritten. process) Starting from version 8.x, these WAL logs are used for Point-In-Time-Recovery (PITR), which restores the database state to a certain state at certain period of time.

AutoVacuum In PostgreSQL database, after performing UPDATE or (automatic DELETE operations on the data, the database will not clearing immediately delete the old version of the data. Instead, process) the data will be marked as deleted by PostgreSQL's multi-version mechanism. If these old versions of data are being accessed by other transactions, it is necessary to retain them temporarily. After the transaction is submitted, the old versions of the data are no longer required (dead tuples) and the AutoVacuum process cleans up the data to make room.

Ty Name Description pe

PgStat This process collects statistical information, such as the (statistics number of table additions, deletions, or updates, the collection number of data blocks, and changes in indexes. The process) collected statistical information is used for the query optimizer to make correct judgment and choose the best execution plan. The pg_statistic catalog stores the information collected by the PgStat process.

CheckPoint The checkpointer is a sequence of transaction points set (checkpoint by the system. Setting the checkpoint ensures that the process) WAL log information before the checkpoint is flushed to drives.

Development language: C/C++

Brief description: object-relational database management system

Recommended Version

PostgreSQL 11.3

7.2 Environment Requirements

Hardware Requirements

Table 7-2 lists the hardware requirements.

Table 7-2 Hardware requirements

Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

OS Requirements

Table 7-3 lists the OS requirements.

Table 7-3 OS requirements Item Version

CentOS 7.6 for Arm or later. This document uses 7.6 as an example.

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Requirements The following software packages are required for configuring the PostgreSQL compilation environment: ● OS image Obtain the dependencies from the OS image. automake zlib zlib-devel bzip2 bzip2-devel bzip2-libs readline readline- devel gcc gcc-c++ bison ncurses ncurses-devel libaio-devel gmp gmp- devel mpfr mpfr-devel libmpc libmpc-devel ● CMake, GCC, and PostgreSQL source code packages URL of the CMake 3.5.2 source code: https://cmake.org/files/v3.5/ cmake-3.5.2.tar.gz URL of the GCC 7.3.0 source code: https:// mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz URL of the PostgreSQL source code: https://www.postgresql.org/ftp/source/

7.3 Compilation and Installation Process Figure 7-2 shows the compilation and installation process of the PostgreSQL source code.

Figure 7-2 PostgreSQL compilation and installation process

7.4 Configuring the Compilation Environment

7.4.1 Configuring the Local Yum Source

Mounting the OS Image Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

#mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart.

Method 2: Use the KVM to mount the OS image, and run the following command to mount the image to the /mnt/ directory.

#mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source

NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. #mv /etc/yum.repos.d /etc/yum.repos.d.bak

Step 2 Create the /etc/yum.repos.d directory for the new Yum source. #mkdir /etc/yum.repos.d

Step 3 Go to the directory created. #cd /etc/yum.repos.d

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file.

Step 5 Make the local source take effect. #yum clean all #yum makecache

----End

Installing OS Dependencies

Step 1 Install dependencies. #yum -y install automake zlib zlib-devel bzip2 bzip2-devel bzip2-libs readline readline-devel gcc gcc-c ++ bison ncurses ncurses-devel libaio-devel gmp gmp-devel mpfr mpfr-devel libmpc libmpc-devel

----End 7.4.2 Upgrading CMake

NO TE

The built-in CMake software of the system cannot meet the MySQL compilation requirements. Upgrade CMake to 3.4.3 or later. The following describes how to upgrade CMake to 3.5.2.

Step 3 Go to the CMake directory decompressed. #cd cmake-3.5.2 Step 4 Upgrade CMake. #./bootstrap

#make -j 96

NO TE

#make install

Step 5 Check whether the CMake version is 3.5.2. #/usr/local/bin/cmake --version

----End 7.4.3 Upgrading GCC

NO TE

Upgrade the GCC version to 5.3 or later. The following describes how to upgrade GCC to 7.3 as an example.

Step 1 Download GCC 7.3.0.

URL: https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-7.3.0/gcc-7.3.0.tar.gz

Step 2 Upload gcc-7.3.0.tar.gz to the /home directory and decompress it.

#cd /home #tar -xvf gcc-7.3.0.tar.gz

Step 3 Install GCC.

Go to the GCC directory decompressed. #cd /home/gcc-7.3.0 1. Perform configuration. #./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-bootstrap

NO TE

If "configure: error: no acceptable C compiler found in $PATH" is displayed, run the yum -y reinstall gcc gcc-c++ command. 2. Compile the GCC source code. #make -j 96

NO TE

3. Perform installation. #make -j 96 install

Step 4 Check whether the GCC version is 7.3.0. #gcc -v

----End

7.5 Compiling and Installing PostgreSQL

Step 1 Download the PostgreSQL 11.3 source code package. URL: https://www.postgresql.org/ftp/source/ (source code package: postgresql-113.tar.gz)

Step 2 Upload the package to the /home directory on the server and decompress it.

Go to the /home directory.

#cd /home

Decompress the source code package.

#tar -xvf postgresql-11.3.tar.gz

Step 3 Go to the PostgreSQL directory decompressed. #cd /home/postgresql-11.3

Step 4 Perform compilation. #./configure -prefix=/usr/local/pgsql

Step 5 Configure performance tuning options.

Run the following command to open the Makefile.global file:

#vi /home/postgresql-11.3/src/Makefile.global

Add -march=armv8-a+crc+lse to the end of line 261 in the file to enable lse, as shown in the following figure.

Step 6 Install PostgreSQL. #make -j 64

#make -j 64 install

NO TE

In the command, -j 64 can make full use of the multi-core feature of the CPUs to accelerate the compilation. The number following -j indicates the number of CPU cores. You can run the cat /proc/cpuinfo | grep processor | wc -l command to query the number of CPU cores. The value specified must be less than or equal to the number of CPU cores.

Step 7 Check the installation directory. #ls /usr/local/pgsql

Step 8 Check the database version. #/usr/local/pgsql/bin/postgres --version

----End

7.6 Troubleshooting

7.6.1 CMake Target Version Does Not Take Effect After the Upgrade

Symptom The target version does not take effect after CMake is upgraded (7.4.2 Upgrading CMake).

Possible Causes The original CMake version is not uninstalled.

Procedure

Step 1 Remove the original CMake. #yum remove cmake -y Step 2 Create a soft link for CMake of the target version. #ln -s /usr/local/bin/cmake /usr/bin Step 3 Check the CMake version. #cmake --version

----End

8 Greenplum 6.0.0 Porting Guide (CentOS 7.6)

8.1 Introduction 8.2 Environment Requirements 8.3 Compilation and Installation Process 8.4 Configuring the Compilation Environment 8.5 Compiling and Installing Greenplum 8.6 Troubleshooting

8.1 Introduction

Greenplum Overview

Greenplum is a relational database catering to data warehouses. It has distinguished advantages in data storage, concurrency, availability, linear expansion, response speed, usability, and cost-effectiveness due to a solid architecture. As a distributed database based on PostgreSQL, Greenplum uses the shared-nothing architecture, which allows the hosts, operating systems, memory, and storage to be self-controlled and do not share resources.

Greenplum is a relational database cluster, a logical database composed by multiple independent database services. Different from the Real Application Cluster (RAC), the Greenplum cluster employs the Massively Parallel Processing (MPP) architecture.

For more information, visit the official Greenplum website.

Brief description: distributed relational database

Recommended Version

Greenplum 6.0.0

NO TE

This document applies to Greenplum 6.0.0, and the compilation of other Greenplum versions.

8.2 Environment Requirements

Hardware Requirements

Table 8-1 lists the hardware requirements.

Table 8-1 Hardware requirements

Item Description

Server TaiShan 200 server (model 2280)

CPU Huawei Kunpeng 920 processor

OS Requirements

Table 8-2 lists the OS requirements.

Table 8-2 OS requirements

Item Version

CentOS 7.6 for Arm or later. This document uses CentOS 7.6 as an example.

NO TE

When installing an operating system, choose Minimal Install and select Development Tools to minimize manual operations.

Software Requirements

Before the compilation of Greenplum, you need to obtain the OS image, Python dependencies, source code packages for configuring the compilation environment such as CMake, zstd-dev, Xerces, re2c, Ninja, and GPORCA, and the Greenplum source code package. Download the software listed in Table 8-3 in advance.

Table 8-3 Version and URL for downloading the dependencies Source Code URL Version Dependency

Setuptools https://github.com/pypa/setuptools 29.0.1

psutil https://pypi.org/project/psutil/#files 5.7.0

PBR https://pypi.org/project/pbr/#files 5.4.4

LockFile https://pypi.org/project/lockfile/ 0.12.2 #files

pycparser https://pypi.org/project/pycparser/ 2.20 #files

CFFI https://pypi.org/project/cffi/#files 1.14.0

bcrypt https://pypi.org/project/bcrypt/#files 3.1.7

six https://pypi.org/project/six/#files 1.14.0

PyNaCl https://pypi.org/project/PyNaCl/ 1.3.0 #files

cryptography https://pypi.org/project/ 2.8 cryptography/#files

ipaddress https://pypi.org/project/ipaddress/ 1.0.23 #files

enum34 https://pypi.org/project/enum34/ 1.1.10 #files

Paramiko https://pypi.org/project/paramiko/ 2.7.1 #files

Epydoc https://pypi.org/project/epydoc/ 3.0.1 #files

CMake https://cmake.org/files/v3.5/ 3.5.2 cmake-3.5.2.tar.gz

zstd-dev https://github.com/facebook/zstd 1.4.3

Xerces https://github.com/greenplum- 3.1.2 db/gp-xerces

re2c https://github.com/skvadrik/re2c 2.0.3

Ninja https://github.com/ninja-build/ninja 1.10.1

Source Code URL Version Dependency

GPORCA https://github.com/greenplum-db/ Greenplum and gporca/tree/v3.65.3 GPORCA have strict version dependency. Greenplum 6.0.0 depends on GPORCA 3.65.XX. For details about how to check the version, see 8.4.8 Installing GPORCA.

Greenplum https://github.com/greenplum-db/ 6.0.0 gpdb/tree/6.0.0

The following two types of Greenplum dependency libraries need to be installed: ● Local Yum source curl-devel bzip2-devel python-devel openssl-devel readline-devel perl- ExtUtils-Embed libxml2-devel openldap-devel pam pam-devel perl-devel apr- devel libevent-devel libyaml libyaml-devel libedit-devel libffi-devel bison flex flex-devel You can obtain these dependencies from the ISO image. ● Source code dependencies You need to download Python dependencies and the CMake, zstd-dev, Xerces, re2c, Ninja, and GPORCA dependencies, and recompile and install the source code.

8.3 Compilation and Installation Process Figure 8-1 shows the compilation and installation process of the Greenplum source code.

Figure 8-1 Greenplum compilation and installation process

8.4 Configuring the Compilation Environment

8.4.1 Configuring the Local Yum Source

Mounting the OS Image Mount the OS image using either of the following methods:

Method 1: Upload the OS image to the /home directory and mount the image to the /mnt/ directory.

mount /home/CentOS-7-aarch64-Everything-1810.iso /mnt/

NO TE

Change the .iso file name based on actual situation. The operation takes effect only once and becomes invalid after the restart.

Method 2: Use the KVM to mount the OS image, and run the following command to mount the image to the /mnt/ directory.

mount /dev/sr0 /mnt/

NO TE

If the system displays a message indicating that no image is available after you run the command, mount the image again and then run the command.

Configuring the Local Yum Source

NO TE

Configure the local source if the server cannot obtain dependencies from the Internet using yum commands.

Step 1 Back up the original Yum source configuration file of the system. mv /etc/yum.repos.d /etc/yum.repos.d.bak

Step 2 Create the /etc/yum.repos.d directory for the new Yum source. mkdir /etc/yum.repos.d

Step 3 Go to the directory created. cd /etc/yum.repos.d

CA UTION

baseurl specifies the directory to which the image is mounted, that is, the /mnt directory in Mounting the OS Image.

3. Save and close the local.repo file.

Step 5 Make the local source take effect. yum clean all yum makecache

----End

Installing OS Dependencies

Step 1 Install dependencies. yum -y install curl-devel bzip2-devel python-devel openssl-devel readline-devel perl-ExtUtils-Embed libxml2-devel openldap-devel pam pam-devel perl-devel apr-devel libevent-devel libyaml libyaml- devel libedit-devel libffi-devel bison flex flex-devel

----End

8.4.2 Installing Python Dependencies

Installing Setuptools

Step 1 Use Google Chrome to download the setuptools dependency. URL: https://github.com/pypa/setuptools Download version 29.0.1 from Releases, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip setuptools-29.0.1.zip

Step 3 Install Setuptools. cd /home/setuptools-29.0.1 python bootstrap.py python setup.py install

----End

Installing psutil

Step 1 Use Google Chrome to download the psutil dependency.

URL: https://pypi.org/project/psutil/#files

Download version 5.7.0, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf psutil-5.7.0.tar.gz

Step 3 Install psutil. cd /home/psutil-5.7.0 python setup.py install

----End

Installing pbr

Step 1 Use Google Chrome to download the pbr dependency.

URL: https://pypi.org/project/pbr/#files

Download version 5.4.4, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf pbr-5.4.4.tar.gz Step 3 Install pbr. cd /home/pbr-5.4.4 python setup.py install

----End

Installing LockFile

Step 1 Use Google Chrome to download the LockFile dependency. URL: https://pypi.org/project/lockfile/#files Download version 0.12.2, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf lockfile-0.12.2.tar.gz Step 3 Install LockFile. cd /home/lockfile-0.12.2 python setup.py install

----End

Installing Pycparser

Step 1 Use Google Chrome to download the Pycparser dependency. URL: https://pypi.org/project/pycparser/#files Download version 2.20, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf pycparser-2.20.tar.gz

Step 3 Install pycparser. cd /home/pycparser-2.20 python setup.py install

----End

Installing CFFI

Step 1 Use Google Chrome to download the CFFI dependency.

URL: https://pypi.org/project/cffi/#files

Download version 1.14.0, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf cffi-1.14.0.tar.gz

Step 3 Install CFFI. cd /home/cffi-1.14.0 python setup.py install

----End

Installing Six

Step 1 Use Google Chrome to download the Six dependency. URL: https://pypi.org/project/six/#files Download version 1.14.0, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf six-1.14.0.tar.gz Step 3 Install Six. cd /home/six-1.14.0 python setup.py install

----End

Installing bcrypt

Step 1 Use Google Chrome to download the bcrypt dependency.

URL: https://pypi.org/project/bcrypt/#files

Download version 3.1.7, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf bcrypt-3.1.7.tar.gz Step 3 Install bcrypt. cd /home/bcrypt-3.1.7 python setup.py install

----End

Installing PyNaCl

Step 1 Use Google Chrome to download the PyNaCl dependency. URL: https://pypi.org/project/PyNaCl/#files Download version 1.3.0, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf PyNaCl-1.3.0.tar.gz Step 3 Install PyNaCl. cd /home/PyNaCl-1.3.0 python setup.py install

----End

Installing ipaddress

Step 1 Use Google Chrome to download the ipaddress dependency. URL: https://pypi.org/project/ipaddress/#files Download version 1.0.23, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf ipaddress-1.0.23.tar.gz Step 3 Install ipaddress. cd /home/ipaddress-1.0.23 python setup.py install

----End

Installing enum34

Step 1 Use Google Chrome to download the enum34 dependency. URL: https://pypi.org/project/enum34/#files Download version 1.1.10, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf enum34-1.1.10.tar.gz Step 3 Install enum34. cd /home/enum34-1.1.10 python setup.py install ----End

Installing cryptography Step 1 Use Google Chrome to download the cryptography dependency. URL: https://pypi.org/project/cryptography/#files Download version 2.8, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf cryptography-2.8.tar.gz Step 3 Install cryptography. cd /home/cryptography-2.8 python setup.py install

----End

Installing paramiko

Step 1 Use Google Chrome to download the paramiko dependency. URL: https://pypi.org/project/paramiko/#files Download version 2.7.1, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf paramiko-2.7.1.tar.gz Step 3 Install paramiko. cd /home/paramiko-2.7.1 python setup.py install

----End

Installing epydoc

Step 1 Use Google Chrome to download the epydoc dependency. URL: https://pypi.org/project/epydoc/#files Download version 3.0.1, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home tar -zxvf epydoc-3.0.1.tar.gz

Step 3 Install epydoc. cd /home/epydoc-3.0.1 python setup.py install

----End 8.4.3 Upgrading CMake Upgrade CMake to 3.1 or later. This document uses the upgrade to 3.5.2 as an example.

Step 1 Download the CMake 3.5.2 package.

URL: https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz

Step 2 Upload the package to the /home directory on the server and decompress it.

cd /home tar -zxvf cmake-3.5.2.tar.gz

Step 3 Go to the directory generated after the package is decompressed. cd cmake-3.5.2 Step 4 Upgrade CMake. ./bootstrap

make

make install

Step 5 Check the CMake version. /usr/local/bin/cmake --version

----End 8.4.4 Installing zstd-dev

Step 1 Use Google Chrome to download the zstd-dev software package. URL: https://github.com/facebook/zstd Download version 1.4.3, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip zstd-1.4.3.zip

Step 3 Go to the directory generated after the package is decompressed. cd /home/zstd-1.4.3/

Step 4 Compile and install zstd-dev. make

make install

ln -s /usr/local/lib/libzstd.so.1.4.3 /usr/lib64/libzstd.so.1

----End 8.4.5 Installing Xerces

Step 1 Use Google Chrome to download the Xerces software package.

URL: https://github.com/greenplum-db/gp-xerces Download version 3.1.2-p1, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip gp-xerces-3.1.2-p1.zip Step 3 Go to the directory generated after the package is decompressed. cd /home/gp-xerces-3.1.2-p1/ Step 4 Compile and install gp-xerces. ./configure

make

make install

----End 8.4.6 Installing re2c

Step 1 Use Google Chrome to download the re2c software package. URL: https://github.com/skvadrik/re2c Download version 2.0.3, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip re2c-2.0.3.zip Step 3 Go to the directory generated after the package is decompressed. cd /home/re2c-2.0.3/ Step 4 Compile and install re2c. ./autogen.sh

./configure

make

make install

----End 8.4.7 Installing Ninja

Step 1 Use Google Chrome to download the Ninja software package.

URL: https://github.com/ninja-build/ninja

Download version 1.10.1, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip ninja-1.10.1.zip

Step 3 Go to the directory generated after the package is decompressed. cd /home/ninja-1.10.1 Step 4 Compile and install Ninja. ./configure.py --bootstrap

cp ninja /usr/bin/

----End 8.4.8 Installing GPORCA

Step 1 Use Google Chrome to download the GPORCA software package. URL: https://github.com/greenplum-db/gporca/tree/v3.65.3 Download version 3.65.3, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip gporca-3.65.3.zip Step 3 Go to the directory generated after the package is decompressed. cd gporca-3.65.3 Step 4 Compile and install GPORCA. cmake -GNinja -H. -Bbuild

vi libgpos/src/common/CStackDescriptor.cpp Comment out line 167.

ninja install -C build

Step 5 Add /usr/local/lib to the /etc/ld.so.conf file. echo /usr/local/lib >> /etc/ld.so.conf ldconfig

NO TE

If the error message "ldconfig: /lib64/libstdc++.so.6.0.24-gdb.py is not an ELF file - it has the wrong magic bytes at the start." is displayed when you run the ldconfig command, run the following command to delete the file: rm -rf /lib64/libstdc++.so.6.0.24-gdb.py

----End

8.5 Compiling and Installing Greenplum

Obtaining Source Code

Step 1 Use Google Chrome to download the Greenplum software package. URL: https://github.com/greenplum-db/gpdb/tree/6.0.0 Download version 6.0.0, as shown in the following figures.

Step 2 Upload the package to the /home directory on the server and decompress it. cd /home unzip gpdb-6.0.0.zip ----End

Compiling and Installing Greenplum Step 1 Go to the /home/gpdb-6.0.0 source code directory. cd /home/gpdb-6.0.0 Step 2 Perform compilation. ./configure --with-perl --with-python --with-libxml --prefix=/usr/local/gpdb

Table 8-4 Major parameters involved in compilation and installation Parameter Description

prefix Specifies the software installation path. In this example, it is /usr/local/gpdb.

with-python Builds the Python module (PL/Perl).

with-xml Supports XML.

Step 3 Perform installation. make

make install

----End

8.6 Troubleshooting

8.6.1 Greenplum Compilation Failed Due to Incorrect ORCA Version

Symptom

The following error message is displayed during Greenplum compilation: Your ORCA version is expected to be 3.74.XXX.

Possible Cause

The Greenplum version is incompatible with the current GPORCA version.

Procedure

Step 1 Check the GPORCA version required. cd gpdb-6.0.0/depends cat conanfile_orca.txt

Step 2 Reinstall GPORCA of the correct version. For details, see 8.4.8 Installing GPORCA. After the installation is complete, run the following commands to load the dynamic library:

echo /usr/local/lib >/etc/ld.so.conf ldconfig Step 3 Recompile Greenplum.

----End

9 Kunpeng BoostKit for Database Porting Guide

9.1 Introduction 9.2 Common Atomic Operations (C Language) 9.3 PostgreSQL 9.4 MySQL

9.1 Introduction

The migration of the x86 platform to the Kunpeng computing platform involves the conversion of the underlying instruction set. For example, if the code is written in advanced languages such as C/C++, the GCC is used for compilation on the Kunpeng computing platform. The GCC of a later version supports the ARM v8 instruction set to ensure normal compilation. In database scenarios, a large number of atomic function operations are required to ensure the consistency and performance of concurrent programming. To adapt to different versions of GCC, the database code on the x86 platform often assembles atomic operations. In addition, the SSE4.x instruction set provides some advanced instructions, which are common on the x86 platform.

The assembly instructions and advanced instruction sets specific to the x86 platform need to be modified to adapt to the Kunpeng computing platform. This document uses the open-source databases (PostgreSQL and MySQL) as examples to describe the differences between common platforms for developers who need to port database source code.

In this document, gcc7.3 or later is recommended for compilation. The C11 memory model and new ARM v8 instructions are supported.

9.2 Common Atomic Operations (C Language)

GCC provides atomic operations for the C language. It supports cross-platform compilation.

9.2.1 C11 Memory Model Before GCC 4.7, the default memory model in the __sync primitive is the full barrier model. The read and write operations before and after the __sync primitive cannot be rearranged. To improve the pipeline execution efficiency, GCC 4.7 integrates with the C11 memory model. The __atomic synchronization primitive is used to control the required barrier level.

__ATOMIC_RELAXED Other read and write operations are not synchronized. Only the atomic operation is ensured.

__ATOMIC_CONSUME Load operation. The fetch operation on which the current thread depends on the atomic variable cannot be reordered before the instruction, and is visible to the store operation (release) of another thread.

__ATOMIC_ACQUIRE Load operation. All fetch operations on the current thread cannot be reordered before the instruction, and is visible to the store operation (release) of another thread.

__ATOMIC_RELEASE Store operation. All fetch operations on the current thread cannot be reordered after the instruction, and is visible to the load operation (consume) of another thread.

__ATOMIC_ACQ_REL Load/store operation: memory_order_acquire + memory_order_release.

__ATOMIC_SEQ_CST memory_order_acq_rel + sequence consistency.

9.2.2 Recommended Memory Barrier

NO TE

If the logic for multiple threads to access the critical area is unclear, you are advised to use the __ATOMIC_SEQ_CST barrier to avoid consistency problems caused by improper use of the barrier. A common database scenario is lock operations. A critical area is formed between lock acquire and release to ensure the consistency of the critical area. ● Lock acquire Use the __ATOMIC_ACQUIRE barrier. ● Lock release Use the __ATOMIC_RELEASE barrier.

9.2.3 Atomic Operations

Operation Type Function

Atomic read type __atomic_load_n(type *ptr,int memmodel)

Atomic write void __atomic_store_n (type *ptr, type val, int memmodel)

Atomic exchange type __atomic_exchange_n (type *ptr, type val, int memmodel)

Atomic CAS bool __atomic_compare_exchange_n (type *ptr, type *expected, type desired, bool weak, int success_memmodel, int failure_memmodel)

Atomic addition, type __atomic_op_fetch (type *ptr, type val, int memorder) subtraction, AND, type __atomic_fetch_op (type *ptr, type val, int memorder) OR, and XOR add/sub/and/xor/or/nand

Atomic TAS bool __atomic_test_and_set (void *ptr, int memorder)

Atomic Clear void __atomic_clear (bool *ptr, int memorder)

9.2.4 Spin Operation ● x86 platform: Use pause or rep;nop for spin operation. #define SPIN() __asm__ __volatile__("pause") #define SPIN() __asm__ __volatile__(" rep; nop \n"); ● AArch64: You can use nop to implement similar operations. #define __nops(n) ".rept" #n"\nnop\n.endr\n" #define SPIN(n) asm volatile(__nops(n))

9.3 PostgreSQL In the multiple platforms supported by PostgreSQL, the x86 platform uses assembly to implement some frequently used atomic operations to shield the compilation efficiency problems that may occur in the gcc version. 9.3.1 Atomic Operations

9.3.1.1 Memory Barrier To prevent CPU instruction rearrangement, memory barriers need to be explicitly added in concurrent programming. ● Memory barrier for x86 platform #define pg_memory_barrier_impl() \ __asm__ __volatile__ ("lock; addl $0,0(%%esp)" : : : "memory", "cc") ● AArch64 platform #define pg_memory_barrier_impl() __atomic_thread_fence(__ATOMIC_SEQ_CST) #define pg_memory_barrier_impl() __sync_synchronize()

9.3.1.2 Spin Function The database does not invoke Pthread_spinlock to monitor spin mutex locks, detect deadlocks, and control spin time. Instead, the database uses the customized mode. The spin operation implementation varies according to the platform. ● x86 platform static __inline__ void pg_spin_delay_impl(void) { __asm__ __volatile__(" rep; nop \n"); } ● AArch64 platform (rep command is not supported) #define pg_spin_delay_impl() ((void)0)

9.3.1.3 TAS & Clear TAS performs test_and_set operation on a bit. ● x86 platform static inline bool pg_atomic_test_set_flag_impl(volatile pg_atomic_flag *ptr) { register char _res = 1; __asm__ __volatile__( " lock \n" " xchgb %0,%1 \n" : "+q"(_res), "+m"(ptr->value) : : "memory"); return _res == 0; } static inline void pg_atomic_clear_flag_impl(volatile pg_atomic_flag *ptr) { /* * On a TSO architecture like x86 it's sufficient to use a compiler * barrier to achieve release semantics. */ __asm__ __volatile__("" ::: "memory"); ptr->value = 0; } ● AArch64 platform static inline bool pg_atomic_test_set_flag_impl(volatile pg_atomic_flag *ptr) { return __sync_lock_test_and_set(&ptr->value, 1) == 0; } static inline void pg_atomic_clear_flag_impl(volatile pg_atomic_flag *ptr) { __sync_lock_release(&ptr->value); }

9.3.1.4 CAS Perform the compare_and_swap operation on a 32-bit or 64-bit variable (a 32-bit variable is used as an example). ● x86 platform

static inline bool pg_atomic_compare_exchange_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 *expected, uint32 newval) { char ret;

/* * Perform cmpxchg and use the zero flag which it implicitly sets when * equal to measure the success. */ __asm__ __volatile__( " lock \n" " cmpxchgl %4,%5 \n" " setz %2 \n" : "=a" (*expected), "=m"(ptr->value), "=q" (ret) : "a" (*expected), "r" (newval), "m"(ptr->value) : "memory", "cc"); return (bool) ret; } ● AArch64 platform static inline bool pg_atomic_compare_exchange_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 *expected, uint32 newval) { /* FIXME: we can probably use a lower consistency model */ return __atomic_compare_exchange_n(&ptr->value, expected, newval, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); }

static inline bool pg_atomic_compare_exchange_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 *expected, uint32 newval) { bool ret; uint32 current; current = __sync_val_compare_and_swap(&ptr->value, *expected, newval); ret = current == *expected; *expected = current; return ret; }

9.3.1.5 Fetch_add

Atomic addition

● x86 platform static inline uint64 pg_atomic_fetch_add_u64_impl(volatile pg_atomic_uint64 *ptr, int64 add_) { uint64 res; __asm__ __volatile__( " lock \n" " xaddq %0,%1 \n" : "=q"(res), "=m"(ptr->value) : "0" (add_), "m"(ptr->value) : "memory", "cc"); return res; } ● AArch64 platform

static inline uint64 pg_atomic_fetch_add_u64_impl(volatile pg_atomic_uint64 *ptr, int64 add_) { return __sync_fetch_and_add(&ptr->value, add_); } 9.3.2 CRC Checksum Algorithm SSE4.2 and ARM v8 provide their own CRC instructions. ● x86 platform (src/port/pg_crc32c_sse42.c) #include Use _mm_crc32_u8/16/32/64 ● AArch64 platform (src/port/pg_crc32c_armv8.c) #include Use __crc32cb/cw/cd/ch

9.4 MySQL

9.4.1 Spin Function ● x86 platform #define UT_RELAX_CPU() __asm__ __volatile__("pause") ● AArch64 platform #define UT_RELAX_CPU() __asm__ __volatile__("" ::: "memory") 9.4.2 CRC Checksum Algorithm ● x86 platform #include _mm_crc32_u8/16/32/64 ● AArch64 platform #include Use __crc32cb/cw/cd/ch

NO TE

The MySQL 5.7 patch is not incorporated into the community: https://github.com/mysql/ mysql-server/pull/136/files 9.4.3 Cache Line Alignment Optimization In multi-concurrency programming, the critical area variable features high write frequency. For example, the lockword in a lock changes the status continuously due to lock-up and lock release operations. Each change causes the CACHE_LINE in other cores to become invalid. To avoid interference between such data structure and other variables, the cache line is aligned in the application so that the variable exclusively occupies a cache line.

MySQL has made a number of 64-byte-aligned cache lines on the x86 platform. The L3 cache line of Huawei Kunpeng 920 processor is 128-byte. Therefore, the alignment in the MySQL source code needs to be changed to 128-byte. The modified alignment data structure is as follows:

Cache line brt_search_latches The TPM is improved by 3% alignment btr_search_sys to 4% ReadView::m_view_list trx_sys_t::rw_trx_list trx_sys_t::mysql_trx_list trx_sys_t::rsegs srv_conc_t::n_active srv_conc_t::n_active lock_sys_t::mutex lock_sys_t::wait_mutex The 64-byte-aligned data structure in MySQL is changed to 128-byte-aligned data structure.

9.4.4 MySQL Read/Write Deadlock Due to ARM Weak Memory Sequence

Problem Description Recently, sysbench is used to perform read and write tests on MySQL 5.7 Community Edition. After a period of time, MySQL 5.7 deadlock occurs and the test stops.

Test Environment ● Network topology

Sysbench is installed on an x86 server and MySQL database is installed on an ARM server and connected via a switch.

- CPU DDR Disk OS Type

Sysbenc E5-2695 v4/2.10 384 GB SATA Euler 2.0 h GHz/72-core (4.16.2)

MySQL HI1620/2.5 GHz/96- 512 GB SSD Ubuntu 16.04 core (4.4.0)

● Dataset The test database contains 64 tables, and each table contains 10,000,000 records. ● Test script Use the oltp.lua file of sysbench to perform the test. The test command is as follows: /ssd/zqsource/rds-perf/source/sysbench-0.5/sysbench/sysbench -- test=/ssd/zqsource/rds-perf/source/sysbench-0.5/sysbench/tests/db/ oltp.lua --oltp-tables-count=64 --oltp-table-size=10000000 --mysql-host= $HOST --mysql-port=$PORT --mysql-db=$TABLE --mysql-user=$USER -- mysql-password=$PASSWD --oltp-read-only=off --oltp-point-selects=10 -- oltp-simple-ranges=1 --oltp-sum-ranges=1 --oltp-order-ranges=1 --oltp- distinct-ranges=1 --max-requests=0 --max-time=$TIME --report-interval=5 --num-threads=$THREAD run ● MySQL configuration file [mysqld_safe] log-error=/data/mysql/log/mysql.log pid-file=/data/mysql/run/mysqld.pid

[client] socket=/data/mysql/run/mysql.sock default-character-set=utf8mb4

[mysqld] basedir=/home/mysql-8.0.17-linux-glibc2.12-x86_64 #basedir=/usr/local/mysql tmpdir=/data/mysql/tmp datadir=/data/mysql/data socket=/data/mysql/run/mysql.sock port=3306 user=root default_authentication_plugin=mysql_native_password innodb_page_size=4k max_connections=2000 back_log=2048 performance_schema=OFF max_prepared_stmt_count=128000 #file innodb_file_per_table innodb_log_file_size=2048M innodb_log_files_in_group=32 innodb_open_files=10000 table_open_cache_instances=64 #buffers innodb_buffer_pool_size=230G innodb_buffer_pool_instances=16 innodb_log_buffer_size=2048M #tune sync_binlog=1 innodb_flush_log_at_trx_commit=1

innodb_use_native_aio=1 innodb_spin_wait_delay=180 innodb_sync_spin_loops=25 innodb_flush_method=O_DIRECT innodb_io_capacity=30000 innodb_io_capacity_max=40000 innodb_lru_scan_depth=9000 innodb_page_cleaners=16 innodb_spin_wait_pause_multiplier=25 #perf special innodb_flush_neighbors=0 innodb_write_io_threads=24 innodb_read_io_threads=16 innodb_purge_threads=32 sql_mode=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_AUTO_VALUE_ON_ZERO,STRICT_A LL_TABLES log-bin=mysql-bin ssl=0 table_open_cache=15000

Reproduction Procedure Procedure for reproducing the fault:

Step 1 Run the mysqld_safe command to start MySQL. Step 2 Run the test command on the sysbench server to perform the read/write test. Step 3 After a long-time test, the TPS returned by sysbench is 0. Connect to the MySQL database on the MySQL server to query data in the test table. No data is returned for a long time.

----End

Analysis When a deadlock occurs, the gdb command is used to view the thread stack information of the MySQL database. Most threads are waiting for locks.

According to code analysis, ARM uses to relaxed memory models, while x86 use a stronger isotonic model based on the total store ordering (TSO) model. For the following pseudocode:

After abstraction, the following pseudocode is obtained:

On x86 servers, r1 = 1 and r2 = 0 are impossible. However, on ARM servers, r1 = 1 and r2 = 0 may occur. MySQL Mutex and RWLock are implemented based on the atomic operation and lock-free algorithm, the out-of-order problem of this module is especially serious. For other modules or programs, if the critical section is protected by a correct lock, the modules or programs can run properly. To ensure the logic correctness, a memory barrier needs to be added to the ARM version, and the read/write sequence of keywords needs to be focused on.

After being tested on Qualcomm ARM servers in 2017, MariaDB uses the __atomic functions to modify the implementation of RWLock and Mutex. The __atomic functions can ensure SC (memory sequence needs to be specified) and can be used across platforms. The read/write sequence of keywords such as lock_word and lock->waiters can be protected to resolve the MySQL deadlock problem.

Conclusion ● Currently, only MariaDB supports ARM servers. https://mariadb.com/resources/blog/mariadb-server-102-now-available- qualcomm-centriqtm-2400-server-processor ● The MySQL Community Edition and Enterprise Edition do not support ARM. https://www.mysql.com/support/supportedplatforms/database.html

● Available MariaDB versions: http://yum.mariadb.org/10.3/rhel7-aarch64/rpms/

A Change History

Date Description

2021-03-30 This issue is the fourth official release. Changed the name of MySQL 8.0.x Porting Guide (CentOS 7.6) to (Recommended) MySQL 8.0.x Porting Guide (CentOS 7.6).

2020-09-24 This issue is the third official release. Added the MySQL 8.0.17 Porting Guide (openEuler 20.03).

2020-05-09 This issue is the second official release. MariaDB 10.3.8 Porting Guide (CentOS 7.6): Modified the MariaDB architecture in 4.1 Introduction. PostgreSQL 11.3 Porting Guide (CentOS 7.6): Modified the PostgreSQL architecture in 7.1 Introduction.

2020-03-20 This issue is the first official release.