Porting Guides (Third-Party Dependency Libraries)

Kunpeng BoostKit for Big Data

Issue 06 Date 2020-12-30

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2021. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied.

Contents

1 CheckSo User Guide...... 1 1.1 Introduction...... 1 1.2 Environment Requirements...... 1 1.3 Scanning Files...... 2 1.4 Viewing the Scan Results...... 2 1.5 Analyze the Scanning Result...... 3 2 commons-crypto-1.0.0.jar Porting Guide (CentOS 7.6)...... 7 2.1 Introduction...... 7 2.2 Requirements...... 7 2.3 Configuring the Compilation Environment...... 8 2.4 Compilation Process...... 11 2.5 Rectifying Faults...... 11 2.5.1 Failed to Find the Library Files...... 12 2.5.2 Java AES Error...... 12 2.5.3 OpenSSL Error...... 13 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide (CentOS 7.6)...... 15 3.1 Introduction...... 15 3.2 Environment Requirements...... 15 3.3 Configuring the Compilation Environment...... 17 3.4 Compiling and Installing Software...... 20 4 HBase-shaded-netty-2.1.0 Porting Guide (CentOS 7.6)...... 22 4.1 Introduction...... 22 4.2 Requirements...... 22 4.3 Configuring the Compilation Environment...... 23 4.4 Compilation HBase-shaded-netty-2.1.0...... 26 4.5 Rectifying Faults...... 27 4.5.1 Failed to Run the patch Command...... 27 4.5.2 Failed to Find libnetty_transport_native_epoll_x86_64.so...... 28 5 Jansi-1.4.jar Porting Guide (CentOS 7.6)...... 29 5.1 Introduction...... 29 5.2 Requirements...... 29 5.3 Configuring the Compilation Environment...... 30

5.4 Compiling Jansi-1.4.jar ...... 33 5.5 Rectifying Faults...... 34 5.5.1 Error Occurs During Compilation...... 34 5.5.2 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed...... 35 6 Jansi-1.11.jar Porting Guide (CentOS 7.6)...... 36 6.1 Introduction...... 36 6.2 Requirements...... 36 6.3 Configuring the Compilation Environment...... 37 6.4 Compilation...... 40 6.5 Troubleshooting...... 41 6.5.1 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed...... 41 6.5.2 Error Occurs During Compilation...... 42 6.5.3 java.lang.OutOfMemoryError: PermGen space...... 42 6.5.4 no description for @param...... 43 7 Jansi-1.9,1.12,1.16.jar Porting Guide (CentOS 7.6)...... 44 7.1 Introduction...... 44 7.2 Environment Requirements...... 44 7.3 Configuring the Compilation Environment...... 45 7.4 Compilation Process...... 48 7.5 Rectifying Faults...... 50 7.5.1 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed...... 50 7.5.2 Error Occurs During Compilation...... 51 7.5.3 java.lang.OutOfMemoryError: PermGen space...... 51 8 jline-2.10.5.jar Porting Guide (CentOS 7.6)...... 52 8.1 Introduction...... 52 8.2 Environment Requirements...... 52 8.3 Configuring the Compilation Environment...... 53 8.4 Compiling JLine...... 56 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Guide (CentOS 7.6)...... 58 9.1 Introduction...... 58 9.2 Requirements...... 58 9.3 Configuring the Compilation Environment...... 59 9.4 Compilation Process...... 62 9.5 Troubleshooting...... 65 9.5.1 Failed to Download doxia-module-apt-1.6.jar During the Compilation of jline-2.14.3...... 66 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide (CentOS 7.6)...... 67 10.1 Introduction...... 67 10.2 Environment Requirements...... 68 10.3 Configuring the Compilation Environment...... 68 10.4 Performing Porting Analysis...... 71

10.5 Compiling Jython...... 72 10.5.1 Compiling jython-standalone-2.7.0...... 72 10.5.2 Compiling jython-standalone-2.7.1...... 73 10.6 Troubleshooting...... 74 10.6.1 Failure of the First Compilation...... 74 10.6.2 Failed to Replacing the jython-standalone-2.7.0 Dependencies Identified by the Tool...... 74 10.6.3 Failed to Replacing the jython-standalone-2.7.1 Dependencies Identified by the Tool...... 75 11 leveldbjni-all-1.8.jar Porting Guide (CentOS 7.6)...... 77 11.1 Introduction...... 77 11.2 Environment Requirements...... 77 11.3 Configuring the Compilation Environment...... 78 11.4 Compilation Process...... 83 11.5 Performing the Verification...... 86 11.6 Rectifying Faults...... 87 11.6.1 error: 'class leveldb::DB' has no member named 'ResumeCompactions'...... 87 11.6.2 error: required file 'autotools/compile' not found...... 91 11.6.3 error: 'const char* leveldb::Slice::data_' is private...... 92 11.6.4 java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path...... 92 12 lz4-java-x.x.x.jar Porting Guide (CentOS 7.6)...... 96 12.1 Introduction...... 96 12.2 Requirements...... 97 12.3 Configuring the Compilation Environment...... 97 12.4 Compiling lz4-java...... 99 12.5 Rectifying Faults...... 101 12.5.1 "BUILD FAILED" Is Displayed During Compilation...... 101 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guide (CentOS 7.6)..102 13.1 Introduction...... 102 13.2 Requirements...... 102 13.3 Configuring the Compilation Environment...... 103 13.4 Compiling Netty...... 106 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS 7.6)...... 109 14.1 Introduction...... 109 14.2 Requirements...... 109 14.3 Configuring the Compilation Environment...... 110 14.4 Compiling netty-all-4.0.52.Final.jar...... 113 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS 7.6)...... 118 15.1 Introduction...... 118 15.2 Requirements...... 118 15.3 Configuring the Compilation Environment...... 119 15.4 Compiling netty-all-4.1.17.Final.jar...... 122

15.5 Troubleshooting...... 124 15.5.1 Failed to Find jni.h and jni_md.h...... 124 15.5.2 Failed to Find linux-aarch_64-fedora.2.0.7.Final.jar During Compilation...... 125 15.5.3 Failed to Download apr-1.6.3 During Compilation...... 126 15.5.4 Failed to Download libressl-2.5.5 During Compilation...... 126 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide (CentOS 7.6)...... 128 16.1 Introduction...... 128 16.2 Environment Requirements...... 128 16.3 Configuring the Compilation Environment...... 129 16.4 Compiling and Installing RocksDB JNI...... 133 16.5 Troubleshooting...... 134 16.5.1 Size of the Compiled JAR Package Is Greater Than the Package Downloaded From the Maven Repository...... 134 16.5.2 jni.h or jni_md.h File Cannot Be Found...... 135 16.5.3 Compilation Error...... 135 16.5.4 Failed to Compare va_list and nullptr...... 136 16.5.5 Connect to central.maven.org:80 connection timed out...... 136 16.5.6 assertj-core-1.7.1.jar error in opening zip file 1 error...... 137 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Guide (CentOS 7.6)...... 138 17.1 Introduction...... 138 17.2 Requirements...... 138 17.3 Configuring the Compilation Environment...... 139 17.4 Compiling scala-compiler ...... 142 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS 7.6)...... 145 18.1 Introduction...... 145 18.2 Requirements...... 145 18.3 Configuring the Compilation Environment...... 146 18.4 Compiling snappy-java...... 149 18.5 Troubleshooting...... 150 18.5.1 Failed to Decompress the snappy-1.0.4.tar.gz Installation Package...... 150 18.5.2 Failed to Find Library Files During Compilation...... 151 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, 1.1.2.6.jar Porting Guide (CentOS 7.6)...... 152 19.1 Introduction...... 152 19.2 Requirements...... 152 19.3 Configuring the Compilation Environment...... 153 19.4 Compiling snappy-java...... 156 19.5 Rectifying Faults...... 160 19.5.1 Failed to Decompress snappy-1.1.0.tar.gz During the Compilation of snappy-java-1.1.0.1...... 160 19.5.2 Failed to Decompress snappy-1.1.1.tar.gz During the Compilation of snappy- java-1.1.1.3/1.1.1.6/1.1.1.7...... 161

19.5.3 Failed to Download sbt-launch.jar During the Compilation of snappy- java-1.1.1.3/1.1.1.6/1.1.1.7/1.1.2.1/1.1.2.6...... 162 19.5.4 Failed to Find Library Files During Compilation...... 163 20 wildfly-openssl-1.0.4.Final.jar Porting Guide (CentOS 7.6)...... 164 20.1 Introduction...... 164 20.2 Environment Requirements...... 164 20.3 Configuring the Compilation Environment...... 165 20.4 Compiling wildfly-openssl-1.0.4.Final.jar...... 169 20.5 Troubleshooting...... 174 20.5.1 wildfly-openssl-1.0.4.Final.jar Compilation Error...... 174 20.5.2 No ARM-based JAR Package Is Generated After wildfly-openssl-1.0.4.Final.jar Is Compiled...... 175 21 chimera-0.9.2.jar Porting Guide (CentOS 7.6)...... 182 21.1 Introduction...... 182 21.2 Requirements...... 182 21.3 Configuring the Compilation Environment...... 183 21.4 Compiling chimera...... 186 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Guide (CentOS 7.6)...... 188 22.1 Introduction...... 188 22.2 Environment Requirements...... 188 22.3 Configuring the Compilation Environment...... 190 22.4 Compiling and Installing flink-shaded-netty...... 193 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide (CentOS 7.6)...... 196 23.1 Introduction...... 196 23.2 Requirements...... 196 23.3 Configuring the Compilation Environment...... 197 23.4 Compiling the frocksdbjni Source Code...... 200 23.5 Troubleshooting...... 200 23.5.1 Size of the Compiled JAR Package Is Greater Than the Package Downloaded From the Maven Repository...... 200 23.5.2 jni.h or jni_md.h File Cannot Be Found...... 201 23.5.3 "unknown value 'native' for -march" Is Displayed During Compilation...... 201 23.5.4 Failed to Compare va_list and nullptr...... 202 23.5.5 Connect to central.maven.org:80 connection timed out...... 202 23.5.6 assertj-core-1.7.1.jar error in opening zip file 1 error...... 203 24 jna-3.0.9,4.1.0.jar Porting Guide (CentOS 7.6)...... 204 24.1 Introduction...... 204 24.2 Environment Requirements...... 204 24.3 Configuring the Compilation Environment...... 205 24.4 Compiling jna-3.0.9, 4.1.0.jar...... 207 24.5 Rectifying Faults...... 208 24.5.1 X11/Xlib.h Cannot Be Found...... 208

24.5.2 X11/Intrinsic.h Cannot Be Found...... 209 24.5.3 Test Case Failed Due to the Use of the Default Code in Compilation...... 209 25 server-metrics-0.2.8.jar Porting Guide (CentOS 7.6)...... 211 25.1 Introduction...... 211 25.2 Environment Requirements...... 211 25.3 Configuring the Compilation Environment...... 212 25.4 Compiling and Installing server metrics...... 216 26 incremental-compiler-0.13.15.jar Porting Guide (CentOS 7.6)...... 217 26.1 Introduction...... 217 26.2 Environment Requirements...... 217 26.3 Configuring the Compilation Environment...... 218 26.4 Compiling sbt-republish...... 222 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Guide (CentOS 7.6)...... 224 27.1 Introduction...... 224 27.2 Requirements...... 224 27.3 Configuring the Compilation Environment...... 225 27.4 Compiling and Installing flink-shaded-netty...... 228 28 flink-shaded-netty-tcnative-dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS 7.6)...... 231 28.1 Introduction...... 231 28.2 Requirements...... 231 28.3 Configuring the Compilation Environment...... 232 28.4 Compiling flink-shaded-release-7.0...... 235 29 jruby-1.7.27.jar Porting Guide (CentOS 7.6)...... 239 29.1 Introduction...... 239 29.2 Environment Requirements...... 239 29.3 Configuring the Compilation Environment...... 240 29.3.1 Configuring the Local Yum Source...... 240 29.3.2 Installing OpenJDK...... 241 29.3.3 Installing Maven...... 241 29.4 Compiling JRuby...... 242 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS 7.6)...... 246 30.1 Introduction...... 246 30.2 Requirements...... 246 30.3 Configuring the Compilation Environment...... 247 30.4 Compiling and Installing Netty...... 250 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide (CentOS 7.6)...... 254 31.1 Introduction...... 254 31.2 Requirements...... 254 31.3 Configuring the Compilation Environment...... 255

31.4 Compiling and Installing Netty...... 258 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guide (CentOS 7.6).. 260 32.1 Introduction...... 260 32.2 Requirements...... 260 32.3 Configuring the Compilation Environment...... 261 32.4 Compiling and Installing CMake...... 264 32.5 Installing ninja-build...... 265 32.6 Installing Go...... 265 32.7 Performing Compilation and Installation...... 266 32.8 Troubleshooting...... 268 32.8.1 Failed to Download the go get golang.org/x Package...... 268 32.8.2 Compilation Failed Due to an Alarm Generated During the Compilation...... 268 32.8.3 Compilation Error Occurs Because TLS_ST_SR_CLNT_HELLO Is Not Defined...... 270 33 hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guide (CentOS 7.6)...... 271 33.1 Introduction...... 271 33.2 Requirements...... 271 33.3 Configuring the Compilation Environment...... 272 33.4 Compilation Process...... 276 33.5 Rectifying Faults...... 277 33.5.1 Failed to Download hive-jdbc...... 278 A Change History...... 279

1 CheckSo User Guide

1.1 Introduction 1.2 Environment Requirements 1.3 Scanning Files 1.4 Viewing the Scan Results 1.5 Analyze the Scanning Result

1.1 Introduction

The CheckSo script scans software packages, identifies the x86 .so and binary files and .jar packages that contain x86 .so files, and outputs the scan result.

You can download the CheckSo script from:

https://mirrors.huaweicloud.com/kunpeng/archive/kunpeng_solution/bigdata/ Tools/checkSo.zip

1.2 Environment Requirements

Table 1-1 lists the hardware requirements.

Table 1-1 Hardware requirements

Item Description

CPU Kunpeng 920 processor

Network Ethernet-10GE

Storage SATA 1 TB

Memory 512GB 2400MHz

1.3 Scanning Files

Step 1 Place the folders or compressed files to be checked in a folder (folder different from the directory where the tool script is located). The following uses Spark as an example.

mv spark-2.3.0-bin-hadoop2.7.tgz /opt/test_dir/ mv checkSo /opt/

Step 2 Go to the CheckSo directory. cd /opt/checkSo

Step 3 Run the script. ./main.sh /opt/test_dir

----End

1.4 Viewing the Scan Results

File Description Table 1-2 describes JarResult.log and NonJarResult.log, which contain the scan result.

Table 1-2 Scan result description

File Description

JarResult.log Provides information about the .jar packages that contain x86 .so files, including each .jar package name and .so file names in the .jar package.

NonJarResult.lo Provides information about x86 binary files that contain .so g file, including the full paths of the .so or binary files.

Description of JarResult.log In this example, only the JarResult.log file contains useful information, which means that Spark has only the JAR packages that contain x86 .so files and does not have an independent .so file. Table 1-3 describes the JarResult.log file.

Table 1-3 Description of JarResult.log

JAR Package SO File

commons-crypto-1.0.0.jar libcommons-crypto.so

jline-2.12.1.jar libjansi.so

leveldbjni-all-1.8.jar libleveldbjni.so

JAR Package SO File

netty-all-4.1.17.Final.jar libnetty_transport_native_epoll_x86_64 .so

scala-compiler-2.11.12.jar libjansi.so

spark-2.3.0-yarn-shuffle.jar libcommons-crypto.so

spark-2.3.0-yarn-shuffle.jar libleveldbjni.so

spark-2.3.0-yarn-shuffle.jar libnetty_transport_native_epoll_x86_64 .so

Description of NonJarResult.log The following uses the Ambari package as an example and lists some .so and binary files.

tmp_unzip/ambari-release-2.4.1-aarch64-dist.tar.gz/ambari-release-2.4.1-aarch64/ambari-metrics/ambari- metrics-assembly/target/embedded/hadoop-2.7.1.2.3.4.0-3347/lib/native/libhadoop.so tmp_unzip/ambari-release-2.4.1-aarch64-dist.tar.gz/ambari-release-2.4.1-aarch64/ambari-metrics/ambari- metrics-assembly/target/embedded/hadoop-2.7.1.2.3.4.0-3347/lib/native/libsnappy.so.1 tmp_unzip/ambari-release-2.4.1-aarch64-dist.tar.gz/ambari-release-2.4.1-aarch64/ambari-metrics/ambari- metrics-grafana/target/grafana/grafana-2.6.0/bin/grafana-server tmp_unzip/ambari-release-2.4.1-aarch64-dist.tar.gz/ambari-release-2.4.1-aarch64/ambari-server/target/rpm/ ambari-server/RPMS/aarch64/ambari-server-2.4.1.0-0.aarch64.rpm/usr/lib/ambari-server/lib/ ambari_simplejson/_speedups.so

1.5 Analyze the Scanning Result

JarResult.log Figure 1-1 uses the Spark package as an example to describe the analysis process.

Figure 1-1 Analysis process

As illustrated in Figure 1-1, the following operations need to be performed:

Step 1 Determine the source of the jar package and locate the source code that generates the jar package.

Step 2 If multiple jar packages contain the same .so file (as listed in 1.4 Viewing the Scan Results), check whether the source code has dependency on the jar packages.

For example, spark-2.3.0-yarn-shuffle.jar is compiled from the Spark code, and the pom.xml file that generates the jar package has dependency on commons- crypto-1.0.0.jar, leveldbjni-all-1.8.jar, and netty-all-4.1.17.Final.jar. Therefore, the .so file in spark-2.3.0-yarn-shuffle.jar comes from the three dependent JAR packages.

Step 3 Determine the dependencies to be compiled, as listed in Table 1-4.

Table 1-4 Dependencies to be compiled

JAR Package SO File

commons-crypto-1.0.0.jar libcommons-crypto.so

jline-2.12.1.jar libjansi.so

leveldbjni-all-1.8.jar libleveldbjni.so

netty-all-4.1.17.Final.jar libnetty_transport_native_epoll_x86_64.so

scala-compiler-2.11.12.jar libjansi.so

The jline-2.12.1.jar and scala-compiler-2.11.12.jar packages contain libjansi.so, which is from jline-2.12.1.jar. Therefore, you need to compile jline-2.12.1.jar, used the complied file to replace the file in the Maven repository, and compile scala-compiler-2.11.12.jar. Step 4 Use the compiled file to replace the file in the remote repository created. Delete the x86 JAR package from the local repository. During the compilation, the jar package of the remote repository is automatically obtained or manually replace the package in the local repository.

----End

NonJarResult.log Analyze the modules to which the .so and binary files belong based on the .so and binary files obtained in 1.4 Viewing the Scan Results. Table 1-5 describes the modules, to which the .so and binary files belong.

Table 1-5 Modules containing the .so and binary files Module .so File

hadoop-2.7.1.2.3.4.0-3347 libhadoop.so

hadoop-2.7.1.2.3.4.0-3347 libsnappy.so.1

grafana-2.6.0 grafana-server

ambari-server-2.4.1.0-0.aarch64.rpm _speedups.so

1. Locate the modules (decompressed packages or RPM packages) from which the .so or binary files are scanned. 2. Analyze the source code of the corresponding compressed packages or RPM packages to check whether it has dependencies on other modules. Example: a. After the Ambari Server source code is scanned, it is found that _speedups.so is from ambari-common. b. However, the ambari-common source code does not generate the .so file. Check the src/main/python/ambari_simplejson/README.txt file. It is found that the .so file is generated by the C language.

c. Query the source of the _speedups.so file. It is found that the source is the markupsafe module. 3. Determine the modules to be recompiled (as listed in Table 1-6) in this way.

Table 1-6 Modules to be compiled Module .so File

hadoop-2.7.1.2.3.4.0-3347 libhadoop.so

libsnappy.so.1

grafana-2.6.0 grafana-server

markupsafe-1.1.1 _speedups.so

2 commons-crypto-1.0.0.jar Porting Guide (CentOS 7.6)

2.1 Introduction 2.2 Requirements 2.3 Configuring the Compilation Environment 2.4 Compilation Process 2.5 Rectifying Faults

2.1 Introduction Apache Commons Crypto is a cryptographic library optimized with AES-NI (Advanced Encryption Standard New Instructions). It provides Java API for both cipher level and Java stream level. Developers can use it to implement high performance AES encryption and decryption with the minimum code and effort. For more information about Apache Commons Crypto, visit the official website. Type: encryption

2.2 Requirements

Hardware Requirements Table 2-1 lists the hardware requirements.

Table 2-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Item Description

Storage No requirement

Memory No requirement

OS Requirements Table 2-2 lists the requirements for the operating systems (OSs).

Table 2-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

2.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Step 6 Resolve the -fsigned-char problem (by modifying the GCC). 1. Search for the directory where GCC is located. Generally, the directory is /usr/bin/gcc. command -v gcc 2. Change the name of original GCC file, for example, to gcc-impl. mv /usr/bin/gcc /usr/bin/gcc-impl 3. Create a new GCC file. vi /usr/bin/gcc Add the following information to the file and save the file: #! /bin/sh /usr/bin/gcc-impl -fsigned-char "$@" 4. Add the execute permission for the GCC file. chmod +x /usr/bin/gcc 5. Check whether the GCC is available. gcc --version The installation is successful if information similar to the following is displayed:

Step 7 Resolve the -fsigned-char problem (by modifying G++). 1. Search for the directory where G++ is located. Generally, the directory is /usr/bin/g++. command -v g++ 2. Change the original G++ file name, for example, to g++-impl. mv /usr/bin/g++ /usr/bin/g++-impl 3. Create a new G++ file. vi /usr/bin/g++ Add the following information to the file and save the file: #! /bin/sh /usr/bin/g++-impl -fsigned-char "$@" 4. Add the execute permission for the G++ file. chmod +x /usr/bin/g++ 5. Check whether the G++ file is available. g++ --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK Step 1 Download and decompress the installation package to a directory (for example, /opt/tools/installed). wget https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u252-b09/ OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz tar -zxf OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz mkdir -p /opt/tools/installed/ mv jdk8u252-b09 /opt/tools/installed/ Step 2 Configure Java environment variables. vim /etc/profile Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether OpenJDK is successfully installed. java -version The installation is successful if information similar to the following is displayed:

----End

Installing Maven Step 1 Download and install the installation package to a directory (for example, /opt/ tools/installed). wget https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz tar -zxf apache-maven-3.5.4-bin.tar.gz mv apache-maven-3.5.4 /opt/tools/installed/ Step 2 Modify the Maven environment variables. vim /etc/profile Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

The default local repository directory is ~/.m2/. If you want to change the directory to a specified one, modify the localRepository tag. You do not need to modify this parameter unless otherwise specified. Add the following content to the tag to configure the remote repository (change the repository to the Maven repository that you have built. If the Maven repository does not exist, configure it based on the following example):

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

2.4 Compilation Process

Step 1 Download the commons-crypto-1.0.0 source code package. wget https://github.com/apache/commons-crypto/archive/CRYPTO-1.0.0.tar.gz Step 2 Decompress the source code package. tar -zxvf CRYPTO-1.0.0.tar.gz Step 3 Go to the directory generated after the decompression. cd commons-crypto-CRYPTO-1.0.0 Step 4 Perform compilation. mvn clean install -DskipTests

----End

2.5 Rectifying Faults

2.5.1 Failed to Find the Library Files

Symptom The following error message is displayed during compilation:

Handling Procedure

Step 1 Go to the commons-crypto-CRYPTO-1.0.0 directory. cd commons-crypto-CRYPTO-1.0.0 Step 2 Add the -Wl,-Bdynamic compilation option of G++ to the Makefile file.

----End 2.5.2 Java AES Error

Symptom The following error message is displayed during compilation:

Possible Cause The AES key length required by the test is 256. However, the supported maximum key length is 128.

Handling Procedure

Step 1 Download the jce_policy-8.zip package and decompress it. Then replace the Java library file with the decompressed file. unzip jce_policy-8.zip

Step 2 The decompressed folder contains two JAR packages: local_policy.jar and US_export_policy.jar.

Step 3 Copy local_policy.jar and US_export_policy.jar to $JAVA_HOME/jre/lib/security.

You can check the JAVA_HOME directory in the /etc/profile file.

NO TE

If you replace the static library with a dynamic library, the library may not be found during the running. You can install the corresponding static library to solve the problem.

Step 4 Use the Yum source to install the required glibc and libstdc++-static. yum install libstdc++-static.aarch64 -y yum install glibc* -y

----End 2.5.3 OpenSSL Error

Symptom

An error "Error fatal error:openssl/aes.h: No such or directory" is displayed during compilation. The error is related to OpenSSL.

See the following figure.

Handling Procedure Install openssl.static.aarch64 and then perform compilation and installation.

yum install openssl.aarch64

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 14 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

3 flink-shaded-netty-4.0.27.Final.jar Porting Guide (CentOS 7.6)

3.1 Introduction 3.2 Environment Requirements 3.3 Configuring the Compilation Environment 3.4 Compiling and Installing Software

3.1 Introduction flink-shaded-netty Overview

The flink-shaded-netty module is a big data component.

Recommended Version

4.0.27.Final-2.0

3.2 Environment Requirements

Hardware Requirements

Table 3-1 lists the hardware requirements.

Table 3-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 15 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Item Description

Storage No requirement

Memory No requirement

OS Requirements Table 3-2 lists the requirements for the operating systems (OSs).

Table 3-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

Software Requirements Table 3-3 lists the software requirements.

Table 3-3 Software requirements Item Version How to Obtain

GCC 4.8.5 -

OpenJDK 1.8.0_252 https://github.com/AdoptOpenJDK/ openjdk8-binaries/releases/download/ jdk8u252-b09/OpenJDK8U- jdk_aarch64_linux_hotspot_8u252b09.tar.gz

Maven 3.5.4 https://archive.apache.org/dist/maven/ maven-3/3.5.4/binaries/apache- maven-3.5.4-bin.tar.gz

Apr 1.5.2 https://archive.apache.org/dist/apr/ apr-1.5.2.tar.gz

Netty-tcnative 1.1.32.Fork1 https://codeload.github.com/netty/netty- tcnative/tar.gz/netty-tcnative-1.1.32.Fork1

Netty 4.0.27 https://github.com/netty/netty/archive/ netty-4.0.27.Final.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 16 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

3.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name.

Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 17 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

Step 1 Download and decompress the installation package to a directory (for example, /opt/tools/installed). wget https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u252-b09/ OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz tar -zxf OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz mkdir -p /opt/tools/installed/ mv jdk8u252-b09 /opt/tools/installed/ Step 2 Configure Java environment variables. vim /etc/profile Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether OpenJDK is successfully installed. java -version

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 18 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 1 Download and install the installation package to a directory (for example, /opt/ tools/installed). wget https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz tar -zxf apache-maven-3.5.4-bin.tar.gz mv apache-maven-3.5.4 /opt/tools/installed/ Step 2 Modify the Maven environment variables. vim /etc/profile Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 19 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

3.4 Compiling and Installing Software

Compiling and Installing apr-1.5.2

Step 1 Download the apr-1.5.2 source code package and decompress it. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz tar -zxvf apr-1.5.2.tar.gz

Step 2 Go to the directory generated after the decompression. cd apr-1.5.2

Step 3 Compile and install apr-1.5.2. ./configure make make install

----End

Compiling and installing netty-tcnative-1.1.32.Fork1

Step 1 Download the netty-tcnative-1.1.32.Fork1 source code package. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-1.1.32.Fork1

Step 2 Decompress the source code package. mv netty-tcnative-1.1.32.Fork1 netty-tcnative-1.1.32.Fork1.tar.gz tar -zxvf netty-tcnative-1.1.32.Fork1.tar.gz

Step 3 Switch to the directory where the netty-tcnative source code package is decompressed. cd netty-tcnative-netty-tcnative-1.1.32.Fork1

Step 4 Compile and package the source code, and place the package in the local Maven repository. mvn install

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 20 Kunpeng BoostKit for Big Data 3 flink-shaded-netty-4.0.27.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Compiling and Installing netty-all-4.0.27.Final

Step 1 Download the netty-4.0.27 source code package. wget https://github.com/netty/netty/archive/netty-4.0.27.Final.tar.gz Step 2 Decompress the source code package. tar -zxvf netty-4.0.27.Final.tar.gz Step 3 Switch to the directory where the Netty source code package is decompressed. cd netty-netty-4.0.27.Final Step 4 Compile and package netty-all-4.0.27.Final.jar to the all/target directory. mvn install -DskipTests

----End

Compiling flink-shaded-netty

Step 1 Download the flink-shaded-2.0 installation package. wget https://codeload.github.com/apache/flink-shaded/tar.gz/2.0 Step 2 Decompress the installation package. mv 2.0 flink-shaded-2.0.tar.gz tar -zxvf flink-shaded-2.0.tar.gz Step 3 Go to the directory generated after the decompression. cd flink-shaded-2.0 Step 4 Modify the pom.xml file. You only need to compile and package flink-shaded- netty-4. Therefore, comment out the modules that are not required.

Step 5 Compile and package flink-shaded-netty-4.0.27.Final-2.0.jar to the flink- shaded-netty-4/target/ directory. mvn install package -DskipTests

----End

4 HBase-shaded-netty-2.1.0 Porting Guide (CentOS 7.6)

4.1 Introduction 4.2 Requirements 4.3 Configuring the Compilation Environment 4.4 Compilation HBase-shaded-netty-2.1.0 4.5 Rectifying Faults

4.1 Introduction

Hbase-shaded-netty-2.1.0 is a third-party library required for HBase packaging.

4.2 Requirements

Hardware Requirements

Table 4-1 lists the hardware requirements.

Table 4-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 4-2 lists the requirements for the operating systems (OSs).

Table 4-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

4.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0 Step 4 Make the Yum source configuration take effect. yum clean all yum makecache Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim Step 6 Resolve the -fsigned-char problem (by modifying the GCC). 1. Search for the directory where GCC is located. Generally, the directory is /usr/bin/gcc. command -v gcc

2. Change the name of original GCC file, for example, to gcc-impl. mv /usr/bin/gcc /usr/bin/gcc-impl 3. Create a new GCC file. vi /usr/bin/gcc Add the following information to the file and save the file: #! /bin/sh /usr/bin/gcc-impl -fsigned-char "$@" 4. Add the execute permission for the GCC file. chmod +x /usr/bin/gcc 5. Check whether the GCC is available. gcc --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

Add the following content to the tag to configure the remote repository (change the repository to the Maven repository that you have built. If the Maven repository does not exist, configure it based on the following example):

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

4.4 Compilation HBase-shaded-netty-2.1.0

Step 1 Download the source code. wget https://github.com/apache/hbase-thirdparty/archive/rel/2.1.0.tar.gz

Step 2 Decompress the source code package. tar -zxf 2.1.0.tar.gz

Step 3 Go to the directory generated after the decompression. cd hbase-thirdparty-rel-2.1.0

Step 4 Modify the pom.xml file. vim pom.xml

Add the Kunpeng Maven repository source. kunpengmaven kunpeng maven https://mirrors.huaweicloud.com/kunpeng/maven

Step 5 Change x86_64 to aarch_64 in the hbase-shaded-netty/pom.xml file.

Step 6 Perform compilation. mvn clean install

The compiled package is in the hbase-thirdparty-rel-2.1.0/hbase-shaded-netty/ target directory.

----End

4.5 Rectifying Faults

4.5.1 Failed to Run the patch Command

Symptom

Failed to run the patch command.

Handling Procedure The patch is missing. You need to install it manually. Run the following command to install the patch:

yum -y install patch 4.5.2 Failed to Find libnetty_transport_native_epoll_x86_64.so

Symptom Failed to find libnetty_transport_native_epoll_x86_64.so.

Handling Procedure Change x86_64 to aarch_64 in the hbase-shaded-netty/pom.xml file.

5 Jansi-1.4.jar Porting Guide (CentOS 7.6)

5.1 Introduction 5.2 Requirements 5.3 Configuring the Compilation Environment 5.4 Compiling Jansi-1.4.jar 5.5 Rectifying Faults

5.1 Introduction Jansi is a Java class library for text and image processing.

5.2 Requirements

Hardware Requirements Table 5-1 lists the hardware requirements.

Table 5-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 5-2 lists the requirements for the operating systems (OSs).

Table 5-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

5.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Add the following information to the file and save the file: #! /bin/sh /usr/bin/gcc-impl -fsigned-char "$@" 4. Add the execute permission for the GCC file. chmod +x /usr/bin/gcc 5. Check whether the GCC is available. gcc --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

5.4 Compiling Jansi-1.4.jar

Step 1 Compile the Jansi-native source code. 1. Download the Jansi-native1.0 source code package. wget https://github.com/fusesource/jansi-native/archive/jansi-native-1.0.tar.gz 2. Decompress the source code package. tar -zxvf jansi-native-1.0.tar.gz 3. Go to the directory where the Jansi-native source code is located. cd jansi-native-jansi-native-1.0 4. Perform compilation. mvn install -Dplatform=linux64 package A JAR package is generated in the ./target/jansi-native-1.0-linux64.jar directory.

5. Copy the jansi-native-1.0.jar and jansi-native-1.0-linux64.jar packages to the local Maven repository LOCAL_M2_PATH/org/fusesource/jansi/jansi- native/1.0/. Step 2 Compile the Jansi1.4 source code. 1. Download the Jansi 1.4 source code package from GitHub. wget https://github.com/fusesource/jansi/archive/jansi-1.4.tar.gz

2. Decompress the source code package. tar -zxvf jansi-1.4.tar.gz 3. Go to the directory where the Jansi-1.4 source code is located. cd jansi-jansi-1.4 4. Modify the pom.xml file by commenting out the following information in line 125 to line 148:

5. Perform compilation. mvn clean install A JAR package is generated in the ./target/jansi-1.4.jar directory.

----End

5.5 Rectifying Faults

5.5.1 Error Occurs During Compilation

Symptom

The following error message is displayed during compilation:

[INFO] /usr/share/automake-1.13/am/ltlibrary.am: warning: 'libjansi.la': linking libtool libraries using a non- POSIX [INFO] /usr/share/automake-1.13/am/ltlibrary.am: archiver requires 'AM_PROG_AR' in 'configure.ac' [INFO] Makefile.am:20: while processing Libtool library 'libjansi.la' [INFO] autoreconf: automake failed with exit status: 1

Procedure Find the configure.ac file in the compiled library directory. Open the file by using a text editor and add AM_PROG_AR to the end of the following information.

AC_PROG_CC AC_PROG_INSTALL AC_PROG_LIBTOOL([disable-static])

5.5.2 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed

Symptom Error message "config.status: error: cannot find input file: 'Makefile.in' " is displayed.

Procedure Go to the ./target/native-build directory, run the following commands, and perform the compilation again:

aclocal automake --add-missing

NO TE

The aclocal and automake versions are 1.13.4.

6 Jansi-1.11.jar Porting Guide (CentOS 7.6)

6.1 Introduction 6.2 Requirements 6.3 Configuring the Compilation Environment 6.4 Compilation 6.5 Troubleshooting

6.1 Introduction

Jansi is a Java class library for text and image processing.

6.2 Requirements

Hardware Requirements

Table 6-1 lists the hardware requirements.

Table 6-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 6-2 lists the requirements for the operating systems (OSs).

Table 6-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

6.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

----End

Installing OpenJDK

Step 1 Install the Java Development Kit (JDK) 1.7. yum install -y java-1.7.0-openjdk java-1.7.0-openjdk-devel Step 2 Configure Java environment variables. vim /etc/profile Add the following to the end of the file:

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.261-2.6.22.2.el7_8.aarch64 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether the environment variables take effect. java -version

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

6.4 Compilation

Table 6-3 Jansi version dependencies

Jansi Version Jansi Native Version

Jansi 1.11 Jansi-native 1.5

Step 1 Compile the Jansi-native 1.5 source code. 1. Download the Jansi-native1.5 source code package. wget https://github.com/fusesource/jansi-native/archive/jansi-native-1.5.tar.gz 2. Decompress the source code package. tar -zxvf jansi-native-1.5.tar.gz 3. Go to the directory where the Jansi-native source code is located. cd jansi-native-jansi-native-1.5 4. Perform the compilation. mvn install -Dplatform=linux64 A JAR package is generated in the ./target directory. 5. Copy jansi-linux64-1.5.jar to the local Maven repository LOCAL_M2_PATH/org/fusesource/jansi/jansi-linux64/1.5/.

Step 2 Compile the Jansi 1.11 source code. 1. Download the Jansi 1.11 source code package. wget https://github.com/fusesource/jansi/archive/jansi-project-1.11.tar.gz

2. Decompress the source code package. tar -zxvf jansi-project-1.11.tar.gz 3. Go to the directory where the Jansi-1.11 source code is located. cd jansi-jansi-project-1.11/jansi 4. Compile the source code. mvn install -Dmaven.javadoc.skip=true

A JAR package is generated in the ./jansi/target/jansi-1.11.jar directory.

----End

6.5 Troubleshooting

6.5.1 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed

Symptom

Error message "config.status: error: cannot find input file: 'Makefile.in' " is displayed.

Procedure

Go to the ./target/native-build directory, run the following commands, and perform the compilation again:

aclocal automake --add-missing

NO TE

The aclocal and automake versions are 1.13.4.

6.5.2 Error Occurs During Compilation

Symptom

The following error message is displayed during compilation:

Procedure

Find the configure.ac file in the compiled library directory. Open the file by using a text editor and add AM_PROG_AR to the end of the following information.

AC_PROG_CC AC_PROG_INSTALL AC_PROG_LIBTOOL([disable-static])

6.5.3 java.lang.OutOfMemoryError: PermGen space

Symptom

In the Bootstrapped websit gen for Jansi compilation phase, the following error occurs:

java.lang.OutOfMemoryError: PermGen space

Procedure

Step 1 Add the following content to the /etc/profile file. export MAVEN_OPTS="-Xmx512M -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512M"

Step 2 Make the configuration take effect. source /etc/profile

----End

6.5.4 no description for @param

Symptom

Possible Cause The DocLint feature is added to Java 8. This feature is used to check the Javadoc comment errors before the Javadoc is generated in the development phase and link to the source code. If the comments of the Javadoc are incorrect, the Javadoc is not generated.

Procedure Skip the Javadoc check during compilation, and perform the compilation again.

mvn clean package -Dmaven.javadoc.skip=true

7 Jansi-1.9,1.12,1.16.jar Porting Guide (CentOS 7.6)

7.1 Introduction 7.2 Environment Requirements 7.3 Configuring the Compilation Environment 7.4 Compilation Process 7.5 Rectifying Faults

7.1 Introduction Jansi is a Java class library for text and image processing.

7.2 Environment Requirements

Hardware Requirements Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements Item Version

CentOS 7.6

OS kernel 4.14.0

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

CMake 3.12.4

7.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

1. Search for the directory where GCC is located. Generally, the directory is /usr/bin/gcc. command -v gcc 2. Change the name of original GCC file, for example, to gcc-impl. mv /usr/bin/gcc /usr/bin/gcc-impl 3. Create a new GCC file. vi /usr/bin/gcc Add the following information to the file and save the file: #! /bin/sh /usr/bin/gcc-impl -fsigned-char "$@" 4. Add the execute permission for the GCC file. chmod +x /usr/bin/gcc 5. Check whether the GCC is available. gcc --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

Step 1 Installing OpenJDK yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel

Step 2 Configure the Java environment variables. 1. Open the /etc/profile file. vi /etc/profile 2. Add the following code to the end of the /etc/profile file: export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.aarch64/jre export PATH=$JAVA_HOME/bin:$PATH 3. Press Esc and run the :wq! command to save the file and exit. 4. Make the modified environment variables take effect. source /etc/profile

----End

Installing Maven

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven

huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

7.4 Compilation Process

Table 7-1 Jansi version dependencies Jansi Version Jansi Native Version

Jansi 1.9 Jansi-native 1.3

Jansi 1.12 Jansi-native 1.6

Jansi 1.16 Jansi-native 1.7

Compiling jansi-1.9

Step 1 Compile the Jansi-native source code. 1. Download the Jansi-native1.3 source code package. wget https://github.com/fusesource/jansi-native/archive/jansi-native-1.3.tar.gz 2. Decompress the source code package. tar -zxvf jansi-native-1.3.tar.gz 3. Go to the directory where the Jansi-native source code is located. cd jansi-native-jansi-native-1.3 4. Perform the compilation. mvn -Dplatform=linux64 package A JAR package is generated in the ./target directory. 5. Copy the jansi-native-1.3.jar and jansi-native-1.3-linux64.jar packages to the local Maven repository LOCAL_M2_PATH/org/fusesource/jansi/jansi- native/1.3/. Step 2 Compile the Jansi 1.9 source code. 1. Download the Jansi-1.9 source code package. wget https://github.com/fusesource/jansi/archive/jansi-project-1.9.tar.gz

2. Decompress the source code package. tar -zxvf jansi-project-1.9.tar.gz 3. Go to the directory where the Jansi-1.9 source code is located. cd jansi-jansi-project-1.9 4. Perform the compilation. mvn clean install -Dmaven.javadoc.skip=true A JAR package is generated in the ./jansi/target/jansi-1.9.jar directory.

----End

Compiling jansi-1.12

Step 1 Compile the Jansi-native 1.6 source code. 1. Download the Jansi-native1.6 source code package. wget https://github.com/fusesource/jansi-native/archive/jansi-native-1.6.tar.gz 2. Decompress the source code package. tar -zxvf jansi-native-1.6.tar.gz 3. Go to the directory where the Jansi-native source code is located. cd jansi-native-jansi-native-1.6 4. Perform the compilation. mvn install -Dplatform=linux64 A JAR package is generated in the ./target directory. 5. Copy jansi-linux64-1.6.jar to the local Maven repository LOCAL_M2_PATH/org/fusesource/jansi/jansi-linux64/1.6/. Step 2 Compile the Jansi1.12 source code. 1. Download the Jansi 1.12 source code package. wget https://github.com/fusesource/jansi/archive/jansi-project-1.12.tar.gz 2. Decompress the source code package. tar -zxvf jansi-project-1.12.tar.gz 3. Go to the directory where the Jansi-1.12 source code is located. cd jansi-jansi-project-1.12 4. Perform the compilation. mvn install -Dmaven.javadoc.skip=true A JAR package is generated in the ./jansi/target/jansi-1.12.jar directory. 5. Copy jansi-1.12.jar to the org/fusesource/jansi/jansi/directory of the local repository.

----End

Compiling jansi-1.16

Step 1 Compile the Jansi-native 1.7 source code. 1. Download the Jansi-native1.7 source code package. wget https://github.com/fusesource/jansi-native/archive/jansi-native-1.7.tar.gz 2. Decompress the source code package. tar -zxvf jansi-native-1.7.tar.gz 3. Go to the directory where the Jansi-native source code is located. cd jansi-native-jansi-native-1.7 4. Perform the compilation. mvn install -Dplatform=linux64

A JAR package is generated in the ./target directory. 5. Copy jansi-linux64-1.7.jar to the local Maven repository LOCAL_M2_PATH/org/fusesource/jansi/jansi-linux64/1.7/.

Step 2 Compile the Jansi 1.16 source code. 1. Download the Jansi 1.16 source code package. wget https://github.com/fusesource/jansi/archive/jansi-project-1.16.tar.gz 2. Decompress the source code package. tar -zxvf jansi-project-1.16.tar.gz 3. Go to the directory where the Jansi-1.16 source code is located. cd jansi-jansi-project-1.16 4. Perform the compilation. mvn install -Dmaven.javadoc.skip=true A JAR package is generated in the ./jansi/target/jansi-1.16.jar directory. 5. Copy jansi-1.16.jar to the org/fusesource/jansi/jansi/directory of the local repository.

----End

7.5 Rectifying Faults

7.5.1 Error Message "config.status: error: cannot find input file: 'Makefile.in' " Is Displayed

Symptom

Error message "config.status: error: cannot find input file: 'Makefile.in' " is displayed.

Procedure

Go to the ./target/native-build directory, run the following commands, and perform the compilation again:

aclocal automake --add-missing

NO TE

The aclocal and automake versions are 1.13.4.

7.5.2 Error Occurs During Compilation

Symptom

The following error message is displayed during compilation:

Procedure

Find the configure.ac file in the compiled library directory. Open the file by using a text editor and add AM_PROG_AR to the end of the following information.

AC_PROG_CC AC_PROG_INSTALL AC_PROG_LIBTOOL([disable-static])

7.5.3 java.lang.OutOfMemoryError: PermGen space

Symptom

In the Bootstrapped websit gen for Jansi compilation phase, the following error occurs:

java.lang.OutOfMemoryError: PermGen space

Procedure

Step 1 Add the following content to the /etc/profile file. export MAVEN_OPTS="-Xmx512M -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512M"

Step 2 Make the configuration take effect. source /etc/profile

----End

8 jline-2.10.5.jar Porting Guide (CentOS 7.6)

8.1 Introduction 8.2 Environment Requirements 8.3 Configuring the Compilation Environment 8.4 Compiling JLine

8.1 Introduction

JLine is a Java class library used to process console input.

8.2 Environment Requirements

Hardware Requirements

Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements

Item Version

CentOS 7.6

Item Version

OS kernel 4.14.0

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

CMake 3.12.4

8.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

8.4 Compiling JLine

NO TE

Dependency: jline-2.10.5 --> jansi-1.4 --> jansi-native-1.0

Step 1 Compile jansi-native-1.0 (see the Jansi-1.4.jar Porting Guide (CentOS 7.6)), and copy the generated jansi-native-1.0-linux64.jar file to the /home directory. Step 2 Create the jansi-native-1.0 directory in the /home directory and decompress the jansi-native-1.0-linux64.jar package to the directory. mkdir /home/jansi-native-1.0 cp jansi-native-1.0-linux64.jar /home/jansi-native-1.0 cd /home/jansi-native-1.0 jar -xf jansi-native-1.0-linux64.jar Step 3 Create the jline-2.10.5 directory, and download and decompress jline-2.10.5.jar. mkdir /home/jline-2.10.5 cd /home/jline-2.10.5 wget https://repo1.maven.org/maven2/org/scala-lang/jline/2.10.5/jline-2.10.5.jar jar -xf jline-2.10.5.jar rm -rf jline-2.10.5.jar Step 4 Replace the libjansi.so file of jline-2.10.5 with the libjansi.so file of jansi- native-1.0. cd /home/jline-2.10.5 cp /home/jansi-native-1.0/META-INF/native/linux64/libjansi.so META-INF/native/linux64/libjansi.so

Step 5 Package jline-2.10.5.jar again. cd /home/jline-2.10.5 jar -cf jline-2.10.5.jar ./

----End

NO TE

The value of groupId is org.scala-lang and the value of md5 is 2a9314e4fd985f848e2e170349d2a1cc.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 57 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Guide (CentOS 7.6)

9.1 Introduction 9.2 Requirements 9.3 Configuring the Compilation Environment 9.4 Compilation Process 9.5 Troubleshooting

9.1 Introduction

JLine is a Java class library used to process console input.

9.2 Requirements

Hardware Requirements

Table 9-1 lists the hardware requirements.

Table 9-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 58 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

OS Requirements Table 9-2 lists the requirements for the operating systems (OSs).

Table 9-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

9.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 59 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 60 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 61 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

9.4 Compilation Process

Compiling jline-2.11

NO TE

Dependency: jline-2.11 > jansi-1.11 > jansi-native-1.5

Step 1 Compile jansi-native-1.5 and jansi1.11 by referring to the Jansi-1.11.jar Porting Guide (CentOS 7.6).

NO TICE

During compilation, the command is changed to the following: mvn package -Dmaven.javadoc.skip=true After the compilation is complete, copy the generated jansi-1.11.jar file to the local Maven repository.

Step 2 Download the Jline-2.11 source code package. wget https://github.com/jline/jline2/archive/jline-2.11.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 62 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 3 Decompress the source code package. tar -zxvf jline-2.11.tar.gz Step 4 Go to the directory generated after the decompression. cd jline2-jline-2.11 Step 5 Add the Kunpeng Maven repository in the pom.xml file. vim pom.xml

kunpengmaven kunpeng maven https://mirrors.huaweicloud.com/kunpeng/maven Step 6 Perform compilation. mvn install package A JAR package is generated in the ./target/jline-2.11.jar directory.

----End

Compiling jline-2.12

NO TE

Dependency: jline-2.12 > jansi-1.11 > jansi-native-1.5

Step 1 Compile jansi-native-1.5 and jansi1.11 by referring to the Jansi-1.11.jar Porting Guide (CentOS 7.6).

NO TICE

Step 2 Download the Jline-2.12 source code package. wget https://github.com/jline/jline2/archive/jline-2.12.tar.gz Step 3 Decompress the source code package. tar -zxvf jline-2.12.tar.gz Step 4 Go to the directory generated after the decompression. cd jline2-jline-2.12 Step 5 Perform compilation.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 63 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

mvn install package A JAR package is generated in the ./target/jline-2.12.jar directory.

----End

Compiling jline-2.12.1 NO TE

Dependency: jline-2.12.1 > jansi-1.11 > jansi-native-1.5

Step 1 Compile jansi-native-1.5 and jansi1.11 by referring to the Jansi-1.11.jar Porting Guide (CentOS 7.6).

NO TICE

Step 2 Download the Jline-2.12.1 source code package. wget https://github.com/jline/jline2/archive/jline-2.12.1.tar.gz Step 3 Decompress the source code package. tar -zxvf jline-2.12.1.tar.gz Step 4 Go to the directory generated after the decompression. cd jline2-jline-2.12.1 Step 5 Perform compilation. mvn install package A JAR package is generated in the ./target/jline-2.12.1.jar directory.

----End

Compiling jline-2.13 NO TE

Dependency: jline-2.13 > jansi-1.11 > jansi-native-1.5

Step 1 Compile jansi-native-1.5 and jansi1.11 by referring to the Jansi-1.11.jar Porting Guide (CentOS 7.6).

NO TICE

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 64 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 2 Download the Jline-2.13 source code package. wget https://github.com/jline/jline2/archive/jline-2.13.tar.gz

Step 3 Decompress the source code package. tar -zxvf jline-2.13.tar.gz

Step 4 Go to the directory generated after the decompression. cd jline2-jline-2.13

Step 5 Perform compilation. mvn install package -DskipTests -Dmaven.javadoc.skip=true

A JAR package is generated in the ./target/jline-2.13.jar directory.

----End

Compiling jline-2.14.3

NO TE

Dependency: jline-2.14.3 > jansi-1.12 > jansi-native-1.6

Step 1 Compile jansi-native-1.6 and jansi1.12 by referring to the Jansi-1.11.jar Porting Guide (CentOS 7.6).

NO TICE

During compilation, the command is changed to the following: mvn package -Dmaven.javadoc.skip=true After the compilation is complete, copy the generated jansi-1.12.jar file to the local Maven repository.

Step 2 Download the Jline-2.14.3 source code package. wget https://github.com/jline/jline2/archive/jline-2.14.3.tar.gz

Step 3 Decompress the source code package. tar -zxvf jline-2.14.3.tar.gz

Step 4 Go to the directory generated after the decompression. cd jline2-jline-2.14.3

Step 5 Perform compilation. mvn install -Dmaven.javadoc.skip=true -DskipTests

A JAR package is generated in the ./target/jline-2.14.3.jar directory.

----End

9.5 Troubleshooting

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 65 Kunpeng BoostKit for Big Data 9 jline-2.11, 2.12, 2.12.1, 2.13, 2.14.3.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

9.5.1 Failed to Download doxia-module-apt-1.6.jar During the Compilation of jline-2.14.3

Symptom A message is displayed indicating that the doxia-module-apt-1.6.jar fails to be downloaded during the compilation of jline-2.14.3.

Procedure Manually download the JAR package and pom.xml file to the local repository.

wget https://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-module-apt/1.6/doxia-module- apt-1.6.jar wget https://repo1.maven.org/maven2/org/apache/maven/doxia/doxia-module-apt/1.6/doxia-module- apt-1.6.pom

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 66 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide (CentOS 7.6)

10.1 Introduction 10.2 Environment Requirements 10.3 Configuring the Compilation Environment 10.4 Performing Porting Analysis 10.5 Compiling Jython 10.6 Troubleshooting

10.1 Introduction

Jython Overview Jython is a complete programming language — not a Java translator and not just a Python compiler, but a complete implementation of Python in Java. Jython has a sizable library of modules inherited from CPython. Unlike CPython and many other high-level languages, Jython gives you access to everything in its implementation language, Java. So, not only does Jython give you the Python libraries, it also gives you all available Java classes.

Recommended Version jython-standalone-2.7.0

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 67 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

10.2 Environment Requirements

Hardware Requirements Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements Item Version

CentOS 7.6

OS kernel 4.14.0

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

CMake 3.12.4

10.3 Configuring the Compilation Environment

Installing Dependencies Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 68 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 69 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

5. Check whether the G++ file is available. g++ --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

----End

Installing Maven

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 70 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

10.4 Performing Porting Analysis Use the Kunpeng Porting Advisor to scan the software package to be ported and obtain information about the third-party dependencies to be ported. For details,

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 71 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

see the Kunpeng Porting Advisor Case Study. Table 10-1 lists the third-party dependencies to be ported.

Table 10-1 Third-party dependencies to be ported

Original JAR Package SO File

jffi-arm-Linux.jar libjffi-1.2.so

jffi-i386-Linux.jar libjffi-1.2.so

jffi-i386-SunOS.jar libjffi-1.2.so

jffi-sparcv9-SunOS.jar libjffi-1.2.so

jffi-x86_64-FreeBSD.jar libjffi-1.2.so

jffi-x86_64-Linux.jar libjffi-1.2.so

jffi-x86_64-SunOS.jar libjffi-1.2.so

jline-2.12.1.jar libjansi.so

jython.jar libjansi.so

jython.jar libjffi-1.2.so

jython-standalone.jar libjansi.so

jython-standalone.jar libjffi-1.2.so

10.5 Compiling Jython

10.5.1 Compiling jython-standalone-2.7.0

Compiling jline-2.12.1.jar Perform compilation jline-2.12.1.jar by referring to the jline-2.11,2.12,2.12.1,2.14.3.jar Porting Guide (CentOS 7.6)

Compiling jffi-arm-Linux.jar

Step 1 Download jffi-1.2.19.tar.gz. wget https://github.com/jnr/jffi/archive/jffi-1.2.19.tar.gz Step 2 Decompress jffi-1.2.19.tar.gz. tar -xf jffi-1.2.19.tar.gz cd jffi-jffi-1.2.19 Step 3 Compile the source code. ant jar && ant archive-platform-jar && mvn package Step 4 After the compilation is successful, use jffi-aarch64-Linux.jar in the dist directory to replace jffi-arm-Linux.jar in the jython-2.7.0/extlibs directory.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 72 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

cp jffi-aarch64-Linux.jar jython-2.7.0/extlibs/jffi-arm-Linux.jar

----End

Compiling jython-2.7.0

Step 1 Download the source code. wget https://github.com/jythontools/jython/archive/v2.7.0.tar.gz

Step 2 Decompress the source code package. tar -xf v2.7.0.tar.gz

Step 3 Go to the directory where the Jython source code package is decompressed. mv frozen-mirror-2.7.0 jython-2.7.0 cd jython-2.7.0

NO TICE

Copy the jline-2.12.1.jar file generated in Compiling jline-2.12.1.jar to the extlibs directory of jython-2.7.0. cp jline-2.12.1.jar jython-2.7.0/extlibs/

Step 4 Compile the source code. ant jar-standalone

After the compilation is successful, the dist directory is generated.

----End 10.5.2 Compiling jython-standalone-2.7.1

Compiling jline-2.14.3.jar

Perform compilation by referring to the jline-2.11,2.12,2.12.1,2.14.3.jar Porting Guide (CentOS 7.6)

Compiling jffi-arm-Linux.jar

Step 1 Download jffi-1.2.15.tar.gz. wget https://github.com/jnr/jffi/archive/jffi-1.2.15.tar.gz

Step 2 Decompress the jffi-1.2.15 package and go to the directory generated after the decompression. tar -xf jffi-1.2.15.tar.gz cd jffi-jffi-1.2.15

Step 3 Compile the source code. ant jar && mvn package

Step 4 After the compilation is successful, use jffi-aarch64-Linux.jar in the dist directory to replace jffi-arm-Linux.jar in the jython-2.7.1/extlibs directory. cp dist/jffi-aarch64-Linux.jar jython-2.7.1/extlibs/jffi-arm-Linux.jar

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 73 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Compiling jython-2.7.1

Step 1 Download the source code. wget https://github.com/jythontools/jython/archive/v2.7.1.tar.gz

Step 2 Decompress the source code package. tar -xf v2.7.1.tar.gz

Step 3 Go to the directory where the Jython source code package is decompressed. mv frozen-mirror-2.7.1 jython-2.7.1 cd jython-2.7.1

NO TICE

Copy the jline-2.14.3.jar file generated in Compiling jline-2.14.3.jar to the extlibs directory of jython-2.7.1. cp jline-2.14.3.jar jython-2.7.1/extlibs/

Step 4 Compile the source code. ant jar-standalone

After the compilation is successful, the dist directory is generated.

----End

10.6 Troubleshooting

10.6.1 Failure of the First Compilation

Symptom

The first compilation fails.

Solution

After the first compilation fails, run the ant jar-standalone command again. The second compilation will be successful. 10.6.2 Failed to Replacing the jython-standalone-2.7.0 Dependencies Identified by the Tool

Symptom

The jython-standalone-2.7.0 dependencies identified by the Kunpeng Porting Advisor failed to be replaced.

Solution

Replace the related JAR packages before compilation.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 74 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 1 Use the jline-2.12.1.jar file compiled on the ARM-based server to replace the jline-2.12.1.jar file in the extlibs directory of Jython. Step 2 Use jffi-aarch64-Linux.jar in the dist directory of jffi-1.2.19 to replace jffi-arm- Linux.jar in the jython-2.7.0/extlibs directory. Step 3 After the replacement, run the compilation command twice. ant jar-standalone Step 4 After the JAR package is replaced, use the tool to scan the new package again. The figure below shows the result (the jffi packages for other CPU architectures are not replaced).

----End 10.6.3 Failed to Replacing the jython-standalone-2.7.1 Dependencies Identified by the Tool

Symptom The jython-standalone-2.7.0 dependencies identified by the Kunpeng Porting Advisor failed to be replaced.

Solution Replace the related JAR packages before compilation.

Step 1 Use the jline-2.14.3.jar file compiled on the ARM-based server to replace the jline-2.14.3.jar file in the extlibs directory of Jython. Step 2 Use jffi-aarch64-Linux.jar in the dist directory of jffi-1.2.19 to replace jffi-arm- Linux.jar in the jython-2.7.1/extlibs directory. Step 3 After the replacement, run the compilation command twice. ant jar-standalone Step 4 After the JAR package is replaced, use the tool to scan the new package again. The figure below shows the result (the jffi packages for other CPU architectures are not replaced).

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 75 Kunpeng BoostKit for Big Data 10 jython-standalone-2.7.0, 2.7.1.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

----End

11 leveldbjni-all-1.8.jar Porting Guide (CentOS 7.6)

11.1 Introduction 11.2 Environment Requirements 11.3 Configuring the Compilation Environment 11.4 Compilation Process 11.5 Performing the Verification 11.6 Rectifying Faults

11.1 Introduction LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

For more information about Leveldb, visit the official website.

Type: jni

11.2 Environment Requirements

Hardware Requirements

Table 11-1 lists the hardware requirements.

Table 11-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Item Description

Storage No requirement

Memory No requirement

OS Requirements Table 11-2 lists the requirements for the operating systems (OSs).

Table 11-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

11.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

----End

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Compiling snappy-1.1.5

NO TE

Before compiling the leveldbjni-all-1.8.jar package, compile Snappy and LevelDB into static libraries. Compile the two static libraries into leveldbjni.so, and then the leveldbjni- all-1.8.jar package is generated.

Step 1 Download the source code. wget https://github.com/google/snappy/archive/1.1.5.tar.gz Step 2 Decompress the source code package. tar -zxvf 1.1.5.tar.gz Step 3 Switch to the directory where the source code package is decompressed. cd snappy-1.1.5 Step 4 Perform compilation. ./autogen.sh ./configure --with-pic make -j20

Step 5 After the compilation is complete, the libsnappy.so and libsnappy.a files are generated in the snappy-1.1.5/.libs directory. Copy the libsnappy.a file to the upper-level directory. cp .libs/libsnappy.a ./

Step 6 Set SNAPPY_HOME. export SNAPPY_HOME=`pwd`

NO TE

Copy the static library to the corresponding directory to prepare for compiling the static library into leveldbjni.so.

----End

Compiling leveldb-1.20

Step 1 Download the source code. wget https://github.com/google/leveldb/archive/v1.20.tar.gz

Step 2 Decompress the source code package. tar -zxvf v1.20.tar.gz

Step 3 Switch to the directory where the source code package is decompressed. cd leveldb-1.20

Step 4 Modify the build_detect_platform file. vim build_detect_platform

Modify lines 250 to 252. The content in lines 250 to 251 is modified, and the content in line 252 is added.

echo "PLATFORM_CCFLAGS=$PLATFORM_CCFLAGS $PLATFORM_SHARED_CFLAGS" >> $OUTPUT echo "PLATFORM_CXXFLAGS=$PLATFORM_CXXFLAGS $PLATFORM_SHARED_CFLAGS" >> $OUTPUT echo "PLATFORM_SHARED_CFLAGS=" >> $OUTPUT

Step 5 Apply the corresponding patch to the code by referring to 11.6.1 error: 'class leveldb::DB' has no member named 'ResumeCompactions', and then perform compilation. make -j20

Step 6 After the compilation is complete, the dynamic library libleveldb.so and other binary files are generated in the out-shared directory, and the libleveldb.a static library and other binary files are generated in the out-static directory. Copy the static library in the out-static directory to the current directory and set LEVELDB_HOME.

export LEVELDB_HOME=`pwd` cp out-static/libleveldb.a ./

NO TE

Copy the static library to the corresponding directory to prepare for compiling the static library into leveldbjni.so.

----End

11.4 Compilation Process

Step 1 Download the source code. git clone https://github.com/fusesource/leveldbjni.git Step 2 Go to the current directory and set LEVELDBJNI_HOME. cd leveldbjni export LEVELDBJNI_HOME=`pwd` Step 3 Modify the pom.xml file. 1. Modify the pom.xml file in the current directory. a. Comment out irrelevant OS types (freebsd64, sunos64-amd64, sunos64- sparcv9, and linux64-ppc64le) to avoid compilation failures. You can comment out the corresponding contents by using the following method: Add "" at the end of these lines.

b. Add lines 60 and 61 to add the leveldbjni-all and linux64 modules to the target module.

2. Modify line 77 in the leveldbjni-linux64-aarch64/pom.xml file, as shown in the following figure.

3. Modify the leveldbjni-all/pom.xml file. a. Comment out irrelevant OS types (freebsd64, sunos64-amd64, sunos64- sparcv9, and linux64-ppc64le) to avoid compilation failures. Comment out the dependencies related to "dependency".

b. Comment out the native code part of the build by doing the following operations: Copy the SunOS, FreeBSD, and processor = ppc64le part (line 154 to line 157); paste it behind the Bundle-Native Code tags, and comment out this part. Then perform the modification as shown in the following figure and change "x86-64" to "aarch64". By doing this, the .so file in the corresponding linux64 directory in leveldbjni-all-1.8.jar is also "aarch64". During the startup of the NodeManager of Ambari, the .so file in this directory is used by default.

4. Change the version numbers in all pom.xml files from 99-master-SNAPSHOT to 1.8. sed -i 's/99-master-SNAPSHOT/1.8/g' `find . -name pom.xml` Step 4 The following five modules are automatically compiled: leveldbjni-project, leveldbjni, leveldbjni-linux64, leveldbjni-linux64-aarch64 and leveldbjni-all.

mvn clean install package -P download -P linux64-aarch64 -DskipTests ----End

11.5 Performing the Verification Step 1 Check the compilation result of leveldbjni-all-1.8.jar. find . -name *.jar

Step 2 Verify whether the .so file is correct. Decompress the package and check whether the CPU architecture of the .so file in the directory corresponding to the JAR package is correct.

Step 3 Verify the static library. 1. Check whether libleveldbjni.so contains the static libraries libsnappy.a and libleveldb.a. According to the result, the dynamic library links of the snappy and leveldb modules are not included. ldd aarch64/libleveldbjni.so ldd linux64/libleveldbjni.so

2. Select a function in leveldb.so. View the symbol table of leveldbjni.so, and check whether the corresponding function exists in the .so file. nm aarch64/libleveldbjni.so | grep ResumeCompactions nm linux64/libleveldbjni.so | grep ResumeCompactions

The result shows that the two .so files contain the function "TT" that correspond to the functions in the leveldb.so file.

NO TE

– "T" indicates that the symbol is defined in the current file. – "U" indicates that the symbol is defined in another file. As a result, the system searches the .so file in the /usr/lib64 directory.

----End

11.6 Rectifying Faults

11.6.1 error: 'class leveldb::DB' has no member named 'ResumeCompactions'

Symptom

The following error message is displayed during compilation:

[INFO] src/leveldbjni.cpp: In function 'void Java_org_fusesource_leveldbjni_internal_NativeDB_00024DBJNI_ResumeCompactions(JNIEnv*, jclass, jlong)': [INFO] src/leveldbjni.cpp:426:35: error: 'class leveldb::DB' has no member named 'ResumeCompactions' [INFO] ((leveldb::DB *)(intptr_t)arg0)->ResumeCompactions(); [INFO] src/leveldbjni.cpp: In function 'void Java_org_fusesource_leveldbjni_internal_NativeDB_00024DBJNI_SuspendCompactions(JNIEnv*, jclass, jlong)': [INFO] src/leveldbjni.cpp:435:35: error: 'class leveldb::DB' has no member named 'SuspendCompactions'

Possible Cause

This error is reported when the leveldbjni-all-1.8.jar package is compiled. The two native interfaces are added to the Java source code of the JAR package to invoke the methods in the underlying leveldb.so. However, the leveldb.so source code is not modified accordingly.

Procedure

Manually integrate the patch of the leveldb.so source code. For details, see GitHub.

Only one patch is submitted on the patch submission page. In the previous code submission, another patch needs to be integrated.

The modification method is as follows:

Step 1 Add lines 127 to 131 and lines 1479 to 1511 in the db/db_impl.cc file. // add begin suspend_cv(&suspend_mutex), suspend_count(0), suspended(false), // add end

// add begin void DBImpl::SuspendCompactions() { MutexLock l(& suspend_mutex); env_->Schedule(&SuspendWork, this); suspend_count++; while( !suspended ) { suspend_cv.Wait(); } } void DBImpl::SuspendWork(void* db) { reinterpret_cast(db)->SuspendCallback(); } void DBImpl::SuspendCallback() { MutexLock l(&suspend_mutex); Log(options_.info_log, "Compactions suspended"); suspended = true; suspend_cv.SignalAll(); while( suspend_count > 0 ) { suspend_cv.Wait(); } suspended = false; suspend_cv.SignalAll(); Log(options_.info_log, "Compactions resumed"); } void DBImpl::ResumeCompactions() { MutexLock l(&suspend_mutex); suspend_count--; suspend_cv.SignalAll(); while( suspended ) { suspend_cv.Wait(); } } // add end

Step 2 Add lines 45 to 48 and lines 141 to 148 in the db/db_impl.h file. // add begin virtual void SuspendCompactions(); virtual void ResumeCompactions(); // add end

// add begin port::Mutex suspend_mutex; port::CondVar suspend_cv; int suspend_count; bool suspended; static void SuspendWork(void* db); void SuspendCallback(); // add end

Step 3 Add lines 1870 to 1873 in the db/db_test.cc file. // add begin virtual void SuspendCompactions() {} virtual void ResumeCompactions() {} // add end

Step 4 Add lines 144 to 147 in the include/leveldb/db.h file. // add begin virtual void SuspendCompactions() = 0; virtual void ResumeCompactions() = 0; // add end

NO TE

If the include/leveldb/db.h file is not modified, the error is still reported even if the interface is not exposed to the upper-layer Java for invoking.

----End 11.6.2 error: required file 'autotools/compile' not found

Symptom

The following error message is displayed during compilation:

Possible Cause

The compile module is missing during compilation.

Procedure

Step 1 Go to the leveldbjni-linux64/target/native-build directory and perform the installation. cd leveldbjni-linux64/target/native-build automake --add-missing Step 2 Return to the previous source code directory. cd - Step 3 Perform compilation. mvn package -P download -P linux64-aarch64 -DskipTests

----End 11.6.3 error: 'const char* leveldb::Slice::data_' is private

Symptom The following error message is displayed during compilation: leveldb/include/leveldb/slice.h:80:15: error: 'const char* leveldb::Slice::data_' is private [INFO] const char* data_; [INFO] src/leveldbjni_structs.cpp:256:78: error: within this context [INFO] env->SetLongField(lpObject, NativeSliceFc.data_, (jlong)(intptr_t)lpStruct->data_);

Possible Cause If an error occurs in the private variable during compilation, comment out the keyword "private".

Procedure Modify the source code of the include/leveldb/slice.h file in the LevelDB project to set the variables to public variables.

11.6.4 java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path

Symptom The following error is reported during compilation: "java.lang.UnsatisfiedLinkError: Could not load library". Reasons: [no leveldbjni64-1.8 in java.library.path

java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path, no leveldbjni-1.8 in java.library.path, no leveldbjni in java.library.path, /var/lib/ambari-agent/tmp/

hadoop_java_io_tmpdir/libleveldbjni-64-1-6674485803454070000.8: libleveldb.so.1: cannot open shared object file: No such file or directory

When an Ambari cluster is deployed, NodeManager cannot be started during service startup, and a message is displayed indicating that the corresponding .so file cannot be found.

Possible Cause 1. The libleveldbjni.so file is missing in the java.library.path directory of the system. java -XshowSettings:properties

The error persists after libleveldbjni.so is copied to the java.library.path directory. The focus is shifted to leveldb.so.1. 2. Analyze the reason why leveldb.so.1 is missing. According to the stack information, the following process is performed when NodeManager is started: a. The Hadoop code calls YarnShuffleService in the Spark code. b. The third-party JAR package leveldbjni-all-1.8.jar is loaded when LevelDB is initialized by Spark code. c. In leveldbjni-all-1.8.jar, the LevelDB module is called through the JNI module (implemented in C and packed into leveldb.so by default). d. The jni module in the levedbjni-all-1.8.jar package is packed into the libleveldbjni.so file. Therefore, the general process of identifying the .so file that is missing is as follows:

Compile leveldbjni-all-1.8.jar again. Search for the installation directory that contains the JAR package on the faulty agent node and replace the JAR package. find /usr/hdp -name leveldbjni-all-1.8.jar

Ensure that the symbol table of libleveldbjni.so in the compiled JAR package contains information libleveldb.so. nm libleveldbjni.so | grep SuspendCompactions

After the replacement, the problem persists. More and more logs are checked during this process, and no problem is found. 3. Write a demo program to test the regenerated JAR package. Based on the stack information, extract the LevelDBProvider.java code from Spark, and use the JNI to call the method in leveldb.so to test whether the JAR package can pass. Obtain LevelDBProvider.java from https://github.com/ apache/spark/blob/v2.2.0/common/network-common/src/main/java/org/ apache/spark/network/util/LevelDBProvider.java. Copy the recompiled leveldbjni-all-1.8.jar package to the directory where LevelDBProvider.java is stored and run the following commands to test the JAR package: javac -cp leveldbjni-all-1.8.jar LevelDBProvider.javajava -Djava.ext.dirs=./ LevelDBProvider The preceding analysis shows that the JAR package is not faulty. The test is passed on several agent nodes of the x86 and TaiShan servers. The test is also passed on other non-cluster computers. 4. Add a new node and test the startup of NodeManager on the new node. After a new node is added and NodeManager is installed, the RPM packages related to Spark and Ranger need to be installed during startup again. After the installation, the problem described in this section persists. This indicates that there may be leveldbjni-all-1.8.jar packages in other places and these packages contain incorrect .so files. During the startup, the RPM package for Spark is required, and the stack shows that the JNI is called from Spark. So, decompress the spark2_3_1_0_0_78-yarn- shuffle-2.3.2.3.1.0.0-78.aarch64.rpm package. The spark-2.3.2.3.1.0.0-78- yarn-shuffle.jar file contains libleveldbjni.so, and the JAR package is stored in the /usr/hdp/3.1.0.0-78/spark2/aux/ installation directory on the agent node.

find /usr/hdp -name *spark*.jar

Decompress the JAR package and check the .so file in the Linux64 directory. It is found that libleveldbjni.so in the JAR package dynamically links libleveldb.so.1. Check the symbol table. TU is also displayed. ldd libleveldbjni.so

nm libleveldbjni.so | grep SuspendCompactions Therefore, the problem is caused by libleveldb.so.1 that libleveldbjni.so in spark-2.3.2.3.1.0.0-78-yarn-shuffle.jar is dynamically linked to.

5. Conclusion: The .so file is dynamically linked with libleveldb.so.1. Therefore, the libleveldb.so.1 file must be stored in the /usr/lib64 directory for the service to properly run. Check the symbol table and we can find that the symbol types are "TU". "U" indicates that the symbol is not defined in this .so file and needs to be obtained from another file. The tracing result shows that the faulty .so file is from the V1.0 version. In this version, the libleveldb.so.1 file is not loaded to the libleveldbjni.so file as a static library.

Procedure In the Maven repository, replace the faulty leveldbjni-all-1.8.jar package with the correct one. Recompile Spark and pack the RPM package. To quickly verify the correctness, copy the recompiled spark-2.3.2.3.1.0.0-78-yarn-shuffle.jar file to the /usr/hdp/3.1.0.0-78/spark2/aux/ directory and restart the NodeManager service. The service can be started properly now.

12 lz4-java-x.x.x.jar Porting Guide (CentOS 7.6)

12.1 Introduction 12.2 Requirements 12.3 Configuring the Compilation Environment 12.4 Compiling lz4-java 12.5 Rectifying Faults

12.1 Introduction lz4-java is a compression algorithm applicable to Java based on Yann Collet's work available at http://code.google.com/p/lz4/. This library provides access to two compression methods that both generate a valid LZ4 stream: ● Fast scan (LZ4): – Low memory footprint (about 16 KB) – Very fast (fast scan with skipping heuristics in case the input looks incompressible) – Reasonable compression ratio (depending on the redundancy of the input) ● High compression (LZ4 HC): – Medium memory footprint (about 256 KB) – Rather slow (about 10 times slower than LZ4) – Good compression ratio (depending on the size and the redundancy of the input) The streams produced by those 2 compression algorithms use the same compression format with fast decompressing speed, and can be decompressed by the same decompressor instance. Type: decompression

For more information about lz4-java, visit the LZ4 official website.

12.2 Requirements

Hardware Requirements

Table 12-1 lists the hardware requirements.

Table 12-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements

Table 12-2 lists the requirements for the operating systems (OSs).

Table 12-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

12.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name.

Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Add the following information to the file and save the file: #! /bin/sh /usr/bin/g++-impl -fsigned-char "$@" 4. Add the execute permission for the G++ file. chmod +x /usr/bin/g++ 5. Check whether the G++ file is available. g++ --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

12.4 Compiling lz4-java

NO TE

The following uses lz4-java-1.2.0.jar as an example. When compiling lz4-java-1.3.0.jar, you only need to change the version number.

Installing ant-1.9.14

Step 1 Download the ant-1.9.14 installation package from the official website. wget https://archive.apache.org/dist/ant/binaries/apache-ant-1.9.14-bin.zip

Step 2 Decompress the installation package. unzip apache-ant-1.9.14-bin.zip

Step 3 Install Ant to a directory (for example, /opt/tools/installed). mv apache-ant-1.9.14 /opt/tools/installed/

Step 4 Configure Ant environment variables. 1. Open the /etc/profile file. vim /etc/profile 2. Add the following code to the end of the /etc/profile file: export ANT_HOME=/opt/tools/installed/apache-ant-1.9.14 export PATH=$ANT_HOME/bin:$PATH 3. Press Esc and run the :wq! command to save the file and exit. 4. Make the environment variables take effect. source /etc/profile

Step 5 Check the Ant version. ant -version

----End

Compiling lz4-java

Step 1 Download the source code. wget https://github.com/lz4/lz4-java/archive/1.2.0.zip

Step 2 Decompress the source code package. unzip 1.2.0.zip

Step 3 Go to the directory generated after the decompression. cd lz4-java-1.2.0

Step 4 Modify the build.xml file. vim lz4-java-1.2.0/build.xml 1. Change "http" in line 60 to "https".

2. Change the ivy version number in line 29 to 2.5.0.

Step 5 Compile lz4-java. ant ivy-bootstrap ant The compiled package is in the dist folder of the root directory of compilation.

NO TICE

Before compilation, ensure that the /root/.ivy2/ directory is empty.

----End

12.5 Rectifying Faults

12.5.1 "BUILD FAILED" Is Displayed During Compilation

Symptom The following error message is displayed during compilation:

Procedure This problem occurs because ivy-2.5.0.jar is not found. Download ivy-2.5.0.jar.

ant ivy-bootstrap

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 101 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

13 netty-all-4.0.23.Final,4.0.28.Final, 4.0.29.Final.jar Porting Guide (CentOS 7.6)

13.1 Introduction 13.2 Requirements 13.3 Configuring the Compilation Environment 13.4 Compiling Netty

13.1 Introduction Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

13.2 Requirements

Hardware Requirements Table 13-1 lists the hardware requirements.

Table 13-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 102 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

OS Requirements Table 13-2 lists the requirements for the operating systems (OSs).

Table 13-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

13.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 103 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 104 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 105 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

13.4 Compiling Netty

Compiling netty-all-4.0.23.Final.jar

Step 2 Install netty-tcnative-1.1.30.Fork2. 1. Download the netty-tcnative-1.1.30.Fork2 source code. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-1.1.30.Fork2 mv netty-tcnative-1.1.30.Fork2 netty-tcnative-1.1.30.Fork2.tar.gz 2. Decompress the source code package. tar -zxvf netty-tcnative-1.1.30.Fork2.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 106 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

3. Go to the directory generated after the decompression. cd netty-tcnative-netty-tcnative-1.1.30.Fork2 4. Compile and package the file to the local Maven repository. mvn install Step 3 Install netty-4.0.23. 1. Download the netty-4.0.23 source code. wget https://github.com/netty/netty/archive/netty-4.0.23.Final.tar.gz 2. Decompress the source code package. tar -zxvf netty-4.0.23.Final.tar.gz 3. Go to the directory where the source code package is decompressed. cd netty-netty-4.0.23.Final 4. Compile and generate netty-all-4.0.23.Final.jar in the all/target directory, and netty-transport-native-epoll-4.0.23.Final.jar and netty-transport- native-epoll-4.0.23.Final-linux-aarch64_64.jar in the transport-native- epoll/target directory. mvn package -DskipTests

----End

Compiling netty-all-4.0.28.Final Step 1 Install apr-1.5.2. 1. Download the apr-1.5.2 source code. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz 2. Decompress the source code package. tar -zxvf apr-1.5.2.tar.gz 3. Go to the directory generated after the decompression. cd apr-1.5.2 4. Compile and install apr-1.5.2. ./configure make make install Step 2 Install netty-tcnative-1.1.33.Fork2. 1. Download the netty-tcnative-1.1.33.Fork2 source code. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-1.1.33.Fork2 mv netty-tcnative-1.1.33.Fork2 netty-tcnative-1.1.33.Fork2.tar.gz 2. Decompress the source code package. tar -zxvf netty-tcnative-1.1.33.Fork2.tar.gz 3. Go to the directory generated after the decompression. cd netty-tcnative-netty-tcnative-1.1.33.Fork2 4. Compile and package the file to the local Maven repository. mvn install Step 3 Install netty-4.0.28. 1. Download the netty-4.0.28 source code. wget https://github.com/netty/netty/archive/netty-4.0.28.Final.tar.gz 2. Decompress the source code package. tar -zxvf netty-4.0.28.Final.tar.gz 3. Go to the directory where the source code package is decompressed. cd netty-netty-4.0.28.Final 4. Compile and generate netty-all-4.0.28.Final.jar in the all/target directory, and netty-transport-native-epoll-4.0.28.Final.jar and netty-transport-

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 107 Kunpeng BoostKit for Big Data 13 netty-all-4.0.23.Final,4.0.28.Final,4.0.29.Final.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

native-epoll-4.0.28.Final-linux-aarch64_64.jar in the transport-native- epoll/target directory. mvn package -DskipTests

----End

Compiling netty-all-4.0.29.Final.jar

Step 1 Install apr-1.5.2. 1. Download the apr-1.5.2 source code. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz 2. Decompress the source code package. tar -zxvf apr-1.5.2.tar.gz 3. Go to the directory generated after the decompression. cd apr-1.5.2 4. Compile and install apr-1.5.2. ./configure make make install Step 2 Install netty-tcnative-1.1.33.Fork3. 1. Download the netty-tcnative-1.1.33.Fork3 source code package. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-1.1.33.Fork3 mv netty-tcnative-1.1.33.Fork3 netty-tcnative-1.1.33.Fork3.tar.gz 2. Decompress the source code package. tar -zxvf netty-tcnative-1.1.33.Fork3.tar.gz 3. Go to the directory generated after the decompression. cd netty-tcnative-netty-tcnative-1.1.33.Fork3 4. Compile and package the file to the local Maven repository. mvn install Step 3 Install netty-4.0.29. 1. Download the netty-4.0.29 source code. wget https://github.com/netty/netty/archive/netty-4.0.29.Final.tar.gz 2. Decompress the source code package. tar -zxvf netty-4.0.29.Final.tar.gz 3. Go to the directory where the source code package is decompressed. cd netty-netty-4.0.29.Final 4. Compile and generate netty-all-4.0.29.Final.jar in the all/target directory, and netty-transport-native-epoll-4.0.29.Final.jar and netty-transport- native-epoll-4.0.29.Final-linux-aarch64_64.jar in the transport-native- epoll/target directory. mvn package -DskipTests

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 108 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

14 netty-all-4.0.52.Final.jar Porting Guide (CentOS 7.6)

14.1 Introduction 14.2 Requirements 14.3 Configuring the Compilation Environment 14.4 Compiling netty-all-4.0.52.Final.jar

14.1 Introduction Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

14.2 Requirements

Hardware Requirements Table 14-1 lists the hardware requirements.

Table 14-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 109 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

OS Requirements Table 14-2 lists the requirements for the operating systems (OSs).

Table 14-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

14.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 110 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 111 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 112 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

14.4 Compiling netty-all-4.0.52.Final.jar

Installing APR

Step 1 Download the apr-1.5.2 source code. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz

Step 2 Decompress apr-1.5.2.tar.gz. tar -xvzf apr-1.5.2.tar.gz

Step 3 Go to the apr-1.5.2 directory. cd apr-1.5.2

Step 4 Install APR to the /usr/local/apr directory. ./configure --prefix=/usr/local/apr make && make install

----End

Compiling and Installing netty-tcnative-parent-2.0.6.Final

Step 1 Download the netty-tcnative-parent-2.0.6.Final source code. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-parent-2.0.6.Final mv netty-tcnative-parent-2.0.6.Final netty-tcnative-parent-2.0.6.Final.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 113 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Decompress the source code package. tar -zxvf netty-tcnative-parent-2.0.6.Final.tar.gz

Step 3 Go to the directory generated after the decompression. cd netty-tcnative-netty-tcnative-parent-2.0.6.Final

Step 4 Modify the pom.xml file to shield the compilation of BoringSSL.

Step 5 Change the APR reference directory to the installation directory. vim /opt/tools/netty-tcnative-netty-tcnative-parent-2.0.6.Final/openssl-dynamic/pom.xml

Step 6 Compile and package the file to local Maven repository. mvn install

Step 7 In the pom.xml file, comment out the following two parts for downloading the apr-1.6.2 package.

Position 1:

Position 2:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 114 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 8 Download the apr-1.6.2.tar.gz package to the following directory: wget https://archive.apache.org/dist/apr/apr-1.6.2.tar.gz mv apr-1.6.2.tar.gz netty-tcnative-netty-tcnative-parent-2.0.6.Final/openssl-static/target Step 9 Compile and package the file to the local Maven repository. mvn install

Step 10 In the openssl-static/pom.xml file, comment out the following three parts for downloading openssl-1.0.2l. Position 1:

Position 2:

Position 3:

Step 11 Download the openssl-1.0.2l.tar.gz package to the following directory: wget https://ftp.openssl.org/source/old/1.0.2/openssl-1.0.2l.tar.gz mv openssl-1.0.2l.tar.gz netty-tcnative-netty-tcnative-parent-2.0.6.Final/openssl-static/target Step 12 Compile and package the file to the local Maven repository. mvn install

Step 13 Copy the apr-1.6.2 package downloaded in Step 8 to the netty-tcnative-netty- tcnative-parent-2.0.6.Final/libressl-static/target directory.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 115 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

cp apr-1.6.2.tar.gz netty-tcnative-netty-tcnative-parent-2.0.6.Final/libressl-static/target

Step 14 Compile and package the file to the local Maven repository. mvn install

Step 15 In the libressl-static/pom.xml file, comment out the following two parts for downloading the libressl-2.5.5 package:

Step 16 Download the libressl-2.5.5.tar.gz package to the following directory: wget http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.5.5.tar.gz mv libressl-2.5.5.tar.gz netty-tcnative-netty-tcnative-parent-2.0.6.Final/libressl-static/target

Step 17 Compile and package the file to the local Maven repository. mvn install

----End

Compiling and Installing netty-4.0.52

Step 1 Download the netty-4.0.52 source code. wget https://github.com/netty/netty/archive/netty-4.0.52.Final.tar.gz

Step 2 Decompress the source code package. tar -zxvf netty-4.0.52.Final.tar.gz

Step 3 Go to the directory generated after the decompression. cd netty-netty-4.0.52.Final

Step 4 Comment out the following code in netty-netty-4.0.52.Final/pom.xml:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 116 Kunpeng BoostKit for Big Data 14 netty-all-4.0.52.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 5 Compile and generate netty-all-4.0.52.Final.jar in the all/target directory, and netty-transport-native-epoll-4.0.52.Final.jar and netty-transport-native- epoll-4.0.52.Final-linux-aarch64_64.jar in the transport-native-epoll/target directory. mvn package -DskipTests

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 117 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

15 netty-all-4.1.17.Final.jar Porting Guide (CentOS 7.6)

15.1 Introduction 15.2 Requirements 15.3 Configuring the Compilation Environment 15.4 Compiling netty-all-4.1.17.Final.jar 15.5 Troubleshooting

15.1 Introduction

Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

15.2 Requirements

Hardware Requirements

Table 15-1 lists the hardware requirements.

Table 15-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 118 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

OS Requirements Table 15-2 lists the requirements for the operating systems (OSs).

Table 15-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

15.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 119 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 120 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 121 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

15.4 Compiling netty-all-4.1.17.Final.jar

Installing APR

Step 1 Download the apr-1.5.2 source code. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz

Step 2 Decompress apr-1.5.2.tar.gz. tar -xvzf apr-1.5.2.tar.gz

Step 3 Go to the apr-1.5.2 directory. cd apr-1.5.2

Step 4 Install APR to the /usr/local/apr directory. ./configure --prefix=/usr/local/apr make && make install

----End

Compiling and Installing netty-tcnative-parent-2.0.7.Final

Step 1 Download the netty-tcnative-parent-2.0.7.Final source code. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-parent-2.0.7.Final mv netty-tcnative-parent-2.0.7.Final netty-tcnative-parent-2.0.7.Final.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 122 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Decompress the source code package. tar -zxvf netty-tcnative-parent-2.0.7.Final.tar.gz Step 3 Go to the directory generated after the decompression. cd netty-tcnative-netty-tcnative-parent-2.0.7.Final Step 4 Modify the pom.xml file to shield the compilation of BoringSSL.

Step 5 Compile and package the code to the local Maven Repository. The following error information is displayed: mvn install

Step 6 In the openssl-static/pom.xml file, comment out the following three parts for downloading openssl-1.0.2l. Position 1:

Position 2:

Position 3:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 123 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 7 Download the openssl-1.0.2l.tar.gz package to the following directory: wget https://ftp.openssl.org/source/old/1.0.2/openssl-1.0.2l.tar.gz mv openssl-1.0.2l.tar.gz netty-tcnative-netty-tcnative-parent-2.0.7.Final/openssl-static/target Step 8 Compile and package the file to the local Maven repository. mvn install

----End

Compiling and Installing netty-4.1.17

Step 1 Download the netty-4.1.17 source code package. wget https://github.com/netty/netty/archive/netty-4.1.17.Final.tar.gz Step 2 Decompress the source code package. tar -zxvf netty-4.1.17.Final.tar.gz Step 3 Go to the directory generated after the decompression. cd netty-netty-4.1.17.Final Step 4 In the pom.xml file, comment out the following code:

Step 5 Compile and generate netty-all-4.1.17.Final.jar in the all/target directory, and netty-transport-native-epoll-4.1.17.Final.jar and netty-transport-native- epoll-4.1.17.Final-linux-aarch64_64.jar in the transport-native-epoll/target directory. mvn package -DskipTests

----End

15.5 Troubleshooting

15.5.1 Failed to Find jni.h and jni_md.h

Symptom The following error is reported:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 124 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Procedure Open the ${netty-netty-4.1.17.Final}/transport-native-unix-common/pom.xml file, search for the keyword "CFLAGS", and add the following parameters to the file:

-I/usr/lib/jvm/java/include –I/usr/lib/jvm/java/linux/include

15.5.2 Failed to Find linux-aarch_64-fedora.2.0.7.Final.jar During Compilation

Symptom linux-aarch_64-fedora.2.0.7.Final.jar cannot be found during compilation.

Procedure In the pom.xml file, delete fedora, and perform the compilation again.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 125 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

15.5.3 Failed to Download apr-1.6.3 During Compilation

Symptom

Failed to download apr-1.6.3.

Procedure

Step 1 In the pom.xml file, comment out the following two parts for downloading the apr-1.6.3 package.

Position 1:

Position 2:

Step 2 Download the apr-1.6.3.tar.gz package to the following directory: wget https://archive.apache.org/dist/apr/apr-1.6.3.tar.gz mv apr-1.6.3.tar.gz netty-tcnative-netty-tcnative-parent-2.0.7.Final/openssl-static/target

Step 3 Perform the compilation again.

----End 15.5.4 Failed to Download libressl-2.5.5 During Compilation

Symptom

Failed to download libressl-2.5.5 during the compilation process.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 126 Kunpeng BoostKit for Big Data 15 netty-all-4.1.17.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Procedure

Step 1 Save the apr-1.6.3.tar.gz package downloaded in Step 2 in the following directory: cp apr-1.6.3.tar.gz netty-tcnative-netty-tcnative-parent-2.0.7.Final/libressl-static/target Step 2 In the libressl-static/pom.xml file, comment out the following part for downloading the libressl-2.5.5 package.

Step 3 Download the libressl-2.5.5.tar.gz package to the following directory: wget http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.5.5.tar.gz mv libressl-2.5.5.tar.gz netty-tcnative-netty-tcnative-parent-2.0.7.Final/libressl-static/target Step 4 Compile and package the file to the local Maven repository. mvn install

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 127 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide (CentOS 7.6)

16.1 Introduction 16.2 Environment Requirements 16.3 Configuring the Compilation Environment 16.4 Compiling and Installing RocksDB JNI 16.5 Troubleshooting

16.1 Introduction

RocksDB JNI provides a Java interface to the RocksDB C++ library, which is an embeddable persistent key-value store for fast storage.

16.2 Environment Requirements

Hardware Requirements

Table 16-1 lists the hardware requirements.

Table 16-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 128 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

OS Requirements Table 16-2 lists the requirements for the operating systems (OSs).

Table 16-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

Software Requirements Table 16-3 lists the software requirements.

Table 16-3 Software requirements Item Version How to Obtain

GCC 4.8.5 --

Maven 3.5.4 https://archive.apache.org/dist/maven/ maven-3/3.5.4/binaries/apache- maven-3.5.4-bin.tar.gz

Rocksdbjni 5.7.3,5.14.2 https://codeload.github.com/facebook/ rocksdb/zip/v5.7.3 https://codeload.github.com/facebook/ rocksdb/zip/v5.14.2

16.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 129 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 130 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

5. Check whether the G++ file is available. g++ --version The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

----End

Installing Maven

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 131 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 132 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

16.4 Compiling and Installing RocksDB JNI

Compiling rocksdbjni-5.7.3

Step 1 Download the rocksdbjni-5.7.3 source code. wget https://codeload.github.com/facebook/rocksdb/zip/v5.7.3 Step 2 Decompress the source code package. mv v5.7.3 rocksdbjni-5.7.3.zip unzip rocksdbjni-5.7.3.zip Step 3 Go to the directory generated after the decompression. cd rocksdb-5.7.3 Step 4 In the Makefile file, change the value of DEBUG_LEVEL to 0.

Step 5 Change the Maven repository address in the java/Makefile file to https:// repo1.maven.org/maven2/. vim java/Makefile

Step 6 Compile the source code and package it into rocksdbjni-5.7.3-linux64.jar in the rocksdb-5.7.3/java/target directory. PORTABLE=1 make rocksdbjava -j8

----End

Compiling rocksdbjni-5.14.2

Step 1 Download the rocksdbjni-5.14.2 source code. wget https://codeload.github.com/facebook/rocksdb/zip/v5.14.2 Step 2 Decompress the source code package. mv v5.14.2 rocksdbjni-5.14.2.zip unzip rocksdbjni-5.14.2.zip Step 3 Go to the directory generated after the decompression. cd rocksdb-5.14.2 Step 4 In the Makefile file, change the value of DEBUG_LEVEL to 0.

Step 5 Change the Maven repository address in the java/Makefile file to https:// repo1.maven.org/maven2/. vim java/Makefile

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 133 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 6 Compile the source code and package rocksdbjni-5.14.2-linux64.jar to the rocksdb-5.14.2/java/target directory. PORTABLE=1 make rocksdbjava -j8

----End

Compiling rocksdbjni-5.15.10

Step 1 Download the rocksdbjni-5.15.10 source code. wget https://codeload.github.com/facebook/rocksdb/zip/v5.15.10 Step 2 Decompress the source code package. mv v5.15.10 rocksdbjni-5.15.10.zip unzip rocksdbjni-5.15.102.zip Step 3 Go to the directory generated after the decompression. cd rocksdb-5.15.10 Step 4 In the Makefile file, change the value of DEBUG_LEVEL to 0.

Step 5 Change the Maven repository address in the java/Makefile file to https:// repo1.maven.org/maven2/. vim java/Makefile

Step 6 Compile the source code and package rocksdbjni-5.15.10-linux64.jar to the rocksdb-5.15.10/java/target directory. PORTABLE=1 make rocksdbjava -j8

----End

16.5 Troubleshooting

16.5.1 Size of the Compiled JAR Package Is Greater Than the Package Downloaded From the Maven Repository

Symptom The size of the compiled JAR package is greater than the package downloaded from the Maven repository.

Cause Analysis The librocksdbjni-linux64.so file in the JAR package contains symbol information.

Procedure Run following command to remove the symbol information:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 134 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

strip 16.5.2 jni.h or jni_md.h File Cannot Be Found

Symptom An error message is displayed indicating that the jni.h or jni_md.h file cannot be found.

Possible Cause The path for JAVA_HOME is not configured.

Procedure Configure the path for JAVA_HOME.

If the problem persists, move the jni.h and jni_md.h files in the JDK used for compilation to the root directory for compilation output. 16.5.3 Compilation Error

Symptom "unknown value 'native' for -march" is displayed during compilation.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 135 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Solution Run the following command to perform compilation:

PORTABLE=1 make rocksdbjava -j8 16.5.4 Failed to Compare va_list and nullptr

Symptom An error message is displayed during compilation indicating that va_list and nullptr cannot be compared.

Procedure Comment out line 134 in java/rocksjni/loggerjnicallback.cc and recompile the source code.

16.5.5 Connect to central.maven.org:80 connection timed out

Symptom "Connect to central.maven.org:80 connection timed out" is displayed during compilation.

Procedure Change the Maven repository address in line 190 of the java/Makefile file to https://repo1.maven.org/maven2/.

vim java/Makefile

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 136 Kunpeng BoostKit for Big Data 16 rocksdbjni-5.7.3,5.14.2,5.15.10.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

16.5.6 assertj-core-1.7.1.jar error in opening zip file 1 error

Symptom An error message is displayed during compilation indicating that assertj- core-1.7.1.jar cannot be opened.

Procedure Manually download assertj-core-1.7.1.jar and assertj-core-1.7.1.pom from the Maven repository at https://repo1.maven.org/maven2/org/assertj/assertj-core/ 1.7.1/ to the /org/assertj/assertj-core/1.7.1 directory of the local repository.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 137 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Guide (CentOS 7.6)

17.1 Introduction 17.2 Requirements 17.3 Configuring the Compilation Environment 17.4 Compiling scala-compiler

17.1 Introduction Scala-compiler is a compiler for the Scala language.

17.2 Requirements

Hardware Requirements Table 17-1 lists the hardware requirements.

Table 17-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 138 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

OS Requirements Table 17-2 lists the requirements for the operating systems (OSs).

Table 17-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

17.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 139 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 140 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 141 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

17.4 Compiling scala-compiler

Installing sbt

Step 1 Download sbt-0.13.15 to a specified directory (for example, /opt/tools/installed) and decompress it. wget https://sbt-downloads.cdnedge.bluemix.net/releases/v0.13.15/sbt-0.13.15.tgz tar -zxf sbt-0.13.15.tgz mv sbt /opt/tools/installed Step 2 Create the .sbt folder in the root directory of the current user and copy the repositories file to the .sbt folder.

NO TE

The repositories section defines the sequence, in which the Maven repositories are used during sbt compilation. mkdir ~/.sbt cp repositories ~/.sbt The content of the repositories file is as follows:

[repositories] local kunpeng-maven: https://mirrors.huaweicloud.com/kunpeng/maven/ huawei-maven: https://mirrors.huaweicloud.com/repository/maven/

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 142 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

sbt-releases-repo: https://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/ (scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext] sbt-plugins-repo: https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/, [organization]/[module]/ (scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]

Step 3 Configure the SBT_HOME environment variable. vim /etc/profile

Add the following information to the profile file:

export SBT_HOME=/opt/tools/installed/sbt export PATH=${SBT_HOME}/bin:${PATH}

Step 4 Make the environment variable take effect. source /etc/profile

----End

Compiling scala-compiler-2.11.8

Step 1 Copy the compiled jline-2.12.1.jar file, the dependency of Scala compilation, to the ~/.ivy2/cache/jline/jline/jars/ directory to replace the existing one. For details, see the jline-2.11,2.12,2.12.1,2.14.3.jar Porting Guide (CentOS 7.6).

Step 2 Download the scala-2.11.8 source code. git clone https://github.com/scala/scala.git

Step 3 Go to the source code directory and switch to the version fork. cd scala git checkout v2.11.8 sed -i "48,48s%)%),\n\ \ \ \ Keys.\`package\`\ := bundle.value%g" project/Osgi.scala

Step 4 Compile the source code. sbt package

The scala-compiler.jar file is generated in the build-sbt/pack/lib directory.

----End

Compiling scala-compiler-2.11.12

Step 1 Copy the compiled jline-2.14.3 file, the dependency of Scala compilation, to the ~/.ivy2/cache/jline/jline/jars/ directory to replace the existing one. For details, see the jline-2.11,2.12,2.12.1,2.14.3.jar Porting Guide (CentOS 7.6).

Step 2 Download the scala-2.11.12 source code. git clone https://github.com/scala/scala.git

Step 3 Go to the source code directory and switch to the version fork. cd scala git checkout v2.11.12 sed -i "50,50s%)%),\n\ \ \ \ Keys.\`package\`\ := bundle.value%g" project/Osgi.scala

Step 4 Compile the source code. sbt package

The scala-compiler.jar file is generated in the build/pack/lib directory.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 143 Kunpeng BoostKit for Big Data 17 scala-compiler-2.11.8, 2.11.12, 2.12.7.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

----End

Compiling scala-compiler-2.12.7

Step 1 Copy the compiled jline-2.14.6.jar file, the dependency of Scala compilation, to the ~/.ivy2/cache/jline/jline/jars/ directory and replace the existing one. For details, see the jline-2.11,2.12,2.12.1,2.14.3.jar Porting Guide (CentOS 7.6). Step 2 Download the scala-2.12.7 source code. git clone https://github.com/scala/scala.git Step 3 Go to the source code directory and switch to the version fork. cd scala git checkout v2.12.7 Step 4 Change "http" in the repositories file to "https". vim ~/.sbt/repositories

Step 5 Compile the source code. sbt package The scala-compiler.jar file is generated in the build/pack/lib directory.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 144 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS 7.6)

18.1 Introduction 18.2 Requirements 18.3 Configuring the Compilation Environment 18.4 Compiling snappy-java 18.5 Troubleshooting

18.1 Introduction

Snappy is an open-source compression and decompression library of Google. snappy-java encapsulates its interfaces.

18.2 Requirements

Hardware Requirements

Table 18-1 lists the hardware requirements.

Table 18-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 145 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

OS Requirements Table 18-2 lists the requirements for the operating systems (OSs).

Table 18-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

18.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 146 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 147 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 148 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

18.4 Compiling snappy-java

Compiling snappy-java-1.0.4.1 Step 1 Download the source code package. wget https://github.com/xerial/snappy-java/archive/snappy-java-1.0.4.1.tar.gz --no-check-certificate Step 2 Decompress the source code package. tar -zxvf snappy-java-1.0.4.1.tar.gz mv snappy-java-snappy-java-1.0.4.1 snappy-java-1.0.4.1 Step 3 Go to the directory generated after the decompression. cd snappy-java-1.0.4.1 Step 4 Modify the Makefile file. vim Makefile 1. In the Makefile file, comment out the original download URL of Snappy and add the new URL. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -o$@ http://src.fedoraproject.org/lookaside/pkgs/snappy/snappy-1.0.4.tar.gz/ b69151652e82168bc5c643bcd6f07162/snappy-$(VERSION).tar.gz 2. In the Makefile file, comment out the original decompression command and add the new command. # $(TAR) xvfz $< -C $(TARGET) $(TAR) xvf $< -C $(TARGET)

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 149 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 5 Compile the source code. make The generated JAR package after the compilation is stored in target/snappy- java-1.0.4.1.jar. ----End

Compiling snappy-java-1.0.5 Step 1 Download the source code. wget https://codeload.github.com/xerial/snappy-java/zip/1.0.5 mv 1.0.5 snappy-java-1.0.5.zip Step 2 Decompress the source code package. unzip snappy-java-1.0.5.zip Step 3 Go to the directory generated after the decompression. cd snappy-java-1.0.5 Step 4 In the Makefile file, comment out the "curl" script statement in line 21 and add a new web page link for downloading snappy-1.0.5.tar.gz. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -k -o$@ https://src.fedoraproject.org/lookaside/pkgs/snappy/snappy-$(VERSION).tar.gz/ 4c0af044e654f5983f4acbf00d1ac236/snappy-$(VERSION).tar.gz NO TE

The original Snappy download website snappy.googlecode.com cannot be accessed. Therefore, you need to use another download URL. Step 5 Go to the directory where the snappy-java source code package is decompressed. cd snappy-java-1.0.5 Step 6 Compile the source code. make ----End

18.5 Troubleshooting

18.5.1 Failed to Decompress the snappy-1.0.4.tar.gz Installation Package

Symptom The following error message is displayed during compilation:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 150 Kunpeng BoostKit for Big Data 18 snappy-java-1.0.4.1, 1.0.5 Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Procedure

Step 1 Go to the snappy-java-1.0.4.1/target directory and delete the TAR package that failed to be downloaded. cd snappy-java-1.0.4.1/target rm -rf snappy-1.0.4.tar.gz Step 2 Comment out the download URL of the Snappy package in the Makefile file and manually download the snappy-1.0.4.tar.gz package.

wget http://src.fedoraproject.org/lookaside/pkgs/snappy/snappy-1.0.4.tar.gz/ b69151652e82168bc5c643bcd6f07162/snappy-1.0.4.tar.gz --no-check-certificate

----End 18.5.2 Failed to Find Library Files During Compilation

Symptom The following error message is displayed during compilation:

Procedure If you replace the static library with a dynamic library, the dynamic library may not be found at runtime. You can install the corresponding static library to solve the problem. Use the Yum source to install the required glibc and libstdc++-static.

yum install libstdc++-static.aarch64 -y yum install glibc* -y

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 151 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, 1.1.2.6.jar Porting Guide (CentOS 7.6)

19.1 Introduction 19.2 Requirements 19.3 Configuring the Compilation Environment 19.4 Compiling snappy-java 19.5 Rectifying Faults

19.1 Introduction

Snappy is an open-source compression and decompression library of Google. snappy-java encapsulates its interfaces.

19.2 Requirements

Hardware Requirements

Table 19-1 lists the hardware requirements.

Table 19-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 152 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

OS Requirements Table 19-2 lists the requirements for the operating systems (OSs).

Table 19-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

19.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 153 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 154 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 155 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

19.4 Compiling snappy-java

Installing sbt

NO TE

The repositories section defines the sequence, in which the Maven repositories are used during sbt compilation. mkdir ~/.sbt cp repositories ~/.sbt The content of the repositories file is as follows:

[repositories] local kunpeng-maven: https://mirrors.huaweicloud.com/kunpeng/maven/ huawei-maven: https://mirrors.huaweicloud.com/repository/maven/

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 156 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

export SBT_HOME=/opt/tools/installed/sbt export PATH=${SBT_HOME}/bin:${PATH} Step 4 Make the environment variable take effect. source /etc/profile ----End

Compiling snappy-java-1.1.0.1 Step 1 Download the source code package. wget https://github.com/xerial/snappy-java/archive/1.1.0.1.zip --no-check-certificate mv 1.1.0.1.zip snappy-java-1.1.0.1.zip Step 2 Decompress the source code package. unzip snappy-java-1.1.0.1.zip Step 3 Go to the directory generated after the decompression. cd snappy-java-1.1.0.1 Step 4 Modify the Makefile file. vim Makefile 1. In the Makefile file, comment out the original download URL of Snappy and add the new URL. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -o$@ https://src.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.0.tar.gz/ c8f3ef29b5281e78f4946b2d739cea4f/snappy-$(VERSION).tar.gz 2. In the Makefile file, comment out the original decompression command and add the new command. # $(TAR) xvfz $< -C $(TARGET) $(TAR) xvf $< -C $(TARGET)

Step 5 Compile the source code. make The generated JAR package after the compilation is stored in target/snappy- java-1.1.0.1-SNAPSHOT.jar. Step 6 Modify the pom file. vim pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 157 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

Step 7 Perform the compilation again. make

The generated JAR package after the compilation is stored in target/snappy- java-1.1.0.1.jar.

----End

Compiling snappy-java-1.1.1.3

Step 1 Download the source code package. wget https://github.com/xerial/snappy-java/archive/1.1.1.3.zip --no-check-certificate mv 1.1.1.3.zip snappy-java-1.1.1.3.zip

Step 2 Decompress the source code package. unzip snappy-java-1.1.1.3.zip

Step 3 Go to the directory generated after the decompression. cd snappy-java-1.1.1.3

Step 4 Modify the Makefile file. vim Makefile 1. In the Makefile file, comment out the original download URL of Snappy and add the new URL. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -o$@ http://src.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.1.tar.gz/ 8887e3b7253b22a31f5486bca3cbc1c2/snappy-$(VERSION).tar.gz 2. In the Makefile file, comment out the original decompression command and add the new command. # $(TAR) xvfz $< -C $(TARGET) $(TAR) xvf $< -C $(TARGET)

Step 5 Compile the source code. make

The generated JAR package after the compilation is stored in target/snappy- java-1.1.1.3.jar.

----End

Compiling snappy-java-1.1.1.6

Step 1 Download the source code from GitHub. wget https://github.com/xerial/snappy-java/archive/1.1.1.6.zip --no-check-certificate mv 1.1.1.6.zip snappy-java-1.1.1.6.zip

Step 2 Decompress the source code package. unzip snappy-java-1.1.1.6.zip

Step 3 Go to the directory generated after the decompression. cd snappy-java-1.1.1.6

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 158 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

Step 5 Compile the source code. make The generated JAR package after the compilation is stored in target/snappy- java-1.1.1.6.jar.

----End

Compiling snappy-java-1.1.1.7

Step 1 Download the source code package. wget https://github.com/xerial/snappy-java/archive/1.1.1.7.zip mv 1.1.1.7.zip snappy-java-1.1.1.7.zip Step 2 Decompress the source code package. unzip snappy-java-1.1.1.7.zip Step 3 Go to the directory generated after the decompression. cd snappy-java-1.1.1.7 Step 4 Modify the Makefile file. vim Makefile 1. In the Makefile file, comment out the original download URL of Snappy and add the new URL. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -o$@ http://src.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.1.tar.gz/ 8887e3b7253b22a31f5486bca3cbc1c2/snappy-$(VERSION).tar.gz 2. In the Makefile file, comment out the original decompression command and add the new command. # $(TAR) xvfz $< -C $(TARGET) $(TAR) xvf $< -C $(TARGET)

Step 5 Compile the source code. make

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 159 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

The generated JAR package after the compilation is stored in target/snappy- java-1.1.1.7.jar.

----End

Compiling snappy-java-1.1.2.6 Step 1 Download the source code from GitHub. wget https://github.com/xerial/snappy-java/archive/1.1.2.6.zip --no-check-certificate mv 1.1.2.6.zip snappy-java-1.1.2.6.zip Step 2 Decompress the source code package. unzip snappy-java-1.1.2.6.zip Step 3 Go to the directory generated after the decompression. cd snappy-java-1.1.2.6 Step 4 Modify the Makefile file. vim Makefile 1. In the Makefile file, comment out the original download URL of Snappy and add the new URL. #curl -o$@ http://snappy.googlecode.com/files/snappy-$(VERSION).tar.gz curl -o$@ http://repository.timesys.com/buildsources/s/snappy/snappy-1.1.2/snappy-$(VERSION).tar.gz 2. In the Makefile file, comment out the original decompression command and add the new command. # $(TAR) xvfz $< -C $(TARGET) $(TAR) xvf $< -C $(TARGET)

Step 5 Compile the source code. make The generated JAR package after the compilation is stored in target/snappy- java-1.1.2.6.jar.

----End

19.5 Rectifying Faults

19.5.1 Failed to Decompress snappy-1.1.0.tar.gz During the Compilation of snappy-java-1.1.0.1

Symptom The following error message is displayed during compilation:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 160 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

Procedure

Step 1 Go to the snappy-java-1.1.0.1/target directory and delete the TAR package that failed to be downloaded. cd snappy-java-1.1.0.1/target rm -rf snappy-1.1.1.tar.gz Step 2 Comment out the download URL of the Snappy package in the Makefile file and manually download the snappy-1.1.0.tar.gz package.

wget http://src.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.0.tar.gz/ c8f3ef29b5281e78f4946b2d739cea4f/snappy-1.1.0.tar.gz --no-check-certificate

----End 19.5.2 Failed to Decompress snappy-1.1.1.tar.gz During the Compilation of snappy-java-1.1.1.3/1.1.1.6/1.1.1.7

Symptom The following error message is displayed during compilation:

Procedure The following uses snappy-java-1.1.1.3 as an example.

Step 1 Go to the snappy-java-1.1.1.3/target directory and delete the TAR package that failed to be downloaded. cd snappy-java-1.1.1.3/target rm -rf snappy-1.1.1.tar.gz Step 2 Comment out the download URL of the Snappy package in the Makefile file and manually download the snappy-1.1.1.tar.gz package.

wget http://src.fedoraproject.org/repo/pkgs/snappy/snappy-1.1.1.tar.gz/ 8887e3b7253b22a31f5486bca3cbc1c2/snappy-1.1.1.tar.gz --no-check-certificate

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 161 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

19.5.3 Failed to Download sbt-launch.jar During the Compilation of snappy- java-1.1.1.3/1.1.1.6/1.1.1.7/1.1.2.1/1.1.2.6

Symptom The following error message is displayed during compilation:

Procedure The following uses snappy-java-1.1.1.3 as an example.

Step 1 Go to the /root/.sbt/launchers/0.13.5 directory. cd /root/.sbt/launchers/0.13.5

NO TE

The directory for the 1.1.1.6 and 1.1.1.7 versions is /root/.sbt/launchers/0.13.6. For version 1.1.2.1, the directory is /root/.sbt/launchers/0.13.8. For version 1.1.2.6, the directory is /root/.sbt/launchers/0.13.9. Step 2 Manually download the JAR package in the current directory. wget http://dl.bintray.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.5/sbt-launch.jar

NO TE

The download URL for version 1.1.1.6 and 1.1.1.7 is http://dl.bintray.com/typesafe/ivy- releases/org.scala-sbt/sbt-launch/0.13.6/sbt-launch.jar. The download URL for version 1.1.2.1 is http://dl.bintray.com/typesafe/ivy-releases/ org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar. The download URL for version 1.1.2.6 version is http://dl.bintray.com/typesafe/ivy-releases/ org.scala-sbt/sbt-launch/0.13.9/sbt-launch.jar. Step 3 Comment out the following line in the sbt file in the source code directory: -sbt-launch-repo) require_arg path "$1" "$2" && sbt_launch_repo="$2" && shift 2 ;;

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 162 Kunpeng BoostKit for Big Data 19 snappy-java-1.1.0.1, 1.1.1.3, 1.1.1.6, 1.1.1.7, Porting Guides (Third-Party Dependency Libraries) 1.1.2.6.jar Porting Guide (CentOS 7.6)

----End 19.5.4 Failed to Find Library Files During Compilation

Symptom The following error message is displayed during compilation:

yum install libstdc++-static.aarch64 -y yum install glibc* -y

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 163 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

20 wildfly-openssl-1.0.4.Final.jar Porting Guide (CentOS 7.6)

20.1 Introduction 20.2 Environment Requirements 20.3 Configuring the Compilation Environment 20.4 Compiling wildfly-openssl-1.0.4.Final.jar 20.5 Troubleshooting

20.1 Introduction

Generic OpenSSL bindings for Java.

Official website: https://github.com/wildfly/wildfly-openssl/releases/tag/ 1.0.4.Final

Category: Toolkit

20.2 Environment Requirements

Hardware Requirements

Table 20-1 lists the hardware requirements.

Table 20-1 Hardware requirements

Item Description

CPU Kunpeng 916 or 920 processor

Network Ethernet-10GE

Storage SATA 1 TB

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 164 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Item Description

Memory 512 GB, 2400 MHz

OS Requirements Table 20-2 lists the requirements for the operating systems (OSs).

Table 20-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

Software Requirements Table 20-3 lists the software requirements.

Table 20-3 Software requirements Item Version

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

20.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 165 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 166 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 167 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 168 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

20.4 Compiling wildfly-openssl-1.0.4.Final.jar

Step 1 Download the wildfly-openssl-1.0.4.Final source code package and decompress it. wget https://github.com/wildfly/wildfly-openssl/archive/1.0.4.Final.tar.gz tar -zxf 1.0.4.Final.tar.gz Step 2 Go to the directory generated after the decompression. cd wildfly-openssl-1.0.4.Final Step 3 Create the linux-aarch64 directory based on the directory format after the decompression. mkdir linux-aarch64 Step 4 Copy the Makefile and pom.xml files in the linux-x86_64 directory to the linux- aarch64 directory. cp linux-x86_64/Makefile linux-aarch64/ cp linux-x86_64/pom.xml linux-aarch64/ Step 5 Modify the ./pom.xml, ./combined/pom.xml, ./java/pom.xml, ./linux-aarch64/ pom.xml, and ./linux-aarch64/Makefile files as required. ● combined/pom.xml sed -i '122a\ \nlinux-aarch64<\/id>\n\n\nlinux<\/family> \n<\/os>\n\nsun.arch.data.model<\/name>\n64<\/value>\n<\/ property>\n<\/activation>\n\n\norg.wildfly.openssl<\/ groupId>\nwildfly-openssl-linux-aarch64<\/artifactId>\n\${project.version}< \/version>\n<\/dependency>\n<\/dependencies>\n<\/profile>' combined/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 169 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

sed -i '240a\ \norg.wildfly.openssl<\/groupId>\nwildfly- openssl-linux-aarch64<\/artifactId>\n\${project.version}<\/version>\n<\/dependency>' combined/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 170 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● java/pom.xml sed -i '169a\\nlinux-aarch64<\/id>\n\n\nlinux<\/family> \n<\/os>\n\nsun.arch.data.model<\/name>\n64<\/value>\n<\/ property>\n<\/activation>\n\n\norg.wildfly.openssl<\/ groupId>\nwildfly-openssl-linux-aarch64<\/artifactId>\n\${project.version}< \/version>\n<\/dependency>\n<\/dependencies>\n<\/profile>' java/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 171 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● linux-aarch64/pom.xml sed -i 's/x86_64/aarch64/' linux-aarch64/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 172 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● Makefile sed -i 's/x86_64/aarch64/' linux-aarch64/Makefile

● pom.xml sed -i '154a\\nlinux-aarch64<\/id>\n\n\nlinux<\/family> \n<\/os>\n\nsun.arch.data.model<\/name>\n64<\/value>\n<\/ property>\n<\/activation>\n\nlinux-aarch64<\/module>\n<\/modules>\n<\/ profile>' pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 173 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 6 Perform compilation. mvn install -DskipTests=true -Dmaven.javadoc.skip=true

NO TE

If the .jar package still does not contain ARM-based .so files after compilation, modify the source code and perform the compilation again. For details, see 20.5.2 No ARM-based JAR Package Is Generated After wildfly-openssl-1.0.4.Final.jar Is Compiled.

----End

20.5 Troubleshooting

20.5.1 wildfly-openssl-1.0.4.Final.jar Compilation Error

Symptom An error is reported when wildfly-openssl-1.0.4.Final.jar is compiled.

../libwfssl/include/wfssl.h:41:17: fatal error: jni.h: No such file or directory

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 174 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Possible Cause

The header file jni.h is not found in ./libwfssl/include/. The header file is an .h file provided by Java.

Solution

Run the find command to perform global search, copy the .h file in the Java- related path to the ./libwfssl/include/ directory, and recompile the file.

You can use this method to solve similar problems. 20.5.2 No ARM-based JAR Package Is Generated After wildfly- openssl-1.0.4.Final.jar Is Compiled

Symptom

No ARM-based JAR package is generated after wildfly-openssl-1.0.4.Final.jar is compiled.

Possible Cause

The linux-aarch64 module is not defined in the pom.xml file in the source code.

Solution

Step 1 Create the linux-aarch64 directory based on the directory format after the decompression. cd wildfly-openssl-1.0.4.Final mkdir linux-aarch64

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 175 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 2 Copy the Makefile and pom.xml files in the linux-x86_64 directory to the created linux-aarch64 directory.

Step 3 Modify or add related content to the ./pom.xml, ./combined/pom.xml, ./java/ pom.xml, ./linux-aarch64/pom.xml, and ./linux-aarch64/Makefile files. ● ./combined/pom.xml vim ./combined/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 176 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● ./combined/pom.xml vim ./combined/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 177 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● ./java/pom.xml vim ./java/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 178 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● ./linux-aarch64/pom.xml vim ./linux-aarch64/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 179 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

● ./linux-aarch64/Makefile vim ./linux-aarch64/Makefile

● ./pom.xml vim ./pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 180 Kunpeng BoostKit for Big Data 20 wildfly-openssl-1.0.4.Final.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 4 Run the compilation command again. Step 5 View the compilation result.

----End

21 chimera-0.9.2.jar Porting Guide (CentOS 7.6)

21.1 Introduction 21.2 Requirements 21.3 Configuring the Compilation Environment 21.4 Compiling chimera

21.1 Introduction

Chimera is an encryption library optimized with Advanced Encryption Standard New Instructions (AES-NI) It provides Java APIs for both password level and Java stream level. Developers can use it to implement high-performance AES encryption and decryption. Chimera does not directly implement encryption algorithms such as AES. Instead, it uses OpenSSL or JCE to implement the algorithms.

21.2 Requirements

Hardware Requirements

Table 21-1 lists the hardware requirements.

Table 21-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Item Description

Memory No requirement

OS Requirements Table 21-2 lists the requirements for the operating systems (OSs).

Table 21-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

21.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

----End

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

21.4 Compiling chimera

Step 1 Download the source code. wget https://github.com/intel-hadoop/chimera/archive/0.9.2.tar.gz

Step 2 Decompress the source code package. tar -zxvf 0.9.2.tar.gz

Step 3 Go to the directory where the Jansi-native source code is located. cd chimera-0.9.2

Step 4 Install dependencies. yum install glibc-static libstdc++-static -y

Step 5 Perform compilation. mvn clean install

The result is ./target/chimera-0.9.2.jar, which is installed in the local Maven repository.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 187 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

22 flink-shaded- netty-4.1.32.Final-6.0.jar Porting Guide (CentOS 7.6)

22.1 Introduction 22.2 Environment Requirements 22.3 Configuring the Compilation Environment 22.4 Compiling and Installing flink-shaded-netty

22.1 Introduction flink-shaded-netty Overview

The flink-shaded-netty module is a big data component.

Recommended Version

4.1.32.Final-6.0

22.2 Environment Requirements

Hardware Requirements

Table 22-1 lists the hardware requirements.

Table 22-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 188 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Item Description

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 22-2 lists the requirements for the operating systems (OSs).

Table 22-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

Software Requirements Table 22-3 lists the software requirements.

Table 22-3 Software requirements Item Version How to Obtain

GCC 4.8.5 -

OpenJDK 1.8.0_252 https://github.com/AdoptOpenJDK/ openjdk8-binaries/releases/download/ jdk8u252-b09/OpenJDK8U- jdk_aarch64_linux_hotspot_8u252b09.tar.gz

Maven 3.5.4 https://archive.apache.org/dist/maven/ maven-3/3.5.4/binaries/apache- maven-3.5.4-bin.tar.gz

Apr 1.6.5 https://archive.apache.org/dist/apr/ apr-1.6.5.tar.gz

Netty-tcnative parent-2.0.2 https://codeload.github.com/netty/netty- 0.Final tcnative/tar.gz/netty-tcnative- parent-2.0.20.Final

Netty 4.1.32.Final https://github.com/netty/netty/archive/ netty-4.1.32.Final.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 189 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

22.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name.

Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Step 3 Modify the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Configure the local Yum source. [Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 190 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

The installation is successful if information similar to the following is displayed:

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 191 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 192 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

22.4 Compiling and Installing flink-shaded-netty

Compiling and Installing apr-1.6.5

Step 1 Download the apr-1.6.5 source code. wget https://archive.apache.org/dist/apr/apr-1.6.5.tar.gz Step 2 Decompress the package and compile and install apr-1.6.5. tar -zxvf apr-1.6.5.tar.gz cd apr-1.6.5 ./configure make make install

----End

Compiling and installing netty-tcnative-parent-2.0.20.Final

Step 1 Download the netty-tcnative-parent-2.0.20.Final source code. wget https://codeload.github.com/netty/netty-tcnative/tar.gz/netty-tcnative-parent-2.0.20.Final mv netty-tcnative-parent-2.0.20.Final netty-tcnative-parent-2.0.20.Final.tar.gz Step 2 Decompress the source code package. tar -zxvf netty-tcnative-parent-2.0.20.Final.tar.gz Step 3 Go to the directory where the source code package is decompressed. cd netty-tcnative-netty-tcnative-parent-2.0.20.Final Step 4 Modify the pom.xml file to shield the compilation of boringssl-static.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 193 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 5 Compile and package the modules, and place the package in the local Maven repository. mvn install

----End

Compiling and Installing netty-all-4.1.32.Final

Step 1 Download the netty-4.1.32.Final source code. wget https://github.com/netty/netty/archive/netty-4.1.32.Final.tar.gz

Step 2 Decompress the source code package. tar -zxvf netty-4.1.32.Final.tar.gz

Step 3 Go to the directory where the Netty source code package is decompressed. cd netty-netty-4.1.32.Final

Step 4 Compile and package netty-all-4.1.32.Final.jar to the all/target directory. mvn install -DskipTests

----End

Compiling flink-shaded-netty

Step 1 Download the flink-shaded-release-6.0 installation package. wget https://codeload.github.com/apache/flink-shaded/tar.gz/release-6.0

Step 2 Decompress the installation package. mv release-6.0 flink-shaded-release-6.0.tar.gz tar -zxvf flink-shaded-release-6.0.tar.gz

Step 3 Go to the directory generated after the decompression. cd flink-shaded-release-6.0

Step 4 Modify the pom.xml file. You only need to compile and package flink-shaded- netty-4. Therefore, comment out the modules that are not required.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 194 Kunpeng BoostKit for Big Data 22 flink-shaded-netty-4.1.32.Final-6.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 5 Compile and package flink-shaded-netty-4.1.32.Final-6.0.jar to the flink- shaded-netty-4/target/ directory. mvn install package -DskipTests

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 195 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide (CentOS 7.6)

23.1 Introduction 23.2 Requirements 23.3 Configuring the Compilation Environment 23.4 Compiling the frocksdbjni Source Code 23.5 Troubleshooting

23.1 Introduction

frocksdbjni is a rocksdbjni encapsulation library customized for Flink.

23.2 Requirements

Hardware Requirements

Table 23-1 lists the hardware requirements.

Table 23-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 196 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

OS Requirements Table 23-2 lists the requirements for the operating systems (OSs).

Table 23-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

23.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 197 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

----End

Installing OpenJDK

Step 1 Installing OpenJDK yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel Step 2 Configure the Java environment variables. 1. Open the /etc/profile file. vi /etc/profile

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 198 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

2. Add the following code to the end of the /etc/profile file: export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-7.b13.el7.aarch64/jre export PATH=$JAVA_HOME/bin:$PATH 3. Press Esc and run the :wq! command to save the file and exit. 4. Make the modified environment variables take effect. source /etc/profile

----End

Installing Maven

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 199 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

23.4 Compiling the frocksdbjni Source Code

Step 1 Download the frocksdbjni 5.17.2 source code. wget https://github.com/dataArtisans/frocksdb/archive/v5.17.2-artisans-1.0.tar.gz Step 2 Decompress the source code package. tar -zxvf v5.17.2-artisans-1.0.tar.gz Step 3 Go to the directory where the frocksdb source code is located. cd frocksdb-5.17.2-artisans-1.0 Step 4 Modify the Makefile file and change the value of DEBUG_LEVEL to 0.

Step 5 Compile the source code and package the corresponding files. The generated rocksdbjni-5.17.2-linux64.jar file is in the frocksdb-5.17.2-artisans-1.0/java/ target directory. make rocksdbjava -j8

----End

23.5 Troubleshooting

23.5.1 Size of the Compiled JAR Package Is Greater Than the Package Downloaded From the Maven Repository

Symptom The size of the compiled JAR package is greater than the package downloaded from the Maven repository.

Possible Cause The librocksdbjni-linux64.so file in the JAR package contains symbol information.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 200 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Solution

Run following command to remove the symbol information.

strip 23.5.2 jni.h or jni_md.h File Cannot Be Found

Symptom

An error message is displayed indicating that the jni.h or jni_md.h file cannot be found.

Possible Cause

The path for JAVA_HOME is not configured.

Solution

Configure the path for JAVA_HOME.

If the problem persists, move the jni.h and jni_md.h files in the JDK used for compilation to the root directory for compilation output. 23.5.3 "unknown value 'native' for -march" Is Displayed During Compilation

Symptom

An error message is displayed during compilation.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 201 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Solution Run the following command to perform compilation:

PORTABLE=1 make rocksdbjava -j8 23.5.4 Failed to Compare va_list and nullptr

Symptom An error message is displayed during compilation indicating that va_list cannot be compared with nullptr.

Solution Comment out line 151 in java/rocksjni/loggerjnicallback.cc and recompile the source code.

23.5.5 Connect to central.maven.org:80 connection timed out

Symptom "Connect to central.maven.org:80 connection timed out" is displayed during compilation.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 202 Kunpeng BoostKit for Big Data 23 frocksdbjni-5.17.2-artisans-1.0.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Solution Change the Maven repository address in line 193 of the java/Makefile file to http://repo1.maven.org/maven2/.

vim java/Makefile

23.5.6 assertj-core-1.7.1.jar error in opening zip file 1 error

Symptom A message is displayed during compilation indicating a failure in opening assertj- core-1.7.1.jar.

Solution Manually download assertj-core-1.7.1.jar and assertj-core-1.7.1.pom from the Maven repository to the org/assert/assert-core/1.7.1 directory of the local repository. The download URL is as follows: https://repo1.maven.org/maven2/org/assertj/assertj-core/1.7.1/

24 jna-3.0.9,4.1.0.jar Porting Guide (CentOS 7.6)

24.1 Introduction 24.2 Environment Requirements 24.3 Configuring the Compilation Environment 24.4 Compiling jna-3.0.9, 4.1.0.jar 24.5 Rectifying Faults

24.1 Introduction Java Native Access (JNA) provides a set of Java tool classes for dynamically accessing the system native library (such as the Windows DLL) during the running without compiling any native/JNI code. Developers only need to describe the functions and structures of the target native library in a Java interface. The JNA automatically maps the Java interface to the native function.

24.2 Environment Requirements

Hardware Requirements Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements Item Version

CentOS 7.6

OS kernel 4.14.0

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

CMake 3.12.4

24.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

----End

Installing OpenJDK

Step 1 Download and decompress the installation package to a directory (for example, /opt/tools/installed).

wget https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u252-b09/ OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz tar -zxf OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz mkdir -p /opt/tools/installed/ mv jdk8u252-b09 /opt/tools/installed/ Step 2 Configure Java environment variables. vim /etc/profile Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether OpenJDK is successfully installed. java -version The installation is successful if information similar to the following is displayed:

----End

24.4 Compiling jna-3.0.9, 4.1.0.jar

Installing Ant-1.10.8

Step 1 Download and decompress the Ant1.10.8 installation package. wget https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.tar.gz tar -zxvf apache-ant-1.10.8-bin.tar.gz -C /opt/tools/installed/ Step 2 Modify the /etc/profile file. vim /etc/profile Add Ant environment variables at the end of the file. export ANT_HOME=/opt/tools/installed/apache-ant-1.10.8 export PATH=$ANT_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check the Ant version. ant -version

----End

Compiling jna-3.0.9

Step 1 Download the jna-3.0.9 source code and decompress it.

wget https://github.com/java-native-access/jna/archive/3.0.9.tar.gz

Step 2 Decompress the source code package. tar -zxf 3.0.9.tar.gz

Step 3 The libffi version corresponding to jna-3.0.9 does not support aarch64. Download the libffi-3.2.1 source code and replace the jna-3.0.9/jnalib/native/libffi directory. wget https://gcc.gnu.org/pub/libffi/libffi-3.2.1.tar.gz tar -zxf libffi-3.2.1.tar.gz rm -rf jna-3.0.9/jnalib/native/libffi/* cp -r libffi-3.2.1/* jna-3.0.9/jnalib/native/libffi/

Step 4 Go to the jna-3.0.9/jnalib directory. cd jna-3.0.9/jnalib

Step 5 Perform compilation. ant

After the compilation is complete, the jna.jar file is generated in the jna-3.0.9/ jnalib/build-d64 directory.

----End

Compiling jna-4.1.0

Step 1 Download the jna-4.1.0 source code and decompress it. wget https://github.com/java-native-access/jna/archive/4.1.0.tar.gz

Step 2 Decompress the source code package. tar -zxf 4.1.0.tar.gz

Step 3 Go to the jna-4.1.0 directory. cd jna-4.1.0

Step 4 Perform compilation. ant

After the compilation is complete, the jna.jar file is generated in the build directory.

----End

24.5 Rectifying Faults

24.5.1 X11/Xlib.h Cannot Be Found

Symptom

During compilation, the following error message is displayed: fatal error: X11/ Xlib.h: No such file or directory.

Handling Procedure Install libXpm-devel.

yum install libXpm-devel 24.5.2 X11/Intrinsic.h Cannot Be Found

Symptom During compilation, the following error message is displayed: fatal error: X11/ Xlib.h: No such file or directory.

Handling Procedure Install libXt-devel.

yum install libXt-devel 24.5.3 Test Case Failed Due to the Use of the Default Code in Compilation

Symptom During compilation, the following error message is displayed: Libarary'lib? ??.so' at /tmp/jna-3506402/lib?????.so could not be loaded: java.lang.UnsatisfiedLinkError.

Handling Procedure

Step 1 In the test/com/sun/jna/LibaryLoadTest.java file in the source code directory, comment out the corresponding code.

Step 2 Run the compile command. ant

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 210 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

25 server-metrics-0.2.8.jar Porting Guide (CentOS 7.6)

25.1 Introduction 25.2 Environment Requirements 25.3 Configuring the Compilation Environment 25.4 Compiling and Installing server metrics

25.1 Introduction

server metrics is a functional component used to monitor cluster resources.

25.2 Environment Requirements

Hardware Requirements

Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements

Item Version

CentOS 7.6

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 211 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Item Version

OS kernel 4.14.0

GCC 4.8.5

OpenJDK 1.8.0_252

Maven 3.5.4

CMake 3.12.4

25.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 212 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 213 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 214 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Installing ant-1.9.14

Step 1 Download the ant-1.9.14 installation package from the official website. wget https://archive.apache.org/dist/ant/binaries/apache-ant-1.9.14-bin.zip

Step 2 Decompress the installation package. unzip apache-ant-1.9.14-bin.zip

Step 3 Install Ant to a directory (for example, /opt/tools/installed). mv apache-ant-1.9.14 /opt/tools/installed/

Step 5 Check the Ant version. ant -version

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 215 Kunpeng BoostKit for Big Data 25 server-metrics-0.2.8.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

25.4 Compiling and Installing server metrics

Step 1 Download the server metrics source code. wget --no-check-certificate https://github.com/metamx/server-metrics/archive/server- metrics-0.2.8.tar.gz Step 2 Download the dependency SO source code. wget --no-check-certificate https://github.com/hyperic/sigar/archive/sigar-1.6.zip Step 3 Decompress the sigar package. unzip sigar-1.6.zip Step 4 Enter the sigar-sigar-1.6/bindings/java/hyperic_jni/src/org/hyperic/jni directory, and modify ArchNameTask.java. vim sigar-sigar-1.6/bindings/java/hyperic_jni/src/org/hyperic/jni/ArchNameTask.java

Step 5 Go to the sigar-sigar-1.6/bindings/java/ directory to perform compilation. cd sigar-sigar-1.6/bindings/java/ ant Step 6 Copy libsigar-aarch64-linux.so to the server-metrics-server-metrics-0.2.8/src/ main/resources/ directory. cd - tar -zxf server-metrics-0.2.8.tar.gz cp sigar-sigar-1.6/bindings/java/sigar-bin/lib/libsigar-aarch64-linux.so server-metrics-server- metrics-0.2.8/src/main/resources/ Step 7 Compile server metrics. cd server-metrics-server-metrics-0.2.8 mvn install package -DskipTests

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 216 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

26 incremental-compiler-0.13.15.jar Porting Guide (CentOS 7.6)

26.1 Introduction 26.2 Environment Requirements 26.3 Configuring the Compilation Environment 26.4 Compiling sbt-republish

26.1 Introduction

incremental-compiler republishes sbt incremental compiler for use in Maven- based build tools.

26.2 Environment Requirements

Hardware Requirements

Table 26-1 lists the hardware requirements.

Table 26-1 Hardware requirements

Item Description

Server TaiShan servers

CPU Kunpeng 916 or 920 processor

Network Accessible to the Internet

Storage 10 GB or higher

Memory 10 GB or higher

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 217 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

OS Requirements Table 26-2 lists the OS requirements.

Table 26-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

Java 1.8.0_252

Sbt 0.13.15

Maven 3.5.4

26.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 218 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

----End

Installing OpenJDK Step 1 Download and decompress the installation package to a directory (for example, /opt/tools/installed).

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 219 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 220 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Installing sbt

NO TE

The repositories section defines the sequence, in which the Maven repositories are used during sbt compilation. mkdir ~/.sbt cp repositories ~/.sbt The content of the repositories file is as follows:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 221 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext] sbt-plugins-repo: https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/, [organization]/[module]/ (scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]

Step 3 Configure the SBT_HOME environment variable. vim /etc/profile

Add the following information to the profile file:

export SBT_HOME=/opt/tools/installed/sbt export PATH=${SBT_HOME}/bin:${PATH}

Step 4 Make the environment variable take effect. source /etc/profile

----End

26.4 Compiling sbt-republish

Step 1 Download the source code. wget https://github.com/typesafehub/sbt-republish/archive/master.zip -O sbt-republish-master.zip

NO TE

Currently, the version 0.13.15 is only available in the master branch. Pay attention to the version number change when compiling version 0.13.15 by referring to this document.

Step 2 Install the required components. yum install autoconf unzip -y

Step 3 Decompress the source code package. unzip sbt-republish-master.zip

Step 4 Go to the directory where the sbt-republish source code is located. cd sbt-republish-master

Step 5 Compile the source code. mkdir -p ~/.ivy2/cache/jline/jline/jars/ wget http://mirrors.huaweicloud.com/kunpeng/maven/jline/jline/2.14.3/jline-2.14.3.jar -O ~/.ivy2/ cache/jline/jline/jars/jline-2.14.3.jar wget http://mirrors.huaweicloud.com/kunpeng/maven/jline/jline/2.14.3/jline-2.14.3.pom -O ~/.ivy2/ cache/jline/jline/jars/jline-2.14.3.pom sbt package

The operation is successful if information similar to the following is displayed:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 222 Kunpeng BoostKit for Big Data 26 incremental-compiler-0.13.15.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

NO TE

The compiled incremental-compiler-0.13.15.jar file is in incremental-compiler/target/ incremental-compiler-0.13.15.jar.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 223 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

27 flink-shaded- netty-4.1.32.Final-7.0.jar Porting Guide (CentOS 7.6)

27.1 Introduction 27.2 Requirements 27.3 Configuring the Compilation Environment 27.4 Compiling and Installing flink-shaded-netty

27.1 Introduction

Overview

It is a component of the big data component flink-shaded.

Recommended Version

4.1.32.Final-7.0

27.2 Requirements

Hardware Requirements

Table 27-1 lists the hardware requirements.

Table 27-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 224 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Item Description

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 27-2 lists the requirements for the operating systems (OSs).

Table 27-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

27.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 225 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

yum clean all yum makecache Step 5 Use the Yum source to install GCC related software. yum -y install gcc.aarch64 gcc-c++.aarch64 gcc-gfortran.aarch64 libgcc.aarch64 make cmake libtool autoconf automake ant wget git vim Step 6 Resolve the -fsigned-char problem (by modifying the GCC). 1. Search for the directory where GCC is located. Generally, the directory is /usr/bin/gcc. command -v gcc 2. Change the name of original GCC file, for example, to gcc-impl. mv /usr/bin/gcc /usr/bin/gcc-impl 3. Create a new GCC file. vi /usr/bin/gcc Add the following information to the file and save the file: #! /bin/sh /usr/bin/gcc-impl -fsigned-char "$@" 4. Add the execute permission for the GCC file. chmod +x /usr/bin/gcc 5. Check whether the GCC is available. gcc --version The installation is successful if information similar to the following is displayed:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 226 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 227 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

27.4 Compiling and Installing flink-shaded-netty

Step 1 Download the flink-shaded-release-7.0 source code package. wget https://codeload.github.com/apache/flink-shaded/tar.gz/release-7.0 mv release-7.0 flink-shaded-release-7.0.tar.gz Step 2 Decompress the source code package. tar -zxvf flink-shaded-release-7.0.tar.gz

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 228 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

Step 3 Switch to the directory where the source code package is decompressed. cd flink-shaded-release-7.0 Step 4 Modify the flink-shaded-release-7.0/pom.xml file. You only need to compile and build flink-shaded-netty-4 and comment out other modules. vim pom.xml

Step 5 Modify the flink-shaded-release-7.0/flink-shaded-netty-4/pom.xml file. vim flink-shaded-netty-4/pom.xml Add the Kunpeng Maven repository in the first line under repositories.

kunpengmaven kunpeng maven https://mirrors.huaweicloud.com/kunpeng/maven

Step 6 Compile and package the modules, and place the package in the local Maven repository. mvn install If the operation is successful, information similar to the following is displayed:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 229 Kunpeng BoostKit for Big Data 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Porting Guides (Third-Party Dependency Libraries) Guide (CentOS 7.6)

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 230 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

28 flink-shaded-netty-tcnative- dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS 7.6)

28.1 Introduction 28.2 Requirements 28.3 Configuring the Compilation Environment 28.4 Compiling flink-shaded-release-7.0

28.1 Introduction

It is a module of the big data component flink-shaded.

28.2 Requirements

Hardware Requirements

Table 28-1 lists the hardware requirements.

Table 28-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 231 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

OS Requirements Table 28-2 lists the requirements for the operating systems (OSs).

Table 28-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

28.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 232 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 233 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 234 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

28.4 Compiling flink-shaded-release-7.0

Step 1 Download the flink-shaded-release-7.0 source code. wget https://codeload.github.com/apache/flink-shaded/tar.gz/release-7.0 mv release-7.0 flink-shaded-release-7.0.tar.gz Step 2 Decompress the source code package. tar -zxvf flink-shaded-release-7.0.tar.gz Step 3 Go to the directory generated after the decompression. cd flink-shaded-release-7.0 Step 4 Modify the flink-shaded-release-7.0/pom.xml file. You only need to compile and build flink-shaded-netty-tcnative-dynamic. Therefore, comment out other modules. vim pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 235 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 5 Modify the flink-shaded-release-7.0/flink-shaded-netty-tcnative-dynamic/ pom.xml file. vim flink-shaded-netty-tcnative-dynamic/pom.xml 1. Insert the following code below line 94: io.netty netty-tcnative ${netty.tcnative.version} linux-aarch_64 jar false ${project.build.directory}/native_libs

2. Insert the following code below line 156:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 236 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

3. Comment out lines 175 to 198.

4. In line 211, add a reference to the Kunpeng Maven repository. kunpengmaven kunpeng maven https://mirrors.huaweicloud.com/kunpeng/maven

Step 6 Compile and build the software package in the local Maven repository. mvn install The operation is successful if information similar to the following is displayed:

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 237 28 flink-shaded-netty-tcnative- Kunpeng BoostKit for Big Data dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

29 jruby-1.7.27.jar Porting Guide (CentOS 7.6)

29.1 Introduction 29.2 Environment Requirements 29.3 Configuring the Compilation Environment 29.4 Compiling JRuby

29.1 Introduction JRuby is a Ruby-oriented interpreter based on the Java Virtual Machine (JVM). It combines the simplicity of the Ruby language and the powerful JVM execution mechanism, including full integration with the Java library. jruby.jar is an intermediate product of jruby-core-1.7.27.jar. That is, jruby.jar is compiled from jruby-core-1.7.27.jar.

29.2 Environment Requirements

Hardware Requirements Item Remarks

Server TaiShan server

CPU Huawei Kunpeng 920 processor or Huawei Kunpeng 916 processor

Drive partition No requirement for drive partitions

Network Accessible to the Internet

Software Requirements

Item Version

CentOS 7.6

OS kernel 4.14.0

OpenJDK 1.8.0_252

Maven 3.5.4

29.3 Configuring the Compilation Environment

29.3.1 Configuring the Local Yum Source

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name.

Step 2 Back up the Yum repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to agree to the deletion.

Step 3 Create the /etc/yum.repos.d/Local.repo file. vi /etc/yum.repos.d/Local.repo

Add the following content to the file to configure the local Yum source:

[Local] name=CentOS-7.6 Local baseurl=file:///media/ enabled=1 gpgcheck=0

Step 4 Make the Yum source configuration take effect. yum clean all yum makecache

Step 5 Use the Yum source to install related software. yum -y install wget vim

----End

29.3.2 Installing OpenJDK Step 1 Download and decompress the installation package to a directory (for example, /opt/tools/installed/). wget https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u252-b09/ OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz tar -zxf OpenJDK8U-jdk_aarch64_linux_hotspot_8u252b09.tar.gz mkdir -p /opt/tools/installed/ mv jdk8u252-b09 /opt/tools/installed/ Step 2 Configure Java environment variables. vim /etc/profile Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether OpenJDK is successfully installed. java -version The installation is successful if information similar to the following is displayed:

----End 29.3.3 Installing Maven Step 1 Download the installation package and install Maven to a directory (for example, /opt/tools/installed/). wget https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz tar -zxf apache-maven-3.5.4-bin.tar.gz mv apache-maven-3.5.4 /opt/tools/installed/ Step 2 Modify the Maven environment variables. vim /etc/profile Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH Step 3 Make the environment variables take effect. source /etc/profile Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml.

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to settings.xml:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

29.4 Compiling JRuby

NO TE

The jruby.jar package, on which the earlier Logstash version depends, does not support aarch64. As a result, the error message "Unsupported platform: unknown-linux" will be displayed when Logstash is started.

Step 1 Download the source code and decompress it. wget https://github.com/jruby/jruby/archive/1.7.27.tar.gz -O jruby-1.7.27.tar.gz tar -zxvf jruby-1.7.27.tar.gz Step 2 Go to the directory decompressed. cd jruby-1.7.27 Step 3 Modify the source code core/src/main/java/org/jruby/ext/ffi/Platform.java. vim core/src/main/java/org/jruby/ext/ffi/Platform.java 1. Add "AARCH64" to line 87.

2. Add the AARCH64 fork to lines 149 to 150.

3. Add the AARCH64 fork to line 209.

Step 4 Save the file and exit. Step 5 Modify the pom.xml file. vim pom.xml In line 71, change the jffi version from 1.2.12 to 1.2.13. (The jffi version 1.2.13 supports aarch64.)

1.2.13

Step 6 Modify the core/pom.xml file. vim core/pom.xml In line 101, change the version of jnr-ffi from 2.0.9 to 2.1.0.

jnr-ffi 2.1.0

Step 7 Perform the compilation. mvn package The operation is successful if information similar to the following is displayed:

The compiled jruby.jar file is in the lib directory.

Step 8 Use the Kunpeng Porting Advisor to scan jruby.jar.

NO TE

jruby.jar is compatible with multiple platforms. It contains both x86 and aarch64 .so files. Therefore, it can be used if x86 .so files are found. For details about how to use the Kunpeng Porting Advisor, see the Kunpeng Porting Advisor Case Study.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 245 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

30 netty-all-4.1.3.Final.jar Porting Guide (CentOS 7.6)

30.1 Introduction 30.2 Requirements 30.3 Configuring the Compilation Environment 30.4 Compiling and Installing Netty

30.1 Introduction Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

30.2 Requirements

Hardware Requirements Table 30-1 lists the hardware requirements.

Table 30-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 246 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

OS Requirements Table 30-2 lists the requirements for the operating systems (OSs).

Table 30-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

30.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 247 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 248 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 249 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

30.4 Compiling and Installing Netty

Installing APR

Step 1 Download the apr-1.5.2 source code. wget https://archive.apache.org/dist/apr/apr-1.5.2.tar.gz Step 2 Decompress apr-1.5.2.tar.gz. tar -xvzf apr-1.5.2.tar.gz Step 3 Go to the apr-1.5.2 directory. cd apr-1.5.2 Step 4 Install APR to the /usr/local/apr directory. ./configure --prefix=/usr/local/apr make && make install

----End

Compiling and Installing netty-tcnative-1.1.33.Fork19.jar

Step 1 Download the netty-tcnative-1.1.33.Fork19.zip source code package and decompress it. wget https://github.com/netty/netty-tcnative/archive/netty-tcnative-1.1.33.Fork19.zip unzip netty-tcnative-1.1.33.Fork19.zip

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 250 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 2 Modify the link for downloading the apr-1.5.2.tar.gz package in the pom.xml file to prevent access timeout. cd netty-tcnative-netty-tcnative-1.1.33.Fork19 vim pom.xml

Modify the following:

Replace http://www.us.apache.org with https://archive.apache.org.

Step 3 Download openssl-1.0.2h to netty-tcnative-netty-tcnative-1.1.33.Fork19/ openssl-static/target. wget https://ftp.openssl.org/source/old/1.0.2/openssl-1.0.2h.tar.gz mkdir -p netty-tcnative-netty-tcnative-1.1.33.Fork19/openssl-static/target mv openssl-1.0.2h.tar.gz netty-tcnative-netty-tcnative-1.1.33.Fork19/openssl-static/target

Step 4 Comment out the compilation part of the boringssl-static module in the pom.xml file. vim netty-tcnative-netty-tcnative-1.1.33.Fork19/pom.xml

Step 5 Compile and install netty-tcnative-1.1.33.Fork19.jar to the local Maven repository. mvn install -DskipTests

The compiled package netty-tcnative-1.1.33.Fork19.jar is in the openssl- dynamic/target/ directory.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 251 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Compiling netty-all-4.1.3.Final.jar

Step 1 Download the source code package and decompress it. wget https://github.com/netty/netty/archive/netty-4.1.3.Final.tar.gz tar -zxvf netty-4.1.3.Final.tar.gz

Step 2 Modify the pom.xml file. vim netty-netty-4.1.3.Final/pom.xml 1. In the pom.xml file, comment out the JDK information related to the x86_64 version.

2. Comment out the content of Fedora in the pom.xml file.

Step 3 Compile netty-all-4.1.3.Final.jar. cd netty-netty-4.1.3.Final mvn package -DskipTests

You can view the compiled netty-all-4.1.3.Final.jar package in the netty- netty-4.1.3.Final/all/target directory.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 252 Kunpeng BoostKit for Big Data 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS Porting Guides (Third-Party Dependency Libraries) 7.6)

Step 4 Use the Kunpeng Porting Advisor to scan the package generated after the compilation and ensure that the package does not contain the x86 .so or .jar packages.

NO TE

The compiled netty-all.4.1.3.Final.jar must be scanned by using the Kunpeng Porting Advisor to ensure that no x86 .so or .jar packages are contained. If the compiled package contains x86 .so or .jar packages, the component functions may be affected. For details about how to use the Kunpeng Porting Advisor, see the Kunpeng Porting Advisor Case Study.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 253 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

31 netty-tcnative-1.1.33.Fork19.jar Porting Guide (CentOS 7.6)

31.1 Introduction 31.2 Requirements 31.3 Configuring the Compilation Environment 31.4 Compiling and Installing Netty

31.1 Introduction Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

31.2 Requirements

Hardware Requirements Table 31-1 lists the hardware requirements.

Table 31-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 254 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

OS Requirements Table 31-2 lists the requirements for the operating systems (OSs).

Table 31-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

31.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 255 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 256 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 257 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

31.4 Compiling and Installing Netty

Installing APR

----End

Compiling and Installing netty-tcnative-1.1.33.Fork19.jar

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 258 Kunpeng BoostKit for Big Data 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide Porting Guides (Third-Party Dependency Libraries) (CentOS 7.6)

Step 2 Modify the link for downloading the apr-1.5.2.tar.gz package in the pom.xml file to prevent access timeout. cd netty-tcnative-netty-tcnative-1.1.33.Fork19 vim pom.xml

Modify the following:

Replace http://www.us.apache.org with https://archive.apache.org.

Step 4 Comment out the compilation part of the boringssl-static module in the pom.xml file. vim netty-tcnative-netty-tcnative-1.1.33.Fork19/pom.xml

Step 5 Compile and install netty-tcnative-1.1.33.Fork19.jar to the local Maven repository. mvn install -DskipTests

The compiled package netty-tcnative-1.1.33.Fork19.jar is in the openssl- dynamic/target/ directory.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 259 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

32 netty-tcnative-boringssl- static-1.1.33.Fork23.jar Porting Guide (CentOS 7.6)

32.1 Introduction 32.2 Requirements 32.3 Configuring the Compilation Environment 32.4 Compiling and Installing CMake 32.5 Installing ninja-build 32.6 Installing Go 32.7 Performing Compilation and Installation 32.8 Troubleshooting

32.1 Introduction Netty is an asynchronous event-driven network application framework for quickly developing maintainable high-performance protocol servers and clients.

32.2 Requirements

Hardware Requirements Table 32-1 lists the hardware requirements.

Table 32-1 Hardware requirements

Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 260 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Item Description

Network Accessible to the Internet

Storage No requirement

Memory No requirement

OS Requirements Table 32-2 lists the requirements for the operating systems (OSs).

Table 32-2 OS requirements

Item Version

CentOS 7.6

Kernel 4.14.0

32.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 261 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 262 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

----End

Installing OpenJDK

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 263 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 4 Check whether Maven is successfully installed. mvn -v The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file. Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

32.4 Compiling and Installing CMake Step 1 Download and decompress the CMake source code. wget https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz tar -zxf cmake-3.5.2.tar.gz Step 2 Compile the source code and install CMake. cd cmake-3.5.2 ./bootstrap && make && make install

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 264 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 3 Clear the hash of the system. Otherwise, the CMake of an earlier version may be referenced. hash -r Step 4 Check whether CMake is successfully installed. cmake --version The installation is successful if information similar to the following is displayed:

----End

32.5 Installing ninja-build Step 1 Install epel-release. yum install -y epel-release Step 2 Install ninja-build. yum install -y ninja-build Step 3 Create a symbolic link. ln -sf /bin/ninja-build /bin/ninja Step 4 Check whether ninja-build is successfully installed. ninja --version The installation is successful if information similar to the following is displayed:

----End

32.6 Installing Go

NO TE

To compile BoringSSL, the Go version must be 1.13 or later.

Step 1 Download and decompress the Go installation package. wget https://dl.google.com/go/go1.14.2.linux-arm64.tar.gz tar -zxf go1.14.2.linux-arm64.tar.gz Step 2 Create an installation path and perform deployment. mkdir -p /opt/install mv go /opt/install Step 3 Set environment variables. vim /etc/profile Add the following content to the end of the file:

export GO_HOME=/opt/install/go export PATH=$GO_HOME/bin:$PATH

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 265 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 4 Make the environment variables take effect. source /etc/profile Step 5 Check whether Go is successfully installed. go version The installation is successful if information similar to the following is displayed:

----End

32.7 Performing Compilation and Installation

Installing APR

----End

Compiling netty-tcnative-boringssl-static-1.1.33.Fork23.jar

Step 1 Download and decompress the netty-tcnative-parent-1.1.33.Fork23 source code. wget https://github.com/netty/netty-tcnative/archive/netty-tcnative-parent-1.1.33.Fork23.tar.gz tar -zxvf netty-tcnative-parent-1.1.33.Fork23.tar.gz Step 2 Modify the apr-1.5.2.tar.gz download link in the pom.xml file to avoid access timeout. cd netty-tcnative-netty-tcnative-parent-1.1.33.Fork23 vim pom.xml Modify the following: Replace http://www.us.apache.org with https://archive.apache.org.

Step 3 Modify the link for downloading the BoringSSL source code in the pom.xml file to avoid the access timeout problem. vim boringssl-static/pom.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 266 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Content to be modified is as follows: Replace https://boringssl.googlesource.com/boringssl with https://github.com/ google/boringssl.

Step 4 Download openssl-1.0.2j and place it in netty-tcnative-netty- tcnative-1.1.33.Fork19/openssl-static/target. wget https://ftp.openssl.org/source/old/1.0.2/openssl-1.0.2j.tar.gz mkdir -p openssl-static/target mv openssl-1.0.2j.tar.gz openssl-static/target Step 5 Perform the compilation. mvn package -DskipTests

The compilation target netty-tcnative-boringssl-static-1.1.33.Fork23.jar is in the netty-tcnative-netty-tcnative-parent-1.1.33.Fork23/boringssl-static/target directory.

Step 6 Use Kunpeng Porting Advisor to scan the package generated after the compilation and ensure that the package does not contain the x86 .so or .jar packages.

NO TE

The compiled netty-tcnative-boringssl-static-1.1.33.Fork23.jar package must be scanned by Kunpeng Porting Advisor to ensure that the package does not contain the x86 .so or .jar packages. If the compiled directory contains x86 .so or .jar packages, the component functions may be affected. For details about how to use the Kunpeng Porting Advisor, see the Kunpeng Porting Advisor Case Study.

----End

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 267 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

32.8 Troubleshooting

32.8.1 Failed to Download the go get golang.org/x Package

Symptom Failed to download the go get golang.org/x package.

Procedure Configure Go-related environment variables and use a proxy to resolve the network access timeout issue.

vim /etc/profile Configure the following environment variables:

export GOPROXY=https://goproxy.io export GO111MODULE=on 32.8.2 Compilation Failed Due to an Alarm Generated During the Compilation

Symptom The compilation fails due to an alarm.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 268 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Procedure

Step 1 Delete the compilation parameter -Wshadow/–Werror. Other compilation modules share the openssl-dynamic code. You only need to modify the code in this module. vim openssl-dynamic/src/main/native-package/configure.ac

Delete the -Werror compilation parameter.

Step 2 Delete the -Werror compilation parameter. vim openssl-dynamic/src/main/native-package/m4/apr_common.m4

Step 3 Delete the -Wshadow compilation parameter. vim boringssl-static/target/boringssl-chromium-stable/CMakeLists.txt

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 269 Kunpeng BoostKit for Big Data 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")

----End 32.8.3 Compilation Error Occurs Because TLS_ST_SR_CLNT_HELLO Is Not Defined

Symptom TLS_ST_SR_CLNT_HELLO is not defined. As a result, a compilation error is reported.

Procedure vim openssl-dynamic/src/main/c/sslutils.c Change TLS_ST_SR_CLNT_HELLO to 20.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 270 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

33 hive-warehouse-connector- assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guide (CentOS 7.6)

33.1 Introduction 33.2 Requirements 33.3 Configuring the Compilation Environment 33.4 Compilation Process 33.5 Rectifying Faults

33.1 Introduction A third-party dependency library that uses LLAP to read and write Apache Hive.

33.2 Requirements

Hardware Requirements Table 33-1 lists the hardware requirements.

Table 33-1 Hardware requirements Item Description

CPU Kunpeng 916 processor or Kunpeng 920 processor

Network Accessible to the Internet

Storage No requirement

Memory No requirement

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 271 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

OS Requirements Table 33-2 lists the requirements for the operating systems (OSs).

Table 33-2 OS requirements Item Version

CentOS 7.6

Kernel 4.14.0

33.3 Configuring the Compilation Environment

Installing Dependencies

Step 1 Mount the OS image. mount YOUR_OS.iso /media -o loop

NO TE

Replace YOUR_OS.iso with the actual iso package name. Step 2 Back up the repo file and clear the /etc/yum.repos.d/ directory. cp -r /etc/yum.repos.d /etc/yum.repos.d-bak rm /etc/yum.repos.d/*

NO TICE

Ensure that all repo files have been backed up. Enter y on the rm deletion page to delete the files.

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 272 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

----End

Installing OpenJDK

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 273 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 2 Configure Java environment variables. vim /etc/profile

Add the following to the end of the file: export JAVA_HOME=/opt/tools/installed/jdk8u252-b09 export PATH=$JAVA_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether OpenJDK is successfully installed. java -version

The installation is successful if information similar to the following is displayed:

----End

Installing Maven

Step 2 Modify the Maven environment variables. vim /etc/profile

Add the following at the end of the /etc/profile file: export MAVEN_HOME=/opt/tools/installed/apache-maven-3.5.4 export PATH=$MAVEN_HOME/bin:$PATH

Step 3 Make the environment variables take effect. source /etc/profile

Step 4 Check whether Maven is successfully installed. mvn -v

The installation is successful if information similar to the following is displayed:

Step 5 Modify the local repository path and remote repository in the Maven configuration file.

Configuration file path: /opt/tools/installed/apache-maven-3.5.4/conf/ settings.xml

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 274 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

NO TE

huaweimaven huawei maven https://mirrors.huaweicloud.com/repository/maven/ central

If the compilation environment cannot access Internet, add the following proxy configuration to the settings.xml file:

optional true http Username Password Proxy server URL Proxy server port local.net|some.host.com

----End

Installing sbt

Step 2 Create the .sbt folder in the root directory of the current user and copy the repositories file to the .sbt folder.

NO TE

The repositories section defines the sequence, in which the Maven repositories are used during sbt compilation. mkdir ~/.sbt cp repositories ~/.sbt

The content of the repositories file is as follows:

[repositories] local kunpeng-maven: https://mirrors.huaweicloud.com/kunpeng/maven/ huawei-maven: https://mirrors.huaweicloud.com/repository/maven/ sbt-releases-repo: https://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/ (scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext] sbt-plugins-repo: https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/, [organization]/[module]/ (scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 275 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

Step 3 Configure the SBT_HOME environment variable. vim /etc/profile Add the following information to the profile file:

export SBT_HOME=/opt/tools/installed/sbt export PATH=${SBT_HOME}/bin:${PATH} Step 4 Make the environment variable take effect. source /etc/profile

----End

33.4 Compilation Process

Prerequisites You have downloaded and decompressed the hive-warehouse-connector- assembly source package.

Compiling hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78.jar

Step 1 Go to the source code directory. cd hive-warehouse-connector-release-HDP-3.1.0.0-78-tag Step 2 Modify the build.sbt file. vim build.sbt 1. Change the version number at the end of line 7. val versionString = sys.props.getOrElse("version","1.0.0.3.1.0.0-78")

2. Change the version numbers of Hadoop, Hive, and Tez in lines 15, 16, and 18. val hadoopVersion = sys.props.getOrElse("hadoop.version", "3.1.1.3.1.0.0-78") val hiveVersion = sys.props.getOrElse("hive.version", "3.1.0.3.1.0.0-78") .... val tezVersion = sys.props.getOrElse("tez.version", "0.9.1.3.1.0.0-78")

3. Add the Kunpeng Maven and Huawei Maven in lines 338 and 339. resolvers += "Kunpeng Maven" at "https://mirrors.huaweicloud.com/kunpeng/maven" resolvers += "huawei Maven" at "https://mirrors.huaweicloud.com/repository/maven"

4. Add the following statement to line 342: externalResolvers := Resolver.withDefaultResolvers(resolvers.value, mavenCentral = false)

Step 3 Press Esc to exit the editing mode and save the modification. :wq Step 4 Perform compilation. sbt assembly

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 276 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

The compiled package is stored in the ./target/scala-2.11/ directory.

----End

Compiling hive-warehouse-connector-assembly-1.0.0.3.0.1.0-187.jar

Step 1 Go to the source code directory. cd hive-warehouse-connector-release-HDP-3.0.1.0-187-tag Step 2 Modify the build.sbt file. vim build.sbt 1. Change the version number at the end of line 7. val versionString = sys.props.getOrElse("version","1.0.0.3.0.1.0-187")

2. Change the version numbers of Hadoop, Hive, and Tez in lines 15, 16, and 18. val hadoopVersion = sys.props.getOrElse("hadoop.version", "3.1.1.3.0.1.0-187") val hiveVersion = sys.props.getOrElse("hive.version", "3.1.0.3.0.1.0-187") .... val tezVersion = sys.props.getOrElse("tez.version", "0.9.1.3.0.1.0-187")

4. Add the following statement to line 342: externalResolvers := Resolver.withDefaultResolvers(resolvers.value, mavenCentral = false)

Step 3 Press Esc to exit the editing mode and save the modification. :wq Step 4 Perform compilation. sbt assembly

The compiled package is stored in the ./target/scala-2.11/ directory.

----End

33.5 Rectifying Faults

Issue 06 (2020-12-30) Copyright © Huawei Technologies Co., Ltd. 277 33 hive-warehouse-connector- Kunpeng BoostKit for Big Data assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guides (Third-Party Dependency Libraries) Porting Guide (CentOS 7.6)

33.5.1 Failed to Download hive-jdbc

Symptom Failed to download hive-jdbc.

Handling Procedure If the download fails due to the network environment, perform the compilation again. Then, you can download it successfully.

sbt assembly

NO TE

Problems such as download failure can be resolved by recompiling.

A Change History

Date Description

2020-12-30 This issue is the sixth official release. Changed "Kunpeng Code Scanner" to "Kunpeng Porting Advisor".

2020-11-05 This issue is the fifth official release. Added: ● 29 jruby-1.7.27.jar Porting Guide (CentOS 7.6) ● 30 netty-all-4.1.3.Final.jar Porting Guide (CentOS 7.6) ● 31 netty-tcnative-1.1.33.Fork19.jar Porting Guide (CentOS 7.6) ● 32 netty-tcnative-boringssl-static-1.1.33.Fork23.jar Porting Guide (CentOS 7.6) ● 33 hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78, 1.0.0.3.0.1.0-187.jar Porting Guide (CentOS 7.6)

2020-08-14 This issue is the fourth official release. Added: ● 27 flink-shaded-netty-4.1.32.Final-7.0.jar Porting Guide (CentOS 7.6) ● 28 flink-shaded-netty-tcnative-dynamic-2.0.25.Final-7.0.jar Porting Guide (CentOS 7.6)

2020-06-23 This issue is the second official release. Added: 26 incremental-compiler-0.13.15.jar Porting Guide (CentOS 7.6)

2020-05-23 This issue is the second official release. Added the path for obtaining the CheckSo script in 1.1 Introduction of the 1 CheckSo User Guide.

2020-03-20 This issue is the first official release.