Landslide Susceptibility Mapping Using the Slope Unit for Southeastern Helong City, Jilin Province, China: a Comparison of ANN and SVM

S S symmetry

Article Landslide Susceptibility Mapping Using the Slope Unit for Southeastern Helong City, Jilin Province, China: A Comparison of ANN and SVM

Chenglong Yu 1,2 and Jianping Chen 1,*

1 College of Construction Engineering, Jilin University, Changchun 130026, China; [email protected] 2 Jilin Team of Geological Survey Center of China Building Materials Industrial, Changchun 130026, China * Correspondence: [email protected]; Tel.: +86-0431-8850-2353

Received: 1 June 2020; Accepted: 21 June 2020; Published: 23 June 2020

Abstract: The purpose of this study is to produce a landslide susceptibility map of Southeastern Helong City, Jilin Province, Northeastern China. According to the geological hazard survey (1:50,000) project of Helong city, a total of 83 landslides were mapped in the study area. The slope unit, which is classified based on the curvature watershed method, is selected as the mapping unit. Based on field investigations and previous studies, three groups of influencing Factors—Lithological factors, topographic factors, and geological environment factors (including ten influencing factors)—are selected as the influencing factors. Artificial neural networks (ANN’s) and support vector machines (SVM’s) are introduced to build the landslide susceptibility model. Five-fold cross-validation, the receiver operating characteristic curve, and statistical parameters are used to optimize model. The results show that the SVM model is the optimal model. The landslide susceptibility maps produced using the SVM model are classified into five grades—very high, high, moderate, low, and very low—and the areas of the five grades were 127.43, 151.60, 198.77, 491.19, and 506.91 km2, respectively. The very high and high susceptibility areas included 79.52% of the total landslides, demonstrating that the landslide susceptibility map produced in this paper is reasonable. Consequently, this study can serve as a guide for landslide prevention and for future land planning in the southeast of Helong city.

Keywords: landslide susceptibility mapping; artiﬁcial neural networks; support vector machines; ﬁve-fold cross-validation; receiver operating characteristic curve; statistical parameters

1. Introduction Landslides are one of the most common geological disasters in the mountainous areas of China [1–3]. In recent years, the Chinese economy has developed steadily and rapidly, and has become the second largest economy in the world. With such rapid development of the economy, many previously inaccessible areas have carried out corresponding infrastructure construction and various engineering activities in succession. Therefore, it is necessary to evaluate the risk of natural disasters such as landslides in these areas. Landslide susceptibility mapping is the basis of landslide hazard and risk assessment [4]. The purpose of landslide susceptibility mapping is to answer the question “what is the geological background of landslides and where are the areas which are most prone to them?” [2]. Based on the review of relevant papers, it was found that landslide susceptibility mapping mainly includes the following [5,6]: (a) landslide inventory data acquisition; (b) mapping unit selection; (c) inﬂuencing factor selection; (d) establishment of the evaluation model; and (e) production of the landslide susceptibility map. Landslide inventory data is the basis of landslide susceptibility mapping. The early acquisition of landslide inventory data mainly depends on ﬁeld geological disaster surveys. With the development

Symmetry 2020, 12, 1047; doi:10.3390/sym12061047 www.mdpi.com/journal/symmetry Symmetry 2020, 12, 1047 2 of 23 of remote sensing technology, landslide interpretation based on remote sensing technology has become one of the most popular tools to quickly acquire landslide inventory data [7,8]. Remote sensing techniques commonly used in landslide interpretation mainly include (a) visible optical remote sensing and (b) InSAR (Interferometric Synthetic Aperture Radar) remote sensing. The key to remote sensing interpretation of landslides based on visible optics is to interpret the landslide based on the special topographic and geomorphic features of the landslide in the remote sensing image. However, this method is greatly affected by the spatial resolution of remote sensing data and is unable to identify the surface deformation characteristics (i.e., the potential and slow deformation of the landslide); thus, it has certain limitations. However, landslide interpretation based on InSAR technology can effectively identify the surface deformation and identify the potential landslides that are under deformation [9], which makes up for the limitations of visible optical remote sensing interpretation. The combination of these two methods has become a common method for landslide interpretation. As for mapping units, they are the basic units in landslide susceptibility mapping [10]. At present, the commonly used mapping units are the grid unit, watershed unit, slope unit, region unit, and uniform condition unit [4]. Among these, the grid unit is the most widely used but, due to its lack of connection with the topography and other geological information, the slope unit [11,12] has become more and more respected by relevant scholars. The selection of influencing factors is one of the key issues in the study of landslide susceptibility mapping [13]. The choice of influencing factors used in the evaluation will affect the final mapping results. The selection of influencing factors mostly relies on field investigation, cause analysis, expert experience, and related research of landslides; the factors with the highest correlation with the occurrence of landslides in the study area are typically selected as the influencing factors to evaluate the susceptibility to landslides [2]. At present, many scholars also use methods of automatic forward selection of parameters that weight and evaluate the effectiveness of each influencing factor and make use only of the “best” ones [14,15]. The evaluation models applied to landslide susceptibility mapping are mainly divided into qualitative and quantitative evaluation [6,16]. Qualitative evaluation is mainly based on the knowledge and experience of relevant experts, where the susceptibility to geological hazards such as landslides is determined by means of grading and weighting [17,18]. Quantitative evaluation methods include statistics, machine learning, deep mining, and other methods [6]. With the rapid improvement of calculation ability, an increasing number of quantitative evaluation methods have been applied to landslide susceptibility mapping. In addition, evaluation methods of the performance of landslide susceptibility models have also been developed rapidly [16,19]. Finally, the landslide susceptibility map can be produced according to the established model. According to the basic steps of landslide susceptibility mapping, this paper carries out a relative evaluation of the southeast of Helong City, Jilin Province, northeastern China. Firstly, according to the geological hazard survey (1:50,000) project, 83 landslides were mapped in the study. Then, the slope unit classified by the curvature watershed method was applied as the mapping unit, which can identify the horizontal surface and the inclined surface. Moreover, the area of the slope unit is more concentrated, the shapes mostly range between isosceles triangles and squares, and there are few elongated units. Next, ten influencing factors were selected. Artificial neural networks (ANN) and support vector machines (SVM) were introduced to build the landslide susceptibility model. Five-fold cross-validation, the receiver operating characteristic curve (ROC), and statistical parameters were used to optimize the model. Finally, the landslide susceptibility map was divided into five classes. This study can serve as a guide for landslide prevention and for future land planning in the southeast of Helong city.

2. Study Area The study area is in the Chongshan and Nanping counties of the southeast Helong city, Jinlin province, China, along the Tumen river. The study area is marked by valleys and rugged topography, and covers an area of 1476 km2 (Figure1a–c). More than 90% of the total area of Symmetry 2020, 12, 1047 3 of 23 the study area consists of mountainous areas. In the study area, the maximum elevation is 1450 m, and the minimum elevation is 350 m. The southern part of the study area is the Chinese and Korean quasi-platform, and the northern part is the Jihei fold system in the Tianshan-Xingan geosyncline fold area, which are bounded by the deep and large fault of Gudong River. The study area belongs to the temperate monsoon sub-humid climate zone. Based on rainfall data collected from 1960 to 2012, the maximum daily rainfall of the study area is 164.8 mm. The perennial average temperature is 5.6 ◦C. The vegetation coverage in the study area is relatively high. The seismic intensity of the study area has a degree of VI on the modified Mercalli index. At present, no earthquake-induced landslides have been detected in the study area. According to the field investigation, the landslides in the study area are mainly distributed along the Tumen River. According to a geological map (downloaded using the 91 Weitu software, with a scale of 1:500,000) (Figure1d), it can be seen that the mainly exposed strata in the study area are Quaternary (Q), Neogene (N), Cretaceous (K), Jurassic (J), Middle Proterozoic (Pt), and New Archean (Ar). The lithological information of the study area is listed in Table1. There are several reverse faults in the study area, which affect the regional stability. The Tumen river and its tributariesSymmetry flow through2020, 12, x FOR the PEER study REVIEW area, which has a significant impact on landslide risk.4 of 24

Figure 1. GeographicalFigure 1. Geographical position position and and landslide landslide inventory inventory of the study study area. area. (a) and (a,b (b):): geographicalgeographical position position of the study area; (c) landslide inventory of the study area; (c) geological map of the study of the study area; (c) landslide inventory of the study area; (d) geological map of the study area; area; (e) and (f) typical landslides of the study area. (e,f) typical landslides of the study area. 3. Methodology Figure 2 shows the three main steps of this paper, namely (a) data preparation, (b) landslide susceptibility modelling, and (c) validation and selection of the optimal models.

Symmetry 2020, 12, 1047 4 of 23

Table 1. Description of the Lithology.

System Code Lithology Quaternary Q Alluvial–Diluvial, Gravel, Sub-Sandy Soil, Sub-Clay, and Basalt Neogene N Sandstone, Conglomerate, and Siltstone with Basalt Cretaceous K Sandstone, Conglomerate, Siltstone with Limestone, and Oil Limestone Jurassic J Andesite and Tuﬀ Middle Proterozoic Pt Marble New Archean Ar Black Cloud Amphibolic Granulite and Granulite

3. Methodology Figure2 shows the three main steps of this paper, namely (a) data preparation, (b) landslide susceptibility modelling, and (c) validation and selection of the optimal models. Symmetry 2020, 12, x FOR PEER REVIEW 5 of 24

Original data Influencing factors DEM Google images 5-cross-validation geological map Field surveying and repots Slope unit map

Data preparation

ANN model SVM model

Landslide susceptibility modeling

Statistical Analyzing Method Receiver Operating Characteristic Curve

Validation and selection of the optimal models

Figure 2.2. Flowchart of this study.

3.1.3.1. The Mapping Unit ReasonableReasonable mapping units shouldshould bebe determineddetermined beforebefore landslidelandslide susceptibilitysusceptibility mappingmapping isis carriedcarried outout [20[20].]. WhetherWhether the the mapping mapping unit unit is reasonableis reasonable or notor not directly directly affects affects the accuracythe accuracy of the of final the evaluationfinal evaluation result. result. Slope unitsSlope are units often are used often to studyused to landslide study landslide susceptibility susceptibility mapping duemapping to their due close to relationshiptheir close relationship with the topography. with the topography. The dividing The principle dividing of slopeprinciple units of is slope to divide units the is researchto divide area the intoresearch many area small into areas many with small diff erentareas sizeswith bydifferent cutting sizes along by the cutting ridge andalong valley the ridge lines [and21]. valley At present, lines hydrologic[21]. At present, analysis hydrologic is the most analysis used is method the most for us dividinged method slope for units. dividing However, slope units. this method However, cannot this identifymethod thecannot boundary identify between the boundary the horizontal between and the thehorizontal inclined and surface the inclined and generates surface a and large generates number ofa large parallel number river of networks, parallel river which networks, increases which the di ffiincreasesculty of the manual difficulty modification of manual [21 modification]. By comparing [21]. theBy comparing slope unit division the slope results unit division of the curvature results of watershed the curvature method watershed and the hydrologicmethod and analysis the hydrologic method, itanalysis was found method, that it the was slope found unit that divided the slope based unit on divided the curvature based on watershed the curvature method watershed has a uniform method size,has a auniform regular size, shape, a regular and small shape, terrain and small variation terrain inside, variation which inside, is obviously which is betterobviously than better the result than basedthe result on thebased hydrological on the hydrological method. Therefore, method. Ther we choseefore, thewe curvaturechose the curvature watershed watershed method to method divide theto divide slope the units slope in thisunits study. in this Slopestudy. units Slope are unit divideds are divided by valley by valley and ridge and ridge lines, lines, where where there there are abruptare abrupt changes changes in slope in slope angle angle and and slope slope aspect. aspect Therefore,. Therefore, slope slope units units can can be dividedbe divided according according to theto the change change of slopeof slope angle angle and and slope slope aspect. aspect. Profile Profile curvature curvature and and plan plan curvature curvature are are the the derivatives derivatives of slopeof slope angle angle and and slope slope aspect, aspect, respectively. respectively. Their Thei maximumr maximum and and minimum minimum values values cancan reflectreflect abruptabrupt changes of slope angle and slope aspect. Therefore, the curvature can be used to divide the slope units. The specific process of dividing slope units based on curvature is shown in Figure 3. It is mainly used to identify the boundary of concave terrain and convex terrain by the curvature and reverse curvature, respectively, to divide the slope units. The classification steps of the slope units can be divided into two parts: positive relief extraction and negative relief extraction. The original curvature was used to extract the positive relief, and the flip curvature was used to extract the negative relief, which can be obtained by multiplying the curvature by −1. Before calculating the curvature, to remove the influence of the roughness of the original Digital Elevation Model (DEM) on the curvature calculation results, focal statistics were made on the DEM with three pixels as the radius. After the flow direction and sink calculations, the positive and negative relief boundary can be obtained. The results of dividing slope units can be obtained by merging the positive and negative relief boundaries and manually modifying the unreasonable units.

Symmetry 2020, 12, 1047 5 of 23 changes of slope angle and slope aspect. Therefore, the curvature can be used to divide the slope units. The specific process of dividing slope units based on curvature is shown in Figure3. It is mainly used to identify the boundary of concave terrain and convex terrain by the curvature and reverse curvature, respectively, to divide the slope units. The classification steps of the slope units can be divided into two parts: positive relief extraction and negative relief extraction. The original curvature was used to extract the positive relief, and the flip curvature was used to extract the negative relief, which can be obtained by multiplying the curvature by 1. Before calculating the curvature, to remove the influence − of the roughness of the original Digital Elevation Model (DEM) on the curvature calculation results, focal statistics were made on the DEM with three pixels as the radius. After the flow direction and sink calculations, the positive and negative relief boundary can be obtained. The results of dividing slope units can be obtained by merging the positive and negative relief boundaries and manually modifying Symmetrythe unreasonable 2020, 12, x FOR units. PEER REVIEW 6 of 24

DEM

Focal statistics

Calculator Curvature Flip curvature

Flow direction map

Sink map

Catchment map Flip Catchment map

Raster to Polygon Polygon merging

Modify unreasonable cell

Slope units

Figure 3. ConstructionConstruction process process of of the slope units based on the curvature watershed method. 3.2. Landslide Inventory 3.2. Landslide Inventory Landslide inventory data is the basis of landslide susceptibility mapping. In order to grasp Landslide inventory data is the basis of landslide susceptibility mapping. In order to grasp the the spatial distribution characteristic information and develop a mechanism of landslide hazards spatial distribution characteristic information and develop a mechanism of landslide hazards in the in the study area, the Jilin team of the Geological Survey Center of China Industrial Building study area, the Jilin team of the Geological Survey Center of China Industrial Building Materials Materials carried out the geological hazard survey (1:50,000) project of Helong city. The characteristic carried out the geological hazard survey (1:50,000) project of Helong city. The characteristic geomorphological features of the landslides, such as the chair-like landform of the landslide, a color geomorphological features of the landslides, such as the chair-like landform of the landslide, a color difference between the landslide and the surrounding environment, and a change of vegetation before difference between the landslide and the surrounding environment, and a change of vegetation and after the landslide, can be clearly identified in a remote sensing image. Thus, the project team before and after the landslide, can be clearly identified in a remote sensing image. Thus, the project preliminarily identified 112 landslides in the study area using the visible optical remote sensing team preliminarily identified 112 landslides in the study area using the visible optical remote sensing technology and Google images. Then, during the field geological survey, the remote sensing technology and Google images. Then, during the field geological survey, the remote sensing interpretation results were reviewed, and 42 interpretation results were excluded (such as exposed interpretation results were reviewed, and 42 interpretation results were excluded (such as exposed steep cliffs of bedrock and a waste rock dump in a quarry) and the uninterpreted landslides were steep cliffs of bedrock and a waste rock dump in a quarry) and the uninterpreted landslides were supplemented. Finally, a total of a total of 83 landslides were mapped in the study area (Figure1c), supplemented. Finally, a total of a total of 83 landslides were mapped in the study area (Figure 1c), including soil and rock slope deformation, soil slide, and collapse. Figure1e,f are examples of including soil and rock slope deformation, soil slide, and collapse. Figure 1e,f are examples of the the landslides. Table A1 shows the basic information for the landslides. According to Table A1, landslides. Table A1 shows the basic information for the landslides. According to Table A1, the the landslides in the study area are mostly rock landslides, which are mainly developed in crystalline landslides in the study area are mostly rock landslides, which are mainly developed in crystalline rocks. These landslides are mainly controlled by the rock mass structural plane, and a small number of landslides are controlled by the contact surface between the overburden layer and the rock mass. The scale of landslide is mainly small and medium size. The failure mode of the landslides is mainly pull-type and toppling-type.

3.3. Influencing Factors The occurrence of a landslide is a complicated process. Therefore, the factors influencing the occurrence of a landslide are also varied. Pourghasemi and Rossi [13] reviewed a total of 220 related papers and found that the factors with the highest application frequency in landslide susceptibility mapping were slope angle, lithology, slope aspect, land use, distance to river, elevation, distance to faults, curvature, distance to road, and soil type, among others. The selection of influencing factors for the landslide susceptibility mapping in the study area should be based on the understanding of the characteristics of the landslide and the geological environment. Therefore, the relationship between the landslide and the geological environment in the study area was analyzed as follows.

Symmetry 2020, 12, 1047 6 of 23 rocks. These landslides are mainly controlled by the rock mass structural plane, and a small number of landslides are controlled by the contact surface between the overburden layer and the rock mass. The scale of landslide is mainly small and medium size. The failure mode of the landslides is mainly pull-type and toppling-type.

3.3. Influencing Factors The occurrence of a landslide is a complicated process. Therefore, the factors influencing the occurrence of a landslide are also varied. Pourghasemi and Rossi [13] reviewed a total of 220 related papers and found that the factors with the highest application frequency in landslide susceptibility mapping were slope angle, lithology, slope aspect, land use, distance to river, elevation, distance to faults, curvature, distance to road, and soil type, among others. The selection of influencing factors for the landslide susceptibility mapping in the study area should be based on the understanding of the characteristics of the landslide and the geological environment. Therefore, the relationship between theSymmetry landslide 2020, 12 and, x FOR the PEER geological REVIEW environment in the study area was analyzed as follows. 7 of 24

3.3.1.3.3.1. RelationshipRelationship betweenBetween GeologicalGeological EnvironmentEnvironment and Landslides Relationship between Topographic Features and Landslides Relationship Between Topographic Features and Landslides The topography is the basic condition of a landslide and determines the formation and development The topography is the basic condition of a landslide and determines the formation and of landslide to a great extent. For example, convex and bedding slopes are prone to landslides. Based development of landslide to a great extent. For example, convex and bedding slopes are prone to onlandslides. ﬁeld investigation Based on andfield analysis, investigation it was and found analysis, that most it was landslides found that in the most study landslides area occurred in the instudy hills andarea low occurred mountains in hills with and an low elevation mountains range with of 200–1000 an elevation m and range a slope of angle200–1000m more and than a 30 slope◦ (Figure angle4). Furthermore,more than 30° the (Figure number 4). ofFurthermore, landslides increasedthe number with of landslides the increase increased of the slope with angle.the increase The number of the ofslope landslides angle. The in a number convex slopeof landslides was much in a largerconvex than slope that was in much concave larger slope. than The that micro-landform in concave slope. of theThe landslide micro-landform site was of a steepthe landslide slope or site steep wa coasts a steep (Table slope A1 or). steep coast (Table A1).

FigureFigure 4.4. ((a)) Elevation map; ( b)) slope slope angle angle map. map.

RelationshipRelationship between Between Lithological Lithological FeaturesFeatures andand LandslidesLandslides TheThe lithology lithology of of landslides landslides in in the the study study area area was was mainly mainly crystalline crystalline rocks, rocks, such assuch granite, as granite, diorite, anddiorite, other and magmatic other magmatic rocks (Table rocks A1 (Table). The A1). number The nu ofmber landslides of landslides in the in crystalline the crystalline rock inrock the in study the areastudy was area the was largest the and largest was and obviously was obviously higher than high thater inthan other that geotechnical in other geotechnical rock type distribution rock type areas.distribution This is areas. because This the is crystallinebecause the rock crystalline is usually rock strongly is usually inﬂuenced strongly influenced by joints, cracks,by joints, and cracks, faults. Theand presence faults. The of presence these structural of these planesstructural greatly planes reduces greatly the reduces shear strengththe shear ofstrength the rock of mass,the rock resulting mass, inresulting the sliding in the of thesliding upper of the rock upper mass rock or overlying mass or overlying loose rock. loose rock.

Relationship Between Geologic Features and Landslides The number of landslides in fault geological tectonic units was the largest in the study area. This is mainly because the in-situ stress of the fracture is stronger than that of other parts, and its influence range is larger and wider. The rock in these parts has deformations such as bending, squeezing, and tearing, which reduces the structural mechanical properties of the rock mass.

Relationship Between Rainfall Features and Landslides According to statistics, the landslides that occurred in the study area were mostly caused by heavy rainfall, and the two are positively correlated. Continuous heavy rainfall, particularly during the flood season, results in a high incidence of landslides.

Relationship Between Other Features and Landslides Landslides occurred more frequently in areas with low vegetation coverage in the study area. Landslides occurred extensively along the Tumen river, which is the main river in the study area. Earthquakes are an important trigger of landslides. They damage the integrity of rock and soil mass and make slopes prone to landslide. However, there is no historical record of landslides triggered by

Symmetry 2020, 12, 1047 7 of 23

Relationship between Geologic Features and Landslides The number of landslides in fault geological tectonic units was the largest in the study area. This is mainly because the in-situ stress of the fracture is stronger than that of other parts, and its inﬂuence range is larger and wider. The rock in these parts has deformations such as bending, squeezing, and tearing, which reduces the structural mechanical properties of the rock mass.

Relationship between Rainfall Features and Landslides According to statistics, the landslides that occurred in the study area were mostly caused by heavy rainfall, and the two are positively correlated. Continuous heavy rainfall, particularly during the ﬂood season, results in a high incidence of landslides.

Relationship between Other Features and Landslides Landslides occurred more frequently in areas with low vegetation coverage in the study area. Landslides occurred extensively along the Tumen river, which is the main river in the study area. Earthquakes are an important trigger of landslides. They damage the integrity of rock and soil mass and make slopes prone to landslide. However, there is no historical record of landslides triggered by earthquakes in the study area. Thus, seismic activity has had little eﬀect on the occurrence of landslides in the study area.

3.3.2. Selection of Influencing Factors Based on the above analysis, and the statistical results of Pourghasemi and Rossi [13], three groups of influencing factors were selected for this study: geologic factors, topographic factors, and environment factors. The geologic factors only consist of geology, while there were five topographic factors: elevation, slope angle, slope aspect, topographic relief, and curvature. Finally, there were four environment factors: land use, rainfall, distance to river, and distance to faults. In this study, all the influencing factor maps were extracted using the slope unit. Furthermore, the dominant category of lithology, land use, was estimated as the influencing factor value of each slope unit, with the average value used for the other factors.

3.3.3. Extraction of Inﬂuencing Factors

Geologic Factor The study area is covered with six types of geologic formations, composed of different lithologies pertaining to different geologic ages. The combination of these two characteristics is one of the most important factors influencing the occurrence of landslides [22]. The shear strength, weathering resistance, and crushing degree of different lithologies are greatly different [23]. In addition, the same lithology with different structural planes will have different effects on the stability of the slope. A rock mass with joint and fissure development is more prone to landslides than a complete rock mass. A highly weathered rock mass is also less stable than a fresh rock mass. In this study, the geology map was obtained based on a geological map with a scale of 1:500,000 (downloaded using 91 Weitu software); the geology map of the study area is shown in Figure5a. Symmetry 2020, 12, 1047 8 of 23 Symmetry 2020, 12, x FOR PEER REVIEW 9 of 24

Figure 5. Influencing factor maps of the study area: (a) lithology (downloaded using 91 Weitu software), (b)Figure elevation, 5. Influencing (c) slope angle, factor and maps (d) slopeof the aspect.study area: (a) lithology (downloaded using 91 Weitu software), (b) elevation, (c) slope angle, and (d) slope aspect. Topographic Factors Topographic Factors Elevation usually affects the depth of the water table aquifer [1,24]. In addition, the higher the elevation,Elevation the usually less human affects activitythe depth takes of the place, water which table aquifer is also more[1,24]. conduciveIn addition, to the the higher stability the of theelevation, slope. Regarding the less human the slope activity angle, takes a place, change which of slope is also angle more changes conducive the to original the stability stress of state the of slope. Regarding the slope angle, a change of slope angle changes the original stress state of the slope the slope [25]. Within a certain range of slope, the self-gravity stress and shear stress in the slope [25]. Within a certain range of slope, the self-gravity stress and shear stress in the slope generally generally increase with an increase of the slope angle, and the probability of slope instability increases increase with an increase of the slope angle, and the probability of slope instability increases accordingly. The intensity of light exposure, the type and extent of vegetation cover, and the supply of accordingly. The intensity of light exposure, the type and extent of vegetation cover, and the supply surface water vary greatly with differing slope aspect [2,26]. For example, the illumination time on a of surface water vary greatly with differing slope aspect [2,26]. For example, the illumination time on sunny slope is much longer than that on a shady slope; therefore, the temperature difference between a sunny slope is much longer than that on a shady slope; therefore, the temperature difference daybetween and night day on and a sunnynight on slope a sunny is also slope larger is also than larger that on than a shady that on slope a shady and theslope dry–wet and the cycle dry–wet is also faster.cycle In is thisalso case, faster. the In weathering this case, the strength weathering of the strength rock mass of the on rock a sunny mass slope on a sunny is larger slope than is thatlarger on a shadythan slope, that on which a shady reduces slope, the which strength reduces and stabilitythe strength of rock and and stability soil mass of rock on theand sunny soil mass slope, on in the turn increasingsunny slope, the probability in turn increasing of landslide the [4 ].probabilit The topographicy of landslide Relief—The [4]. The diff topographicerence between Relief—The the highest anddifference lowest Points—Can between the reflect highest the and degree lowest of Points—Can relief in a specific reflect area the [degree4]. Curvature of relief can in a reflect specific the area shape of[4]. a slope Curvature body. Accordingcan reflect tothe the shape shape of characteristics,a slope body. According a slope can to be the divided shape characteristics, into three forms: a slope convex slope,can be straight divided slope, into three and concaveforms: convex slope slope, [27]. Astraight DEM (Digitalslope, and Elevation concave Model)slope [27]. with A DEM a resolution (Digital of 10 Elevation10 m was Model) used with to extract a resolution the five of influencing10 × 10 m was factor used mapsto extract through the five the influencing slope unit factor using maps ArcGis × software.through The the fiveslope influencing unit using ArcGis factor mapssoftware. are shownThe five in influencing Figures5b–d factor and maps6a,b. are shown in Figure 5b–d and Figure 6a,b.

Symmetry 2020, 12, 1047 9 of 23 Symmetry 2020, 12, x FOR PEER REVIEW 10 of 24

FigureFigure 6. 6.Inﬂuencing Influencing factors factors maps maps of of the the studystudy area:area: ( (aa)) topographic topographic relief, relief, (b (b) curvature,) curvature, (c) ( cland-use,) land-use, andand (d )(d rainfall.) rainfall.

EnvironmentEnvironment Factors Factors VegetationVegetation has has a positivea positive e ffeffectect on on the the stabilitystability ofof aa slope, which can can enhance enhance the the resistance resistance of of thethe slope slope surface surface to to water water erosion erosion and and influence influence rainfallrainfall infiltration.infiltration. Rainfall Rainfall has has a asignificant significant impact impact onon slope slope risk. risk. Rainwater Rainwater infiltration infiltration willwill increaseincrease the gravity of of rock rock and and soil soil mass, mass, reducing reducing their their shearshear strength strength parameters, parameters, which which will will have have anan adverseadverse effect effect on slope slope stability. stability. The The distance distance to toriver river factorfactor has has an importantan important influence influence on the on occurrencethe occurre ofnce landslides of landslides [28]. The[28]. downcut The downcut of a river of increasesa river theincreases slope angle the alongslope angle the river. along To the adapt river. to To the adapt rapid to downcut, the rapid disastersdowncut, such disast asers landslides such as landslides often occur alongoften the occur river, along thus the reducing river, thethus slope reducing angle. the The slope existence angle. ofThe faults existence leads of to faults the development leads to the of jointsdevelopment and fissures of joints in the and surrounding fissures in rockthe surrounding mass, which rock leads mass, to fragmentation which leads to offragmentation the rock mass of andthe a reductionrock mass of weatheringand a reduction resistance of weathering [29]. Therefore, resistance many [29]. Therefore, geological many hazards geological such as hazards landslides such often as landslides often develop around deep and large fissures. These four influencing factors are extracted develop around deep and large fissures. These four influencing factors are extracted through the slope through the slope unit using ArcGis software. The five influencing factor maps are shown in Figure unit using ArcGis software. The five influencing factor maps are shown in Figure6c,d and Figure7a,b. 6c,d and Figure 7a,b.

Symmetry 2020, 12, 1047 10 of 23

Symmetry 2020, 12, x FOR PEER REVIEW 11 of 24

FigureFigure 7. 7.Inﬂuencing Influencing factors factors maps maps of of the the studystudy area:area: ( a)) distance distance to to river river and and (b (b) distance) distance to tofault. fault.

3.4.3.4. Landslide Landslide Susceptibility Susceptibility Modeling Modeling AtAt present, present, an increasingan increasing number number of models of models are being are applied being toapplied the study to the of landslide study of susceptibility landslide mappingsusceptibility [30–33 ].mapping Each of[30–33]. these modelsEach of hasthes itse models advantages has its and advantages disadvantages; and disadvantages; combined with thecombined complexity with of the the complexity factors aff ofecting the factors slope stabilityaffecting andslope the stability different and geologicalthe different environments geological in dienvironmentsfferent regions, in different it is impossible regions, to it applyis impossi one modelble to apply to the one study model of landslide to the study susceptibility of landslide in all areas.susceptibility Therefore, inin all the areas. relevant Therefore, evaluation in the releva of thent studyevaluation area, of the the evaluation study area, model the evaluation adopted model should beadopted optimized should to determine be optimized the mostto determine suitable the landslide most suitable susceptibility landslide evaluation susceptibility model. evaluation Recently, somemodel. machine Recently, learning some approachesmachine learning have approaches been developed have been with developed the aim ofwith evaluating the aim of the evaluating landslide susceptibilitythe landslide [21 susceptibility]. Due to its [21]. strong Due learning to its strong ability, learning fast calculation ability, fast speed, calculation and strong speed, fault and tolerance, strong thefault artificial tolerance, neural the network artificial (ANN) neural model network has (ANN) become model one ofhas the become most used one of machine the most learning used machine methods learning methods in landslide susceptibility mapping. By comparison, the support vector machine in landslide susceptibility mapping. By comparison, the support vector machine (SVM) model requires (SVM) model requires fewer modeling data and is suitable for binary classification problems. Thus, fewer modeling data and is suitable for binary classification problems. Thus, the SVM model may be the SVM model may be more suitable for landslide susceptibility prediction with fewer data, such as more suitable for landslide susceptibility prediction with fewer data, such as in the case of this study. in the case of this study. Therefore, we selected the artificial neural network (ANN) and support Therefore, we selected the artificial neural network (ANN) and support vector machine (SVM) models vector machine (SVM) models to optimize the evaluation model of landslide susceptibility in the to optimizestudy area. the evaluation model of landslide susceptibility in the study area. 3.4.1. Artificial Neural Network (ANN) 3.4.1. Artificial Neural Network (ANN) AsAs a a machine machine learninglearning method, method, the the artificial artificial neural neural network network model model has been has widely been widelyused in the used instudy the study of landslide of landslide susceptibility susceptibility mapping mapping [26,34,35]. [26 The,34 ,ANN35]. model The ANN has the model following has the advantages following advantages[36]: (a) better [36]: non-linear (a) better non-linearmapping capability; mapping (b) capability; highly self-learning (b) highly self-learningand adaptive; and (c) adaptive;strong (c)generalization strong generalization ability; and ability; (d) strong and (d)fault strong tolera faultnce. Thus, tolerance. the ANN Thus, model the can ANN simulate model the can complex simulate thenon-linear complex non-linearinteractions interactions between influencing between influencing factors and factors landslides and through landslides the through interaction the interactionbetween betweenneurons. neurons. Moreover, Moreover, in an ANN in an model ANN it model is not itnece is notssary necessary to describe to describethe interactions the interactions between factors between factorsby complex by complex mathematical mathematical formulae formulae [37]. [ 37The]. Thefitting fitting effect eff ectis relatively is relatively good, good, which which is isespecially especially suitablesuitable for for the the simulation simulation of complex of complex geological geological phenomena phenomena influenced influenced by many by many factors factors and presents and advantagespresents advantages for the simulation for the simulation of phenomena of phenom suchena as landslides,such as landslides, in which in variouswhich various factors factors interact andinteract havecomplex and have relationships. complex relationships. A complete A artificial complete neural artificial network neural model network is usually model composedis usually of ancomposed input layer, of an an output input layer, andan output one or layer, more and hidden one layers.or more Figure hidden8 shows layers. a schematicFigure 8 shows diagram a of aschematic simple artificial diagram neuralof a simple network artificial model. neural Each network layercontains model. Each several layer of contains the model’s several basic of units:the neurons.model’s The basic data units: processing neurons. and The storage data processing of an artificial and neural storage network of an artificial are represented neural network as the mutual are relationshipsrepresented and as the connections mutual relationships between neurons. and conn Throughections between training neurons. using samples, Through the training ANN changesusing samples, the ANN changes the weights of its internal connections to minimize the error between the the weights of its internal connections to minimize the error between the output value and the target output value and the target value, to achieve the purpose of accurate modeling. Each node in the value, to achieve the purpose of accurate modeling. Each node in the input layer corresponds to a input layer corresponds to a predictive variable, while the node in the output layer corresponds to predictive variable, while the node in the output layer corresponds to the target variable. A hidden the target variable. A hidden layer is a regular layer connecting the input layer and output layer, and

Symmetry 2020, 12, 1047 11 of 23 Symmetry 2020, 12, x FOR PEER REVIEW 12 of 24 thelayer number is a regular of the layerhidden connecting layers and the the input number layer of andnodes output in each layer, layer and determine the number the complexity of the hidden of thelayers network. and the The number process of of nodes forecasting in each with layer an determine ANN can the be complexity divided into of thea learning network. process The process and a predictionof forecasting process with [38]. an ANNThe learni canng be dividedprocess, intoin which a learning a large process number and of learning a prediction samples process and [the38]. iterativeThe learning function process, of the in network which a largeare used number to train of learning the network samples by andoptimizing the iterative the process function via of minimizingthe network the are usednetwork to train error, the includes network bythe optimizing forward transmission the process via of minimizing input information the network and error, the reverseincludes transmission the forward of transmission error [34]. The of prediction input information process is and the the process reverse of substituting transmission the of unknown error [34]. sampleThe prediction into the processmodel after is the training. process According of substituting to the the rules unknown of the learning sample into sample, the model the process after training. finally obtainsAccording the tooutput the rules result of theof learningthe sample sample, through the processthe forward ﬁnally transmission obtains the output of the resultinput ofinformation the sample [39].through the forward transmission of the input information [39].

1 Landslide

0 Non-Landslide Output layers

Input layers Hidden layers

FigureFigure 8. 8. SchematicSchematic diagram diagram of of a a simple simple artificial artiﬁcial neural neural network network (ANN) (ANN) model. model.

3.4.2.3.4.2. Support Support Vector Vector Machine Machine (SVM) (SVM) TheThe support support vector machine (SVM)(SVM) modelmodel isis alsoalso aa machine machine learning learning model. model. Its Its theoretical theoretical basis basis is isstatistical statistical learning learning theory theory [40 [40–43].–43]. The The SVM SVM model model has has the the following following advantages advantages [40, 44[40,44]:]: (a) low (a) datalow datavolume volume requirement; requirement; (b) strong (b) generalizationstrong generalization ability; (c)ability; strong (c) optimization strong optimization ability; (d) adaptabilityability; (d) adaptabilityto high-dimensional to high-dimensional samples; and samples; (e) strong and (e) learning strong abilitylearning and ability fast and convergence. fast convergence. It can realize It can realizethe linear the segmentationlinear segmentation of data of by data transforming by transfor eachming evaluation each evaluation index from index low from dimensional low dimensional space to spacehigh dimensionalto high dimensional space. Thus, space. it can Thus, analyze it can and analyze evaluate and non-linear evaluate problemsnon-linear in problems low dimensional in low dimensionalspace. Due to space. its low Due requirement to its low inrequirement terms of data in term volume,s of data it has volume, also been it widelyhas also used been in widely the study used of inlandslide the study susceptibility of landslide mapping.susceptibility The mapping. process of Th modelinge process for of landslidemodeling susceptibilityfor landslide susceptibility evaluation by evaluationSVM is summarized by SVM is assummarized follows [40 ,as44 ]:follows [40,44]: Consider some linearly separable data points x (i = 1, 2, ... ,n) that fall into two diﬀerent classes Consider some linearly separable data points xii (i = 1, 2,…,n) that fall into two different classes y = 1. The goal of SVM is to ﬁnd a hyperplane in n-dimensional data space which can separate the two yi i= ±1.± The goal of SVM is to find a hyperplane in n-dimensional data space which can separate the twoclasses classes of data of data based based on on the the maximum maximum interval. interval. The The hyperplane hyperplane cancan bebe expressed mathematically mathematically asas follows: follows: 1 2 L = w , (1) 𝐿=2||𝑤|||| || , (1) which should satisfy the following constraint conditions: which should satisfy the following constraint conditions:

yi((w xi) + b) 1, (2) 𝑦((𝑤∙𝑥· ) +𝑏)≥1≥ , (2) wherewhere ||ww||||is theis the norm norm of theof the normal normal vector vector of the of hyperplane,the hyperplane,b is ab scalar,is a scalar, and represents and represents the scalar the scalarproduct. product. Based Based on the on Lagrange the Lagrange multiplier, multiplier, the cost the function cost function can be can expressed be expressed as follows: as follows:

𝐿= ||𝑤|| −X∑ n 𝜆 (𝑦 (𝑤∙𝑥) +𝑏−1), (3) 1 2 L = w λi(yi((w xi) + b) 1), (3) 2|| || − i = 1 · − where λi is the Lagrange multiplier.

Symmetry 2020, 12, 1047 12 of 23

where λi is the Lagrange multiplier. In the case of linear indivisibility, the constraint condition can introduce a slack variable ξi, which can be expressed as follows: y ((w x ) + b) 1 ξ . (4) i · i ≥ − i Then, Equation (1) can be converted into the following form: X 1 2 1 n LL = w ξi. (5) 2|| || − vn i = 1 In Equation (5), v in [0,1] is introduced to consider the case of misclassiﬁcation. In addition, the kernel function K (xi, yi) is introduced to explain the non-linear decision boundary problem in SVM.

3.5. Data for Landslide Susceptibility Modeling When using an artificial neural network or support vector machine to establish a landslide susceptibility model, the same amount of data is needed for landslide units and non-landslide units. In this study, we determined the number of slope units with landslides according to the division results of slope units in the study area and the landslide inventory map. An equal number of units were randomly selected at a minimum distance of 800 m from these units, in order to avoid the effects of landslides [37]. The five-fold cross-validation method [39,40] was used to validate the models and to overcome the shortage of landslide data and the problem of model overfitting. All the ten influencing factors were involved in landslide susceptibility model building.

3.6. Validation Method

3.6.1. Receiver Operating Characteristic Curve (ROC) The receiver operating characteristic curve (ROC) [18,45] is a quantitative analysis method to evaluate the prediction accuracy of the landslide susceptibility model. This method evaluates the prediction accuracy of the model using the area under the curve (AUC). The AUC value lies between 0 and 1, and the greater its value, the higher the prediction accuracy of the model.

3.6.2. Statistical Analysis Method Statistical indices are also widely used for evaluating the prediction ability of landslide susceptibility mapping models [21]. The most used statistical indices are the following:

Ac = (TP + TN)/(TP + TN + FP + FN), (6)

Sen = TP/(TP + FN), (7)

Sp = TN/(TN + FP), (8)

PPV = TP/(TP + FP), (9)

NPV = TN/(TN + FN) (10) where Ac is the accuracy; Sen is the sensitivity; Sp is the speciﬁcity; PPV is the positive predictive value; NPV is the negative positive value; TP is the true positive; TN is the true negative; FP is the false positive; and FN is the false negative.

4. Results

4.1. Division Result of the Slope Units In this paper, a DEM with a resolution of 10 10 m was adopted to the divided slope units of × the study area. In the process of division, we found that the size of slope units divided by the curvature Symmetry 2020, 12, 1047 13 of 23

Symmetrywatershed 2020 method, 12, x FOR was PEER related REVIEW to the DEM resolution. Therefore, the DEM resolution was converted14 of 24 to 10 10 m, 30 30 m, 50 50 m, 80 80 m, 100 100 m, and 120 120 m for slope classification. × × × × × × classification.By comparing By the comparing slope unit the classification slope unit resultsclassification with Google results imageswith Google of the images studyarea, of the it study was found area, itthat was the found slope that unit the classification slope unit resultclassification was the resu mostlt consistentwas the most with consistent the actual with terrain the withactual the terrain DEM with athe resolution DEM with of 80a resolution80 m. A of total 80 × of 80 2956 m. A slope total units of 2956 were slope obtained units (Figurewere obtained9). The maximum(Figure 9). area The × maximumof a slope unitarea was of a 18.45slope unit105 wasm2 and 18.45 the × 10 minimum5 m2 and areathe minimum was 0.11 area105 wasm2. 0.11 × 105 m2. × ×

Figure 9. DivisionDivision of of slope slope units of the study area.

4.2. Model Model Fitting Fitting Results Results According to the landslide inventory map, there are 83 slope units containing the entire known landslide body. Therefore, the 83 slope units that experienced landslides were used as the modeling data. ToTo meetmeet the the modeling modeling requirements, requirements, the samethe sa numberme number of non-landslide of non-land unitsslide (83) units were (83) randomly were randomlyselected at selected least 800 at mleast away 800 fromm away the from landslide the landslide units (Figure units9 ).(Figure To establish 9). To establish the ANN the model, ANN model,the numbers the numbers of input, of hidden,input, hidden, and output and layersoutput should layers beshould determined be determined first. In first. this study,In this eachstudy, of eachthe input, of the hidden, input, hidden, and output and layersoutput consisted layers consis of ated single of layer.a single Secondly, layer. Secondly, the number the ofnumber neurons of neuronsin each layer in each was layer determined, was determined, with the numberwith the of numb neuronser of in neurons the input in layerthe input the same layer as the the same number as the of numberinfluencing of influencing factors. There factors. were tenThere influencing were ten factors influencing used for factors modeling used in for this modeling study, thus, in thethis number study, thus,of neurons the number in the inputof neurons layer wasin the ten. input The outputlayer was layer ten. was The used output to determine layer was whether used to adetermine landslide whetheroccurs, so a thelandslide number occurs, of neurons so the was number two. Theof neurons number was of hiddentwo. The layer number neurons of hidden can be determinedlayer neurons by canthe followingbe determined empirical by the formula following [46 ]:empirical formula [46]:

N 𝑁== √√A𝐴++𝐵B ++𝑘k (11)(11) where N is the recommended value for the number of neurons in the hidden layer, A is the number where N is the recommended value for the number of neurons in the hidden layer, A is the number of neurons in the input layer, B is the number of neurons in the output layer, and k is an empirical of neurons in the input layer, B is the number of neurons in the output layer, and k is an empirical coefficient with value between 0 and 10. According to the empirical formula, the ideal number of coeﬃcient with value between 0 and 10. According to the empirical formula, the ideal number of hidden layer neurons in the artificial neural network in this study ranged from 4 to 14. By using all hidden layer neurons in the artiﬁcial neural network in this study ranged from 4 to 14. By using all the data to establish the ANN model, the number of hidden layer neurons was optimized. It can be the data to establish the ANN model, the number of hidden layer neurons was optimized. It can be seen seen from Figure 10 that, when the number of hidden layer neurons is 6, the ANN model has the highest prediction accuracy. Thus, the number of neurons in the hidden layer was finally selected as 6.

Symmetry 2020, 12, 1047 14 of 23

from Figure 10 that, when the number of hidden layer neurons is 6, the ANN model has the highest Symmetryprediction 2020 accuracy., 12, x FOR PEER Thus, REVIEW the number of neurons in the hidden layer was ﬁnally selected as 6.15 of 24

0.90 0.89 0.88 0.87 0.86 0.85 0.84 0.83 4 5 6 7 8 9 10 11 12 13 14 AUC 0.885 0.887 0.897 0.879 0.859 0.876 0.882 0.888 0.869 0.873 0.872

FigureFigure 10. 10. AArearea under under the the curve curve (AUC) (AUC) va valueslues of of model model evaluation evaluation parameters parameters when when the the hidden hidden neuronsneurons of of ANN ANN model model varied. varied. For the ANN model, the learning rate, momentum, and training time were set as 0.3, 0.3, For the ANN model, the learning rate, momentum, and training time were set as 0.3, 0.3, and and 500, respectively [34,37]. The choice of kernel function affects the prediction accuracy of SVM. 500, respectively [34,37]. The choice of kernel function affects the prediction accuracy of SVM. In this In this study, the radial basis function (RBF) was selected as the kernel function, which was influenced by study, the radial basis function (RBF) was selected as the kernel function, which was influenced by the regularization parameter (C) and the kernel parameter (g). C and g were set as 0.8 and 0.5 [34,47,48], the regularization parameter (C) and the kernel parameter (g). C and g were set as 0.8 and 0.5 respectively. All the parameters were obtained based on the previous research experience and experiments [34,47,48], respectively. All the parameters were obtained based on the previous research experience in the calculation process. The model fitting results are shown in Table2. and experiments in the calculation process. The model fitting results are shown in Table 2. Table 2. Performance of the Two Slope-Unit-Fitted Landslide Susceptibility Models (%). Table 2. Performance of the Two Slope-Unit-Fitted Landslide Susceptibility Models (%). Stage Method Statistical Index K = 1 K = 2 K = 3 K = 4 K = 5 Mean Standard Deviation Stage Method Statistical Index K = 1 K = 2 K = 3 K = 4 K = 5 Mean Standard Deviation AUC 88.20 91.10 90.30 84.70 88.70 88.60 2.48 AUC 88.20 91.10 90.30 84.70 88.70 88.60 2.48 AC 86.57 89.39 85.07 73.48 81.82 83.27 6.11 SEAC 91.53 86.57 90.6389.39 83.1085.07 73.48 72.46 81.82 82.81 83.27 84.11 6.11 7.68 ANN SPSE 82.67 91.53 88.2490.63 87.3083.10 72.46 74.60 82.81 80.88 84.11 82.74 7.68 5.49 ANN PPSP 80.60 82.67 87.8888.24 88.0687.30 74.60 75.76 80.88 80.30 82.74 82.52 5.49 5.33 NP 92.54 90.91 82.09 71.21 83.33 84.02 8.49 Training PP 80.60 87.88 88.06 75.76 80.30 82.52 5.33 AUCNP 92.70 92.54 93.4090.91 92.3082.09 71.21 93.20 83.33 94.50 84.02 93.22 8.49 0.83 Training AC 87.31 89.39 86.57 88.64 90.91 88.56 1.71 SEAUC 89.06 92.70 91.9493.40 87.6992.30 93.20 90.48 94.50 93.55 93.22 90.54 0.83 2.31 SVM SPAC 85.71 87.31 87.1489.39 85.5186.57 88.64 86.96 90.91 88.57 88.56 86.78 1.71 1.24 PPSE 85.07 89.06 86.3691.94 85.0787.69 90.48 86.36 93.55 87.88 90.54 86.15 2.31 1.16 SVM NP 89.55 92.42 88.06 90.91 93.94 90.98 2.31 SP 85.71 87.14 85.51 86.96 88.57 86.78 1.24 AUCPP 83.20 85.07 87.0086.36 82.1085.07 86.36 88.20 87.88 82.10 86.15 84.52 1.16 2.88 AC 71.88 70.59 68.75 71.43 73.53 71.23 1.75 NP 89.55 92.42 88.06 90.91 93.94 90.98 2.31 SE 73.33 81.82 65.00 66.67 72.22 71.81 6.62 ANN SPAUC 70.59 83.20 65.2287.00 75.0082.10 88.20 78.57 82.10 75.00 84.52 72.88 2.88 5.13 PPAC 68.75 71.88 52.9470.59 81.2568.75 71.43 82.35 73.53 76.47 71.23 72.35 1.75 12.10 Testing NPSE 75.00 73.33 88.2481.82 56.2565.00 66.67 64.71 72.22 70.59 71.81 70.96 6.62 11.94 ANN AUCSP 88.70 70.59 89.6065.22 91.2075.00 78.57 91.30 75.00 87.90 72.88 89.74 5.13 1.50 ACPP 81.25 68.75 88.2452.94 84.3881.25 82.35 85.29 76.47 82.35 72.35 84.30 12.10 2.72 SE 81.25 93.33 82.35 83.33 82.35 84.52 4.98 SVM NP 75.00 88.24 56.25 64.71 70.59 70.96 11.94 Testing SP 81.25 84.21 86.67 87.50 82.35 84.40 2.69 PPAUC 81.25 88.70 82.3589.60 87.5091.20 91.30 88.24 87.90 82.35 89.74 84.34 1.50 3.26 NPAC 81.25 81.25 94.1288.24 81.2584.38 85.29 82.35 82.35 82.35 84.30 84.26 2.72 5.54 SE Note: K 81.25 is the number93.33 of82.35 cross validations.83.33 82.35 84.52 4.98 SVM SP 81.25 84.21 86.67 87.50 82.35 84.40 2.69 4.3. Landslide Susceptibility MappingPP Results 81.25 82.35 87.50 88.24 82.35 84.34 3.26 NP 81.25 94.12 81.25 82.35 82.35 84.26 5.54 By comparing the five statistical parameters and AUC values of the two models, the SVM model Note: K is the number of cross validations. was determined to be the optimal model. Therefore, the SVM model was used in this study to produce 4.3.the Landslide landslide Susceptibility susceptibility Mapping map of Results the study area. The final model was established using the model with high accuracy and AUC value in the process of five-fold cross-validation. The natural breaks By comparing the five statistical parameters and AUC values of the two models, the SVM model was determined to be the optimal model. Therefore, the SVM model was used in this study to produce the landslide susceptibility map of the study area. The final model was established using the model with high accuracy and AUC value in the process of five-fold cross-validation. The natural breaks method was used to divide the landslide susceptibility of the study area into five grades: Very

Symmetry 2020, 12, 1047 15 of 23 methodSymmetry was 2020 used, 12, x FOR to divide PEER REVIEW the landslide susceptibility of the study area into ﬁve grades: Very16 of 24 low, low, moderate, high, and very high. The landslide susceptibility map of the study area is shown low, low, moderate, high, and very high. The landslide susceptibility map of the study area is shown in Figure 11. in Figure 11.

FigureFigure 11. 11.Landslide landslide susceptibility susceptibility map. map. (a )( ANNa) ANN model; model; and and (b) support(b) support vector vector machine machine (SVM) (SVM) model. model. Figure 11 and Table3 show that the areas of the five susceptibility classes for the ANN model (very high,Figure high, 11 and moderate, Table 3 low,show and that verythe areas low) of were the 146.24,five susceptibility 297.95, 423.33, classes 310.44, for the and ANN 297.95 model km 2, 2 respectively.(very high, Forhigh, landslide moderate, occurrence, low, and thevery number low) were of landslides 146.24, 297.95, in the 423.33, five susceptibility 310.44, and 297.95 classes km were, 43,respectively. 18, 10, 6, and For 6, landslide respectively. occurrence, For the the SVM number model, of landslides the areas ofin the five five susceptibility susceptibility classes classes were were 43, 18, 10, 6, and 6, respectively. For the SVM model, the areas of the five susceptibility classes were 127.43, 151.60, 198.77, 491.19, and 506.91 km2, respectively. For landslide occurrence, the number of 127.43, 151.60, 198.77, 491.19, and 506.91 km2, respectively. For landslide occurrence, the number of landslides in the five susceptibility classes were 52, 14, 8, 4, and 5, respectively. landslides in the five susceptibility classes were 52, 14, 8, 4, and 5, respectively. Table 3. Statistical Results of the Landslide Susceptibility Map. Table 3. Statistical Results of the Landslide Susceptibility Map. Landslide Occurred Total Study Area Model Susceptibility Landslide Occurred Total Study Area Model Susceptibility Count Ratio Area (km2) Ratio Count Ratio Area(km2) Ratio VeryVery Low Low 6 6 7.23% 7.23% 297.95 297.95 20.19% 20.19% Low 6 7.23% 310.44 21.03% Low 6 7.23% 310.44 21.03% ANN Moderate 10 12.05% 423.33 28.68% ANN HighModerate 1810 12.05% 21.69% 423.33 297.95 28.68% 20.19% Very HighHigh 4318 21.69% 51.81% 297.95 146.24 20.19% 9.91% VeryVery Low High 5 43 51.81% 6.02% 146.24 506.91 9.91% 34.35% LowVery Low 4 5 6.02% 4.82% 506.91 491.19 34.35% 33.28% SVM ModerateLow 84 4.82% 9.64% 491.19 198.77 33.28% 13.47% SVM HighModerate 148 9.64% 16.87% 198.77 151.60 13.47% 10.27% Very HighHigh 5214 16.87% 62.65% 151.60 127.43 10.27% 8.63% Very High 52 62.65% 127.43 8.63% 5. Discussion 5. Discussion 5.1. Slope Unit Classification Results 5.1. Slope Unit Classification Results To evaluate the effect of dividing slope units, the area and shape indices of slope units were statisticallyTo evaluate analyzed. the Theeffect ramp of dividing unit was slope used units, to extract the area the and various shape influences, indices of so slope the areaunits it were covers shouldstatistically be approximately analyzed. The the ramp same unit size was overall. used to Theextract uniformity the various of influences, slope unit so area the can area be it reflectedcovers byshould the distribution be approximately of slope the unit same area. size It overall. can be The seen, uniformity from Figure of slope 12a, unit that area the can slope be unitreflected area by was concentratedthe distribution between of slope 4 and unit 8 area.105 Itm can2, accounting be seen, from for 44.2%Figure for12a, the that total the number slope unit of slopearea was units. × 5 2 If theconcentrated slope unit between is too flat, 4 and or there 8 × 10 is m an, elongatedaccounting unit, for 44.2% the uniformity for the total inside number the unitof slope will units. be destroyed If the slope unit is too flat, or there is an elongated unit, the uniformity inside the unit will be destroyed to a great extent. The shape index can be used to evaluate the shape of the slope unit. The shape index of the slope units can be calculated using the following equation:

Symmetry 2020, 12, 1047 16 of 23 to a great extent. The shape index can be used to evaluate the shape of the slope unit. The shape index Symmetryof the slope 2020, units12, x FOR can PEER be calculated REVIEW using the following equation: 17 of 24

= S 𝑆L =ˆ2 𝐿^2/4𝜋𝐴/4πA,, (12) (12) where S is the shape index, index, L isis the the perimeter perimeter of of the the slope slope units, units, and and AA isis the the area area of of the the slope slope units. units. According to the definition deﬁnition of the shape index, the shape index of of a a circle circle is is 1, 1, that that of of a a square square is is 1.27, 1.27, and that of an an equilateral equilateral triangle is 1.59. Based Based on on Figure Figure 12b,12b, the the slope slope un unitsits with with shape shape index index below below 1.59 accountedaccounted for for 81.0%, 81.0%, which which means means that that most most of the of slope the unitsslope were units between were circlebetween and circle equilateral and equilateraltriangle shapes; triangle however, shapes; there however, were athere few elongatedwere a few slope elongated units. Thus, slope the units. eﬀect Thus, of dividing the effect slope of dividingunits was slope reasonable. units was reasonable.

50% 100% 70% 100% Area/105m2 Shape index 45% 90% 90% 60% 40% 80% 80%

35% 70% 50% 70%

30% 60% 60% 40% 25% 50% 50% 30% 20% 40% 40%

15% 30% 20% 30% 10% 20% 20% 10% 5% ab10% 10% 0% 0% 0% 0% 0.511.522.533.54 820 1 1.27 1.59 1.9 2.22 2.54 2.86 3.18 3.5 20 AREA 0.4% 2.2% 4.4% 5.3% 7.5% 6.7% 7.7% 7.2% 44.2% 14.3% SHAPE 0.0% 17.3% 63.7% 13.8% 3.2% 1.0% 0.3% 0.2% 0.1% 0.2% AREAC 0.4% 2.6% 7.0% 12.3% 19.8% 26.5% 34.3% 41.5% 85.7%100.0% SHAPEC 0.0% 17.3% 81.0% 94.9% 98.1% 99.2% 99.5% 99.7% 99.8% 100.0 Figure 12. Statistics of morphological characteristics of of slope units. ( (aa)) Slope Slope unit unit area distribution diagram; ( b) slope slope unit shape index distribution diagram.

InIn addition,addition, in in the the process process of dividingof dividing slope slope units, units, it was foundit was thatfound compared that compared with the hydrologic with the hydrologicsubdivision subdivision method, there method, is no large there amount is no large of parallel amount river of parallel network river in the network process in of dividingthe process slope of dividingunits by theslope curvature units by watershed the curvature method. watershed Moreover, method. the number Moreover, of unreasonable the number units of unreasonable generated is unitsmuch generated less than theis much hydrologic less than analysis the hydrologic method, so analysis the later method, manual so modification the later manual work ismodification much less. work is much less. 5.2. Comparison between ANN and SVM Model 5.2. ComparisonLandslide susceptibilitybetween ANN mappingand SVM Model is a popular research issue due to its non-linear characteristics. AlthoughLandslide many susceptibility mathematical mapping methods is a popular have been research applied issue to due landslide to its non-linear susceptibility characteristics. mapping, Althoughthe prediction many accuracy mathematical of these methods models is have not very been stable. applied Therefore, to landslide the same susceptibility model cannot mapping, be applied the predictionto all studies. accuracy As a result,of these one models of the is key not problemsvery stable. in Therefore, landslide susceptibilitythe same model mapping cannot isbe to applied find a tolandslide all studies. susceptibility As a result, model one of which the key is suitable problems for in a landslide specific study susceptibility area. In mapping this study, is theto find ANN a landslideand SVM susceptibility models were introducedmodel which to is establish suitable a for landslide a specific susceptibility study area. modelIn this forstudy, the areathe ANN southeast and SVMof Helong models city. were introduced to establish a landslide susceptibility model for the area southeast of HelongThrough city. the training and testing of the two selected models, a confusion matrix was obtained, and theThrough corresponding the training statistical and testing parameters of the of two each selected model models, were calculated a confusion to evaluate matrix theirwas respectiveobtained, andadvantages the corresponding and disadvantages. statistical From parameters Table2, the of mean each AUC model values were of ANNcalculated and SVM to evaluate models ditheirffer respectivegreatly: in theadvantages training stageand disadvantages. they were 88.60% From and Table 93.22%, 2, the respectively; mean AUC in thevalues testing of ANN stage and they SVM were models84.52% anddiffer 89.74%, greatly: respectively. in the training The stage AUC they values were of 88.60% the two and models 93.22%, decreased respectively; in the in testing the testing stage stage(by 4.08% they forwere the 84.52% ANN modeland 89.74%, and 3.48% respectively. for the SVM The model), AUC values and that of the of the two ANN models model decreased decreased in thesignificantly. testing stage According (by 4.08% to for the the mean ANN AUC model value, and the 3.48% SVM for model the SVM was model), slightly betterand that than of the ANN model decreased (4.62% for thesignificantly. training stage, According 5.22% forto the the mean testing AUC stage). value, For the the SVM standard model deviation was slightly of the better AUC thanvalue the in theANN training model stage, (4.62% that for of the ANNtraining model stage, was 5.22% obviously for the larger testing than stage). that ofFor the the SVM standard model deviation(2.48% for of the the ANN AUC model, value in 0.83% the training for the SVM stage, model). that of Thisthe ANN indicated model that was the obviously stability oflarger the SVMthan thatmodel of the was SVM better model than (2.48% that of for ANN the ANN model model, in the 0.83% training for stage.the SVM In themodel). testing This stage, indicated for the that ANN the stabilitymodel, the of standardthe SVM deviationmodel was of better the AUC than value that (2.88%)of ANN increased model in slightly, the training and similarly stage. In for the the testing SVM stage, for the ANN model, the standard deviation of the AUC value (2.88%) increased slightly, and similarly for the SVM model (1.50%); however, the SVM model was more stable. From the mean AUC value alone, we believe that the SVM model could be the best. For the statistical parameters, in the training stage, the mean accuracies were 83.27% and 88.56%, respectively, for the ANN model and the SVM model; in the testing stage, these were 71.23% and

Symmetry 2020, 12, 1047 17 of 23 model (1.50%); however, the SVM model was more stable. From the mean AUC value alone, we believe that the SVM model could be the best. For the statistical parameters, in the training stage, the mean accuracies were 83.27% and 88.56%, respectively, for the ANN model and the SVM model; in the testing stage, these were 71.23% and 84.30%. The standard deviation of the accuracy, in the training stage, was 6.11% and 1.71%, respectively, for the ANN model and the SVM model; in the testing stage, these were 1.75% and 2.72%. In terms of accuracy, the SVM model performed better than the ANN model (5.29% for training stage, 13.07% for testing stage). In terms of stability (according to accuracy) in the training stage, the SVM model was better than the ANN model (4.40%); however, in the testing stage, the ANN model was better than the SVM model (0.97%). From the aspect of mean accuracy, the SVM model was superior to the ANN model in both prediction accuracy and stability. For the other four statistical parameters, the stability of the two models declined in the testing stage. In particular, the stability of the positive predictive value and negative positive value of the ANN model decreased greatly, which means the accuracy of the ANN model in predicting landslide units and non-landslide units is very unstable. The mean values of the four statistical parameters of the two models declined, but the ANN model showed a large drop (more than 10%). From the aspect of these four statistical parameters, the SVM model could also be considered as the optimal model.

5.3. Comparison with Other Models In the evaluation of related geological disasters in Helong city, we used the information content method (ICM) and analytic hierarchy process (AHP) to evaluate the landslide susceptibility of the whole of Helong city based on both the grid unit and slope unit; the results are shown in Table4 First, from the perspective of diﬀerent mapping units, it can be seen that when ICM and AHP use grid units and slope units, respectively, for the landslide susceptibility evaluation of Helong city, their prediction accuracy is quite diﬀerent. The prediction accuracy of the slope unit is obviously higher than that of the grid unit. This shows that it is reasonable to use the slope unit as the mapping unit of landslide susceptibility in this paper. Furthermore, the prediction accuracy of the two models established in this paper is higher than that of ICM and AHP models. In particular, the prediction accuracy of SVM model is the highest. This is because landslide susceptibility mapping is a dichotomous problem, and the SVM model is better for making predictions in such problems.

Table 4. Modeling Fitting Results of the Study Area.

Mapping Units Method Prediction Accuracy (Mean) Training 89.72% Slope Units ANN Validating 88.08% Training 90.72% Slope Units SVM Validating 88.96% Grid Units ICM - 83.42% Grid Units AHP - 70.93% Slope Units ICM - 87.11% Slope Units AHP - 80.54%

5.4. Landslide Suceptibility Map analysis The landslide susceptibility map should meet the following two requirements [37]: (1) The landslide points should be distributed in the areas with high susceptibility as much as possible. The purpose of this is to evaluate the accuracy of grading the sensitivity of landslides. (2) In the landslide susceptibility map, the points predicted to be of high susceptibility should account for as low a proportion as possible. The purpose of this is to reduce the redundancy of high landslide susceptibility prediction and improve the hit ratio of susceptibility assessment. Symmetry 2020, 12, 1047 18 of 23

Table3 shows that the very high and high susceptibility areas for SVM model had a combined area of 279.03 km2, accounting for 18.90% of the total study area. Regarding the landslide occurrence, the very high and high susceptibility areas had 66 landslides, accounting for 79.52% of the total landslides. For the ANN model, the very high and high susceptibility areas had a combined area of 444.19 km2, accounting for 30.01% of the total study area. Regarding the landslide occurrence, the very high and high susceptibility areas had 61 landslides, accounting for 70.49% of the total landslides. This shows that the landslide susceptibility map produced by the SVM model in this paper is more reasonable than that of the ANN model. According to Figure 11, the very high, high, and moderate susceptibility areas were mainly distributed along rivers. The reason for this is that the erosion caused by a river increases the slope angle along the river. To adapt to the erosion of the river, landslides and other disasters often occur along rivers, thus reducing the slope angle adjacent to the river. Furthermore, for slopes located close to a river, the slope foot is soaked by the river water, which reduces the strength of the rock and soil mass inside, thus leading to landslides. The low susceptibility area was mainly distributed in the northwest of the study area. In this area, the vegetation coverage is high and human engineering activities are weak; thus, the geological environment of this area is relatively stable, and landslides are less common.

6. Conclusions In this study, we produced a landslide susceptibility map of the area to the southeast of Helong city. A total of 83 landslides were mapped in the study area through remote sensing interpretation and field investigation. The slope unit divided by the curvature watershed method was selected as the mapping unit. Based on field investigations and previous studies, three groups of influencing factors—lithological factors, topographic factors, and geological environment factors (a total of ten influencing factors: lithology, elevation, slope angle, slope aspect, topographic relief, curvature, land use, rainfall, distance to river, and distance to faults)—were selected to establish landslide susceptibility models. The ANN and SVM methods were used to build the models. Five-fold cross-validation, the ROC curve, and statistical parameters were used to optimize the models. Finally, the landslide susceptibility map was classified into five classes—very high, high, moderate, low, and very low. According to the model fitting results, the SVM model was the optimal model for our purposes. For the five classes—very high, high, moderate, low, and very low—the areas of the grades from the SVM model were 127.43, 151.60, 198.77, 491.19, and 506.91 km2, respectively. For landslide occurrence, the number of landslides in the five susceptibility classes were 52, 14, 8, 4, and 5, respectively; therefore, the very high and high susceptibility areas included 79.52% of the total landslides. This indicates that the landslide susceptibility map produced in this paper is reasonable. In conclusion, the following inferences were obtained: (1) Because the slope units are more closely related to the real topographic and geomorphic features, the obtained landslide susceptibility map is more reasonable. Thus, it is suggested that the slope unit should be used as the mapping unit in related work. (2) The SVM model needs less data and is more suitable for binary classification, so it can be given priority in the study of landslide susceptibility mapping.

Author Contributions: C.Y. contributed to data analysis and manuscript writing. J.C. proposed the main structure of this study. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the Natural Science Foundations of China grant number [U1702241]. Acknowledgments: Thanks to anonymous reviewers for their valuable feedback on the manuscript. Conﬂicts of Interest: The authors declare no conﬂict of interest. Symmetry 2020, 12, 1047 19 of 23

Appendix A

Table A1. Description of the Landslides.

Slope Slope Slope Slope Landslide Failure No. Lithology Microrelief Angle Aspect Height Shape Scale Mode 1 Granodiorite 60 198 6 Convex Steep Slope Small Pull-Type 2 Granodiorite 79 162 17 Convex Steep Coast Middle Pull-Type 3 Granodiorite 63 162 24 Convex Steep Coast Small Toppling Monzonitic 4 65 50 20 Concave Steep Coast Middle Pull-Type Granite Monzonitic 5 71 45 11 Convex Steep Coast Small Pull-Type Granite 6 Granodiorite 52 215 24 Convex Steep Slope Small Sliding 7 Granodiorite 57 110 15 Concave Steep Slope Middle Pull-Type Monzonitic 8 69 148 17 Convex Steep Coast Small Pull-Type Granite Monzonitic 9 62 221 17 Convex Steep Coast Small Sliding Granite Dioritic 10 52 275 12 Convex Steep Slope Small Pull-Type Porphyrite 11 Basalt 81 178 50 Convex Steep Coast Middle Toppling 12 Basalt 86 202 42 Convex Steep Coast Middle Pull-Type 13 Basalt 75 105 17 Convex Steep Coast Small Sliding 14 Basalt 58 128 23 Concave Steep Slope Middle Pull-Type 15 Basalt 87 178 32 Convex Steep Coast Middle Pull-Type 16 Basalt 82 130 20 Concave Steep Coast Middle Pull-Type 17 Basalt 68 178 18 Convex Steep Coast Small Sliding 18 Basalt 40 270 25 Straight Steep Slope Small Pull-Type 19 Basalt 84 253 12 Convex Steep Coast Small Pull-Type 20 Basalt 86 280 25 Convex Steep Coast Middle Pull-Type 21 Basalt 84 183 20 Convex Steep Coast Middle Pull-Type 22 Basalt 81 210 12 Concave Steep Coast Small Pull-Type 23 Granodiorite 71 193 23 Convex Steep Coast Small Pull-Type 24 Granodiorite 37 195 49 Convex Steep Slope Small Pull-Type 25 Granodiorite 40 196 43 Convex Steep Slope Small Toppling 26 Granodiorite 38 154 25 Concave Steep Slope Small Pull-Type 27 Granodiorite 42 268 9 Straight Steep Slope Middle Pull-Type 28 Granodiorite 55 254 8 Concave Steep Slope Small Pull-Type Monzonitic 29 43 190 12 Straight Steep Slope Small Pull-Type Granite 30 Granite 41 230 11 Convex Steep Slope Middle Pull-Type Monzonitic 31 62 130 10 Convex Steep Coast Small Pull-Type Granite 32 Basalt 87 250 12 Concave Steep Coast Small Pull-Type 33 Diorite 71 155 10 Convex Steep Coast Small Pull-Type 34 Granodiorite 42 136 8 Convex Steep Slope Small Pull-Type 35 Granodiorite 55 152 5 Convex Steep Slope Small Pull-Type 36 Basalt 50 135 14 Straight Steep Slope Small Pull-type 37 Granodiorite 71 90 9 Convex Steep Coast Small Toppling Monzonitic 38 35 186 15 Convex Steep Slope Small Pull-Type Granite Monzonitic 39 36 135 30 Straight Steep Slope Small Pull-Type Granite 40 Diorite 37 174 50 Straight Steep Slope Small Toppling 41 Granodiorite 48 148 16 Convex Steep Slope Small Pull-Type 42 Granodiorite 85 128 12 Convex Steep Coast Small Toppling 43 Granodiorite 77 224 15 Convex Steep Coast Small Toppling 44 Granodiorite 67 188 77 Convex Steep Coast Middle Sliding Monzonitic Staggered 45 57 204 61 Concave Steep Slope Middle Granite Breaking Monzonitic 46 81 228 25 Convex Steep Coast Middle Toppling Granite Monzonitic 47 72 250 52 Convex Steep Coast Middle Toppling Granite Monzonitic 48 72 185 31 Concave Steep Coast Middle Toppling Granite Monzonitic 49 76 190 142 Convex Steep Coast Middle Toppling Granite Monzonitic 50 74 120 13 Convex Steep Coast Small Toppling Granite Monzonitic 51 69 204 108 Convex Steep Coast Large Pull-Splitting Granite Symmetry 2020, 12, 1047 20 of 23

Table A1. Cont.

Slope Slope Slope Slope Landslide Failure No. Lithology Microrelief Angle Aspect Height Shape Scale Mode 52 Diorite 78 172 160 Convex Steep Coast Large Toppling 53 Diorite 62 134 224 Convex Steep Coast Large Toppling 54 Granodiorite 68 24 23 Convex Steep Coast Middle Toppling 55 Granodiorite 65 150 136 Concave Steep Coast Large Sliding 56 Granodiorite 76 162 228 Convex Steep Coast Large Pull-Splitting Monzonitic Staggered 57 70 155 151 Concave Steep Coast Large Granite Breaking Monzonitic 58 51 155 32 Convex Steep Slope Small Toppling Granite 59 Granodiorite 70 72 32 Straight Steep Coast Small Pull-Splitting 60 Basalt 72 125 46 Convex Steep Coast Middle Toppling 61 Basalt 58 200 45 Convex Steep Slope Middle Toppling 62 Basalt 83 218 28 Convex Steep Coast Small Toppling 63 Basalt 60 152 89 Concave Steep Slope Middle Sliding 64 Basalt 52 105 7 Convex Steep Slope Small Pull-Splitting 65 Granodiorite 60 130 23 Convex Steep Slope Small Pull-Splitting 66 Andesite 38 220 65 Convex Steep Slope Middle Pull-Splitting 67 Andesite 50 245 55 Concave Steep Slope Middle Pull-Splitting 68 Andesite 70 135 15 Straight Steep Coast Small Sliding Monzonitic 69 44 215 12 Concave Steep Slope Small Toppling Granite 70 Granodiorite 70 190 24 Straight Steep Coast Small Toppling 71 Granite 53 170 14 Convex Steep Coast Small Pull-Splitting 72 Quartzite 73 130 40 Convex Steep Coast Small Pull-Splitting Staggered 73 Quartzite 72 184 42 Concave Steep Coast Small Breaking Staggered 74 Quartzite 52 154 50 Convex Steep Slope Small Breaking Staggered 75 Quartzite 70 192 45 Convex Steep Coast Middle Breaking Monzonitic Staggered 76 58 195 55 Convex Steep Slope Middle Granite Breaking 77 Granodiorite 80 56 8 Straight Steep Coast Small Pull-Splitting Monzonitic 78 71 170 75 Convex Steep Coast Small Toppling Granite Monzonitic 79 75 138 40 Convex Steep Coast Small Toppling Granite Monzonitic 80 51 85 15 Straight Steep Slope Small Toppling Granite 81 Granodiorite 68 175 7 Straight Steep Coast Small Toppling 82 Basalt 62 184 7 Convex Steep Coast Small Toppling 83 Granodiorite 40 160 19 Convex Steep Slope Small Pull-Splitting Note: Landslide scale: (a) Small; Landslide volume <1 104 m3;(b) Middle: 100 104 m3 < Landslide volume <104 m3. × ×

References

1. Cao, C.; Wang, Q.; Chen, J.; Ruan, Y.; Zheng, L.; Song, S.; Niu, C. Landslide susceptibility mapping in vertical distribution law of precipitation area: Case of the xulong hydropower station reservoir, southwestern China. Water 2016, 8, 270. [CrossRef] 2. Sun, X.; Chen, J.; Bao, Y.; Han, X.; Zhan, J.; Peng, W. Landslide susceptibility mapping using logistic regression analysis along the jinsha river and its tributaries close to derong and deqin county, southwestern China. ISPRS Int. J. Geo-Inf. 2018, 7, 438. [CrossRef] 3. Zhan, J.; Chen, J.; Zhang, W.; Han, X.; Sun, X.; Bao, Y. Mass movements along a rapidly uplifting river valley: An example from the upper Jinsha River, southeast margin of the Tibetan Plateau. Environ. Earth Sci. 2018, 77, 634. [CrossRef] 4. Wang, F.; Xu, P.; Wang, C.; Wang, N.; Jiang, N. Application of a GIS-Based slope Unit Method for landslide susceptibility mapping along the Longzi River, Southeastern Tibetan Plateau, China. ISPRS Int. J. Geo-Inf. 2017, 6, 172. [CrossRef] 5. Hearn, G.J.; Hart, A.B. Landslide susceptibility mapping: A practitioner’s view. Bull. Eng. Geol. Environ. 2019, 78, 5811–5826. [CrossRef] 6. Reichenbach, P.; Rossi, M.; Malamud, B.D.; Mihir, M.; Guzzetti, F. A review of statistically-based landslide susceptibility models. Earth-Sci. Rev. 2018, 180, 60–91. [CrossRef] Symmetry 2020, 12, 1047 21 of 23

7. Guzzetti, F.; Mondini, A.C.; Cardinali, M.; Fiorucci, F.; Santangelo, M.; Chang, K.T. Landslide inventory maps: New tools for an old problem. Earth-Sci. Rev. 2012, 112, 42–66. [CrossRef] 8. Yang, X.J.; Chen, L.D. Geoinformation. Using multi-temporal remote sensor imagery to detect earthquake-triggered landslides. Int. J. Appl. Earth Obs. Geoinf. 2010, 12, 487–495. [CrossRef] 9. Mulas, M.; Corsini, A.; Cuozzo, G.; Callegari, M.; Thiebes, B.; Mair, V. Quantitative Monitoring of Surface Movements on Active Landslides by Multi-Temporal, High-Resolution X-Band SAR Amplitude Information: Preliminary Results; CRC Press: Boca Raton, FL, USA, 2016; pp. 1511–1516. 10. Schlogel, R.; Marchesini, I.; Alvioli, M.; Reichenbach, P.; Rossi, M.; Malet, J.P. Optimizing landslide susceptibility zonation: Effects of DEM spatial resolution and slope unit delineation on logistic regression models. Geomorphology 2018, 301, 10–20. [CrossRef] 11. Rotigliano, E.; Cappadonia, C.; Conoscenti, C.; Costanzo, D.; Agnesi, V. Slope units-based flow susceptibility model: Using validation tests to select controlling factors. Nat. Hazards 2012, 61, 143–153. [CrossRef] 12. Alvioli, M.; Marchesini, I.; Reichenbach, P.; Rossi, M.; Ardizzone, F.; Fiorucci, F.; Guzzetti, F. Automatic delineation of geomorphological slope units with r.slopeunits v1.0 and their optimization for landslide susceptibility modeling. Geosci. Model Dev. 2016, 9, 3975–3991. [CrossRef] 13. Pourghasemi, H.R.; Rossi, M. Landslide susceptibility modeling in a landslide prone area in Mazandarn Province, north of Iran: A comparison between GLM, GAM, MARS, and M-AHP methods. Theor. Appl. Climatol. 2017, 130, 609–633. [CrossRef] 14. Lagomarsino, D.; Tofani, V.; Segoni, S.; Catani, F.; Casagli, N. A Tool for Classification and Regression Using Random Forest Methodology: Applications to Landslide Susceptibility Mapping and Soil Thickness Modeling. Environ. Model. Assess. 2017, 22, 201–214. [CrossRef] 15. Dou, J.; Bui, D.T.; Yunus, A.P.; Jia, K.; Song, X.; Revhaug, I.; Xia, H.; Zhu, Z. Optimization of Causative Factors for Landslide Susceptibility Evaluation Using Remote Sensing and GIS Data in Parts of Niigata, Japan. PLoS ONE 2015, 10, e0133262. [CrossRef] 16. Frattini, P.; Crosta, G.; Carrara, A. Techniques for evaluating the performance of landslide susceptibility models. Eng. Geol. 2010, 111, 62–72. [CrossRef] 17. Chen, W.; Li, W.; Chai, H.; Hou, E.; Li, X.; Ding, X. GIS-based landslide susceptibility mapping using analytical hierarchy process (AHP) and certainty factor (CF) models for the Baozhong region of Baoji City, China. Environ. Earth Sci. 2016, 75, 63. [CrossRef] 18. Cao, C.; Xu, P.; Wang, Y.; Chen, J.; Zheng, L.; Niu, C. Flash Flood Hazard Susceptibility Mapping Using Frequency Ratio and Statistical Index Methods in Coalmine Subsidence Areas. Sustainability 2016, 8, 948. [CrossRef] 19. Xiao, T.; Segoni, S.; Chen, L.; Yin, K.; Casagli, N. A step beyond landslide susceptibility maps: A simple method to investigate and explain the different outcomes obtained by different approaches. Landslides 2020, 17, 627–640. [CrossRef] 20. Bai, S.-B.; Wang, J.; Lue, G.-N.; Zhou, P.-G.; Hou, S.-S.; Xu, S.-N. GIS-based logistic regression for landslide susceptibility mapping of the Zhongxian segment in the Three Gorges area, China. Geomorphology 2010, 115, 23–31. [CrossRef] 21. Sun, X.; Chen, J.; Han, X.; Bao, Y.; Zhan, J.; Peng, W. Application of a GIS-based slope unit method for landslide susceptibility mapping along the rapidly uplifting section of the upper Jinsha River, South-Western China. Bull. Eng. Geol. Environ. 2019.[CrossRef] 22. Segoni, S.; Pappafico, G.; Luti, T.; Catani, F. Landslide susceptibility assessment in complex geological settings: Sensitivity to geological information and insights on its parameterization. Landslides 2020.[CrossRef] 23. Ren, D.; Zhou, D.; Liu, D.; Dong, F.; Ma, S.; Huang, H. Formation mechanism of the Upper Triassic Yanchang Formation tight sandstone reservoir in Ordos Basin-Take Chang 6 reservoir in Jiyuan oil field as an example. J. Pet. Sci. Eng. 2019, 178, 497–505. [CrossRef] 24. Ayalew, L.; Yamagishi, H. The application of GIS-based logistic regression for landslide susceptibility mapping in the Kakuda-Yahiko Mountains, Central Japan. Geomorphology 2005, 65, 15–31. [CrossRef] 25. Lee, S.; Min, K. Statistical analysis of landslide susceptibility at Yongin, Korea. Environ. Geol. 2001, 40, 1095–1113. [CrossRef] 26. Conforti, M.; Pascale, S.; Robustelli, G.; Sdao, F. Evaluation of prediction capability of the artificial neural networks for mapping landslide susceptibility in the Turbolo River catchment (northern Calabria, Italy). Catena 2014, 113, 236–250. [CrossRef] Symmetry 2020, 12, 1047 22 of 23

27. Oh, H.-J.; Pradhan, B. Application of a neuro-fuzzy model to landslide-susceptibility mapping for shallow landslides in a tropical hilly area. Comput. Geosci. 2011, 37, 1264–1276. [CrossRef] 28. Park, S.; Choi, C.; Kim, B.; Kim, J. Landslide susceptibility mapping using frequency ratio, analytic hierarchy process, logistic regression, and artificial neural network methods at the Inje area, Korea. Environ. Earth Sci. 2013, 68, 1443–1464. [CrossRef] 29. Hong, H.; Pradhan, B.; Xu, C.; Tien Bui, D. Spatial prediction of landslide hazard at the Yihuang area (China) using two-class kernel logistic regression, alternating decision tree and support vector machines. Catena 2015, 133, 266–281. [CrossRef] 30. Binh Thai, P.; Dieu Tien, B.; Dholakia, M.B.; Prakash, I.; Ha Viet, P.; Mehmood, K.; Hung Quoc, L. A novel ensemble classifier of rotation forest and Naive Bayer for landslide susceptibility assessment at the Luc Yen district, Yen Bai Province (Viet Nam) using GIS. Geomat. Nat. Hazards Risk 2017, 8, 649–671. [CrossRef] 31. Chen, W.; Xie, X.; Wang, J.; Pradhan, B.; Hong, H.; Bui, D.T.; Duan, Z.; Ma, J. A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility. Catena 2017, 151, 147–160. [CrossRef] 32. Dieu Tien, B.; Tran Anh, T.; Klempe, H.; Pradhan, B.; Revhaug, I. Spatial prediction models for shallow landslide hazards: A comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides 2016, 13, 361–378. [CrossRef] 33. Kumar, D.; Thakur, M.; Dubey, C.S.; Shukla, D.P. Landslide Susceptibility Mapping & Prediction using Support Vector Machine for Mandakini River Basin, Garhwal Himalaya, India. Geomorphology 2017, 295, 115–125. 34. Chen, W.; Pourghasemi, H.R.; Kornejady, A.; Zhang, N. Landslide spatial modeling: Introducing new ensembles of ANN, MaxEnt, and SVM machine learning techniques. Geoderma 2017, 305, 314–327. [CrossRef] 35. Yesilnacar, E.; Topal, T. Landslide susceptibility mapping: A comparison of logistic regression and neural networks methods in a medium scale study, Hendek region (Turkey). Eng. Geol. 2005, 79, 251–266. [CrossRef] 36. Binh Thai, P.; Dieu Tien, B.; Prakash, I.; Dholakia, M.B. Hybrid integration of Multilayer Perceptron Neural Networks and machine learning ensembles for landslide susceptibility assessment at Himalayan area (India) using GIS. Catena 2017, 149, 52–63. [CrossRef] 37. Su, Q.; Zhang, J.; Zhao, S.; Wang, L.; Liu, J.; Guo, J. Comparative Assessment of Three Nonlinear Approaches for Landslide Susceptibility Mapping in a Coal Mine Area. ISPRS Int. J. Geo-Inf. 2017, 6, 228. [CrossRef] 38. Aditian, A.; Kubota, T.; Shinohara, Y. Comparison of GIS-based landslide susceptibility models using frequency ratio, logistic regression, and artificial neural network in a tertiary region of Ambon, Indonesia. Geomorphology 2018, 318, 101–111. [CrossRef] 39. Jiang, P.; Zeng, Z.; Chen, J.; Huang, T. Generalized Regression Neural Networks with K-Fold Cross-Validation for Displacement of Landslide Forecasting. In Advances in Neural Networks—Isnn 2014; Zeng, Z., Li, Y., King, I., Eds.; Springer: Cham, Switzerland, 2014; Volume 8866, pp. 533–541. 40. Jiang, T.; Lei, P.; Qin, Q. An Application of SVM-Based Classification in Landslide Stability. Intell. Autom. Soft Comput. 2016, 22, 267–271. [CrossRef] 41. Li, X.Z.; Kong, J.M.J.N.H.; Sciences, E.S. Application of GA-SVM method with parameter optimization for landslide development prediction. Nat. Hazards Earth Syst. Sci. 2014, 14, 5295–5322. [CrossRef] 42. Pradhan, B.; Sameen, M.I. Manifestation of SVM-Based Rectified Linear Unit (ReLU) Kernel Function in Landslide Modelling; Springer: Singapore, 2018. 43. San, B.T. An evaluation of SVM using polygon-based random sampling in landslide susceptibility mapping: The Candir catchment area (western Antalya, Turkey). Int. J. Appl. Earth Obs. Geoinf. 2014, 26, 399–412. [CrossRef] 44. Ballabio, C.; Sterlacchini, S. Support Vector Machines for Landslide Susceptibility Mapping: The Staffora River Basin Case Study, Italy. Math. Geosci. 2012, 44, 47–70. [CrossRef] 45. Chen, W.; Pourghasemi, H.R.; Panahi, M.; Kornejady, A.; Wang, J.; Xie, X.; Cao, S. Spatial prediction of landslide susceptibility using an adaptive neuro-fuzzy inference system combined with frequency ratio, generalized additive model, and support vector machine techniques. Geomorphology 2017, 297, 69–85. [CrossRef] 46. Zhang, X.; Wang, Q.; Huo, Z.; Yu, T.; Wang, G.; Liu, T.; Wang, W. Prediction of Frost-Heaving Behavior of Saline Soil in Western Jilin Province, China, by Neural Network Methods. Math. Probl. Eng. 2017.[CrossRef] Symmetry 2020, 12, 1047 23 of 23

47. Hong, H.; Pradhan, B.; Jebur, M.N.; Bui, D.T.; Xu, C.; Akgun, A. Spatial prediction of landslide hazard at the Luxi area (China) using support vector machines. Environ. Earth Sci. 2016, 75.[CrossRef] 48. Huang, F.; Yin, K.; Huang, J.; Gui, L.; Wang, P.Landslide susceptibility mapping based on self-organizing-map network and extreme learning machine. Eng. Geol. 2017, 223, 11–22. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).