A Geostatistical Simulation of a Mineral Deposit Using Uncertain Experimental Data

minerals

Article A Geostatistical Simulation of a Mineral Deposit using Uncertain Experimental Data

João Narciso 1,* , Cristina Paixão Araújo 2, Leonardo Azevedo 1 , Ruben Nunes 1 , João Filipe Costa 2 and Amílcar Soares 1

1 CERENA/DECivil, Instituto Superior Técnico, Universidade Técnica de Lisboa, 1049-001 Lisbon, Portugal; [email protected] (L.A.); [email protected] (R.N.); [email protected] (A.S.) 2 Departamento de Engenharia de Minas, Universidade Federal do Rio Grande do Sul, 91509-900 Porto Alegre, Brazil; [email protected] (C.P.A.); [email protected] (J.F.C.) * Correspondence: [email protected]; Tel.: +351-218-417-425

Received: 19 March 2019; Accepted: 19 April 2019; Published: 23 April 2019

Abstract: In the geostatistical modeling and characterization of natural resources, the traditional approach for determining the spatial distribution of a given deposit using stochastic sequential simulation is to use the existing experimental data (i.e., direct measurements) of the property of interest as if there is no uncertainty involved in the data. However, any measurement is prone to error from different sources, for example from the equipment, the sampling method, or the human factor. It is also common to have distinct measurements for the same property with different levels of resolution and uncertainty. There is a need to assess the uncertainty associated with the experimental data and integrate it during the modeling procedure. This process is not straightforward and is often overlooked. For the reliable modeling and characterization of a given ore deposit, measurement uncertainties should be included as an intrinsic part of the geo-modeling procedure. This work proposes the use of a geostatistical simulation algorithm to integrate uncertain experimental data through the use of stochastic sequential simulations with local probability functions. The methodology is applied to the stochastic modeling of a benchmark mineral deposit, where certain and uncertain experimental data co-exist. The uncertain data is modeled by assigning individual probability distribution functions to each sample location. Different strategies are proposed to build these local probability distributions. Each scenario represents variable degrees of uncertainty. The impacts of the different modeling approaches on the final deposit model are discussed. The resulting models of these proposed scenarios are also compared against those retrieved from previous studies that use conventional geostatistical simulation. The results from the proposed approaches showed that using stochastic sequential simulation with local probability functions to represent local uncertainties decreased the estimation error of the resulting model, producing fewer misclassified ore blocks.

Keywords: geostatistical modeling; mineral deposit; uncertain experimental data; direct sequential simulation; local probability functions

1. Introduction In the mining industry, the economic value of a mineral deposit is estimated using a resource model that is built from the existing direct measurements of the resource of interest. These models are greatly inﬂuenced by the quantity, quality, and spatial distribution of the existing direct measurements for the properties of interest (e.g., grades). In general, ore deposit models are built based exclusively on the existing primary data that has been collected from rock core samples and/or logging tools with great precision and accuracy, which are less prone to measurement errors and, therefore, low uncertainty. During the determination of the spatial distribution of this resource, the kinds of data used

Minerals 2019, 9, 247; doi:10.3390/min9040247 www.mdpi.com/journal/minerals Minerals 2019, 9, 247 2 of 13 are generally high quality and provide reliable measurements of the property of interest. Therefore, the data are considered to be reliable hard data with no uncertainty. Later, during quasi-mining operations, more samples of the property of interest become available. These samples are normally used to fill in the informational gaps that were left over from previous rock core samples, and are often collected along several distinct campaigns from blast holes during the production years, prepared using different protocols, sampled with different equipment, and/or analyzed at different laboratories. Ideally, as the spacing between samples decreases, the uncertainty about the spatial distribution of the mineral deposit is also reduced. However, due to the abovementioned reasons, this information does not have the same quality throughout the entire dataset or within the existing rock core samples. Therefore, it is common to observe different statistical populations formed by readings of the same geological attribute when sampled, prepared, and/or assayed by distinct protocols. These analyses often produce different results when compared with the high quality measurements from the core analysis. From a modeling point of view, lower quality data is frequently considered to be secondary variables (i.e., soft data). In the mining industry, it is still a common practice to exclude these uncertain measurements from the long-term and auditable resource models due to their imprecision and the bias they may introduce during the geo-modeling workflow and the final ore deposit model. Whether the soft measurement data should be included or not during the modeling procedure is still an open and relevant scientific question, e.g., [1–7]. Emery et al. [1] discussed the impact of the lack of accuracy within the available set of measurements for resource and reserve evaluation. This work showed that the quantity of data prevailed over its quality (i.e., the more experimental samples that were analyzed, the more accurate the resource and/or reserve models were, independent of the data quality). Therefore, the characterization of a given ore deposit would be improved if all the data available, even with different degrees of uncertainty, are considered during resource modeling. An alternative approach to integrate secondary data was proposed by Babak and Deutsch [2]. Their methodology consisted of merging all secondary data into a single super secondary variable, which is then used in the geo-modeling workflow through collocated co-kriging. Leuangthong and Deutsch [8] transformed multiple variables into univariate and multivariate Gaussian variables with no cross-correlation. The transformed variables were then submitted to independent stochastic Gaussian simulation and back transformed to the original variables. Ribeiro et al. [3] proposed the use of mixed support data (i.e., samples obtained using diamond drill holes, and samples collected using a reverse circulation drilling method) in an iron ore deposit evaluation. Although the results showed a small global bias and high spatial correlation between some variables, in general, the differences in volumetric sample support between the two drilling methods generated a conditional bias and low or no correlation on a local scale. Reuwsaat [4] also discussed the effect of sampling using different supports on the precision and accuracy of iron ore deposit models, demonstrating the importance of the correlation between the variables in the quality of the models. Minnitt and Deutsch [5] showed the importance of using simultaneously primary, accurate, and precise data together with a more abundant, imprecise, and biased secondary dataset, using cokriging as the estimation technique and a linear model of coregionalization to control the use of the secondary data in the estimation of recoverable reserves of mineral deposit grades. Cornah and Machaka [6], and Donovan [7], using a number of specialized geostatistical tools, most of all cokriging-based techniques, also demonstrated that the use of imprecise or biased (or both) secondary data in mineral resource estimations is recommended if the correlation between the data types can be properly estimated. The geostatistical tools used in these works allowed the use of secondary data without transmitting bias and error through to the final models [7], but the uncertainty associated with the measurements was not properly assessed. A usual way to integrate secondary information during the geo-modeling workflow of natural resources relies on stochastic sequential co-simulation algorithms [9–12]. Alternative procedures [13] Minerals 2019, 9, 247 3 of 13 consider the integration of secondary information in the form of geophysical data through Sequential Gaussian Co-simulation (Co-SGS), but do not consider the physical (or geological) processes between variables (i.e., the secondary data is integrated exclusively considering the statistical criteria). Alternative approaches for joint simulation define a hierarchy of variables, in a process known by Collocated Co-simulation [14]. In this paper, we propose an alternative way to integrate uncertain soft data into the natural resource modeling procedure through stochastic sequential simulation with local probability distribution functions [15]. This approach uses the soft data as a probabilistic model by transforming the existing soft data into local probability distribution functions that represent the existing local uncertainty. These local probability functions may be derived exclusively from the existing secondary data, inferred from analogue geological scenarios, provided by expert knowledge, or a combination of all these options. The distribution types should be directly related to the degree of uncertainty expected for the secondary data in order to reproduce the real conditions of the natural resource and its uncertainty measurements. In order to evaluate the advantages of integrating uncertainty experimental data in stochastic simulation methods, we compare the results between models resulting from conventional stochastic simulation algorithms (i.e., direct sequential simulation (DSS) and co-simulation (co-DSS)) and from stochastic sequential simulation with local probability distribution, as a way to integrate uncertain experimental data (soft data). We base our comparisons exclusively on the methods related to DSS to ensure that all scenarios are based on the same type of assumptions.

2. Methodology This study proposed direct sequential simulation with local probability distributions [15] as a modeling technique to include uncertain experimental data consistently during the geo-modeling workflow. DSS with local probability distribution functions can be considered to be a natural extension of the DSS with joint probability distributions [16]. This geostatistical simulation algorithm was designed to integrate data measurements that have variable degrees of uncertainty in the stochastic sequential simulation workflow. This uncertainty can result from the lack of accuracy and precision in the measurements at each location, and can be expressed by a local probability distribution function. As with any stochastic sequential simulation technique, the main idea of this simulation algorithm was to generate spatially-correlated data values in a sequential approach. DSS with local probability distribution functions is based on two main steps. In the first step, locations associated with uncertain samples, (xβ), were visited following a random path, generating Nd values from the collocated local distributions F(z(xβ)). Following the guidelines of direct sequential simulation [12], at each location, (xβ), the local mean and variance were computed using simple kriging conditioned to the existing hard data values, z(xα), and to the previously simulated samples from other locations associated with s uncertain distributions, z (xβ), around a given neighborhood. From the local distribution F(z(xβ)), centered at the local simple kriging estimate: X X s s z = λλα(x0)z(xα) + βλβ(x0)z (xβ), (1) where the kriging weights, λα and λβ, are associated with the hard data and the previously simulated uncertain experimental data, respectively, and the local variance is represented by the simple kriging s variance. From this auxiliary distribution, a value, z (x0), is drawn at the location of the uncertain experimental data [15]. In the second step, after all uncertain locations were visited and a set of Nd values was generated from the local distributions, the stochastic sequential procedure visits the remaining locations of the model using the conventional direct sequential simulation methodology. The resulting models Minerals 2019, 9, 247 4 of 13 reproduced the spatial covariances, and the global and local distributions [15], as inferred from the existing certain and uncertain data. The main question about the usability of this stochastic sequential simulation technique relied on how the local probability distribution functions, F(z(xβ)), were built to properly describe the uncertainty of the existing measurements. These distribution functions can be derived from expert guesses or from auxiliary variables, at another Nd experimental sample locations associated with uncertain measurements of the same property in the framework of the stochastic sequential simulation [15]. In this study,we proposed alternative strategies to build local probability distributions. The impacts of the different strategies in the resulting models were then compared against each other and contrasted against models resulting from a conventional stochastic sequential simulation algorithm. These different scenarios were applied to a mineral deposit from where a reference model was built from the existing data [17]. This dataset is described in the section below. In order to assess the impact of the local probability distribution functions, two different approaches were created:

1. Experimental local probability distribution functions: the local probability distribution functions were built from 15 soft data samples that were the nearest to the location of the uncertain measurement; and 2. Parametric local probability distribution functions: the local probability distribution functions were built using Gaussian distributions with the means and standard deviations inferred from the 15 samples closest to the uncertain location.

The results of each simulation strategy were compared quantitatively by calculating the error by re-blocking each realization along the reference model, and by computing the proportions of the classiﬁed mineral resources as a function of the calculated error. To properly assess the usefulness of the local probability distribution functions and the approaches proposed for a mineral deposit simulation strategy, we compared the following four simulation strategies, using an E-type model (i.e., the mean model of all simulations), the variance model, and the absolute relative errors for each simulation:

Direct sequential simulation only using the hard data to simulate the mineral deposit; • Direct sequential collocated co-simulation using the accurate and precise experimental data • as the hard data and the uncertain experimental data as the secondary data to simulate the mineral deposit; Direct sequential simulation with local probability distribution functions using the accurate • and precise experimental data as the hard data and the uncertain experimental data to build experimental local probability distribution functions; and Direct sequential simulation with local probability distribution functions using the accurate and • precise experimental data as the hard data and the uncertain experimental data to build parametric local probability distribution functions.

3. Application

3.1. Dataset Description The application example shown here to illustrate the proposed strategy uses the Walker Lake dataset [17], an exhaustive dataset of 78,000 point support samples with a regular sampling grid of 1 m 1 m (designated from now on as V_Ref_points) and two numerical variables. The original × variable, V, from this dataset was scaled up into blocks sized 5 m 5 m (Figure1) to represent the × block grade distribution of a real copper mineral deposit (designated in this text as V_Ref_blocks). The upscaling resulted in a grid of 3120 blocks, which were considered in this work to be the true block grades. MineralsMinerals2019 ,20199, 247, 9, x FOR PEER REVIEW 5 of 135 of 13 Minerals 2019, 9, x FOR PEER REVIEW 5 of 13

Figure 1. Exhaustive Walker Lake dataset [17] and the experimental data locations used in this Figure 1. Exhaustive Walker Lake dataset [17] and the experimental data locations used in this work workFigure (red 1. filled Exhaustive circles: Walker samples Lake without dataset [17] measurement and the experimental errors; white data locations filled circles: used in samples this work with (red filled circles: samples without measurement errors; white filled circles: samples with (red filled circles: samples without measurement errors; white filled circles: samples with measurementmeasurement errors). errors). measurement errors). Within the scope of this case study, the existing dataset was modified in order to retrieve Within the scope of this case study, the existing dataset was modified in order to retrieve two Within the scope of this case study, the existing dataset was modified in order to retrieve two twodistinct distinct types types of data of data (Figure (Figure 2). The2). original The original dataset was dataset sampled was into sampled accurate intoand precise accurate experimental and precise distinct types of data (Figure 2). The original dataset was sampled into accurate and precise experimental experimentalsamples, and, samples, therefore, and, with therefore, no uncertainty. with no These uncertainty. data were These regularly data spaced were regularly at 20 m × spaced 20 m at samples, and, therefore, with no uncertainty. These data were regularly spaced at 20 m × 20 m 20 m(designated20 m (designated as V_20 × as 20), V_20 simulating20), simulating diamond-drill diamond-drill hole samples. hole The samples. secondary The samples, secondary (designated× as V_20 × 20), simulating× diamond-drill hole samples. The secondary samples, samples,considered considered to be tosoft be data soft simulating data simulating blast hole blast samples, hole samples, were located were locatedin a 5 m in × a5 5m m regular5 m grid regular considered to be soft data simulating blast hole samples, were located in a 5 m × 5 m regular× grid gridpattern pattern based based on on sampling sampling the the orig originalinal dataset. dataset. Imprecision Imprecision and bias and were bias then were added then to added these todata these pattern based on sampling the original dataset. Imprecision and bias were then added to these data data(adding (adding variance variance and and error error bias) bias) in inorder order toto mimic mimic real real blast blast hole hole samples samples contaminated contaminated with with (adding variance and error bias) in order to mimic real blast hole samples contaminated with measurement errors [18]. To the original sample data, we either added or subtracted a random measurementmeasurement errors errors [18 ].[18]. To theTo originalthe original sample sample data, data, we eitherwe either added added or subtracted or subtracted a random a random relative relative sampling error of 10% and added a 25% bias (increasing the mean of the grades by 25%). samplingrelative error sampling of 10% error and of added 10% an ad 25% added bias a 25% (increasing bias (increasing the mean the of mean the gradesof the grades by 25%). by 25%).

Figure 2. Schematic representation of the modifications performed to the original exhaustive dataset. FigureFigure 2. Schematic 2. Schematic representation representation of of the themodiﬁcations modifications performed performed to to the the original original exhaustive exhaustive dataset. dataset. Two types of data were considered: samples without and with measurement errors. TwoTwo types types of data of data were were considered: considered: samples samples without without andand with measurement measurement errors. errors.

Table1 shows the main statistics of the datasets used in this work, such as the reference point support dataset (V_Ref_points), the reference block grade distribution (V_Ref_blocks), and the sampled Minerals 2019, 9, x FOR PEER REVIEW 6 of 13

Minerals 2019, 9, 247 6 of 13 Table 1 shows the main statistics of the datasets used in this work, such as the reference point support dataset (V_Ref_points), the reference block grade distribution (V_Ref_blocks), and the datasetsampled used dataset for theused experimental for the experimental data without data uncertainty without uncertainty (V_20 20). (V_20 The hard × 20). data The samples hard data had × meansamples values had mean similar values to the similar true mean to the of true the mean reference of the point reference support point dataset, support assuming dataset, that assuming it was accuratethat it was and accurate precise and data, precise with no data, biases with or no imprecision. biases or imprecision. The datasets, The where datasets, bias andwhere imprecision bias and wereimprecision added (designatedwere added within (designated the scope within of this the work scop ase V_5of this5_ work+25%) as in V_5 order × to5_+25%) simulate in situations order to × ofsimulate poor quality situations data, of poor had aquality mean data, and standard had a me deviationan and standard which deviation were 25% which greater were than 25% the greater mean andthanstandard the meandeviation and standard of the deviation reference of pointthe refere distributionnce point dataset distribution (V_Ref_points), dataset (V_Ref_points), thus introducing thus uncertaintyintroducing intouncertainty the grade into estimation the grade process. estimation process.

Table 1.1. Main statistics of the original reference datasetdataset and the uncertainty dataset used in this work.

DataData No. No. of Samplesof Samples Mean Mean Standa Standardrd Deviation Deviation CV CV Minimum Minimum Maximum Maximum V_Ref_pointsV_Ref_points 78,00078,000 2.782.78 2.50 2.50 0.90 0.90 0.00 0.00 16.31 16.31 V_Ref_blocksV_Ref_blocks 31203120 2.78 2.78 2.49 2.49 0.89 0.89 0.00 0.00 15.68 15.68 V_20V_2020 × 20 195195 2.732.73 2.43 2.43 0.89 0.89 0.00 0.00 10.13 10.13 × V_5 5_+25% 2925 3.44 3.12 0.90 0.00 18.30 V_5× × 5_+25% 2925 3.44 3.12 0.90 0.00 18.30

The cross-correlogram of the hard and soft data showedshowed a correlationcorrelation ofof 0.620.62 (Figure(Figure3 3).). InIn thethe mining industry, thisthis correlationcorrelation isis consideredconsidered toto bebe moderatemoderate and suggests that the soft data can be incorporated inin the the model. model. However, However, the the correlation correlation indicates indicates possible possible measurement measurement errors errors that result that fromresult the from diﬀ theerent different techniques techniques and campaigns and campaigns used for used sampling, for sampling, with all with sampling all sampling preparation preparation errors associatederrors associated and poor and preparation poor preparation protocols protocols in laboratory. in laboratory.

Figure 3. Omnidirectional cross-correlogram for hard data and soft data.

The spatial continuity pattern of the property of interest was modeled by combining the hard data (V_20 20) and the soft data (V_5 5_+25%) using a spherical (Sph) variogram model, with the data (V_20 ×× 20) and the soft data (V_5 × 5_+25%) using a spherical (Sph) variogram model, with the major direction deﬁneddefined asas 157.5157.5°◦ and thethe minorminor directiondirection deﬁneddefined asas 67.567.5°:◦: 𝑁157𝑆𝐸N157SE𝑁67𝑆𝐸N67SE 𝑁157𝑆𝐸N157SE𝑁67𝑆𝐸N67SE γv𝛾(h(ℎ)) ==1.5 1.5+ + 3.0 ⋅ 𝑆𝑝ℎSph((11))⋅ , , +5.1⋅𝑆𝑝ℎ(2)⋅+ 5.1 Sph(2) , , (2) (2) · · 3535 𝑚 m 2525 𝑚 m · ·80 𝑚80 m44 𝑚44 m

This variogram model was used in allall thethe stochasticstochastic sequentialsequential simulationsimulation andand co-simulationco-simulation methodologies compared under the scope of this work.work. For each simulation strategy, we generatedgenerated ensembles of of 100 100 realizations realizations in inorder order to compute to compute the statistics the statistics from fromthem themand compare and compare the results the resultsobtained. obtained.

3.2. Results For illustrationillustration purposes,purposes, the the E-type E-type model model is is shown shown for for the the ensemble ensemble of 100 of 100 realizations realizations that wasthat generatedwas generated for each for modeling each modeling strategy strategy (Figure4 ).(Figure Generally, 4). theGenerally, averaged the model averaged for each model ensemble for waseach similarensemble for was all the similar modeling for all strategies. the modeling The main strategies. features The of main interest features (i.e., high of interest grade values) (i.e., high appeared grade invalues) the same appeared locations in the with same approximately locations with theapproxim same spatialately the extent. same spatial The main exte dint.ﬀ erencesThe main between differences the

MineralsMinerals2019 2019, 9,, 2479, x FOR PEER REVIEW 7 of7 13 of 13

between the modeling strategies depended on the grade values. The E-type model obtained from the modelingtraditional strategies co-DSS dependedshowed a onbias the toward grade higher values. gr Theade E-typevalues modeldue to obtainedthe bias that from existed the traditional in the co-DSSsecondary showed data, a bias including toward higherin the gradegeostatistical values duesimulation to the bias (Figure that existed4b). On in thethe secondaryother hand, data, includingincorporating in the the geostatistical uncertain data simulation in the form (Figure of loca4b).l Onprobability the other distributions, hand, incorporating allowed us the touncertain avoid a databias in thetowards form ofthe local higher probability grade values distributions, (Figure allowed4c,d). In us these to avoid cases, a biasthe towardsresultant the model higher better grade valuesresembled (Figure the4c,d). one obtained In these when cases, exclusively the resultant using model the experimental better resembled hard data the without one obtained uncertainty when exclusively(Figure 4a). using the experimental hard data without uncertainty (Figure4a).

(a) (b)

FigureFigure 4. 4.E-type E-type of of the the 100 100simulations simulations generatedgenerated for each simulation strategy strategy used: used: (a) ( adirect) direct sequentialsequential simulation simulation with with only only hard hard data,data, ((bb)) directdirect sequentialsequential collocated collocated co-simulation, co-simulation, (c) ( cdirect) direct sequentialsequential simulation simulation with with experimental experimental point point distributions,distributions, and ( (dd)) direct direct sequential sequential simulation simulation with with parameterizedparameterized point point distributions. distributions.

ContraryContrary to to the the mean mean model, model, the the spatial spatial variabilityvariability of the computed computed variance variance for for each each ensemble ensemble dependeddepended greatly greatly on on the the modeling modeling strategy strategy (Figure(Figure5 5).). TheThe methodmethod to integrate secondary secondary uncertain uncertain datadata was was a critical a critical choice choice here. here. As As expected, expected, whenwhen usingusing only the experimental experimental data data without without uncertainty uncertainty (Figure(Figure5a), 5a), the the variance variance within within the the ensemble ensemble depended depended exclusivelyexclusively on the the relative relative location location of of the the hard data. By using a simple co-DSS (Figure 5b), the spatial variance of the model decreased and the hard data. By using a simple co-DSS (Figure5b), the spatial variance of the model decreased and model parameter space was not widely explored (i.e., the variability within the ensemble of one the model parameter space was not widely explored (i.e., the variability within the ensemble of one hundred realizations was small). However, the use of local probability distribution functions, built MineralsMinerals2019 2019, 9,, 2479, x FOR PEER REVIEW 8 of8 13 of 13

hundred realizations was small). However, the use of local probability distribution functions, built fromfrom the the existing existing uncertain uncertain experimental experimental data data (Figure(Figure5 5c,d),c,d), allowedallowed for a a variable variable degree degree of of variance variance thatthat was was spatially spatially dependent. dependent. Note Note that that the the variance variance was was not not homogeneous homogeneous throughout throughout the the entire entire ﬁeld, butfield, depended but depended directly directly on the on type the and type shape and shape of the of local the local probability probability distributions distributions used used to to model model the uncertaintythe uncertainty associated associated with the with secondary the secondary data. The data. spatial The distributionspatial distribution of the variance of the variance also depended also ondepended the spatial on homogeneity the spatial homogeneity of the experimental of the experime hardntal data hard samples data samples used in used the in simulations. the simulations. In the examplesIn the examples shown in shown Figure in5, Figure high values 5, high of values variance of variance were mainly were locatedmainly atlocated the boundaries at the boundaries between lowbetween and high low grade and high values. grade values.

(a) (b)

FigureFigure 5. 5.Variance Variance of of the the 100 100 simulations simulations forfor eacheach simulationsimulation strategy strategy used: used: (a ()a )direct direct sequential sequential simulationsimulation with with only only hard hard data, data, ( b()b) direct direct sequential sequential collocatedcollocated co-simulation, (c ()c )direct direct sequential sequential simulationsimulation withwith experimental experimental point point distributions, distributions, and (d) direct and (sequentiald) direct simulation sequential with simulation parameterized with parameterizedpoint distributions. point distributions.

InIn order order to to quantitatively quantitatively compare compare thethe resultsresults of each method, method, the the E-type E-type model model obtained obtained from from eacheach simulation simulation technique technique and and the the referencereference model were re-blocked re-blocked into into 5 5m m × 5 m5 mblocks blocks (Table (Table 1).1 ). × ByBy changing changing the the scale, scale, we we were were now now able able toto calculatecalculate the absolute relative relative error error (ARE; (ARE; Equation Equation (3)): (3)):

_ 𝐸𝑟𝑟𝑜𝑟 = , (3) Etype VRe f _Blocks − ErrorReal = , (3) Etype Minerals 2019, 9, 247 9 of 13

Minerals 2019, 9, x FOR PEER REVIEW 9 of 13 where Etype represents the averaged model for each simulation technique and VRef_Blocks represents the where 𝐸 represents the averaged model for each simulation technique and VRef_Blocks represents truethe block true block values. values. FigureFigure6 shows 6 shows the the map map for for the the error error of of each each modelingmodeling strategy computed computed using using Equation Equation (3). (3). TheThe ARE ARE map, map, obtained obtained from from the DSSthe DSS using using only only the samples the samples without without uncertainty uncertainty (Figure (Figure6a), showed 6a), a large,showed spatially-correlated a large, spatially-correlated error. This error. was inThis line was with in line what with was what proposed was proposed by Emery by et Emery al. (2005) et al. [ 1]: the(2005) bigger [1]: the the dataset, bigger the the dataset, better, the even better, though even uncertainty though uncertainty errors exist errors in exist the samples. in the samples. The co-DSS The (Figureco-DSS6b), (Figure where 6b), the where uncertain the uncertain experimental experimental data were data considered were considered as a secondary as a secondary variable, variable, showed that,showed on average, that, on higheraverage, ARE higher values ARE were values located were mainlylocated atmainly theboundaries at the boundaries of the of main the mineralmain bodies.mineral Note bodies. that theseNote modelingthat these absolute modeling relative absolute errors relative are directly errors translatedare directly as biastranslated during as resource bias calculation.during resource When incorporatingcalculation. When the secondary incorporat dataing asthe local secondary probability data distributions, as local probability the relative errordistributions, decreased the for relative the entire error region, decreased inferring for the that entire the region, biasand inferring imprecision that the bias of the and secondary imprecision data wereof the controlled secondary by data the localwere probabilitycontrolled by distributions the local probability used in thedistributions stochastic used sequential in the simulationsstochastic (Figuresequential6c,d). simulations Also of interest, (Figure the 6c,d high). Also ARE of values interest, did the not high appear ARE atvalues theboundaries did not appear of the at mainthe mineralboundaries bodies. of the main mineral bodies.

(a) (b)

FigureFigure 6. 6.Map Map of of the the absolute absolute relative relative errorserrors forfor eacheach simulation strategy strategy used: used: (a ()a direct) direct sequential sequential simulationsimulation with with only only hard hard data, data, ( b(b)) directdirect sequentialsequential collocated co-simulation, co-simulation, (c ()c direct) direct sequential sequential simulationsimulation with with experimental experimental pointpoint distributions,distributions, and and ( (dd)) Direct Direct sequential sequential simulation simulation with with local local distributiondistribution functions. functions.

Minerals 2019, 9, 247 10 of 13

Minerals 2019, 9, x FOR PEER REVIEW 10 of 13 The information from Figure6 is summarized using the histogram and the main statistics of each The information from Figure 6 is summarized using the histogram and the main statistics of each map, as shown in Figure7. To quantify the bias of the models, we selected the median value of each map, as shown in Figure 7. To quantify the bias of the models, we selected the median value of each ARE map as it is a central tendency statistic, less sensitive to extreme values. The error median was 0.61 ARE map as it is a central tendency statistic, less sensitive to extreme values. The error median was for the direct sequential simulation with experimental data without uncertainty (Figure7a), 0.50 for 0.61 for the direct sequential simulation with experimental data without uncertainty (Figure 7a), 0.50 the direct sequential collocated co-simulation (Figure7b), 0.43 for the direct sequential simulation for the direct sequential collocated co-simulation (Figure 7b), 0.43 for the direct sequential simulation withwith experimental experimental point point distributions distributions (Figure(Figure7 7c),c), andand 0.400.40 for the direct sequential sequential simulation simulation with with parameterizedparameterized point point distributions distributions (Figure (Figure7d). 7d). As As expected, expected, the the standard standard deviation deviation and and the the mean mean were smallerwere smaller in the simulations in the simulations that used that the used local the probabilitylocal probability distribution distribution strategy strategy presented presented in this in this work thanwork in thethan other in the methodologies other methodologies investigated investigated in this studyin this for study comparison. for comparison. Thus, theThus, smallest the smallest bias was obtainedbias was for obtained the direct for sequential the direct se simulationquential simulation with parameterized with parameterized local distribution local distribution functions functions associated withassociated the uncertain with the secondary uncertain data. secondary data.

(a) (b)

FigureFigure 7. 7.Histogram Histogram of of the theabsolute absolute errorerror forfor eacheach simulation strategy strategy used: used: (a ()a direct) direct sequential sequential simulationsimulation with with only only hard hard data, data, ( b(b))direct direct sequentialsequential collocated co-simulation, co-simulation, (c ()c direct) direct sequential sequential simulationsimulation with withexperimental experimental pointpoint distributions,distributions, and and ( (dd) )direct direct sequential sequential simulation simulation with with parameterizedparameterized point point distributions. distributions.

QuantifyingQuantifying the the ARE ARE ofof eacheach modelingmodeling strategystrategy was was an an important important step step since since conventional conventional approachesapproaches to to classify classify natural natural resourcesresources andand mineralmineral deposits deposits in in the the mining mining industry industry tend tend to touse use subjectivesubjective criteria criteria in in order order to to deﬁne define thethe limitslimits of measured, indicated, indicated, and and inferred inferred resources resources [19]. [19 ]. TableTable2 shows 2 shows the the resource resource classiﬁcation classification criteria criteria establishedestablished using the calculated calculated ARE ARE for for this this case case study.study. In In this this way, way, we we intended intended to to quantify quantify the the impact impact of of each each modeling modeling technique technique onon thethe evaluationevaluation of theof existing the existing resource. resource.

Minerals 2019, 9, 247 11 of 13

MineralsTable 2019 2., 9,Proportions x FOR PEER REVIEW of the accurately classiﬁed mineral resources as a function of calculated11 of 13 relative error. Table 2. Proportions of the accurately classified mineral resources as a function of calculated relative error. Classiﬁcation of Resource Blocks with Accuracy Error less than

ClassificationMeasured of Resource Blocks with Accuracy 0–30% Error less than IndicatedMeasured 0–30% 31–50% InferredIndicated 31–50%>51% Inferred >51% Figure8 shows the proportions of accurately classified mineral resources using the di fferent simulationFigure sequential 8 shows the strategies proportions and theof accurately criteria shown classified in Table mineral2. Theresources blocks using classified the different by direct simulation sequential strategies and the criteria shown in Table 2. The blocks classified by direct sequential simulation using only hard data (Figure8a) revealed that this was the least accurate sequential simulation using only hard data (Figure 8a) revealed that this was the least accurate method, with 25% of blocks considered to be measured. Using direct sequential co-simulation, the method, with 25% of blocks considered to be measured. Using direct sequential co-simulation, the blocks that were accurately measured increased to 33% (Figure8b). The strategies based on local blocks that were accurately measured increased to 33% (Figure 8b). The strategies based on local probability distributions were more accurate. The resources measured using the direct sequential probability distributions were more accurate. The resources measured using the direct sequential simulationsimulation with with experimental experimental pointpoint distributions (Figure (Figure 8c)8c) increased increased to to 36% 36% accuracy, accuracy, and and with with the the directdirect sequential sequential simulation simulation withwith parameterizedparameterized po pointint distributions distributions (Figure (Figure 8d),8d), the the blocks blocks measured measured accuratelyaccurately increased increased to to 39%. 39%. This This increase increase occurred occurred because, because, when when the the estimates estimates were were done done with with lower errors,lower the errors, block the misclassification block misclassification was improved. was improved.

Classified Mineral Resources Classified Mineral Resources by DSS with only hard data by CODSS

Measured(0-30%) 25% Measured(0-30%) 50% 33% Indicated(31- 50%) Indicated(31- 50%) 59% 16% Inferred(<51%) 17% Inferred(<51%)

(a) (b)

Classified Mineral Resources Classified Mineral Resources by DSS Experimental point by DSS Parametric point distribution distribution

Measured(0-30%) Measured(0-30%) 42% 36% 39% 39% Indicated(31- 50%) Indicated(31- 50%)

22% Inferred(<51%) 22% Inferred(<51%)

FigureFigure 8. 8.Proportions Proportionsof of thethe classiﬁedclassified mineral resources resources as as a afunction function of of the the calculated calculated error error by byusing using didifferentﬀerent strategies: strategies: (a )( directa) direct sequential sequential simulation simulati withon with only hardonly data,hard (data,b) direct (b) sequentialdirect sequential collocated co-simulation,collocated co-simulation, (c) direct sequential (c) direct sequential simulation simulation with experimental with experimental point distributions, point distributions, and (d )and direct sequential(d) direct simulationsequential simulation with parameterized with parameterized point distributions. point distributions.

Minerals 2019, 9, 247 12 of 13

4. Conclusions and Recommendations We investigated the impact of integrating uncertain secondary data into the mineral resource modeling procedure using different DSS modeling strategies. In the example used here, the uncertain experimental data was contaminated by both bias and imprecision in order to mimic real mining conditions. The results showed that, independent of the quality of the data, more data leads to more reliable models. However, traditional geostatistical modeling approaches, where secondary data are integrated through sequential co-simulation, are not able to properly incorporate the uncertainty present in the data within the resulting models, as these are affected by the bias presented in the secondary data. While the influence of the secondary data can be managed, for example by modifying the correlation coefficient between primary and secondary experimental data (i.e., user-defined correlation coefficients), it has an impact on the reserve calculation. By using stochastic sequential simulation with local probability functions at the locations of the uncertain data, we were able to fully incorporate the uncertainty presented in the secondary data. The ability to integrate this uncertainty depends on the shape and size of the local probability distributions at the location of the secondary data samples. In this study, we considered two alternative options: using the uncertain data directly or building Gaussian distributions using the statistics retrieved from the uncertain data. The option selected should always be based on the type and quality of the existing data.

Author Contributions: Conceptualization, J.N. and L.A.; Resources, C.P.A.; Application tests, J.N.; Validation, C.P.A., and L.A.; Writing—original draft preparation, J.N.; Writing—review and editing, R.N., J.F.C. and A.S.; Supervision, L.A. Funding: This research received no external funding. Acknowledgments: The authors gratefully acknowledge the support of the Coordination for Higher Education Staff Development (CAPES), the Mineral Exploration and Mining Planning Research Unit (LPM) at the University Federal do Rio Grande do Sul and the CERENA (strategic project FCT-UID/ECI/04028/2019) at Instituto Superior Técnico. Conflicts of Interest: The authors declare no conflict of interest and that they have no financial and personal relationship of any kind with other people or organizations that could inappropriately influence your work and the position presented in, or the review of, the manuscript entitled.

References

1. Emery, X.; Bertini, J.P.; Ortiz, J.M. Resource and reserve evaluation in the presence of imprecise data. CIM Bull. 2005, 90, 366–377. 2. Babak, O.; Deutsch, C.V. Collocated cokriging based on merged secondary attributes. Math. Geosci. 2009, 41, 921–926. [CrossRef] 3. Ribeiro, D.; Monteiro, C.; Cunha, E.; Catarino, M.; Augusto, V.; Gomes, M. Correlation study between reverse circulation and diamond drilling in iron ore deposits. In Proceedings of the 5th World Conference on Sampling and Blending, Santiago, Chile, 25–28 October 2011; pp. 277–285. 4. Reuwsaat, J. Estudo de Alternativas de Estimativa Para Jazidas de Ferro Utilizando Dados Heterotópicos; Programa de Pós-Graduação em Engenharia de Minas, Metalúrgica e Materiais, Universidade Federal do Rio Grande do Sul: Porto Alegre, Brazil, 2011. (In Portuguese) 5. Minnitt, R.C.A.; Deutsch, C.V. Cokriging for optimal mineral resource estimates in mining operations. J. S. Afr. Inst. Min. Metal. 2014, 114, 189–203. 6. Cornah, A.; Machaka, E. Integration of imprecise and biased data into mineral resource estimates. J. S. Afr. Inst. Min. Metal. 2015, 115, 523–530. 7. Donovan, P.N. Resource Estimation with Multiple Data Types. Master Thesis, University of Alberta, Edmonton, AB, Canada, 2015. 8. Leuangthong, O.; Deutsch, C.V. Stepwise conditional transformation for simulation of multiple variables. Math. Geol. 2003, 35, 155–173. [CrossRef] Minerals 2019, 9, 247 13 of 13

9. Gómez-Hernández, J.J.; Journel, A.G. Joint Sequential Simulation of MultiGaussian Fields. In Quantitative Geology and Geostatistics, Geostatistics Tróia ’92, Tróia, Portugal, 1992; Soares, A., Ed.; Springer: Dordrecht, The Netherlands, 1993; Volume 5, pp. 85–94. 10. Journel, A.G. Modeling uncertainty: Some conceptual thoughts. In Quantitative Geology and Geostatistics, Geostatistics for Next Century; Dimitrakopoulos, R., Ed.; Springer: Dordrecht, The Netherlands, 1994; Volume 6. 11. Isaaks, E.H. The Application of Monte Carlo Methods to the Analysis of Spatially Correlated Data. Ph.D. Thesis, Stanford University, Stanford, CA, USA, 1990. 12. Soares, A. Direct sequential simulation and cosimulation. Math. Geol. 2001, 33, 911–926. [CrossRef] 13. Verly, G.W. Sequential Gaussian Cosimulation: A simulation method integrating several types of information. In Quantitative Geology and Geostatistics, Geostatistics Tróia ’92, Tróia, Portugal, 1992; Soares, A., Ed.; Springer: Dordrecht, The Netherlands, 1993; Volume 5, pp. 543–554. 14. Almeida, A.S.; Journel, A.G. Joint simulation of multiple variables with a Markov-type coregionalization model. Math. Geol. 1994, 26, 565–588. [CrossRef] 15. Soares, A.; Nunes, R.; Azevedo, L. Integration of Uncertain Data in Geostatistical Modeling. Math. Geosci. 2017, 49, 253–273. [CrossRef] 16. Horta, A.; Soares, A. Direct sequential co-simulation with joint probability distributions. Math. Geosci. 2010, 42, 269–292. [CrossRef] 17. Isaaks, E.H.; Srivastava, R.M. An Introduction to Applied Geostatistics; Oxford University Press: New York, NY, USA, 1989; p. 561. 18. Araújo, C.; Costa, J. Integration of diﬀerent-quality data in short-term mining planning. Rem: Revista Escola de Minas. 2015, 68, 221–227. [CrossRef] 19. Araújo, C.; Costa, J.; Koppe, V.; Soares, A. Reducing error in short term grade prediction including imprecise and inaccurate data. In Proceedings of the 24th World Mining Congress, Rio de Janeiro, Brazil, 18–21 October 2016; pp. 80–89.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).