Chapter 6 DATA PROCESSING Introduction This chapter gives detailed information on the data processing functions and their application. If you have not already done so you should follow through the tutorial of Chapter 3 in detail.. Chapter 3’s tutorial explains important concepts behind the Geoplot environment and provides a hands-on introduction to data processing and how this processing interacts with other Geoplot facilities, especially graphics and the statistics reports. This chapter starts by summarising the process functions available and their purpose. This is followed by additional information on using the process functions not covered in the tutorial. A reference section gives a detailed description of each of the process functions and includes conventions used and examples of use. Detailed guidance is then given on how to process resistance, gradiometer, magnetometer and other survey types. This is supported by “Quick-Start” cards which guide you through the processing sequences appropriate for each data type. Examples of how to process data are give in the following case studies section. The final processing techniques section presents useful and novel ways of extending, combining and utilising the flexibility of the process functions. Examples include merging dual FM256 gradiometer data, dealing with difficult periodic errors, generating and overlaying contour lines and providing contrast equalisation for graphics plots. Summary of the Process Functions The Process Menu and Toolbar provides a comprehensive range of functions for manipulation of all data types, together with specific routines to correct for data collection artefacts such as edge matching and drift correction. Some functions are designed specifically for Geoscan Research instrumentation but all may be equally applied to other instrumentation data sets. Mathematically, any real bipolar or monopolar two dimensional data array may be processed. The process functions are listed below, in alphabetical order, along with a brief introduction to their application. Absolute The Absolute function converts all bipolar data to absolute, positive data. It is a general purpose numeric tool with a variety of applications. For example it can be useful in the generation of magnetic-resistance correlation plots. It can operate over the whole of the data set, or any inclusive or exclusive block. Add The Add function adds a positive or negative constant value to the data. It is a general purpose numeric tool with a wide variety of applications, such as editing a single data point, shifting traverse levels or peforming the equivalent of a mesh bias adjustement. It can operate over the whole of the data set, or any inclusive or exclusive block. Clip The Clip function can be used to clip, or limit, data to specified maximum and minimum values. This can improve graphical presentation and also forms a useful pre-process procedure for many other functions. It can operate over the whole of the data set, or any inclusive or exclusive block. 6-2 Data Processing Compress The Compress function applies either Arctangent or Logarithmic weighting to the data. It is used to compress data to fit within the dynamic range of display devices or printers, allowing both large and small magnitude features to be visible at the same time. It can operate over the whole of the data set, or any inclusive or exclusive block. Cut and Combine The Cut and Combine function provides Cut and Paste, Add, Subtract and Multiply operations between two data sets (grids and composites). This can be applied between any block of one composite or grid and any position in the other (or same) composite. Applications include merging composites, splitting a composite in two, generation of correlation plots (multiply) between data sets etc. A powerful application is to examine the effect of a process function (by subtracting the original data set), thereby ensuring that the process function has been applied with the correct parameters. Deslope The Deslope function is used to remove a linear trend within a data set. It is typically used to correct for drift in gradiometer data where the use of the Zero Mean Traverse function is inappropriate. It operates on individual grid positions within a composite data set, in either the X or Y direction. Despike The Despike function can be used to automatically (a) locate and remove random, spurious readings often present in resistance data and (b) locate and remove random "iron spikes" often present in gradiometer and magnetometer data. It operates over the whole of the data set. Destagger The Destagger function is used to correct for displacement of anomalies caused by alternate zig-zag traverses. These displacements are often observable in gradiometer data (collected with zig-zag traverses) if sample interval is less than 1m. Destagger can operate on alternate lines or pairs of lines, in the X direction only. It can operate on all the grids, individual grids or individual lines within a grid. Edge Match The Edge Match function may be used to remove grid edge discontinuities. These are often present in Twin electrode resistance surveys as a result of improper placement of the remote probes. It operates on individual grid positions within the data set. High Pass Filter The High Pass Filter function may be used to remove low frequency, large scale spatial detail. A typical use is the removal of a slowly changing geological "background" response commonly found in resistance surveys. It can operate over the whole of the data set or any inclusive or exclusive block. Interpolate The Interpolate function may be used to increase or decrease the number of data points in a survey. Increasing the number of data points can be used to create a smoother appearance to the data. Interpolate can also be used to make the sample and traverse intervals of differently sampled composites match; this is essential, for example, prior to combining them into one composite or prior to generating a correlation plot. Decreasing the number of data points using Interpolate (line deletion) can be used to investigate the effect of different sampling strategies. Interpolate operates over the whole of the data set. Low Pass Filter The Low Pass Filter function may be used to remove high frequency, small scale spatial detail. It is useful for smoothing data or for enhancing larger weak features. It can operate over the whole of the data set or any inclusive or exclusive block. Median Filter The Median Filter may be used to remove random spurious readings present in survey data and smoothes the data at the same time. It is most useful for high sample density data. It can operate over the whole of the data set or any inclusive or exclusive block. Data Processing 6-3 Multiply The Multiply function multiplies data by a positive or negative constant value. It is a general purpose numeric tool with a wide variety of applications, including normalisation of data (eg contrast adjustment of resistance data prior to edge matching). It can operate over the whole of the data set or any inclusive or exclusive block. Periodic Defect Filter The Periodic Filter function may be used to remove or reduce the amplitude of regular, periodic features. These may be present in the soil (eg plough marks, ridge and furrow) or may arise as operator induced defects during gradiometer data collection – the function is especially designed with this latter application in mind. It can operate over the whole of the data set or any specified grid. Power The Power function raises data to a positive or negative constant power. It is a general purpose tool with a wide variety of applications including conversion of resistivity to conductivity for comparison with EM surveys, and compression or expansion of data to change the viwing contrast. It can operate over the whole of the data set or any inclusive or exclusive block. Randomise The Randomise function replaces the data with random numbers. It may be used to introduce a controlled amount of noise so that surveys performed at different times or with different instruments visually match. It may also be used for testing other functions, especially in the presence of noise. It operates over the whole of the data set. Search and Replace This Search and Replace function looks for numbers in a specified band and replaces them by another specified number. It is a general purpose numeric tool with a wide variety of applications when used in conjunction with other process functions. For example, (when used with clip) regions which are strongly perturbed by nearby iron fences, pipelines etc can be converted to dummy regions, allowing other statistical functions to perform correctly. It can operate over the whole of the data set, or any inclusive or exclusive block. Spectrum The Spectrum function allows you to analyse the frequency spectrum of the data, splitting it into Amplitude, Phase, Real or Imaginary components. The Amplitude spectrum can be used to identify periodic defects in gradiometer data which can then be removed with the Periodic Filter function. It can operate over the whole of the data set or any inclusive block (exclusive blocks are not allowed in Spectrum). Standard Deviation or Variance Map The Standard Deviation / Variance Map function replaces the data set by either the local standard deviation or the local variance. A graphics plot of this new data set indicates areas of statistically different activity. It only operates over the whole of the data set. Statistics The Statistics function provides a statistical analysis of any block of data within the data set : localised mean, standard deviation, minimum, maximum and a localised histogram (this is in addition to the floating statistics report for the whole of the data set). Statistics can often be used to determine appropriate parameters for other process functions.