ECE3340 Introduction to the limit of computer numerical capability – precision and accuracy concepts PROF. HAN Q. LE Computing errors …~ 99 – 99.9% of computing numerical errors are likely due to user’s coding errors (syntax, algorithm)… and the tiny remaining portion may be due to user’s lack of understanding how numerical computation works… Hope this course will help you on this problems Homework/Classwork 1 CHECK OUT THE PERFORMANCE OF YOUR COMPUTER CW Click open this APP Click to ask your machine the smallest number it can handle (this example shows 64-bit x86 architecture) Log base 2 of the number Do the same for the largest HW number Click to ask the computer the smallest relative difference between two number that it can tell apart. This is called machine epsilon If you use an x86 64-bit CPU, the above is what you get. HW-Problem 1 Use any software (C++, C#, MATLAB, Excel,…) but not Mathematica (because it is too smart and can handle the test below) to do this: 1. Let’s denote xmax be the largest number your computer can handle in the APP test. Let x be a number just below xmax , such as ~0.75 xmax. (For example, I choose x=1.5*10^308). Double it (2*x) and print the result. 2. Let xmin be your computer smallest number. Choose x just above it. Then find 0.5*x and print output. Here is an example what happens in Excel: Excel error: instead of giving the correct answer 3*10^308, Number just slightly less than it gives error output. This is my computer xmax caused by Enter a number Column A*2. Column A/2. overflow 4 8 2 5 10 2.5 Note the three key 1.50E+308 #NUM! 7.50E+307 concepts 6.00E-308 1.20E-307 3.00E-308 highlighted in 5.00E-308 1.00E-307 2.50E-308 4.00E-308 8.00E-308 0.00E+00 yellow Excel fails: instead of giving me the correct answer: Number just slightly more than 2*10^-308, it “ ” by giving zero my computer xmin flushes to zero output. This is caused by underflow : a result that is too small for the computer to handle, it sets as zero. You should get similar results in other software and language. HW-Problem 2 Use any software (C++, C#, MATLAB, Excel,…) but not Mathematica (because it is too smart and can handle the test below) to do this. Let denote eps (for ε) be the smallest relative difference that your computer can handle – aka machine epsilon (it is 2.22*10^-16 on my machine, for example). Then, generate an array of 5-20 elements (your choice), with values ranging from above your computer eps to below eps. Denote this array as x array. Then, add 1 to x array, denote it as y: y=1+x; Then, define x0 array: x0=y-1; Print out all 3 arrays: x, y, and x0 and compare. Is your x0 the same as x? Here is an example what happens in Excel: ε array y=1+ ε ε0=y-1 array array small 1 column A+B Column C-A number 1 5.000E-15 1.00000000000001000000E+00 5.10702591327572000000E-15 1 4.000E-15 1.00000000000000000000E+00 3.99680288865056000000E-15 1 3.000E-15 1.00000000000000000000E+00 3.10862446895044000000E-15 1 2.000E-15 1.00000000000000000000E+00 1.99840144432528000000E-15 1 1.900E-15 1.00000000000000000000E+00 1.99840144432528000000E-15 1 1.800E-15 1.00000000000000000000E+00 1.77635683940025000000E-15 1 1.700E-15 1.00000000000000000000E+00 1.77635683940025000000E-15 1 1.600E-15 1.00000000000000000000E+00 0.00000000000000000000E+00 1 1.500E-15 1.00000000000000000000E+00 0.00000000000000000000E+00 Notice that ε and ε0 are not what happens right here? why ε0 equal as expected. cannot gradually go from 1.776*10^-15 to ~ 1.6*10^-15 as expected, but jumps What we see here is the limit of to zero? machine precision which causes the inaccuracy observed. imagine this: there is an investment fund containing gazillion dollars. Asset= 1 gazillion You add to the account your saving of $1 mil, because it promises 50% profit return in 1 year but the investment company uses Excel on 16-bit CPU and your portion is below its precision, hence, it flushes to zero Hence, after one year, your account is deposited with $ this amount At least, it has a lot of zeros! All is not lost, the company kindly sent you a card… anyway! A different kind of precision problem: Sometimes, digits are lost on the other side (overflow) Problem 3 You don’t have to do any calculation, only discuss what you think here: 1. Describe what you observe from doing problems 1 and 2 above and what is your thought on the results? (in other words, try your best to explain what’s wrong). 2. What limits a computer to have a maximum and minimum magnitude for numbers? (this is limited by machine processor and not by software like Mathematica). 3. If a computer can handle a number as small as 3*10^-308, why can’t it handle a difference between 1 and 1+eps when eps is 1*10^-16, which is >> than the smallest number 10^-308 that it can handle? this problem is aka “how your million dollars flushed to zero” $ Click open this tab of Use slider to input a another APP. number Click to choose how many digits you want to see (decimal level of precision) This is the actual number value inside the computer, not what you think you enter. This is the error between what you want to input (for example. 0.3 here), and what it really is (0.2999 whatever…). Why can’t it just record something as simple as 0.3? Problem 4 Use the APP above to enter 3-5 non-zero numbers (the example is 0.3, but you should enter a set of unique numbers for yourself, e. g. 0.155, 0.291, 0.43, 0.68, 0.912 don’t copy this, please – pick your own unique values). Record what the actual numbers are, and the errors. 1. Make a table (like in Excel) a column of numbers you enter, a column of their actual values, and a column of ratio abs(error)/number – this is called relative error (for example, ratio abs(3.7*10^-17)/0.3 = 1.23*10^-16) 2. Enter a few values that are multiple of 1/2^n, where n=1, 2, 3. For example 0.125, 0.375, etc. Do you see any errors? 3. Write why you think there are errors? Why the magnitude of relative errors is ~ the machine epsilon you find in APP 1. Why do you see errors in question 1, but the errors in question 2 are zero (which means the numbers inside the computer are exact). Click this to see an error plot. The input varies from 1+0*10^-16 to 1+10^10-16 = 1+1*10^-15 Instead of giving us a linear straight line like the blue line, the computer actual values are quantized into discrete step values (red curve). Why? The computer can’t tell the difference and hence, it returns to you zero. you deposited $1 mil machine precision problem Problem 5 Calculate or estimate the quantized step size from the APP for your computer (not this demonstrated computer – although very likely you have exactly the same result if you have x86 64-bit CPU). Then, approximate your step size as and obtain the integer 1 value n, (you can take log2 of the step size to get -n). This is the number of significant bits of your 2machine floating point mantissa. From here on, you can use Mathematica. Example of the code is given here. We have 52- bit mantissa wait a minute, I thought the machine is 64- bit. Why the mantissa has only 52 bits? what happens to the other 12 missing bits? someone takes it? Let’s take a break here and ponder why? The reason will be discussed in details in the next lecture. Meanwhile, the next 2 slides give a preview of “why” Slide from next lecture If the input is a FP number, it will show: - 1 sign bit: 0 for >= 0 and 1 for <=0 - 11 exponent bits - 52 bits for the mantissa (which actually has 53 bits. The first bit is always 1 and needs not be stored here). The 8 bytes can be total is a 64-bit word – arranged in 8x8 dot shown here as 8 bytes matrix display Slide from next lecture Problem 3 (bonus) Find floating points or integers (easier) that have bits patterns representing the initials of your first and last name. Example: 16,419,452,012,919,037,084 and 4,123,389,611,252,201,785 will give something. Check it out. Below are illustrations of common patterns: each has a number. Find your “digital signature” with your name initials. Back to our current lecture… Time to look at this again. We see 52 bits here for the mantissa. 10 bits for negative exponent. 10 bits for positive exponent. That means 11 bits for both. We have 1 bit left. It is for the sign Mantissa and Exponent SCIENTIFIC REPRESENTATION OF NUMBERS exponent mantissa save the hassle of all these zeros electron mass and sun mass are 60 orders of magnitude different. But relatively speaking, which mass do we know “better” or more precise? which tool would we want to use to measure this diamond? The caliper has higher precision, as it can give us a finer, or more-digit reading of the size: precision means the ability to give high resolution, more significant digit reading.