sign in sign up
<<
Home , Grouped data, Mode (statistics)

Statistics, Measures of Central TendencyI

We are considering a X with a which has some parameters. We want to get an idea what these parameters are. We perfom an n times and record the outcome. This we have X1,..., Xn i.i.d. random variables, with probability distribution same as X . We want to use the outcome to infer what the parameters are.

Mean The outcomes are x1,..., xn. The Sample is x1+···+xn x := n . Also sometimes called the . The expected value of X , EX , is also called the mean of X . Often denoted by µ. Sometimes called population mean. The so that half the values are below, half above. If the sample is of even size, you take the average of the middle terms. The number that occurs most frequently. There could be several modes, or no mode.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 1 / 24 Statistics, Measures of Central TendencyII

Example You have a coin for which you know that P(H) = p and P(T ) = 1 − p. You would like to estimate p. You toss it n times. You count the number of heads. The sample mean should be an estimate of p.

EX = p, and E(X1 + ··· + Xn) = np. So

X + ··· + X  E 1 n = p. n

Dan Barbasch Math 1105 Chapter 9 Week of September 25 2 / 24 Descriptive StatisticsI

Frequency Distribution Divide into a number of equal disjoint intervals. For each count the number of elements in the sample occuring. see the next slide Grouped Mean Essentially calculate the mean of the distribution. Intervals are used, rather than single values. It is assumed that all these values are located at the midpoint of the interval. The letter xM is used to represent the midpoints and f represents the frequencies: P fi xM,i n Frequency Polygon Connect the middles of the tops of each interval.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 3 / 24 Histogram A histogram is a graphical representation of the distribution of numerical data. It is a kind of bar graph. To construct a histogram, the first step is to ”bin” the of values, that is, divide the entire range of values into a series of intervals, and then count how many values fall into each interval. The bins are usually specified as consecutive, non-overlapping intervals of a variable. The bins (intervals) must be adjacent, and are often (but are not required to be) of equal size.

Bin Count −3.5 − 2.51 9 −2.5 − 1.51 32 −1.5 − 0.51 109 −0.5 − 0.49 180 0.5 − 1.49 132 1.5 − 2.49 34 2.5 − 3.49 4 (−3)·9+(−2)·32+(−1)·109+·(0)180+1·132+2·34+3·4 Mean: 500 Dan Barbasch Math 1105 Chapter 9 Week of September 25 4 / 24 Example The table on the next page gives the number of days in June and July of recent years in which the temperature reached 90 degrees or higher in New Yorks Central Park. Source: The New York Times and Accuweather.com. a. Prepare a with a column for intervals and frequencies. Use seven intervals, starting with [0 4]. b. Sketch a histogram and a frequency polygon, using the intervals in part a. c. Find the mean for the original data. d. Find the mean using the from part a. e. Explain why your answers to parts c and d are different. f. Find the median and the mode for the original data.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 5 / 24 9.1 Frequency Distributions; Measures of 417 a. Use this table to estimate the mean income for white house- Animal Number of Blood Types holds in 2008. Pig 16 b. Compare this estimate with the estimate found in Exercise 39. Discuss whether this provides evidence that white Amer- Cow 12 ican households have higher earnings than African Ameri- Chicken 11 can households. Horse 9 41. Airlines The number of consumer complaints against the top Human 8 U.S. airlines during the first six months of 2010 is given in the Sheep 7 following table. Source: U.S. Department of Transportation. Dog 7 Rhesus monkey 6 Complaints per 100,000 Mink 5 Airline Complaints Passengers Boarding Rabbit 5 Delta 1175 2.19 Mouse 4 American 660 1.56 Rat 4 United 487 1.84 Cat 2 US Airways 428 1.69 Continental 350 1.64 Southwest 149 0.29 General Interest Skywest 77 0.65 44. Temperature The following table gives the number of days in American Eagle 68 0.87 June and July of recent years in which the temperature reached Expressjet 56 0.70 90 degrees or higher in New York’s Central Park. Source: The Alaska 34 0.44 New York Times and Accuweather.com. Temperature Data a. By considering the in the column labeled “Com- plaints,” calculate the mean and median number of com- plaints per airline. Year Days Year Days Year Days b. Explain why the found in part a are not meaningful. 1972 11 1985 4 1998 5 c. Find the mean and median of the numbers in the column 1973 8 1986 8 1999 24 labeled “Complaints per 100,000 Passengers Boarding.” 1974 11 1987 14 2000 3 Discuss whether these averages are meaningful. 1975 3 1988 21 2001 4

Life Sciences 1976 8 1989 10 2002 13 1977 11 1990 6 2003 11 42. Pandas The size of the home ranges (in square kilometers) of several pandas were surveyed over a year’s time, with the fol- 1978 5 1991 21 2004 1 lowing results. 1979 7 1992 4 2005 12 1980 12 1993 25 2006 5 Home Range Frequency 1981 12 1994 16 2007 4 0.1–0.5 11 1982 11 1995 14 2008 10 0.6–1.0 12 1983 20 1996 0 2009 0 1.1–1.5 7 1984 7 1997 10 2010 20 1.6–2.0 6 Dan Barbasch Math 1105 Chapter 9 Week of September 25 6 / 24 2.1–2.5 2 2.6–3.0 1 a. Prepare a frequency distribution with a column for intervals 3.1–3.5 1 and frequencies. Use six intervals, starting with 0–4. b. Sketch a histogram and a frequency polygon, using the a. Sketch a histogram and frequency polygon for the data. intervals in part a. b. Find the mean for the data. c. Find the mean for the original data. 43. Blood Types The number of recognized blood types varies by d. Find the mean using the grouped data from part a. species, as indicated by the table below. Find the mean, median, and mode of this data. Source: The Handy Science e. Explain why your answers to parts c and d are different. Answer Book. f. Find the median and the mode for the original data. Measures of Variation Summary of Section 9.2 Range The difference Largest Data - Smallest Data in a Sample. Deviation from the Mean P x2−nx2 P(x −x)2 1 σ2 = s2 = i = i n−1 √ n−1 2 σ = s = s2 These are random variables called Sample Variance and Sample Standard Deviation. For a random variable X , µ = E(X ) is called the mean. The variance Var(X ) is σ2 = Var(X ) = E((X − µ)2). Main Property/ Explanation for dividing by n − 1: If Xi are P 2 2 (Xi −X ) i.i.d with distribution X , then if you set S = n−1 , its expected value is E(S2) = σ2. This is not true for the standard deviation, E(S) 6= σ. s P 2 2 fi x − nx Grouped Data s = M,i . n − 1

Dan Barbasch Math 1105 Chapter 9 Week of September 25 7 / 24 ExamplesI

Example (Range) Data 15, −3, 4, 7, 18. The smallest is −3, the largest 18 so Range = 18 − (−3) = 21. Always a nonnegative number.

Example (Deviation from the Mean) 15−3+4+7+18 In the previous example, x = 5 = 8.2. So

15 − 8.2 = 6.8, −3 − 8.2 = −11.2, 4 − 8.2 = −3.8, 7 − 8.2 = −1.2, 18 − 8.2 = 9.8.

Example (Variance and Standard Deviation)

2 2 2 2 2 2 2 2 2 2 2 s2 = 6.8 +11.2 +3.8 +1.2 +9.8 = 15 +3 +4 +7 +18 −5·8.2 √ 4 4 s = s2.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 8 / 24 ExamplesII Example () P(X = 1) = p, P(X = 0) = 1 − p. Then µ = E(X ) = p, and σ2 = E((X − p)2) = (1 − p)2p + (0 − p)2(1 − p) = p(1 − p). This is the same as E(X 2 − p2) = (1 − p2)p + (−p2)(1 − p) = (1 − p)p.

Remark: Note that the formula for variance and standard deviation only holds for n > 2. Otherwise, for n = 1, you would be dividing by 0. For one random variable, the variance is defined as Var(X ) = E((X − E(X ))2). For X1, X2,, two independent random variables, Var(X1 + X2) = Var(X1) + Var(X2). Suppose X is a random variable. We can write a table

X a1 a2 ... an

P(X ) p1 p2 ... pn

Dan Barbasch Math 1105 Chapter 9 Week of September 25 9 / 24 ExamplesIII

For the expected value µ = E(X ), you multiply the two terms in each column, and add X ai × pn = a1p1 + ··· + anpn. i

In a spreadsheet program, the data would be in columns and you would add over the products from the rows. You use a command like sumproduct to perform the operation. If you have some other variable like (X − µ)2, 2 you would use the values (ai − µ) and the same pi .

Dan Barbasch Math 1105 Chapter 9 Week of September 25 10 / 24 ExamplesIV Example

X 2 3 −1 1

X 2 4 9 1 1

(X − µ)2 (2 − 1/4)2 (3 − 1/4)2 (−1 − 1/4)2 (1 − 1/4)2

P(X ) 1/2 1/8 1/4 1/8

Computing the expected values is below.

µ = E(X ) = (2) × (1/2) + (3) × (1/8) + (−1) × (1/4) + (1) × (1/8) = 1/4.

Var(X ) =(2 − 1/4)2 · (1/2) + (3 − 1/4)2 · (1/8) + (−1 − 1/4)2 · (1/4)+ +(1 − 1/4)2 · (1/8) = 47/16.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 11 / 24 Grouped Data

Example (Grouped Data)

Interval Frequency Midpoint xM 30-39 1 34.5 40-49 6 44.5 50-59 13 54.5 60-69 22 64.5 70-79 17 74.5 80-89 13 84.5 90-99 8 94.5 Find the standard deviation of these grouped data.

2 In this case you must sum the xM multiplied by the frequencies, and subtract 80 × x where x is for the full sample, (which is not in the table, you must get it from the full data).

Dan Barbasch Math 1105 Chapter 9 Week of September 25 12 / 24 Chebyshev’s and Markov’s InequalityI

E(X ) P(X ≥ a) ≤ Markov. a 1 P (|X − µ| ≥ kσ) ≤ Chebyshev. k2 In words, the probability that X is more than k standard deviations away from the mean is less than 1/k2.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 13 / 24 Chebyshev’s and Markov’s InequalityII

Example (from the practice prelim) 8. (14 points) Assume that the height in inches of American women follows a with mean Mu = 6400 (5’4”) and standard deviation σ = 300. (a) (3 points) How many standard deviations above or below the mean is a height of 72” (6’0”)? (b) (4 points) What fraction of women are taller than 6 feet? (c) (4 points) In a room with 30 women, what is the probability that at least one of them is taller than 6 feet? (d) (3 points) What assumptions did you make when answering part (c)? Are there circumstances under which those assumptions would not be justified?

Dan Barbasch Math 1105 Chapter 9 Week of September 25 14 / 24 Chebyshev’s and Markov’s InequalityIII

Answer. Say we don’t know what distribution it is. We can still use Markov’s and Chebyshev’s inequality. 72−64 8 (a) 3 = 3 , so closer to 3. Use 73 to get 3. 64 (b) Markov’s inequality says P(X ≥ 73) ≤ 73 . To use Chebyshev’s inequality we must write |X − 64| ≥ 3σ. Then

1 P(|X − 64| ≥ 3σ) ≤ . 9 In other words, k = 3. This includes not just X ≥ 73, but also X ≤ 55. Still we can say the probability is less than 1/9, because |X − 64| ≥ 9 is larger than X − 64 ≥ 9. (c) 1 − P( none are taller than 6 ) = 1 − (1 − P( one is not taller than 6 ))30.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 15 / 24 Continuous Random VariablesI

In many applications it useful to assume that the random varuiable X may take any real value. The probability distribution for the case of finitely many values does not work. We assume that the sampe space is S = R all . The typical event (subset of S) is restricted to sets of the form A = {x | x ≤ a} and complements and intersections of such sets. For us they will be at most sets of the form A = {a ≤ x ≤ b}. The probability distribution is given as numbers P(X ≤ a); in other words a function which takes nonnegative values only, and we allow a = −∞ in which case the value is 0, and ∞, in which case the number is 1. For a continuous reandom variable, P(X = a) = 0 always, but the situation is not trivial.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 16 / 24 Continuous Random VariablesII

Example (the uniform distribution of the interval (0, 1)) Let  0 if a ≤ 0  f (x) = 1 if a < 0 < 1 0 if 1 ≤ a Define P(X ≤ a) = area between the x−axis and f (x), and before the vertical line x = a. See the picture in class, or in the text for the normal distribution. So  0 if a ≤ 0  P(X ≤ a) = a if 0 < a < 1 1 if 1 ≤ a

Dan Barbasch Math 1105 Chapter 9 Week of September 25 17 / 24 Continuous Random VariablesIII

Exercise Do the same for  0 if x ≤ 0  f (x) = 2x if 0 ≤ x ≤ 1 0 if 1 < x.

You need the formula for the area of the triangle, Area = (base) × (height)/2.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 18 / 24 Normal DistributionI

Definition Data are said to be normally distributed if the rate at which the frequencies fall off is proportional to the distance of the score from the mean, and to the frequencies themselves.

This definition requires Calculus. We don’t assume or do Calculus in this course. We will however learn how to work with this distribution. It is very useful in that many phenomena can be modeled by this. We will see how the binomial distribution is related to the normal distribution later in the chapter. Suppose you have a random variable X , and you would like to know about its mean µ. So you perform many n independent trials, and draw a histogram. The larger the n, the closer the outcome will look like the (x−µ)2 1 − curve f (x) = √ e 2σ2 . The pictures in the text show what it looks 2πσ like. The resulting probability is called N(µ, σ2), normal with mean µ and

Dan Barbasch Math 1105 Chapter 9 Week of September 25 19 / 24 Normal DistributionII

variance σ2. There is a precise statement called the Central Limit √ Theorem which says that for large n, n(Sn − µ) “looks” like a normal distribution N(0, σ2). it is used in practice to model large populations and “ errors”. There are many examples that can be approximated by normal distributions. Heights of people, and scores on tests are examples. This is not a finite distribution. For a random variable that is normally distributed, we write N(µ, σ2),

P(X ≤ a) = the area under the normal curve from − ∞ to a.

This is tabulated for µ = 0 and σ = 1. The rest is computed by simple formulas involving arithmetic.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 20 / 24 Height ExampleI

Example (from the practice prelim) 8. (14 points) Assume that the height in inches of American women follows a normal distribution with mean mu = 6400 (5’4”) and standard deviation σ = 300. (a) (3 points) How many standard deviations above or below the mean is a height of 73” (6’1”)? (b) (4 points) What fraction of women are taller than 73 inches? (c) (4 points) In a room with 30 women, what is the probability that at least one of them is taller than 73”? (d) (3 points) What assumptions did you make when answering part (c)? Are there circumstances under which those assumptions would not be justified?

Dan Barbasch Math 1105 Chapter 9 Week of September 25 21 / 24 Height ExampleII Answer. (a) same as before 3 standard deviations away.

X − µ (b) P(X ≥ 73) = P(X − 64 ≥ 73 − 64 = 9 = 3σ) = P( ≥ 3) = σ X − µ =1 − P( ≤ 3) =1 − 0.999 = 0.001. σ This is 1/1000. The random variable X has probability distribution N(64, 17). The probability P(X ≥ 73) comes from this normal distribution. To actually X −64 look it up in the tables, you rewrite it in terms of Z = 3 which has probability distribution N(0, 1). This is the one in the tables.

(c) P(at least 1/30 ≥ 73) =1 − P(30/30 ≤ 73) = 1 − P(X ≤ 73)30 = =1 − (0.998)30.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 22 / 24 z−value

The principle is 2 X −µ X normal N(µ, σ ) ⇐⇒ Z = σ normal N(0, 1). So a − µ P(X ≤ a) = P(Z ≤ ). σ a−µ z = σ is called the z−value. This is what you look up in the tables.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 23 / 24 Example with Grades

Example A professor (not this one!) of a course wants to give grades so that A top 8% F bottom 8% B next 20% below A D next 20% above the F C the rest The mean is µ = 67 and the standard deviation is σ = 17. Find the cutoffs.

Answer. P(≤ A) = 0.92 z = 1.41 a = µ + zσ = 67 + 17 · 1.41 = 91 P(≤ B) = 0.72 z = 0.58 a = µ + zσ = 67 + 17 · 0.58 = 77 P(≤ C) = 0.28 z = −.59 a = µ + zσ = 67 + 17 · (−.59) = 57 P(≤ D) = 0.08 z = −1.39 a = µ + zσ = 67 + 17 · (−1.39) = 43 from the tables. In Excel or alike you can write norminv(0.92, 67, 17) =∼ 91.

Dan Barbasch Math 1105 Chapter 9 Week of September 25 24 / 24