DOCSLIB.ORG

Introduction to Machine Learning Lecture 16

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Introduction to Machine Learning Lecture 16 Introduction to Machine Learning Lecture 16 Mehryar Mohri Courant Institute and Google Research [email protected] Ranking Mehryar Mohri - Introduction to Machine Learning page 2 Motivation Very large data sets: • too large to display or process. • limited resources, need priorities. • ranking more desirable than classification. Applications: • search engines, information extraction. • decision making, auctions, fraud detection. Can we learn to predict ranking accurately? Mehryar Mohri - Foundations of Machine Learning page 3 Score-Based Setting Single stage: learning algorithm • receives labeled sample of pairwise preferences; returns scoring function h : U R . • → Drawbacks: • h induces a linear ordering for full set U . • does not match a query-based scenario. Advantages: • efficient algorithms. • good theory: VC bounds, margin bounds, stability bounds (FISS 03, RCMS 05, AN 05, AGHHR 05, CMR 07). Mehryar Mohri - Foundations of Machine Learning page 4 Preference-Based Setting Definitions: • U : universe, full set of objects. V : finite query subset to rank, V U . • ⊆ • τ ∗: target ranking for V (random variable). Two stages: can be viewed as reduction. learn preference function h : U U [0 , 1] . • × → • given V , use h to determine ranking σ of V . Running-time: measured in terms of |calls to h |. Mehryar Mohri - Foundations of Machine Learning page 5 Related Problem Rank aggregation: given n candidates and k voters each giving a ranking of the candidates, find ordering as close as possible to these. • closeness measured in number of pairwise misrankings. • problem NP-hard even for k =4 (Dwork et al., 2001). Mehryar Mohri - Foundations of Machine Learning page 6 This Talk Score-based ranking Preference-based ranking Mehryar Mohri - Foundations of Machine Learning page 7 Score-Based Ranking Training data: sample of i.i.d. labeled pairs drawn from U U according to some distribution D , × S = (x ,x ,y ),...,(x ,x ,y ) U U 1, 0, +1 , 1 1 1 m m m ∈ × ×{− } +1 if xi >pref xi with yi = 0ifxi =pref xi or no information 1ifx <pref xi. − i Problem : find hypothesis h : U R in H with small generalization error → RD(h)= Pr f(x, x) h(x) h(x) < 0 . (x,x ) D − ∼ Mehryar Mohri - Foundations of Machine Learning page 8 Notes Empirical error: m 1 R(h)= 1y (h(x ) h(x ))<0 . m i i − i i=1 The relation x x f ( x, x )=1 may be non- R ⇔ transitive (needs not even be anti-symmetric). Problem different from classification. Mehryar Mohri - Foundations of Machine Learning page 9 Distributional Assumptions Distribution over points: m points (literature). • labels for pairs. • squared number of examples O ( m 2 ) . Distribution over pairs: m pairs. • label for each pair received. • independence assumption. • same (linear) number of examples. Mehryar Mohri - Foundations of Machine Learning page 10 Boosting for Ranking Use weak ranking algorithm and create stronger ranking algorithm. Ensemble method: combine base rankers returned by weak ranking algorithm. Finding simple relatively accurate base rankers often not hard. How should base rankers be combined? Mehryar Mohri - Foundations of Machine Learning page 11 CD RankBoost (Freund et al., 2003; Rudin et al., 2005) X 0 + s H 0, 1 . t + t + t− =1,t (h)= Pr sgn f(x, x)(h(x) h(x)) = s . (x,x ) Dt − ⊆{ } ∼ RankBoost(S =((x1,x1 ,y1) ...,(xm,xm ,ym))) 1 for i 1 to m do ← 1 2 D1(xi,xi ) m 3 for t 1 to T ←do ← + 4 ht base ranker in H with smallest t− t = Ei Dt yi ht(xi ) ht(xi) ← − − ∼ − + 1 t 5 αt 2 log ← t− 0 + 1 6 Zt t +2[t t−] 2 normalization factor 7 for←i 1 to m do ← Dt(xi,xi )exp αtyi ht(xi ) ht(xi) 8 Dt+1(xi,x ) − − i ← Zt 9 ϕ T α h T ← t=1 t t 10 return h =sgn(ϕT ) Mehryar Mohri - Foundations of Machine Learning page 12 Notes Distributions D t over pairs of sample points: • originally uniform. • at each round, the weight of a misclassified example is increased. y[ϕ (x ) ϕ (x)] e− t − t observation: D t+1 ( x, x )= S t Z , since • | | s=1 s t yαt[ht(x) ht(x)] Q y αs[hs(x) hs(x)] Dt(x, x)e− − 1 e− s=1 − Dt+1(x, x)= = P t . Zt S Z | | s=1 s weight assigned to base classifier h t : α t directy depends on the accuracy of h t at round t . Mehryar Mohri - Foundations of Machine Learning page 13 Coordinate Descent RankBoost Objective Function: convex and differentiable. T y[ϕ (x) ϕ (x)] F (α)= e− T − T = exp y α [h (x) h (x)] . − t t − t (x,x ,y) S (x,x ,y) S t=1 ∈ ∈ x e− 0 1pairwiseloss − Mehryar Mohri - Foundations of Machine Learning page 14 • Direction: unit vector e t with dF (α + ηe ) e =argmin t . t dη t η=0 T y s=1 αs[hs(x) hs(x)] yη[ht(x) ht(x)] • Since F ( α + η e t )= e − − e − − , (x,x ,y) S P ∈ T dF (α + ηet) = y[h (x) h (x)] exp y α [h (x) h (x)] dη − t − t − s s − s η=0 (x,x ,y) S s=1 ∈ T = y[h (x) h (x)]D (x, x) m Z − t − t T +1 s (x,x,y) S s=1 ∈ T + = [ −] m Z . − t − t s s=1 Thus, direction corresponding to base classifier selected by the algorithm. Mehryar Mohri - Foundations of Machine Learning page 15 • Step size: obtained via dF (α + ηe ) t =0 dη T y[h (x) h (x)]η y[h (x) h (x)] exp y α [h (x) h (x)] e− t − t =0 ⇔− t − t − s s − s (x,x ,y) S s=1 ∈ T y[h (x) h (x)]η y[h (x) h (x)]D (x, x) m Z e− t − t =0 ⇔− t − t T +1 s (x,x ,y) S s=1 ∈ y[h (x) h (x)]η y[h (x) h (x)]D (x, x)e− t − t =0 ⇔− t − t T +1 (x,x ,y) S ∈ + η η [ e− −e ]=0 ⇔− t − t 1 + η = log t . ⇔ 2 t− Thus, step size matches base classifier weight used in algorithm. Mehryar Mohri - Foundations of Machine Learning page 16 L1 Margin Definitions Definition: the margin of a pair ( x, x ) with label y =0 is T y(ϕ(x) ϕ(x)) y t=1 αt[ht(x) ht(x)] α ∆h(x) ρ(x, x)= − = − = y · . m α α α t=T t 1 1 Definition: the margin of the hypothesis h for a sample S =(( x 1 ,x 1 ,y 1 ) ..., ( x m ,x m ,y m )) is the minimum margin for pairs in S with non-zero labels: α ∆h(x) ρ =min y · . (x,x,y) S α 1 y=0∈ Mehryar Mohri - Foundations of Machine Learning page 17 Ranking Margin Bound (Cortes and MM, 2011) Theorem: let H be a family of real-valued functions. Fix ρ > 0 , then, for any δ > 0 , with probability at least 1 δ over the choice of a sample of size m , the − following holds for all h H : ∈ 2 log 1 R(h) R (h)+ RD1 (H)+RD2 (H) + δ . ≤ ρ ρ m m 2m Mehryar Mohri - Foundations of Machine Learning page 18 RankBoost Margin But, RankBoot does not maximize the margin. smooth-margin RankBoost (Rudin et al. , 2005): log F (α) G(α)= . − α 1 Empirical performance not reported. Mehryar Mohri - Foundations of Machine Learning page 19 Ranking with SVMs see for example (Joachims, 2002) Optimization problem: application of SVMs. m 1 2 min w + C ξi w,ξ 2 i=1 subject to: y w Φ(x ) Φ(x ) 1 ξ i · i − i ≥ − i ξ 0, i [1,m] . i ≥ ∀ ∈ Decision function: h: x w Φ(x)+b. → · Mehryar Mohri - Foundations of Machine Learning page 20 Notes The algorithm coincides with SVMs using feature mapping (x, x) Ψ(x, x)=Φ(x) Φ(x). → − Can be used with kernels. Algorithm directly based on margin bound. Mehryar Mohri - Foundations of Machine Learning page 21 Bipartite Ranking Training data: sample of negative points drawn according toD • − S =(x ,...,x ) U. + 1 m ∈ • sample of positive points drawn according to D+ S =(x1 ,...,xm ) U. − ∈ Problem: find hypothesis h : U R in H with small generalization error → RD(h)= Pr h(x)<h(x) . x D ,x D+ ∼ − ∼ Mehryar Mohri - Foundations of Machine Learning page 22 Notes More efficient algorithm in this special case (Freund et al., 2003). Connection between AdaBoost and RankBoost (Cortes & MM, 04; Rudin et al., 05). • if constant base ranker used. • relationship between objective functions. Bipartite ranking results typically reported in terms of AUC. Mehryar Mohri - Foundations of Machine Learning page 23 ROC Curve (Egan, 1975) Definition: the receiver operating characteristic (ROC) curve is a plot of the true positive rate (TP) vs. false positive rate (FP). • TP: % positive points correctly labeled positive. • FP: % negative points incorrectly labeled positive. 1 .8 .6 θ - + .4 h(x3) h(x14) h(x5) h(x1) h(x23) True positive rate positive .True 2 sorted scores 0 0 .2 .4 .6 .8 1 False positive rate Mehryar Mohri - Foundations of Machine Learning page 24 Area under the ROC Curve (AUC) (Hanley and McNeil, 1982) Definition: the AUC is the area under the ROC curve. Measure of ranking quality. 1 .8 .6 .4 AUC True positive rate positive .True 2 0 0 .2 .4 .6 .8 1 Equivalently, False positive rate 1 m m AUC(h)= 1 =Pr[h(x) >h(x)] h(xi)>h(xj ) mm x D i=1 j=1 ∼ + x D ∼ − =1 R(h).
Load more

Recommended publications
  • Demand STAR Ranking Methodology
    Demand STAR Ranking Methodology The methodology used to assess demand in this tool is based upon a process used by the State of Louisiana’s “Star Rating” system. Data regarding current openings, short term and long term hiring outlooks along with wages are combined into a single five-point ranking metric. Long Term Occupational Projections 2014-2014 The steps to derive a rank for long term hiring outlook (DUA Occupational Projections) are as follows: 1.) Eliminate occupations with a SOC code ending in “9” in order to remove catch-all occupational titles containing “All Other” in the description. 2.) Compile occupations by six digit Standard Occupational Classification (SOC) codes. 3.) Calculate decile ranking for each occupation based on: a. Total Projected Employment 2024 b. Projected Change number from 2014-2024 4.) For each metric, assign 1-10 points for each occupation based on the decile ranking 5.) Average the points for Project Employment and Change from 2014-2024 Short Term Occupational Projection 2015-2017 The steps to derive occupational ranks for the short-term hiring outlook are same use for the Long Term Hiring Outlook, but using the Short Term Occupational Projections 2015-2017 data set. Current Job Openings Current job openings rankings are assigned based on actual jobs posted on-line for each region for a 12 month period. 12 month average posting volume for each occupation by six digit SOC codes was captured using The Conference Board’s Help Wanted On-Line analytics tool. The process for ranking is as follows: 1) Eliminate occupations with a SOC ending in “9” in order to remove catch-all occupational titles containing “All Other” in the description 2) Compile occupations by six digit Standard Occupational Classification (SOC) codes 3) Determine decile ranking for the average number of on-line postings by occupation 4) Assign 1-10 points for each occupation based on the decile ranking Wages In an effort to prioritize occupations with higher wages, wages are weighted more heavily than the current, short-term and long-term hiring outlook rankings.
    [Show full text]
  • Ranking of Genes, Snvs, and Sequence Regions Ranking Elements Within Various Types of Biosets for Metaanalysis of Genetic Data
    Technical Note: Informatics Ranking of Genes, SNVs, and Sequence Regions Ranking elements within various types of biosets for metaanalysis of genetic data. Introduction In summary, biosets may contain many of the following columns: • Identifier of an entity such as a gene, SNV, or sequence One of the primary applications of the BaseSpace® Correlation Engine region (required) is to allow researchers to perform metaanalyses that harness large amounts of genomic, epigenetic, proteomic, and assay data. Such • Other identifiers of the entity—eg, chromosome, position analyses look for potentially novel and interesting results that cannot • Summary statistics—eg, p-value, fold change, score, rank, necessarily be seen by looking at a single existing experiment. These odds ratio results may be interesting in themselves (eg, associations between different treatment factors, or between a treatment and an existing Ranking of Elements within a Bioset known pathway or protein family), or they may be used to guide further Most biosets generated at Illumina include elements that are changing research and experimentation. relative to a reference genome (mutations) or due to a treatment (or some other test factor) along with a corresponding rank and The primary entity in these analyses is the bioset. It is a ranked list directionality. Typically, the rank will be based on the magnitude of of elements (genes, probes, proteins, compounds, single-nucleotide change (eg, fold change); however, other values, including p-values, variants [SNVs], sequence regions, etc.) that corresponds to a given can be used for this ranking. Directionality is determined from the sign treatment or condition in an experiment, an assay, or a single patient of the statistic: eg, up (+) or down(-) regulation or copy-number gain sample (eg, mutations).
    [Show full text]
  • Conduct and Interpret a Mann-Whitney U-Test
    Statistics Solutions Advancement Through Clarity http://www.statisticssolutions.com Conduct and Interpret a Mann-Whitney U-Test What is the Mann-Whitney U-Test? The Mann-Whitney U-test, is a statistical comparison of the mean. The U-test is a member of the bigger group of dependence tests. Dependence tests assume that the variables in the analysis can be split into independent and dependent variables. A dependence tests that compares the mean scores of an independent and a dependent variable assumes that differences in the mean score of the dependent variable are caused by the independent variable. In most analyses the independent variable is also called factor, because the factor splits the sample in two or more groups, also called factor steps. Other dependency tests that compare the mean scores of two or more groups are the F-test, ANOVA and the t-test family. Unlike the t-test and F-test, the Mann-Whitney U-test is a non- paracontinuous-level test. That means that the test does not assume any properties regarding the distribution of the underlying variables in the analysis. This makes the Mann-Whitney U-test the analysis to use when analyzing variables of ordinal scale. The Mann-Whitney U-test is also the mathematical basis for the H-test (also called Kruskal Wallis H), which is basically nothing more than a series of pairwise U-tests. Because the test was initially designed in 1945 by Wilcoxon for two samples of the same size and in 1947 further developed by Mann and Whitney to cover different sample sizes the test is also called Mann–Whitney–Wilcoxon (MWW), Wilcoxon rank-sum test, Wilcoxon–Mann–Whitney test, or Wilcoxon two-sample test.
    [Show full text]
  • Different Perspectives for Assigning Weights to Determinants of Health
    COUNTY HEALTH RANKINGS WORKING PAPER DIFFERENT PERSPECTIVES FOR ASSIGNING WEIGHTS TO DETERMINANTS OF HEALTH Bridget C. Booske Jessica K. Athens David A. Kindig Hyojun Park Patrick L. Remington FEBRUARY 2010 Table of Contents Summary .............................................................................................................................................................. 1 Historical Perspective ........................................................................................................................................ 2 Review of the Literature ................................................................................................................................... 4 Weighting Schemes Used by Other Rankings ............................................................................................... 5 Analytic Approach ............................................................................................................................................. 6 Pragmatic Approach .......................................................................................................................................... 8 References ........................................................................................................................................................... 9 Appendix 1: Weighting in Other Rankings .................................................................................................. 11 Appendix 2: Analysis of 2010 County Health Rankings Dataset ............................................................
    [Show full text]
  • Learning to Combine Multiple Ranking Metrics for Fault Localization Jifeng Xuan, Martin Monperrus
    Learning to Combine Multiple Ranking Metrics for Fault Localization Jifeng Xuan, Martin Monperrus To cite this version: Jifeng Xuan, Martin Monperrus. Learning to Combine Multiple Ranking Metrics for Fault Local- ization. ICSME - 30th International Conference on Software Maintenance and Evolution, Sep 2014, Victoria, Canada. 10.1109/ICSME.2014.41. hal-01018935 HAL Id: hal-01018935 https://hal.inria.fr/hal-01018935 Submitted on 18 Aug 2014 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Learning to Combine Multiple Ranking Metrics for Fault Localization Jifeng Xuan Martin Monperrus INRIA Lille - Nord Europe University of Lille & INRIA Lille, France Lille, France [email protected] [email protected] Abstract—Fault localization is an inevitable step in software [12], Ochiai [2], Jaccard [2], and Ample [4]). Most of these debugging. Spectrum-based fault localization applies a ranking metrics are manually and analytically designed based on metric to identify faulty source code. Existing empirical studies assumptions on programs, test cases, and their relationship on fault localization show that there is no optimal ranking metric with faults [16]. To our knowledge, only the work by Wang for all the faults in practice.
    [Show full text]
  • Concurrent Non-Malleable Zero Knowledge Proofs
    Concurrent Non-Malleable Zero Knowledge Proofs Huijia Lin?, Rafael Pass??, Wei-Lung Dustin Tseng???, and Muthuramakrishnan Venkitasubramaniam Cornell University, {huijia,rafael,wdtseng,vmuthu}@cs.cornell.edu Abstract. Concurrent non-malleable zero-knowledge (NMZK) consid- ers the concurrent execution of zero-knowledge protocols in a setting where the attacker can simultaneously corrupt multiple provers and ver- ifiers. Barak, Prabhakaran and Sahai (FOCS’06) recently provided the first construction of a concurrent NMZK protocol without any set-up assumptions. Their protocol, however, is only computationally sound (a.k.a., a concurrent NMZK argument). In this work we present the first construction of a concurrent NMZK proof without any set-up assump- tions. Our protocol requires poly(n) rounds assuming one-way functions, or O~(log n) rounds assuming collision-resistant hash functions. As an additional contribution, we improve the round complexity of con- current NMZK arguments based on one-way functions (from poly(n) to O~(log n)), and achieve a near linear (instead of cubic) security reduc- tions. Taken together, our results close the gap between concurrent ZK protocols and concurrent NMZK protocols (in terms of feasibility, round complexity, hardness assumptions, and tightness of the security reduc- tion). 1 Introduction Zero-knowledge (ZK) interactive proofs [GMR89] are fundamental constructs that allow the Prover to convince the Verifier of the validity of a mathematical statement x 2 L, while providing zero additional knowledge to the Verifier. Con- current ZK, first introduced and achieved by Dwork, Naor and Sahai [DNS04], considers the execution of zero-knowledge protocols in an asynchronous and con- current setting.
    [Show full text]
  • FOCS 2005 Program SUNDAY October 23, 2005
    FOCS 2005 Program SUNDAY October 23, 2005 Talks in Grand Ballroom, 17th floor Session 1: 8:50am – 10:10am Chair: Eva´ Tardos 8:50 Agnostically Learning Halfspaces Adam Kalai, Adam Klivans, Yishay Mansour and Rocco Servedio 9:10 Noise stability of functions with low influences: invari- ance and optimality The 46th Annual IEEE Symposium on Elchanan Mossel, Ryan O’Donnell and Krzysztof Foundations of Computer Science Oleszkiewicz October 22-25, 2005 Omni William Penn Hotel, 9:30 Every decision tree has an influential variable Pittsburgh, PA Ryan O’Donnell, Michael Saks, Oded Schramm and Rocco Servedio Sponsored by the IEEE Computer Society Technical Committee on Mathematical Foundations of Computing 9:50 Lower Bounds for the Noisy Broadcast Problem In cooperation with ACM SIGACT Navin Goyal, Guy Kindler and Michael Saks Break 10:10am – 10:30am FOCS ’05 gratefully acknowledges financial support from Microsoft Research, Yahoo! Research, and the CMU Aladdin center Session 2: 10:30am – 12:10pm Chair: Satish Rao SATURDAY October 22, 2005 10:30 The Unique Games Conjecture, Integrality Gap for Cut Problems and Embeddability of Negative Type Metrics Tutorials held at CMU University Center into `1 [Best paper award] Reception at Omni William Penn Hotel, Monongahela Room, Subhash Khot and Nisheeth Vishnoi 17th floor 10:50 The Closest Substring problem with small distances Tutorial 1: 1:30pm – 3:30pm Daniel Marx (McConomy Auditorium) Chair: Irit Dinur 11:10 Fitting tree metrics: Hierarchical clustering and Phy- logeny Subhash Khot Nir Ailon and Moses Charikar On the Unique Games Conjecture 11:30 Metric Embeddings with Relaxed Guarantees Break 3:30pm – 4:00pm Ittai Abraham, Yair Bartal, T-H.
    [Show full text]
  • Magic Adversaries Versus Individual Reduction: Science Wins Either Way ?
    Magic Adversaries Versus Individual Reduction: Science Wins Either Way ? Yi Deng1;2 1 SKLOIS, Institute of Information Engineering, CAS, Beijing, P.R.China 2 State Key Laboratory of Cryptology, P. O. Box 5159, Beijing ,100878,China [email protected] Abstract. We prove that, assuming there exists an injective one-way function f, at least one of the following statements is true: – (Infinitely-often) Non-uniform public-key encryption and key agreement exist; – The Feige-Shamir protocol instantiated with f is distributional concurrent zero knowledge for a large class of distributions over any OR NP-relations with small distinguishability gap. The questions of whether we can achieve these goals are known to be subject to black-box lim- itations. Our win-win result also establishes an unexpected connection between the complexity of public-key encryption and the round-complexity of concurrent zero knowledge. As the main technical contribution, we introduce a dissection procedure for concurrent ad- versaries, which enables us to transform a magic concurrent adversary that breaks the distribu- tional concurrent zero knowledge of the Feige-Shamir protocol into non-black-box construc- tions of (infinitely-often) public-key encryption and key agreement. This dissection of complex algorithms gives insight into the fundamental gap between the known universal security reductions/simulations, in which a single reduction algorithm or simu- lator works for all adversaries, and the natural security definitions (that are sufficient for almost all cryptographic primitives/protocols), which switch the order of qualifiers and only require that for every adversary there exists an individual reduction or simulator. 1 Introduction The seminal work of Impagliazzo and Rudich [IR89] provides a methodology for studying the lim- itations of black-box reductions.
    [Show full text]
  • Pairwise Versus Pointwise Ranking: a Case Study
    Schedae Informaticae Vol. 25 (2016): 73–83 doi: 10.4467/20838476SI.16.006.6187 Pairwise versus Pointwise Ranking: A Case Study Vitalik Melnikov1, Pritha Gupta1, Bernd Frick2, Daniel Kaimann2, Eyke Hullermeier¨ 1 1Department of Computer Science 2Faculty of Business Administration and Economics Paderborn University Warburger Str. 100, 33098 Paderborn e-mail: melnikov,prithag,eyke @mail.upb.de, bernd.frick,daniel.kaimann @upb.de { } { } Abstract. Object ranking is one of the most relevant problems in the realm of preference learning and ranking. It is mostly tackled by means of two different techniques, often referred to as pairwise and pointwise ranking. In this paper, we present a case study in which we systematically compare two representatives of these techniques, a method based on the reduction of ranking to binary clas- sification and so-called expected rank regression (ERR). Our experiments are meant to complement existing studies in this field, especially previous evalua- tions of ERR. And indeed, our results are not fully in agreement with previous findings and partly support different conclusions. Keywords: Preference learning, object ranking, linear regression, logistic re- gression, hotel rating, TripAdvisor 1. Introduction Preference learning is an emerging subfield of machine learning that has received increasing attention in recent years [1]. Roughly speaking, the goal in preference learning is to induce preference models from observed data that reveals information Received: 11 December 2016 / Accepted: 30 December 2016 74 about the preferences of an individual or a group of individuals in a direct or indirect way; these models are then used to predict the preferences in a new situation.
    [Show full text]
  • Power Comparisons of the Mann-Whitney U and Permutation Tests
    Power Comparisons of the Mann-Whitney U and Permutation Tests Abstract: Though the Mann-Whitney U-test and permutation tests are often used in cases where distribution assumptions for the two-sample t-test for equal means are not met, it is not widely understood how the powers of the two tests compare. Our goal was to discover under what circumstances the Mann-Whitney test has greater power than the permutation test. The tests’ powers were compared under various conditions simulated from the Weibull distribution. Under most conditions, the permutation test provided greater power, especially with equal sample sizes and with unequal standard deviations. However, the Mann-Whitney test performed better with highly skewed data. Background and Significance: In many psychological, biological, and clinical trial settings, distributional differences among testing groups render parametric tests requiring normality, such as the z test and t test, unreliable. In these situations, nonparametric tests become necessary. Blair and Higgins (1980) illustrate the empirical invalidity of claims made in the mid-20th century that t and F tests used to detect differences in population means are highly insensitive to violations of distributional assumptions, and that non-parametric alternatives possess lower power. Through power testing, Blair and Higgins demonstrate that the Mann-Whitney test has much higher power relative to the t-test, particularly under small sample conditions. This seems to be true even when Welch’s approximation and pooled variances are used to “account” for violated t-test assumptions (Glass et al. 1972). With the proliferation of powerful computers, computationally intensive alternatives to the Mann-Whitney test have become possible.
    [Show full text]
  • Practice of Epidemiology a Family Longevity Selection Score
    American Journal of Epidemiology Advance Access published November 18, 2009 American Journal of Epidemiology ª The Author 2009. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of DOI: 10.1093/aje/kwp309 Public Health. All rights reserved. For permissions, please e-mail: [email protected]. Practice of Epidemiology A Family Longevity Selection Score: Ranking Sibships by Their Longevity, Size, and Availability for Study Paola Sebastiani, Evan C. Hadley*, Michael Province, Kaare Christensen, Winifred Rossi, Thomas T. Perls, and Arlene S. Ash * Correspondence to Dr. Evan Hadley, Gateway Building, MSC 9205, National Institute on Aging, 7201 Wisconsin Avenue, Bethesda, MD 20892-9205 (e-mail: [email protected]). Initially submitted February 10, 2009; accepted for publication September 1, 2009. Family studies of exceptional longevity can potentially identify genetic and other factors contributing to long life and healthy aging. Although such studies seek families that are exceptionally long lived, they also need living members who can provide DNA and phenotype information. On the basis of these considerations, the authors developed a metric to rank families for selection into a family study of longevity. Their measure, the family longevity selection score (FLoSS), is the sum of 2 components: 1) an estimated family longevity score built from birth-, gender-, and nation-specific cohort survival probabilities and 2) a bonus for older living siblings. The authors examined properties of FLoSS-based family rankings by using data from 3 ongoing studies: the New England Centenarian Study, the Framingham Heart Study, and screenees for the Long Life Family Study. FLoSS-based selection yields families with exceptional longevity, satisfactory sibship sizes and numbers of living siblings, and high ages.
    [Show full text]
  • Data Structures Meet Cryptography: 3SUM with Preprocessing
    Data Structures Meet Cryptography: 3SUM with Preprocessing Alexander Golovnev Siyao Guo Thibaut Horel Harvard NYU Shanghai MIT [email protected] [email protected] [email protected] Sunoo Park Vinod Vaikuntanathan MIT & Harvard MIT [email protected] [email protected] Abstract This paper shows several connections between data structure problems and cryptography against preprocessing attacks. Our results span data structure upper bounds, cryptographic applications, and data structure lower bounds, as summarized next. First, we apply Fiat–Naor inversion, a technique with cryptographic origins, to obtain a data structure upper bound. In particular, our technique yields a suite of algorithms with space S and (online) time T for a preprocessing version of the N-input 3SUM problem where S3 T = O(N 6). This disproves a strong conjecture (Goldstein et al., WADS 2017) that there is no data· structure − − that solves this problem for S = N 2 δ and T = N 1 δ for any constant δ > 0. e Secondly, we show equivalence between lower bounds for a broad class of (static) data struc- ture problems and one-way functions in the random oracle model that resist a very strong form of preprocessing attack. Concretely, given a random function F :[N] [N] (accessed as an oracle) we show how to compile it into a function GF :[N 2] [N 2] which→ resists S-bit prepro- cessing attacks that run in query time T where ST = O(N 2−→ε) (assuming a corresponding data structure lower bound on 3SUM). In contrast, a classical result of Hellman tells us that F itself can be more easily inverted, say with N 2/3-bit preprocessing in N 2/3 time.
    [Show full text]