TRACING THE EVOLUTION OF MALE LINEAGES IN BEARS USING GENETIC MARKERS ON THE Y CHROMOSOME Dissertation zur Erlangung des Doktorgrades der Naturwissenschaften vorgelegt beim Fachbereich 15 der Johann Wolfgang Goethe - Universität in Frankfurt am Main von Tobias Bidon aus Tuttlingen Frankfurt am Main, 2015 (D 30) vom Fachbereich 15 (Biowissenschaften) der Johann Wolfgang Goethe - Universität als Dissertation angenommen. Dekanin: Prof. Dr. Meike Piepenbring Gutachter: Prof. Dr. Axel Janke, Prof. Dr. Markus Pfenninger Datum der Disputation: TABLE OF CONTENTS Summary 1 Zusammenfassung 3 Hintergrund 3 Durchgeführte Studien 4 Fazit 7 General introduction 9 The quest for genetic variation 9 Characteristics and structure of the Y chromosome 11 The male perspective 11 Its special features render the Y chromosome a valuable marker in evolutionary studies 13 Sex chromosomes evolved from autosomes 15 Y chromosomes are rich in repetitive sequences 17 Study organisms 18 Phylogenetic relationship of bears 18 Brown and polar bears have different levels of intraspecific variation 20 Thesis objectives 21 General Discussion 23 The Y chromosome as a resource – developing male-specific markers 24 Molecular sex determination in conservation and forensics 24 Identifying male-specific markers is complicated by the structure of the Y chromosome 26 The Y chromosome as an evolutionary marker – uncovering the paternal history of bears 28 Expanding phylogeographic inferences using male-specific markers 28 Lack of patrilineal structuring in brown bears 30 Male-biased gene flow shapes genetic variation in brown bears and other species 31 Divergent Y-chromosomal clades in polar bears are geographically widespread 33 The Y chromosome and ursine speciation 35 Y-chromosomal introgression is unlikely in bears 38 The role of the Y chromosome in speciation might have been underappreciated 39 On the relevance of a single locus in the era of genomics 40 Conclusion 41 References 43 Publications 55 Publication 1: A sensitive and specific multiplex PCR approach for sex identification of ursine and tremarctine bears suitable for non-invasive samples 55 Publication 2: Brown and Polar Bear Y Chromosomes Reveal Extensive Male-Biased Gene Flow within Brother Lineages 65 Publication 3: Bears in a Forest of Gene Trees: Phylogenetic Inference Is Complicated by Incomplete Lineage Sorting and Gene Flow 93 Appendix 123 Genome-wide search identifies 1.9 megabases from the polar bear Y chromosome for evolutionary analyses 123 Acknowledgements 157 Curriculum Vitae 159 Summary SUMMARY The mammalian family of bears (Ursidae) comprises eight extant species, occurring on four different continents. Among them are the iconic and well-known brown and polar bears, both widely distributed across the Northern hemisphere. Their intraspecific genetic structuring has been extensively investigated, albeit with a focus on genetic markers from maternally inherited parts of their genomes (mitochondrial DNA). The evolutionary relationship and divergence time between brown and polar bears have recently triggered an extensive debate, while less focus has been put on to other parts of the ursid phylogeny, particularly to a clade of three Asian bear species. To date, whole genomes of more than 100 bear individuals from four different species have been sequenced. Yet, one fundamental part of the genome has been largely omitted from specific analyses, in bears as well as in most other mammals: the Y chromosome. The mammalian Y chromosome provides a unique perspective on the evolutionary history of organisms due to its distinct features, and specifically reflects the patriline because of its male-specific inheritance. The characteristics of this chromosome make it well suited to complement and contrast evolutionary inferences based on other genetic markers, and to uncover processes like sex-biased gene flow and hybridization. The unique insights that can be gained from analyses of Y-linked genetic variation made me utilize this part of the genome to investigate the evolution of male lineages in bears. Studying the patriline is particularly promising in this taxonomic group because of male-biased dispersal and a complex and fast radiation of bears. The analysis of Y-chromosomal genetic markers is thus the common theme of this dissertation: I present the identification of large amounts of Y-chromosomal sequence, the development of male-specific markers from such sequences, and the application of these markers to trace the evolution of male lineages of different bear species. Specifically, I developed a molecular sex determination system based on the detection of two Y-linked fragments that allows to reliably discriminate between females and males from seven different bear species (Bidon et al. 2013). The approach is highly sensitive, bear-specific, and can be applied in standard molecular 1 Summary laboratories. This makes it valuable in conservation genetics and forensic applications, e.g. to analyze non-invasively collected samples. Furthermore, I used Y-linked markers in a comprehensive and range-wide sample of brown and polar bears, and show that male-biased gene flow plays an important role in distributing genetic material throughout the ranges of both species (Bidon et al. 2014). In brown bears, I detected a lack of paternal population structuring which is in strong contrast to the detailed structuring of the matriline. Analyzing Y-chromosomal sequences from all eight bear species, I present a phylogeny of the patriline that largely resembles the topology from other nuclear markers but is different from the topology of the mitochondrial gene tree (Kutschera et al. 2014). This discordance among loci generates interesting hypotheses about inter-species gene flow, particularly among American and Asiatic black bears. With the identification of almost two million basepairs of Y-chromosomal sequence and the analysis of an unprecedented large male-specific dataset in polar bears, a high-resolution view on the distribution of their intraspecific variation was obtained (Bidon et al. 2015). In particular, two clades that are divergent but do not show pronounced phylogeographic structure were detected, confirming the great dispersal capacity of males of this high arctic species. This dissertation thus represents a comprehensive investigation of Y-linked genetic variation on the intra- and interspecific level in a non-model organism. With my research, I contribute to an increased understanding of the complex evolutionary history of bears. In particular, I show that male-biased gene flow strongly influences the distribution of nuclear genetic variation, and that the contrast between phylogenies of differentially inherited markers can help to understand interspecific hybridization between closely related species. Moreover, my findings demonstrate the potential of Y-chromosomal markers to uncover unknown evolutionary patterns and processes. This applies not only to bears but to many species, even such that are generally well known and well described. 2 Zusammenfassung ZUSAMMENFASSUNG Untersuchung der Evolution von männlichen Erblinien in Bären mithilfe genetischer Marker auf dem Y Chromosom HINTERGRUND Das Y Chromosom ist ein wesentlicher Bestandteil des Säugetiergenoms, und kann als männchen-spezifischer genetischer Marker einzigartige und interessante Einblicke in die Evolutionsgeschichte von Organismen liefern. Es hat einige besondere Eigenschaften, die es von anderen Teilen des Genoms unterscheiden. Dazu gehören unter anderem die Vererbung von Vater zu Sohn (paternale Vererbung), sowie das weitgehende Fehlen von interchromosomaler Rekombination (Jobling et al. 2014). Evolutive Analysen beruhen meist auf der Analyse von mütterlicherseits (maternal) vererbter DNA der Mitochondrien oder auf Erbinformation der Autosomen des Zellgenoms. Letztere werden von beiden Geschlechtern an die Nachkommen weitergeben. Da unterschiedlich vererbte Teile des Genoms unterschiedliche Muster und Prozesse abbilden können, bleibt etwa der durch die Wanderung von Männchen vermittelte Genfluss durch die Analyse von mitochondrieller DNA unerkannt. Die kombinierte Analyse unterschiedlicher Marker ermöglicht es nun, potentielle Kontraste zwischen ihnen, z. B. die Abbildung unterschiedlicher Verwandtschaftsbeziehungen (Topologien der Stammbäume), darzustellen. Es sind diese Kontraste, die uns z. B. auf Hybridisierung zwischen Arten schließen lassen, und damit unser Verständnis für evolutive Muster und Prozesse erweitern können (Fahey et al. 2014). Das Y Chromosom ist bisher nur in sehr wenigen Arten, hauptsächlich in Modellorganismen wie Mensch und Maus, ausgiebig sequenziert und charakterisiert worden. Auch liegen nur für wenige Arten evolutive Analysen basierend auf diesem genetischen Lokus vor. Die Ursachen dafür sind in den spezifischen Eigenschaften des Y Chromosoms zu finden, z.B. in der großen Zahl sich wiederholender (repetitiver) Sequenzabschnitte, oder der geringen Dichte an Protein-kodierenden Genen. Deshalb wurde das Y Chromosom auch in vielen Studien, die genomweite Daten erheben, entweder gar nicht sequenziert (indem Weibchen untersucht wurden) oder zumindest nicht spezifisch analysiert (Greminger et al. 2010). 3 Zusammenfassung Eine Gruppe innerhalb der Säugetiere, für die das Y Chromosom als genetischer Marker vielversprechend ist, ist die Familie der Bären (Ursidae). Es gibt heute acht Bärenarten, die auf vier verschiedenen Kontinenten vorkommen (McLellan & Reiner 1994). Unter ihnen sind die gut untersuchten und bekannten