Phylogeography and Molecular Systematics of Species Complexes in the Genus Genetta (Carnivora, Viverridae) Carlos Alberto Rodrigues Fernandes A thesis submitted to the School of Biosciences, Cardiff University, Cardiff for the degree of Doctor of Philosophy Cardiff University School of Biosciences 2004 UMI Number: U584656 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. Dissertation Publishing UMI U584656 Published by ProQuest LLC 2013. Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code. ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 I declare that I conducted the work in this thesis, that I have composed this thesis, that all quotations and sources of information have been acknowledged and that this work has not previously been accepted in an application for a degree. Carlos A. R. Fernandes Abstract The main aim of this study was to estimate phylogeographic patterns from mitochondrial DNA diversity and relate them with evolutionary structure in two species complexes of genets, Genetta genetta and Genetta “rubiginosa ”, which have fluid morphological variation. Both are widely distributed in sub-Saharan Africa but whereas G. “rubiginosa ” appears in both closed and open habitats, G. genetta is absent from the rainforest and occurs also in the Maghreb, southwest Europe, and Arabia. DNA sequence data, mainly acquired from museum skin samples (about 75% of the total number of samples), was analysed using methods from the fields of phylogenetics and population genetics. The results for G. genetta are compatible with a scenario of allopatric fragmentation in grassland refuges during the Pleistocene climatic cycles as the main factor responsible for geographic genetic structure within Africa. The Arabian isolate showed significant genetic divergence, species-level compatible, and is probably the result of a long-distance dispersal from North Africa. Genetic diversity in Europe is a subset of that found in North Africa and shallow genetic distance is concordant with their anthropogenic introduction into Europe. North Africa seems to be cyclically connected to West and Central Africa during interglacial periods in which the Sahara recedes substantially. For G. “rubiginosa ”, isolated biogeographic relicts in the eastern African coast, possibly unsuspected species, were uncovered. However, the dominant pattern in the evolution of this species complex seems to be ecological differentiation in parapatry after invasion of open habitats from the rainforest ancestral habitat. Three general conclusions may be extracted from this study. Firstly, the use of mitochondrial DNA is clearly informative, but both gene flow among parapatric populations and introgression are confounding factors. Secondly, the Pleistocene in Africa has had variable biogeographical and evolutionary consequences for different taxa, depending on their ecological breadth. Lastly, large-scale utilisation of museum samples in phylogeographies of cryptic taxa has been rarely attempted but should become a standard approach. Acknowledgments This study was funded by a Praxis XXI PhD studentship from the Foundation for Science and Technology, the research council of the Portuguese government. In recognition to their contribution, in different and complementary grounds, for the successful conclusion of the research effort that constitutes this PhD thesis, I would like to express my appreciation and gratitude to the people listed below. Prof. Michael Bruford for the willingness to accept me as PhD student under his supervision, in spite of my initial complete unfamiliarity with both theoretical and practical foundations of his field of expertise, and afterwards for his advice, always wise, and enormous patience along the way. Dr. Margarida Santos-Reis for the receptivity to supervise and support the transformation into a PhD of a roughly outlined proposal involving genets and systematics presented to her by an unknown and freshly graduated student of the same Department at Lisbon University. Dr. Joao Crawford-Cabral for believing that I could withstand the complexities of genet taxonomy and evolution, become his disciple and collaborator, and, finally, contribute with this study to extend the knowledge of this group of animals in which his work was pioneer and seminal. Also for the privilege of learning all I know about genets from the leading authority, for his enthusiasm and insight in the discussion of my results, and constructive criticism of my interpretations. Dr. Stephan Funk for his shared interest in genets, introducing me to many procedures and approaches essential in the successful use of molecular markers applied to Carnivore research, and for his trust during collaborative work. Dr. Philippe Gaubert for the fruitful collaborative framework that resulted from a common scientific interest in genets and which is expected to continue well into the future. Dr. Mark Jervis for his interest in the phylogenetics component of my thesis and for important corrections on earlier versions of this manuscript. All the people, mainly including museum curators, field ecologists, and tissue bank keepers, who by providing the samples used here made possible the present study. Friendly members of the G10 lab, such as Dr. Cristiane Silveira for her help in showing me the basic technical procedures during my initial times in the lab and Dr. Trinidad Perez for her skills with the software that produced the maps with distribution areas presented here. Ms. Ana Elisabete Pires for all the encouragement, help, patience and support, which were essential for the conclusion of this work in its final stages, when panic and tiredness were frequent visitors. Last, but not least, my family for all their friendship, support, belief, and dedication since the beginning. TABLE OF CONTENTS Chapter 1. Introduction 1 1.1 Context 1 1.2 The species complexes 1 1.3 Species concepts, speciation and genetic markers 13 1.4 Phylogeography and mitochondrial DNA 22 1.5 African biogeography and palaeoclimate 26 1.6 Aims 30 Chapter 2. Concepts and methods in phylogeography 31 2.1 Population genetics 31 2.1.1 Coalescent 31 2.1.2 Demographic history 31 2.1.3 Population structure and gene flow 33 2.1.4 Intraspecific networks and Nested Clade Analysis 35 2.2 Molecular phylogenetics 39 2.2.1 Parsimony methods 40 2.2.2 Maximum likelihood methods 41 2.2.3 Models of sequence evolution 42 2.2.4 Distance methods 46 2.2.5 Tree searching 49 2.2.6 Statistical assessment of trees 51 2.2.7 Tree rooting 56 2.2.8 Sampling in phylogenetics 57 Chapter 3. Mitochondrial DNA phylogeography of the small-spotted genet {Genetta genetta) 58 3.1 Introduction 58 3.2 Material and methods 64 3.2.1 DNA samples 64 3.2.2 Laboratory techniques 64 3.2.2.1 DNA extraction 64 3.2.2.2 Preliminary study of common genet mtDNA 65 3.2.2.3 PCR and sequencing 67 3.2.3 Data analysis 69 3.2.3.1 Phylogenetic analysis 69 3.2.3.2 Population genetics and phylogeographic analyses 70 3.3 Results 75 3.3.1 Reliability of DNA sequences from museum specimens 75 3.3.2 Characterisation of DNA sequences 75 3.3.3 Phylogenetic analysis 76 3.3.4 Population genetics and phylogeographic analyses 93 3.4 Discussion 3.4.1 Genetic structure and phylogeographic patterns 104 3.4.2 The felina case 108 3.4.3 The origins of G. genetta 111 3.4.4 Taxonomic findings 112 3.4.5 Final remarks 113 Appendix to Chapter 3 115 Chapter 4. Mitochondrial DNA phylogeography of the large-spotted genets (Viverridae: Genetta) 128 4.1 Introduction 128 4.2 Material and methods 134 4.2.1 DNA samples 134 4.2.2 Laboratory techniques 134 4.2.2.1 DNA extraction 134 4.2.2.2 Preliminary study of large-spotted mtDNA 135 4.2.2.3 PCR and sequencing 138 4.2.3 Data analysis 138 4.2.3.1 Phylogenetic analyses 138 4.2.3.2 Population genetics and phylogeographic analyses 140 4.3 Results 4.3.1 Reliability of DNA sequences from museum specimens 144 4.3.2 Characterisation of DNA sequences 144 4.3.3 Phylogenetic analyses 145 4.3.4 Population genetics and phylogeographic analyses 158 4.4 Discussion 170 4.4.1 Haplotype genealogies, gene flow and habitat dynamics 170 4.4.2 Taxonomy and conservation 175 4.4.3 Final remarks 178 Appendix to Chapter 4 180 Chapter 5. General Discussion 192 5.1 Small-spotted genets 192 5.2 Large-spotted genets 196 5.3 Phylogeography, evolutionary divergence and speciation 199 5.4 Comparative phylogeography, ecology and biogeography 201 5.5 Future directions 204 References 207 Appendices Appendix I Appendix II Appendix III Appendix IV Chapter 1. Introduction 1.1 Context This thesis is the product of my personal research interests in evolution, biogeography and taxonomy, and on a molecular approach to these disciplines, applied to an available, interesting and basically unexplored animal group, the genets. It is also the natural continuation of my previous research in this cryptic taxon of mammalian carnivores and aims both to increase the knowledge about them and use them as a case study to examine questions of evolution and biogeography in Africa. It is widely accepted that morphology is unable to resolve genet taxonomy and that molecular systematics data may prove fundamental to assess putative cases of speciation and described subspecies (see Martinoli et al. in press, for an example of genet species identification using diagnostic genetic characters; Appendix I). On the other hand, and independently of the taxonomic questions, the genet species complexes considered in this study are, due to their continent- wide ranges, very suitable models for phylogeographic and population genetics analyses.