UNIVERSITY OF CALIFORNIA, MERCED Diversification, speciation, and phylogeography of freshwater sculpins (Cottus, Cottopsis) in California A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Quantitative and Systems Biology by Jason David Baumsteiger Chair: Professor Michael Dawson Professor Andres Aguilar Professor Andrew Kinziger Professor Peter Moyle 2013 Copyright Jason David Baumsteiger, 2013 All rights reserved The Dissertation of Jason David Baumsteiger is approved, and is acceptable in quality and form for publication on microfilm and electronically: Advisor: Professor Andres Aguilar Professor Andrew P. Kinziger Professor Peter B. Moyle Chair: Professor Michael N. Dawson Date University of California, Merced 2013 iii DEDICATION I would like to dedicate this dissertation to my parents, who taught me the strength and fortitude necessary to complete a task as arduous as this dissertation. From their perseverance came my success. iv TABLE OF CONTENTS Signature Page ...................................................................................................................... iii Dedication ............................................................................................................................. iv Table of Contents .................................................................................................................. v List of Abbreviations ............................................................................................................ vii List of Tables ........................................................................................................................ viii List of Figures ....................................................................................................................... x Acknowledgements ............................................................................................................... xii Vita........................................................................................................................................ xiv Abstract ................................................................................................................................. xvii Chapter 1: Introduction ......................................................................................................... 1 1.1. Freshwater Sculpin ................................................................................................ 1 1.2. Speciation and Hybridization ................................................................................ 2 1.3. Cryptic and Incipient Species ............................................................................... 2 1.4. Phylogeography .................................................................................................... 2 1.5. Molecular Marker Development ........................................................................... 3 1.6. Objective ............................................................................................................... 4 Chapter 2: Life history and biogeographic diversification of an endemic Western North American freshwater fish clade using a comparative species tree approach... ........................ 5 2.1. Abstract ................................................................................................................. 5 2.2. Introduction ........................................................................................................... 5 2.3. Materials and Methods .......................................................................................... 7 2.4. Results ................................................................................................................... 11 2.5. Discussion ............................................................................................................. 14 2.6. Tables and Figures ................................................................................................ 18 2.7. Supplemental Material .......................................................................................... 26 Chapter 3: Complex phylogeography and historical hybridization in two closely related sister taxa of freshwater sculpin ........................................................................................... 42 3.1. Abstract ................................................................................................................. 42 3.2. Introduction ........................................................................................................... 42 3.3. Materials and Methods .......................................................................................... 44 3.4. Results ................................................................................................................... 47 3.5. Discussion ............................................................................................................. 50 3.6. Tables and Figures ................................................................................................ 56 3.7. Supplemental Material .......................................................................................... 64 v Chapter 4: Crypsis and potential parapatric incipient speciation in a species of freshwater sculpin, Cottus asper ........................................................................................................ 74 4.1. Abstract ................................................................................................................. 74 4.2. Introduction ........................................................................................................... 74 4.3. Materials and Methods .......................................................................................... 76 4.4. Results ................................................................................................................... 78 4.5. Discussion ............................................................................................................. 81 4.6. Tables and Figures ................................................................................................ 86 4.7. Supplemental Material .......................................................................................... 98 Chapter 5: Synthesis and Future Directions .......................................................................... 103 References…………………………………………………………………………………. 107 vi LIST OF ABBREVIATIONS ABI Applied Biosystems Incorporated AIC Akaike Information Criterion AICC Akaike Information Criterion, corrected AMOVA Analysis of Molecular Variance BAY San Jose Region BSA Bovine Serum Albumin CA California CO1 Cytochrome Oxidase 1 CR Control Region or D-loop CYTB Cytochrome B DNA Deoxyribonucleic acid ESS Effective Sample Size G Gamma Distribution HKY Hasegawa, Kishino, and Yano Model HPD Highest Posterior Density HSU Humboldt State University I Invariable sites IBD Isolation by Distance ILS Incomplete Lineage Sorting K Clusters LLR Log Likelihood Ratio MCMC Markov Chain Monte Carlo MDC Minimize Deep Coalescence ML Maximum Likelihood mtDNA Mitochondrial DNA MYA Million Years Ago NCBI National Center for Biotechnology Information nDNA Nuclear DNA NMT Nuclear and Mitochondrial Sequences NNI Nearest Neighbor Interchange NUC Nuclear sequence only OR Oregon PCR Polymerase Chain Reaction PSRF Potential Scale Reduction Factors RM River Mile SNP Single Nucleotide Polymorphism SPR Subtree Pruning and Regrafting TrN Tamura-Nei Model UCSC University of California, Santa Cruz UCLA University of California, Los Angeles µSAT Microsatellite WA Washington vii LIST OF TABLES Table 2.1: Samples collected for molecular systematic analysis, showing identification number, number of samples and locations ............................................................................ 18 Table 2.2: Maker information for each nuclear (Nuc) or mitochondrial (mt) sequence used to generate individual gene trees or concatenated tree (Nmt) ............................................... 19 Table 3.1: Sampling locations and numbers of individuals covering known ranges of each species……………………………………………………………………………………… 56 Table 3.2: Descriptive statistics, by location, for three nuclear (508,517,520), one mitochondrial (CB), and six microsatellites (µsats) loci. Number of individuals (N), substitutions (S), nucleotide diversity (π), number of haplotypes (H), haplotype diversity (Hd), allele number corrected for rarefaction (AR), expected heterozygosity (HE), observed heterozygosity (HO), and inbreeding coefficient (FIS). *outside HWE (p < 0.01) ................ 57 Table 4.1: List of sampling locations, by name, and numbers sampled. Microsatellite number (usat #) refers to numbering used for locations in all microsatellite analyses……. 86 Table 4.2: Overall table of individuals used for either mitochondrial (cytb) or microsatellites analyses, showing general measures of diversity for each marker type. Number of individuals (N), substitutions (S), nucleotide diversity (π), number of haplotypes (H), haplotype diversity (Hd), allele number corrected for rarefaction (AR), expected heterozygosity (HE), observed heterozygosity (HO), and inbreeding coefficient (FIS). * outside HWE (p < 0.01)…………. 87 Table 4.3A: Population diversity measures for unique haplotypes of cytb sequence data. Intermediate locations (Napa and Suisun Bay) included with Inland group..……………… 88 Table 4.3B: Measure