Paleobiology, 30(1), 2004, pp. 108±128 Evolution of hypercarnivory: the effect of specialization on morphological and taxonomic diversity Jill A. Holliday and Scott J. Steppan Abstract.ÐThe effects of specialization on subsequent morphological evolution are poorly under- stood. Specialization has been implicated in both adaptive radiations that result from key inno- vations and evolutionary ``dead ends,'' where specialized characteristics appear to limit subse- quent evolutionary options. Despite much theoretical debate, however, empirical studies remain infrequent. In this paper, we use sister-group comparisons to evaluate the effect of morphological specialization to a particular ecological niche, hypercarnivory, on subsequent taxonomic and mor- phological diversity. Six sets of sister groups are identi®ed in which one clade exhibits hypercar- nivorous characteristics and the sister clade does not. Comparison results are summed across the categories ``hypercarnivore'' and ``sister group.'' We also evaluate whether increasing degrees of specialization are correlated with decreasing phenotypic variation. Results presented here indicate that specialization to hypercarnivory has no effect on taxonomic diversity, but a strong effect on subsequent morphological diversity related to the jaws and dentition, and that increasing special- ization does not correlate with morphological diversity except in the most specialized saber- toothed taxa, which exhibit higher variance than less specialized morphs, possibly due to selection on other characteristics. Jill A. Holliday and Scott J. Steppan. Department of Biological Science, Florida State University, Talla- hassee, Florida 32306-1100. E-mail: [email protected], E-mail: [email protected] Accepted: 21 May 2003 Introduction pattern (e.g., Price and Carr 2000), and others ®nd no effect at all (e.g., Wiegmann et al. 1993; The effect of specialization on subsequent de Queiroz 1999; Janz et al. 2001). taxonomic and morphological evolution is Despite numerous studies of the effect of fundamentally important to the tempo and specialization on taxonomic diversi®cation, mode of evolution and the role of adaptation studies of its effects on subsequent morpho- in macroevolution (e.g., Futuyma and Moreno logical diversity (disparity) are few. In a the- 1988; Janz et al. 2001). However, there is little oretical context, many workers have suggest- consensus as to how specialization affects di- ed that possession of certain morphological versity: does it act to increase or decrease rates character states may reduce the ability to at- of cladogenesis? How does specialization af- tain certain other states (Lauder 1981; Smith et fect probability of extinction? Does it con- al. 1985; Emerson 1988; Futuyma and Moreno strain further adaptation? Certainly, much at- 1988; Werdelin 1996; Donoghue and Ree 2000; tention has been given to the possibility that Wagner and Schwenk 2000), implying that the particular specializations may promote taxo- subsequent evolutionary trajectories of some nomic diversi®cation; that is, a morphological specialized taxa may be limited. At its ex- or behavioral specialization may act as a key treme, therefore, specialization could act as a innovation, leading to an increase in rates of dead end (Moran 1988; Janz et al. 2001), lim- cladogenesis (Liem 1973; Mitter et al. 1988; iting morphological diversi®cation and poten- Farrell et al. 1991; Hodges and Arnold 1995; tially reducing rates of cladogenesis or, alter- de Queiroz 1999; Dodd et al. 1999), but em- natively, increasing extinction rates as special- pirical studies have produced contrasting re- ized taxa reduce their ability to adapt to sults. Some workers have found that speciali- changing conditions. However, few studies zation increases taxonomic diversity (e.g., have directly identi®ed and tested effects of Liem 1973; Mitter et al. 1988; Farrell et al. 1991; specialization on subsequent phenotypic Hodges and Arnold 1995; de Queiroz 1998; change (but see Liem 1973; Moran 1988; War- Dodd et al. 1999), some report the opposite heit et al. 1999). q 2004 The Paleontological Society. All rights reserved. 0094-8373/04/3001-0000/$1.00 EVOLUTION OF HYPERCARNIVORY 109 We determined the effect of dental and cra- coons (Procyonidae), and strict herbivores nial specialization to a meat-only diet, hyper- such as the giant panda. Variation in ecology carnivory, on subsequent morphological and is strongly re¯ected in the dentition (Van Val- taxonomic diversity in mammalian carni- kenburgh 1989), so a more omnivorous/fru- vores. We used an explicitly phylogenetic ap- givorous diet is accompanied by a relative in- proach and applied it to repeated convergenc- crease in grinding surfaces whereas a more es on hypercarnivory, increasing our statisti- highly carnivorous diet is re¯ected by a rela- cal power by evaluating results from multiple tive decrease in grinding surfaces and an in- sister groups. We speci®cally tested the hy- crease in shearing edges. potheses that (1) hypercarnivores have lower Because of the tight correlation between taxonomic and craniodental morphological dentition and ecology, dental characters can diversity than do their sister groups and (2) be used effectively to infer aspects of the diet increasing specialization leads to lower mor- or ecological niche. Van Valkenburgh (1988, phological diversity. To test these hypotheses, 1989) showed that variables including relative we quanti®ed and compared diversity be- blade length, canine tooth shape, premolar tween hypercarnivore clades and their prim- size and shape, and grinding area of the lower itively nonhypercarnivorous sister groups. molars distinguished between dietary types The advantage of using sister groups is that in extant carnivores. She compared guild both groups (when their stem lineages are in- compositions of carnivoran communities, cluded) have by de®nition had equal time to concluding that each guild comprised a diversify. We used both species counts and the broadly similar set of morphotypes occupying methods of Slowinski and Guyer (1993) to as- a limited number of ecological niches (Van sess taxonomic diversity. We compared mor- Valkenburgh 1988, 1989). There is thus a sub- phological diversities (disparities) by compar- stantial overlap in certain regions of mor- ing variances of factor scores obtained from phospace (Crusafont-Pairo and Truyols-San- principal-components analysis (Foote 1992, tonja 1956; Radinsky 1982; Van Valkenburgh 1993; Wills et al. 1994) and by comparing the 1988, 1989) resulting from convergence of un- differences in average frequency of character related taxa to similar ecomorphological change between categories (Sanderson 1993). types, including meat-specialists, bone-crack- Finally, we used discriminant function anal- ers/scavengers, omnivores, and generalists ysis to assign ``degrees of specialization'' to (Van Valkenburgh 1988; Werdelin 1996). Such hypercarnivores and compared the disparity iterative evolution produces natural replicates values of different levels of specialization. and is conducive for comparative study. Of the recognized carnivoran ecomorphs, The Order Carnivora the niche of the meat specialist, or hypercar- The order Carnivora is composed of 11 ex- nivore, is associated with a diet comprising tant and two extinct families of meat-eating more than 70% meat, in contrast to the gen- mammals. The diagnostic character for Car- eralist (Van Valkenburgh 1988, 1989), which nivora is the carnassial pair, the fourth upper may eat 50±60% meat with vegetable matter premolar and ®rst lower molar, which in this and invertebrates making up the remainder of group have been modi®ed as shearing blades the diet. Ecological specialization to hypercar- for effective slicing of meat. Although the nivory is associated morphologically with shearing carnassials are a synapomorphy for speci®c changes in the skull and dentition that this group, members of Carnivora, hereafter include a relative lengthening of the shearing called carnivorans, have diversi®ed to occupy edges, composed of the trigon of the upper a wide range of ecological niches, and include fourth premolar and the trigonid of the lower highly carnivorous clades such as cats (Feli- ®rst molar, and reduction or loss of the po- dae) and weasels and martens (Mustelidae), stcarnassial dentition, the second and third generalists like the dogs and foxes (Canidae), lower molars and ®rst and second upper mo- insectivores like the mongoose (Herpestidae), lars, teeth used for chewing or grinding food omnivores like the bears (Ursidae) and rac- (Van Valkenburgh 1989; Hunt 1998). The facial 110 JILL A. HOLLIDAY AND SCOTT J. STEPPAN portion of the skull frequently shortens as 1982; Werdelin 1983; Van Valkenburgh 1991) well, an alteration thought related to main- or body-size correlates (Van Valkenburgh taining high bite force (Van Valkenburgh and 1990; Gittleman and Purvis 1998; Gardezi and Ruff 1987; Radinsky 1981a,b; Biknevicius and da Silva 1999). Van Valkenburgh 1996). Although the absence There are, of course, various reasons why of dietary data for many fossil taxa suggests any particular clade might not exhibit certain that the term ``hypercarnivore-morph'' may morphologies, including lack of genetic vari- be more appropriate, in this paper those taxa ation, functional constraint, stabilizing selec- with morphological characteristics consistent tion, or competition (Smith et al. 1985; Brooks with a hypercarnivorous diet will be called and McLennan 1991). Additional